Inspect Element Framework for Mobile DAP Content
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing tools are cumbersome for inspecting mobile device screen elements, especially on small screens, and struggle to identify web elements effectively, requiring additional software and hardware setups on desktops.
Innovation Solution
A method and system that captures a screenshot of a mobile device screen, displays it on a larger computer screen, allows users to select and highlight elements, and captures their attributes, distinguishing between native and web elements, with the ability to store these attributes for digital adoption platform content creation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Difficulty of detecting and measuring
If existing tools like UiAutomatorViewer or Xcode Accessibility Inspector are used to inspect mobile screen elements, then element inspection capability is provided, but device complexity increases due to requiring specific software and drivers installation on desktop
Solution Approach 1:
The patent uses a screenshot (visual copy) of the mobile device screen displayed on the desktop instead of requiring direct connection and inspection tools. The screenshot captures the visual state of mobile elements, allowing inspection through image processing and coordinate mapping without needing UiAutomatorViewer or Xcode Accessibility Inspector installed on the desktop system.
2Ease of operation
If content authoring is done directly on mobile device screen, then portability is improved, but ease of operation deteriorates due to small screen size making it cumbersome
Solution Approach 1:
The patent transitions the content authoring interface from the mobile device's small screen to the desktop's larger display by showing a screenshot of the mobile screen on the desktop. This dimensional shift allows content creators to work with the mobile interface layout in a larger space, improving ease of operation while maintaining the ability to create content for mobile devices.
3Difficulty of detecting and measuring
If existing inspection tools are used, then element identification is provided, but adaptability worsens because they cannot identify web elements
Solution Approach 1:
The patent creates a universal inspection system that can identify both native mobile elements and web elements. By using screenshot-based inspection with coordinate mapping and element attribute extraction, the system adapts to different element types (native UI elements, web view elements, hybrid elements) without requiring different tools, thus providing multi-functionality and versatility in element identification.
Data Source
AI summary
A technique for inspecting screen elements of a mobile device with a laptop or desktop computer displays a screen-shot of the mobile device. Visible screen elements are identified as the user moves a user interface pointer over the screen-shot. Individual screen elements are highlighted and a user may select individual screen elements. Additional information regarding screen element identification and attributes may be displayed. The screen element attributes may be captured for use in generating DAP content for a mobile device.