Automated Screen Capture for UI Testing via Object Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automated software testing tools face challenges in effectively capturing and maintaining accurate screen capture images of user interface features, especially when these features change, which can hinder the testing and documentation process.
Innovation Solution
A computer-implemented method and system that identifies a portion of a user interface display associated with a page object, creates a screen capture image, and associates it with a documentation record, allowing for annotation-driven image capture, explicit or implicit scope definition using the Document Object Model, and image cropping to isolate the page object, with the ability to replace prior images when changes occur.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If automated testing tools capture full screen images for documentation, then visual references are complete, but image processing and storage requirements increase significantly
Solution Approach 1:
The patent extracts and captures only the specific portion of the screen that contains the page object under test, rather than capturing the entire screen. This is achieved by identifying the coordinates and dimensions of the target element and cropping the screen capture to that specific region, thereby reducing image data volume while maintaining visual reference completeness for the relevant testing area.
Solution Approach 2:
The patent segments the screen capture process into two distinct parts: the full screen capture and the cropped portion containing the page object. By dividing the imaging task into these segments, the system can maintain complete visual references when needed while efficiently storing and processing only the relevant cropped portions for routine testing documentation.
2Reliability
If screen capture images are taken frequently to ensure accuracy, then visual documentation is up-to-date, but processing time and computational resources increase
Solution Approach 1:
The patent extracts only the necessary visual information by cropping screen captures to the specific page object region. This reduction in image size and complexity directly decreases processing time while maintaining the accuracy and reliability of visual references for automated testing, as only the relevant portion needs to be captured and processed.
3Measurement precision
If manual testing is used to capture and verify screen images, then visual documentation accuracy is high, but labor requirements and testing time increase
Solution Approach 1:
The patent implements self-service automation where the testing system automatically captures screen images, crops them to the relevant page object portion, and associates them with test results without requiring manual intervention. The automated testing tool identifies the target element coordinates, performs the screen capture and cropping operations, and manages the visual documentation independently, thereby maintaining high accuracy while eliminating manual labor requirements.
Solution Approach 2:
The patent replaces the manual mechanical process of screenshot capture and verification with an automated computational system. The automated testing tool uses programmatic methods to capture screens, process images, and verify visual references, substituting human manual operations with automated software processes that achieve the same or better precision with reduced effort.
4Measurement precision
If screen capture annotations are added to track page objects, then image capture precision improves, but documentation complexity increases
Solution Approach 1:
The patent creates a universal annotation system that serves multiple functions: it identifies page object locations for automated capture, stores coordinate information for cropping operations, and provides metadata for documentation association. This multi-functional annotation approach improves image capture precision while avoiding increased complexity by consolidating multiple purposes into a single standardized annotation structure.
Data Source
AI summary
A computer-implemented method, computer program product, and computing system is provided for automatically creating screen captures of a user interface. In an implementation, a method may include identifying a portion of a user interface display associated with a page object. The method may also include creating a screen capture image including the portion of the user interface display associated with the page object. The method may further include associating the screen capture image with a documentation record for the page object.