Cross-Device Operation Playback via Environment Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing solutions for operation recording and playback are not suitable for user operations with respect to an APP on a digital terminal device, as they fail to execute scripts across different system platforms and result in incorrect control positioning due to variations in device screen sizes and resolutions.
Innovation Solution
A method and apparatus for operation recording and playback that records operations on a first device, acquires environment data, and performs user operations on a second device based on the recorded operations, determining the corresponding target control using environment data from both devices to ensure accurate control positioning across different platforms and devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If existing operation recording solutions are used for test scenes, then operation automation is achieved, but control positioning fails across different devices and platforms
Solution Approach 1:
The patent changes the parameter basis for control identification from fixed coordinates to dynamic environment data. By capturing environment data (page screenshots, control attributes, layout information) and using it to locate controls during playback, the system adapts to different devices and platforms while maintaining reliable control positioning.
Solution Approach 2:
The patent introduces environment data as an intermediary between the recorded operation script and the target control. This intermediary layer captures the contextual information needed to accurately locate controls across different environments, resolving the mismatch between recorded operations and actual control positions on various devices.
2Adaptability or versatility
If scripts are developed for different system platforms, then platform compatibility is improved, but development complexity increases
Solution Approach 1:
The patent creates a universal operation recording and playback system that works across multiple platforms and devices. By using platform-agnostic environment data capture and control location methods, a single operation script can be executed on different systems without requiring separate script versions for each platform.
Solution Approach 2:
The patent captures environment data as a copy of the actual device state during recording. This environment data copy contains all necessary information about controls, layouts, and page structures, allowing the playback system to recreate the original operational context without needing the actual source device, thus enabling cross-platform execution.
3Ease of manufacture
If control locations are fixed in recording, then operation recording is simple, but control positioning fails due to screen size and resolution variations
Solution Approach 1:
The patent makes the control location system dynamic by using environment data captured at runtime. Instead of relying on fixed coordinates recorded once, the system dynamically locates controls based on the actual environment data present during playback, adapting to different screen sizes, resolutions, and device configurations.
Data Source
AI summary
A method for recording and playback including: recording an operation of a first device based on an operation recording instruction to obtain a user operation record; acquiring environment data of the first device; and performing user operations of the user operation record sequentially in a second device based on an operation playback instruction, where the performing the user operations of the user operation record includes, for one of the user operations: determining a corresponding target control in the second device based on the environment data of the first device and environment data of the second device, and performing a respective user operation based on the target control.