Automated Device Test Triage via Dynamic Reordering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current testing processes for mobile computing devices are inefficient, leading to delays and human errors due to manual processes and long release cycles for fixing defects in test suites, which can cause critical and costly delays in device approvals.
Innovation Solution
A method and system for accelerated device approval using a test infrastructure system software that enables continuous, automated testing and triaging of tests across multiple devices, allowing for parallel execution of tests and dynamic adjustment of test suites based on real-time results, thereby reducing delays and improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual testing processes are used to ensure thorough device testing, then testing coverage and reliability are improved, but testing time and human error risks increase
Solution Approach 1:
The testing system performs self-service through automated test execution and result analysis. The host computing device automatically manages test suites, executes tests on multiple DUTs, analyzes results, and propagates fixes without human intervention, thereby maintaining reliability while reducing time loss.
Solution Approach 2:
Manual testing processes are replaced with an automated computing system. The mechanical/manual operations of test execution, result collection, and defect tracking are substituted with software-based automated testing infrastructure that operates continuously without human involvement.
2Reliability
If test suites are executed sequentially on each device to ensure thorough testing, then testing completeness is improved, but testing efficiency and productivity decrease
Solution Approach 1:
The testing process is segmented across multiple independent DUTs that can be tested simultaneously. The test suite is divided and distributed to multiple devices, allowing parallel execution while maintaining comprehensive coverage through aggregation of individual test results.
Solution Approach 2:
The testing approach transitions from a single-dimensional sequential process to a multi-dimensional parallel process. Tests are executed across multiple devices concurrently in space, transforming the time-consuming sequential execution into efficient parallel execution without compromising completeness.
3Reliability
If fixes are propagated manually through the ecosystem to maintain control and accuracy, then fix quality is improved, but fix propagation speed decreases
Solution Approach 1:
The system implements automated feedback loops where test results from multiple DUTs are continuously monitored and analyzed. When defects are identified, the system automatically propagates fixes to the appropriate devices and verifies the fixes through re-testing, ensuring both quality control and rapid propagation.
Solution Approach 2:
The host computing device serves as an intermediary that coordinates fix propagation across the ecosystem. It manages the distribution of fixes to multiple DUTs, tracks propagation status, and verifies effectiveness, maintaining control and accuracy while enabling simultaneous propagation to multiple devices.
4Reliability
If comprehensive test suites are used to cover all device functions, then testing coverage is improved, but test execution time and resource consumption increase
Solution Approach 1:
The system employs partial action by selectively executing test suites based on device-specific needs and failure patterns. Instead of running all possible tests on every device, the system adjusts test execution to cover only relevant functions, maintaining adequate coverage while reducing unnecessary execution time and resource consumption.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods and apparatus are provided for testing computing devices. A host computing device is provided for testing devices under test (DUTs) using a test suite that includes first and second tests. The DUTs can include a first group of DUTs with a first DUT and a second group of DUTs with a second DUT. The first and second groups of DUTs can share a common design. The host computing device can determine that the DUTs execute the first test before the second test. The host computing device can receive failing first test results for the first DUT. The host computing device can determine, based on the first test results and that the first and second DUT groups share a common design, to execute the second test before the first test and can subsequently instruct the second DUT to execute the second test before the first test.