Context-Based Device Testing Using Predictive Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Software testing is inefficient due to the need to test applications in numerous contexts, making it challenging to predict which contexts are most likely to uncover test failures, thereby increasing testing time and reducing effectiveness.
Innovation Solution
A test controller uses machine learning techniques to build a predictive model based on initial testing in randomly selected contexts, identifying priority contexts likely to produce test failures, allowing focused testing in these contexts before expanding to additional randomly generated contexts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If testing is performed in numerous contexts to increase testing effectiveness, then testing effectiveness is improved, but testing time increases
Solution Approach 1:
The system performs preliminary testing in randomly selected contexts to gather data before the main testing phase. This preliminary action builds a predictive model that identifies priority contexts, allowing the system to focus subsequent testing efforts on the most problematic areas rather than uniformly testing all possible contexts.
Solution Approach 2:
The system changes the parameter of context selection from random uniform distribution to priority-based distribution. By using the predictive model to identify contexts with higher failure probabilities, the system dynamically adjusts which contexts are tested and how frequently, optimizing the balance between testing effectiveness and time consumption.
2Reliability
If all possible contexts are tested to ensure comprehensive coverage, then testing completeness is improved, but resource allocation efficiency deteriorates
Solution Approach 1:
Instead of applying uniform testing intensity across all contexts, the system applies local quality by concentrating testing resources on specific priority contexts identified by the predictive model. High-risk contexts receive more testing attention while low-risk contexts receive less, optimizing resource allocation based on local needs rather than global uniformity.
Solution Approach 2:
The system performs partial action by focusing on the most critical contexts rather than exhaustively testing all possible contexts. The predictive model identifies a subset of priority contexts that are most likely to reveal failures, allowing the system to achieve high testing effectiveness without the excessive resource consumption of complete context coverage.
3Ease of operation
If testing focuses on randomly selected contexts to simplify context selection, then ease of operation is improved, but testing effectiveness deteriorates
Solution Approach 1:
The predictive model acts as an intermediary between random context selection and priority-based context selection. It processes data from initial random testing and transforms it into prioritized context recommendations, combining the simplicity of random selection with the effectiveness of targeted testing without requiring manual context prioritization.
Data Source
AI summary
Software applications are tested in different contexts, such as on different devices and under different conditions. During initial testing of an application, conditions of contexts are selected randomly, and the application is tested in each resulting context. After obtaining results from a sufficient number of contexts, the results are analyzed to create a predictive model indicating, for any postulated context, whether testing of the application is most likely to fail or to otherwise produce negative test results. The model is then analyzed to identify contexts that are most likely to produce negative results or failures, and those contexts are emphasized in subsequent application testing.