Dynamic Device Clustering for Software Test Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Testing large or complicated software applications on different devices with varying performance results due to hardware and software configurations, leading to incorrect identification of malfunctioning or non-malfunctioning builds.
Innovation Solution
A system that clusters devices with similar performance by measuring statistical values and using a combination of t-tests, consistency tests, and expected consistency tests to identify devices that can be considered as performing similarly, allowing for consistent test results across the cluster.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If additional devices are added to perform tests, then testing capacity and speed are improved, but test result consistency and reliability deteriorate due to device performance variations
Solution Approach 1:
The patent introduces an intermediary layer (device clustering system) between the test execution and result analysis processes. This system mediates by grouping devices into clusters based on their performance characteristics, allowing tests to be distributed across multiple devices while maintaining result consistency through cluster-based normalization. The intermediary clustering mechanism enables scaling testing capacity without sacrificing reliability.
Solution Approach 2:
The system changes the parameter of device grouping from individual device basis to cluster basis. By dynamically forming clusters based on performance parameters (speed, memory, processor cycles) and adjusting which devices belong to which clusters, the system adapts to device variations while maintaining consistent test results across the cluster level.
2Adaptability or versatility
If tests are executed on multiple different devices, then testing coverage is improved, but measurement precision deteriorates due to device performance variations
Solution Approach 1:
The patent applies local quality by treating each device cluster as a distinct group with its own performance characteristics. Instead of applying a uniform measurement standard across all devices, the system creates localized measurement contexts within each cluster. Tests executed within a cluster benefit from consistent performance characteristics, improving measurement precision while maintaining broad coverage through multiple clusters.
Solution Approach 2:
The system adds a new dimension to device classification by introducing cluster membership as an additional layer beyond individual device identity. This dimensional change allows the system to account for performance variations by grouping devices in a new space (cluster space), enabling both broad coverage across many devices and precise measurement within each cluster dimension.
3Reliability
If device clustering is implemented, then test result consistency is improved, but system complexity increases
Solution Approach 1:
The patent segments the device population into distinct clusters based on performance characteristics. This segmentation simplifies the complexity by breaking down the heterogeneous device set into homogeneous groups, making it easier to manage and analyze test results. Each cluster can be independently managed, reducing the overall system complexity compared to managing all devices individually.
Solution Approach 2:
The clustering system is designed to be universal and multi-functional, serving multiple purposes: improving test result consistency, enabling scalable test distribution, providing performance-based device classification, and facilitating efficient resource allocation. By creating a single clustering infrastructure that serves multiple functions, the patent reduces the need for separate systems for each function, thereby managing complexity.
Data Source
AI summary
A cluster of devices can be identified where results from executing a test by any cluster devices can be considered as being from the same device. Thus, instead of waiting for a single device to produce comparable results, multiple devices from the same cluster can simultaneously perform the test and obtain the needed set of test results more quickly. The technology can identify clusters of devices that are all similar to a primary cluster device. A device pair can be considered similar when (1) a mean each of a set of test results from each device are within a first threshold of each other, (2) a measurement of the consistency of each test result set are within a second threshold of each other, and (3) a measurement of the consistency of a combination of the test results sets is between the consistency measurements of the individual test result sets.