Telemetry-Based UX Inference for Unsampled Computing Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Service providers face challenges in collecting and analyzing usage data from a large number of computing devices due to resource and cost limitations, as well as privacy regulations, leading to inaccurate representation of user experience across a broader population when implementing sampling policies.
Innovation Solution
A system that leverages telemetry data from a group of sampled computing devices and uses propensity score matching to identify a representative sampled device for unsampled devices, utilizing configuration data to predict metrics of interest using a regression model and matching algorithms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If sampling policy is implemented to reduce resource and cost limitations, then resource consumption and cost are reduced, but measurement precision of user experience quality deteriorates
Solution Approach 1:
The patent introduces configuration data as an intermediary that bridges sampled and unsampled devices. By matching configuration attributes (device type, OS version, geographic region) between sampled and unsampled devices, the system enables indirect inference of user experience quality for unsampled devices without directly collecting their usage data, thus maintaining measurement precision while reducing resource consumption
Solution Approach 2:
The patent creates virtual copies of unsampled devices by identifying sampled devices with matching configuration profiles. Instead of collecting data from all devices, the system copies the configuration characteristics of unsampled devices from representative sampled devices and applies the collected usage data to these virtual copies, enabling accurate representation of the broader user population
2Object-affected harmful factors
If sampling policy is implemented to comply with privacy regulations, then privacy compliance is improved, but measurement precision of user experience quality deteriorates
Solution Approach 1:
Configuration data serves as a privacy-friendly intermediary that allows the system to generalize findings from sampled devices to unsampled devices without collecting sensitive personal information. The matching process uses aggregated configuration attributes rather than individual user data, maintaining privacy compliance while enabling population-level inferences
Solution Approach 2:
The system creates anonymized virtual representations of unsampled devices based on configuration matching, allowing analysis without accessing or storing sensitive user information. This copying approach enables privacy-compliant measurement by working with aggregated configuration profiles rather than individual device data
3Measurement precision
If telemetry data is collected from all computing devices, then measurement precision of user experience quality is improved, but resource consumption and cost increase
Solution Approach 1:
The patent applies partial action by collecting telemetry data from only a subset of sampled devices rather than all devices. By strategically sampling devices and using configuration matching to represent the broader population, the system achieves sufficient measurement precision for user experience quality assessment without the excessive resource consumption of universal data collection
Solution Approach 2:
The system segments the device population into sampled and unsampled groups, then further segments the sampled group by configuration attributes (device type, OS version, geographic region). This segmentation enables efficient resource allocation by collecting data only from representative subsets while maintaining overall measurement precision through systematic matching
Data Source
AI summary
Disclosed herein is a system for leveraging telemetry data representing usage of a component installed on a group of sampled computing devices to confidently infer the quality of a user experience and/or the behavior of the component (e.g., an operating system) on a larger group of unsampled computing devices. The system is configured to use a propensity score matching approach to identify a sampled computing device that best represents an unsampled computing device using configuration data that is collected from both the sampled and unsampled computing devices. The quality of the user experience and/or the behavior of the component may be captured by a metric of interest (e.g., a QoS value). Accordingly, the system is configured to use the known metric of interest, determined from the telemetry data collected for the sampled computing device, to determine or predict the metric of interest for the unsampled computing device.