Augmented Video Analytics for IoT Device Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Testing Internet of Things (IoT) devices is challenging due to the complexity of interactions among heterogeneous devices and applications, lack of a common protocol, and difficulties in identifying bottlenecks in real-time field trials.
Innovation Solution
A method using video analytics that captures visually and non-visually perceptible data, event logs, and human interactions to overlay markers on video frames, enabling detection of performance, functionality, and usability issues, and indexing to trace root causes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real-time field trials are used to test IoT devices in actual contexts, then testing accuracy and reliability are improved, but testing duration and time consumption increase significantly
Solution Approach 1:
The system performs preliminary automated monitoring and issue detection during real-time field trials before manual analysis is needed. Video analytics and sensors continuously monitor device interactions and detect issues in advance, so when testing concludes, the analysis phase is already substantially complete rather than starting from scratch.
Solution Approach 2:
Manual video review and issue detection by testers are replaced with automated video analytics systems, sensors, and AI-based detection algorithms. This substitution enables continuous automated monitoring during testing without requiring manual intervention for every observation, significantly reducing the time needed for post-testing analysis.
2Measurement precision
If manual monitoring and analysis of video footage is performed to detect issues, then detection accuracy is improved, but labor requirements and operational complexity increase
Solution Approach 1:
The system performs self-service automated issue detection through video analytics and sensor data analysis without requiring manual review of every interaction. The automated system independently identifies usability issues, performance problems, and functional defects, reducing the burden on testers while maintaining high detection accuracy through multiple detection mechanisms.
Solution Approach 2:
An automated video analytics system and sensor network serve as intermediaries between the tested devices and human analysts. These intermediaries continuously monitor device interactions, detect issues, and prepare structured reports, acting as a bridge that maintains detection accuracy while reducing direct human involvement in routine monitoring tasks.
3Loss of information
If multiple sensors and data sources are integrated to comprehensively monitor device interactions, then measurement completeness is improved, but system complexity and data processing requirements increase
Solution Approach 1:
The monitoring system is segmented into specialized components: video cameras for visual interaction capture, sensors for physical measurements, microphones for audio detection, and automated analytics systems for data processing. Each component focuses on specific data collection tasks, and results are integrated into a unified report, making the complex system manageable through functional segmentation.
Solution Approach 2:
The system employs multi-functional integrated monitoring that combines video analytics, sensor data collection, audio detection, and automated issue detection within a single unified platform. This universal approach allows one system to perform multiple monitoring functions simultaneously, reducing the need for separate specialized systems while maintaining measurement completeness across all interaction types.
Data Source
AI summary
An approach is provided for testing an Internet of Things device. First data captured on video and indicating visual device output and second data indicating non-visual device output are received during testing of the device. An event log of the device is received. Human interactions with the device are received. The first and second data, the event log entries, and indications of the human interactions are overlaid onto frames of the video that include the device, so that timings of the frames are correlated with timestamps of the overlaid items. Based on the overlaid items, performance, functionality, and usability issues are detected and markers of the issues are generated and overlaid onto a timeline of the video. Responsive to a user selection of one of the markers, the computer locating and displaying a frame of the video that depicts the issue that corresponds to the selected marker.