Augmented Sensor Data for Neural Network Performance Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems fail to accurately test the peak performance of consumers, such as machine learning models, due to differences in processing capabilities between data producers and consumers, leading to inaccurate determination of performance capabilities and reduced likelihood of efficient system implementation.
Innovation Solution
Generating augmented sensor data instances to supplement actual data when the message queue is empty, allowing for real-time or near real-time data provision to consumers, which includes emulating or modifying previous data to represent expected sensor data, thereby determining peak performance metrics accurately.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional message queue systems are used to process sensor data, then the system operates with simple data flow, but the consumer processing capabilities cannot be accurately tested because the system is limited by the slower sensor production rate
Solution Approach 1:
The system performs preliminary actions by generating augmented sensor data in advance and storing it in a buffer before the actual sensor data becomes available. This allows the consumer to immediately process pre-generated augmented data without waiting for slow sensor production, thereby enabling accurate measurement of peak consumer performance while maintaining simple data flow architecture
Solution Approach 2:
The system introduces an intermediary component that generates and manages augmented sensor data to bridge the gap between slow sensor production and fast consumer processing. This intermediary augmented data mechanism allows the consumer to operate at full capacity without being bottlenecked by the sensor production rate, thus enabling accurate performance measurement while maintaining system simplicity
2Reliability
If manual or automatic generation of sensor data is used, then data can be produced for testing, but it requires significantly more time than the inferencing time required by the consumer, leading to diminished runtime performance
Solution Approach 1:
The system creates copies of sensor data by generating augmented sensor data that mimics the characteristics and format of real sensor data. These copied augmented data instances can be immediately provided to the consumer for processing without requiring time-consuming manual or automatic sensor data generation, enabling reliable performance testing while minimizing time loss
Solution Approach 2:
The system performs preliminary data preparation by pre-generating augmented sensor data that can be immediately used for consumer processing. This eliminates the need for time-consuming data generation during runtime, allowing the consumer to maintain high runtime performance while still enabling reliable performance testing through pre-prepared test data
3Measurement precision
If the message queue waits for new sensor data before responding to consumer requests, then data accuracy is maintained, but the consumer runtime performance is diminished due to blocking waits
Solution Approach 1:
The system introduces augmented sensor data as an intermediary solution that allows the message queue to immediately respond to consumer requests without blocking waits. The augmented data maintains the necessary accuracy characteristics for performance testing while enabling the consumer to process data continuously at high speed, thus resolving the contradiction between data accuracy and processing speed
Data Source
AI summary
In various examples, performance capabilities of a consumer—such as inference rates of a neural network executing on underlying hardware—may be tested or demonstrated for producers that have lower production rates than consumption rates of the consumer. For example, augmented data instances may be leveraged to augment production data from the producer in order to increase a frequency of transmission of data instances to a consumer. As a result, a data set including additional or alternative instances of data may be generated to emulate real-world data for demonstrating potential performance capabilities of a consuming algorithm beyond a currently required performance capability corresponding to a producer.