IoT File Scheduling with Reinforcement Learning Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current IoT data transmission from industrial machines to cloud storage is inefficient due to non-optimized file size constraints, where compressed files often exceed or fall short of the predefined threshold, leading to suboptimal data transit size.
Innovation Solution
A control device with an optimizer module that employs a scheduling algorithm combined with reinforcement learning, specifically using the Multi Armed Bandits Algorithm and Upper Confidence Bound algorithm, to select and compress files, ensuring they meet the size threshold while optimizing data transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If files are compressed and sent regularly under a predefined size threshold, then data transmission constraint is satisfied, but data transit size is not optimized
Solution Approach 1:
The system dynamically adjusts the compression ratio based on file characteristics and transmission conditions. The compression application modifies compression parameters in real-time to optimize the compressed file size, transitioning from static threshold-based compression to dynamic adaptive compression that responds to changing data characteristics.
Solution Approach 2:
The invention changes the compression ratio parameter adaptively based on file type, size, and content analysis. By adjusting this key parameter dynamically, the system optimizes compressed file sizes to be as close as possible to the transmission threshold while ensuring the constraint is never violated, thereby maximizing data transit efficiency.
2Quantity of substance
If compression ratio is increased to reduce file size, then data transit size decreases, but compression time increases
Solution Approach 1:
The system applies partial compression actions by selecting optimal compression levels for different file types and characteristics. Instead of applying maximum compression to all files, the system determines the appropriate compression intensity for each file, avoiding excessive compression time while achieving sufficient size reduction to meet transmission constraints.
Solution Approach 2:
The compression application dynamically adjusts the compression ratio parameter based on file characteristics, balancing compression time and compressed size. By changing this parameter adaptively, the system avoids the time-costly maximum compression for files that don't require it, while applying higher compression where beneficial, thus optimizing the time-size tradeoff.
3Productivity
If files are selected to be sent regularly, then data transmission frequency is maintained, but compressed file size may exceed or fall short of threshold
Solution Approach 1:
The system implements feedback control by monitoring the compressed file size and adjusting the selection of files for transmission. The compression application uses feedback from previous compression results to refine file selection criteria and compression parameters, ensuring that transmitted files consistently meet the size threshold while maintaining regular transmission frequency.
Solution Approach 2:
The file selection and compression process is made dynamic, with the system continuously adapting which files to include in each transmission batch based on current data availability, file characteristics, and compression performance. This dynamic approach allows the system to maintain regular transmission frequency while precisely controlling compressed file sizes to meet the threshold.
Data Source
Figure 1
Figure 2
AI summary
The invention relates to a device for optimizing the scheduling of files to be sent to an application server at regular time intervals, the device being able to: retrieve (S1) a first set of files from a database for a time interval, the first set of files being stored in a priority queue and carrying information sent from sensors linked to industrial machines, apply (S2) the retrieved first set of files to a scheduling algorithm using a compression ratio and combined with a reinforcement learning in order to select a second set of files, compress (S3) the second set of files based on the compression ratio into a compressed file, wherein the size of the compressed file is below a size limit, send (S4) the compressed file to the application server, wherein the compression ratio is updated based on the size of the second set of files and the size of the compressed file, wherein the reinforcement learning uses the size of a previous compressed file from a previous time interval.