Data Stream Transmission with PCA-DTW Change-Based Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large volumes of complex data in industrial and healthcare applications lead to high transmission and storage costs, with existing data compression methods compromising data quality and requiring efficient methods to reduce data volume while maintaining high transfer quality.
Innovation Solution
A computer-implemented method using principal component analysis (PCA) and dynamic time warping (DTW) to detect data changes, where data packages are transmitted only when significant changes are detected, projecting data points onto principal components, aligning, and calculating distances to determine if the data package reveals new information, thereby reducing the total volume of transmitted data without compromising quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If data compression is performed before transmission, then transmission costs are reduced, but data quality deteriorates
Solution Approach 1:
The patent extracts only the essential changes from the data stream by comparing consecutive packages and transmitting only when the change exceeds a threshold. This selective extraction approach reduces transmission volume without compressing the actual data, thereby maintaining data quality while lowering transmission costs.
Solution Approach 2:
Instead of transmitting all data or applying full compression, the patent transmits only the partial information that represents significant changes. This partial action approach sends minimal necessary data packets, reducing transmission costs while preserving the essential information quality.
2Quantity of substance
If data manipulation such as averaging is performed at a central collector, then data volume is reduced, but data quality deteriorates significantly
Solution Approach 1:
The patent creates simplified representations (copies) of data for comparison purposes only. The original data packets are preserved, and only change detection information is transmitted. This copying approach reduces transmission volume while maintaining access to the complete original data quality when needed.
3Reliability
If all data packages are transmitted to data storage, then data availability is maintained, but transmission and storage costs increase
Solution Approach 1:
The patent extracts only the changed portions of data for transmission and storage. By comparing consecutive data packages and identifying only those with significant changes (exceeding the threshold), the system maintains data availability for changed information while dramatically reducing transmission and storage costs.
Solution Approach 2:
The system uses feedback from change detection comparisons to control transmission decisions. By continuously comparing new data packages against previous ones and using threshold-based feedback, the system intelligently determines what needs to be transmitted and stored, maintaining reliability while optimizing costs.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented method for data transmission with a reduced data volume, comprises: - sampling a data stream (10) in consecutive time intervals and generating consecutive data packages (P) which each contain data points sampled in a window of a predetermined number of said consecutive sampling time intervals; - calculating, by means of principal component analysis, for each data package (P) projections of its data points onto a set of principal components; - aligning, by means of dynamic time warping, for each newly generated data package and for each principal component the projected data points to respective projected data points of an earlier generated data package and calculating distances between the aligned data points; - determining a maximum of the calculated distances and comparing it to a threshold value; and - transmitting the newly generated data package to a data sink (11) only if the maximum distance exceeds the threshold value (Th) and otherwise inhibiting the transmission.