Neural Upsampling of Financial Time-Series After Lossy Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression techniques, such as HEVC, result in loss of data during compression and decompression, particularly in financial time-series data, which can lead to inaccurate analysis and management of large datasets.
Innovation Solution
A system and method using a neural network to upsampling decompressed financial time-series data after lossy compression, incorporating a novel AI deblocking network with recurrent layers for feature extraction and a channel-wise transformer with attention to capture complex inter-channel dependencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If lossy compression techniques are used to reduce data size, then storage efficiency and bandwidth utilization are improved, but data accuracy and information completeness deteriorate
Solution Approach 1:
A neural network upsampler is introduced as an intermediary component between the lossy compression decoder and the final data usage. This neural network processes the decompressed data and recovers lost information by learning the mapping between compressed and original data distributions, thereby mediating the information loss caused by compression
Solution Approach 2:
The system changes the parameters of the decompressed data by applying neural network-based upsampling transformations. The neural network learns optimal parameter transformations to reconstruct high-frequency components and restore data characteristics that were lost during compression, effectively changing the data parameters back toward their original state
2Productivity
If high-frequency financial data is compressed for efficient storage and transmission, then productivity and resource utilization are improved, but measurement precision and analysis accuracy deteriorate
Solution Approach 1:
The neural network upsampler performs preliminary restoration of data quality before the data is used for financial analysis. By pre-processing the decompressed data through the neural network, the system recovers measurement precision in advance, ensuring that subsequent analytical operations work with accurate data while maintaining the efficiency benefits of compression
3Device complexity
If lossy compression is applied to financial time-series data, then device complexity and storage requirements are reduced, but reliability and data integrity deteriorate
Solution Approach 1:
The system implements a feedback mechanism where the neural network upsampler continuously processes decompressed data and uses learned patterns to restore integrity. The neural network learns from training data the relationship between compressed and original states, providing feedback-based correction that maintains reliability while allowing lossy compression to reduce storage complexity
Data Source
AI summary
A system and methods for upsampling of decompressed financial time-series data after lossy compression using a neural network that integrates AI-based techniques to enhance compression quality. It incorporates a novel deep-learning neural network that upsamples decompressed data to restore information lost during lossy compression, taking advantage of cross-correlations between time-series data sets.