Time-Tagged Photon Data Compression Using Sorting and Delta Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for compressing Time Tagged data from TCSPC instruments in FLIM applications are inefficient, achieving poor compression ratios and slow encoding and decoding speeds, which are not sufficient for the large datasets generated.
Innovation Solution
A computer-implemented method that separates Time Tagged data records into groups, sorts them by photon arrival time, subtracts the content of each record by its adjacent record, and then compresses the modified records using a conventional compression method.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If standard compression methods (Huffman coding, Deflate) are used on Time Tagged data, then the compression process is simple to implement, but the compression ratio is poor and encoding/decoding speed is slow
Solution Approach 1:
The patent applies preliminary sorting of Time Tagged data records by arrival time before compression. This pre-processing step creates a more predictable and ordered data structure that enables faster encoding/decoding operations while achieving better compression ratios, directly resolving the speed and simplicity contradiction
Solution Approach 2:
The patent transforms the data representation by converting absolute arrival times into time differences (deltas) between consecutive records. This parameter transformation reduces the data range and variability, enabling more efficient compression with higher speeds while maintaining implementation feasibility
2Ease of manufacture
If standard compression methods are used on Time Tagged data, then the implementation is straightforward, but the compression ratio is insufficient for large datasets
Solution Approach 1:
The patent changes the data parameters by computing differences between consecutive time values instead of compressing absolute values. This transformation reduces the numerical range and increases data redundancy, achieving superior compression ratios (reducing 100 GB to much smaller sizes) while keeping the implementation approachable
Solution Approach 2:
By sorting records beforehand, the patent creates an ordered sequence where time differences between adjacent records are smaller and more predictable. This preliminary action significantly improves compression ratio by creating patterns that compression algorithms can exploit more effectively
3Ease of manufacture
If conventional compression libraries are used, then the implementation is easy, but the encoding and decoding speed is too slow to keep up with storage and network speeds
Solution Approach 1:
The patent transforms time values into differences, which have a smaller and more constrained range. This parameter change enables the use of more efficient encoding schemes that operate faster, achieving encoding/decoding speeds that can keep pace with storage and network transfer rates while remaining implementable with standard tools
4Speed
If no compression is applied, then the data can be accessed and analyzed immediately, but the storage requirements and transfer times become prohibitively large
Solution Approach 1:
The patent applies parameter transformation (converting absolute times to time differences) that achieves high compression ratios. This reduces data volume dramatically while maintaining fast access speeds, as the transformed data structure enables rapid decompression and analysis without the burden of handling prohibitively large datasets
Data Source
AI summary
A computer-implemented method for compression of Time Tagged data including Time Tagged data records includes the step of separating the Time Tagged data records into a plurality of groups. The method also includes sorting the Time Tagged data records in at least one of the groups by a photon arrival time. The method also includes subtracting a content of a record by a content of an adjacent record resulting in modified records. The method also includes compressing the modified records with a compression method.