Statistical Streaming for Large Data Display
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Personal computers struggle to process and display large data files in real time due to limitations in solid-state memory, leading to decreased performance when handling files exceeding 256 Mb to 512 Mb, and this issue is expected to persist as data file sizes continue to grow.
Innovation Solution
A method and system that determine display scaling parameters to use statistical values for digital data, allowing for the generation of display signals by reading and processing digital data and/or statistical values for predetermined sized blocks, reducing the need to read entire large data files and enabling faster statistical calculations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data files are increased in size to allow greater measurement precision, then measurement precision is improved, but processing speed deteriorates
Solution Approach 1:
The patent divides large data files into smaller manageable blocks that can be processed independently. By segmenting the data stream into blocks and processing them separately, the system maintains measurement precision while improving processing speed through parallel block processing and reduced memory requirements.
Solution Approach 2:
The patent extracts and stores statistical values (minimum, maximum, mean, standard deviation) from the raw data blocks. These extracted statistical characteristics allow the system to represent large datasets compactly, maintaining measurement precision while significantly reducing processing and memory requirements.
2Measurement precision
If data files are increased in size to allow greater measurement precision, then measurement precision is improved, but display speed deteriorates
Solution Approach 1:
The patent extracts statistical values from data blocks and stores them separately. When displaying data, the system can use these pre-computed statistical values to rapidly generate displays without reading and processing entire large data files, thus maintaining measurement precision while dramatically reducing display time.
Solution Approach 2:
The patent performs preliminary computation of statistical values during data acquisition and storage. This preliminary action allows the display system to quickly render visualizations using pre-computed statistics rather than calculating them in real-time during display, resolving the contradiction between precision and display speed.
3Quantity of substance
If solid-state memory capacity is increased to handle larger data files, then data handling capability is improved, but cost and complexity increase
Solution Approach 1:
The patent extracts statistical characteristics from raw data and stores them as compact representations. This extraction reduces the effective data volume that must be held in memory, allowing the system to handle large measurement datasets with limited memory capacity while avoiding the need for expensive high-capacity solid-state memory upgrades.
Data Source
AI summary
A method of providing a display signal, wherein digital data to be displayed has associated with it statistical values determined for pre-determined sized blocks of the digital data, comprising the steps of: determining display scaling parameters; determining from the display scaling parameters the extent to which the statistical values are suitable to use in place of some or all of the digital data when determining which data points of either the digital data and/or statistical values to display; determining the data points to display; reading the digital data and/or statistical values to be displayed; and generating the display signal from the digital data and/or statistical values.