Spatial Probability Modeling for Low-Memory Data Compression

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current data processing techniques face inefficiencies in modeling large sequences of data, analyzing patterns, reducing redundancy, and lowering average entropy without loss, particularly due to high memory and computational costs, and the inability to efficiently compress random-like data.

Innovation Solution

The method identifies a subset of states within a data system using a spatial statistical model that represents systemic characteristics and relationships, allowing for efficient modeling and encoding of data systems with reduced memory and processing costs, enabling the encoding and decoding of random-like data.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If adaptive modeling is used to improve compression ratios, then memory resources are depleted when the index or dictionary becomes too large

Engineering Contradiction:

Improvecompression ratioVSAvoidmemory resources

Core Design Contradiction:

Loss of informationVSQuantity of substance

Solution Approach 1:

The patent extracts only the essential probability information needed for compression by using a simplified context model that tracks only the most recent symbol and its probability, rather than maintaining a complete adaptive index or dictionary of all possible patterns. This extraction approach keeps memory usage constant while preserving the essential compression benefits.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Instead of building up a large index or dictionary and then trying to manage its size, the patent inverts the approach by starting with a minimal model and only storing probability information for recently observed symbols. This inversion prevents memory depletion from occurring in the first place, rather than trying to solve it after the fact.

Inventive Principle:

Principle #13The other way round (Inversion)

2Loss of information

If adaptive modeling is used to improve compression ratios, then computational cost increases due to constant model updates

Engineering Contradiction:

Improvecompression ratioVSAvoiddata processing speed

Core Design Contradiction:

Loss of informationVSProductivity

Solution Approach 1:

The patent extracts only the essential probability update operation needed for compression, eliminating the need to update and search through large indexes or dictionaries. By focusing only on updating the probability of the most recent symbol based on the current context, the computational cost is reduced to a simple constant-time operation.

Inventive Principle:

Principle #2Taking out (Extraction)

3Loss of information

If the number of patterns in the index or dictionary is increased to improve compression, then any advantage gained by compressing the data is eliminated after the index or dictionary is appended

Engineering Contradiction:

Improvecompression ratioVSAvoidindex or dictionary size

Core Design Contradiction:

Loss of informationVSQuantity of substance

Solution Approach 1:

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS11475343B2Database utilizing spatial probability models for data compression

Publication Date: 2022.10.18 MIRAL LAB LLC

AI summary

A method, article comprising machine-readable instructions and apparatus that processes data systems for encoding, decoding, pattern recognition/matching and data generation is disclosed. State subsets of a data system are identified for the efficient processing of data based, at least in part, on the data system's systemic characteristics.