Event-Based Data Modeling for Big Data Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional modeling approaches are inadequate for handling big data due to their reliance on erroneous assumptions about functional relationships between inputs and outputs, which fail to account for hidden layers, time, and event variations, leading to inefficiencies in extracting meaningful knowledge from large datasets.
Innovation Solution
The system constructs mappings of inputs to outputs in historical data sets to generate conditional distributions of past outcomes, allowing for improved analysis and prediction of future events, recognizing that simple functional forms may not exist in complex systems like finance or healthcare, and leveraging modern computing resources to store and analyze vast amounts of data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional modeling approaches are used to extract knowledge from big data, then functional relationships can be established between inputs and outputs, but the approach fails to account for hidden layers, time variations, and event variations, leading to inaccurate predictions
Solution Approach 1:
The patent transforms static functional models into dynamic event-based models that adapt to changing conditions. Events are defined as discrete occurrences with specific characteristics (time, type, magnitude) that can vary independently, allowing the system to capture temporal dynamics and hidden layers without assuming fixed functional relationships. This enables accurate predictions in complex systems where traditional static models fail.
Solution Approach 2:
The patent segments continuous data into discrete events with specific attributes. By dividing the data stream into identifiable event units (each with timestamp, type, and magnitude), the system can analyze individual occurrences and their sequences rather than treating data as continuous flows. This segmentation reveals hidden patterns and layers that traditional aggregated modeling misses.
2Loss of information
If vast amounts of big data are stored and analyzed, then more information is available for decision making, but the data becomes cumbersome and difficult to parse, increasing the cost of locating valuable information
Solution Approach 1:
The patent extracts only the essential event characteristics (time, type, magnitude) from vast datasets, storing and analyzing only these key attributes rather than entire raw data records. This extraction approach maintains full information availability for reconstruction while dramatically reducing storage requirements and parsing complexity. Events serve as compact representations that capture all necessary information for analysis.
Solution Approach 2:
The patent creates simplified event-based copies of complex data structures. Instead of storing and processing original complex data formats, the system generates standardized event representations that preserve essential information while enabling efficient storage, retrieval, and analysis. These event copies can be regenerated from original data if needed, maintaining information fidelity while reducing complexity.
3Ease of manufacture
If traditional functional models are assumed to describe input-output relationships, then models can be developed for small data sets, but these models fail when applied to big data where simple functional forms may not exist
Solution Approach 1:
The patent changes the fundamental parameters of modeling from continuous functional relationships to discrete event sequences. Instead of assuming smooth continuous functions, the system models data as sequences of discrete events with specific attributes. This parameter change enables reliable modeling of complex systems where functional relationships are nonexistent or highly nonlinear, while maintaining ease of model development through standardized event representations.
Solution Approach 2:
The patent inverts the traditional modeling approach by not starting with assumed functional forms and fitting data to them. Instead, it starts with observed discrete events and builds models directly from event sequences and patterns. This inversion eliminates the need to assume functional relationships exist, allowing reliable modeling of systems where no simple functional form can be derived.
Data Source
AI summary
A system and method for enabling information extraction from large data sets (so-called “big data”) according to a new paradigm is disclosed. This system does not generate functions describing why certain inputs result in certain outputs. Instead, it creates incident mappings of inputs to outputs without regard to why inputs result in outputs. These mappings can be distributions or other data sets representative of different outcomes occurring. This enables several useful operations. For example, by providing a data set indicative of outputs that have historically occurred following a particular input, the disclosed system can be used to predict future outcomes with probabilities. For example, if a particular stock price pattern is provided as an input, the system generates an output data set indicating the probabilities of certain price behaviors following that input pattern. This data set can thus be used to predict future behavior. Other useful operations are disclosed herein.