Streaming Data Marketplace Correlating Multiple Sources
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current streaming analytics technologies provide data in low latency but with low value until the raw data is enhanced by correlating it with additional data, which often resides with multiple entities, making it difficult to increase the utility and availability of streaming data.
Innovation Solution
A streaming data marketplace that allows customers to receive, correlate, and combine streaming data with data from various sources, including persistent data, and republish the combined stream with low latency, enabling enhanced data value and utility by linking multiple data sources in chains or meshes, with customers paying for the amount of data consumed independently of time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If streaming data is provided in low latency, then real-time processing capability is improved, but data value remains low until enhanced by correlating with additional data
Solution Approach 1:
The patent combines multiple independent data streams from different sources into a unified enhanced data stream. The streaming data marketplace merges raw streaming data with additional contextual data, persistent data, and processed data from multiple entities to create higher-value combined streams that maintain low latency while significantly increasing data utility and information content.
Solution Approach 2:
The streaming data marketplace acts as an intermediary platform that facilitates the correlation and combination of data from multiple sources. It provides the infrastructure and mechanisms for data sellers to publish streams and for data buyers to subscribe, process, and combine streams, enabling value enhancement without sacrificing real-time processing capabilities.
2Loss of information
If data is correlated and combined from multiple entities, then data value and utility are enhanced, but system complexity increases
Solution Approach 1:
The streaming data marketplace is designed as a universal platform that handles multiple functions: data publication, subscription management, stream processing, correlation, combination, and delivery. This multi-functional architecture reduces overall system complexity by consolidating diverse data integration tasks into a single standardized system that can handle various data types and sources uniformly.
Solution Approach 2:
The system enables automated data correlation and combination where data buyers can independently subscribe to multiple streams, define their own processing logic, and automatically combine data from multiple sources without manual intervention. The marketplace infrastructure automatically manages the complex coordination of multiple data sources, reducing the burden on individual users.
3Productivity
If streaming data is made available for sale, then data monetization is enabled, but data availability and accessibility to multiple customers may be limited
Solution Approach 1:
The streaming data marketplace segments data into discrete subscribable streams that can be independently purchased and combined. Data is divided into atomic streaming units that can be subscribed to individually, allowing customers to build customized data combinations from multiple sources. This segmentation enables flexible data distribution where the same underlying data can be sold to multiple customers in different combinations, enhancing both monetization and availability.
Solution Approach 2:
The system adds a new dimension to data distribution by enabling multi-dimensional data combinations. Instead of single-source data delivery, the marketplace allows data to be distributed across multiple dimensions: multiple sources, multiple customers, multiple processing levels, and multiple combination scenarios. This dimensional expansion simultaneously increases monetization opportunities and data accessibility.
Data Source
AI summary
A technology for a streaming data marketplace is provided. In one example, a method may include requesting to receive a first stream of data from a first source via the streaming data marketplace. The first stream of data may be received and then correlated and combined with data from a second source as a combined stream to increase a utility of the first stream of data. The data from the second source may be a different type of data than the first stream of data.