Predictive Graph Generation for Big Data Analytics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in efficiently managing and analyzing large volumes of data in real-time, particularly in generating predictive models and performing complex graph-based analytics, due to limitations in data growth rates, data types, and processing capabilities.
Innovation Solution
A big data analysis system that combines graph-based data notation with big data processing capabilities, utilizing a predictive modeling system and predictive graph processing system to transform big data repositories into graph-based data, enabling efficient real-time processing and generation of predictive graphs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If non-big data technologies (relational databases, columnar databases) are used to process data, then structured data analysis can be performed, but data growth rates and volumes are strictly limited
Solution Approach 1:
The system segments data processing into multiple specialized components: a data accumulation layer for receiving and storing raw data from multiple sources, a predictive modeling layer for generating models, and a graph processing layer for real-time analytics. This segmentation allows each component to handle specific data types and processing requirements independently, enabling support for both structured and unstructured data at scale.
Solution Approach 2:
The patent introduces a predictive graph processing system as an intermediary between big data repositories and analytical applications. This intermediary transforms raw big data into graph-based representations that can be efficiently processed in real-time, bridging the gap between voluminous data storage and sophisticated analytics requirements.
2Productivity
If traditional data processing systems are used, then data can be stored and analyzed, but real-time processing of huge volumes of complex data is not achievable
Solution Approach 1:
The system performs preliminary actions by pre-processing and transforming raw data into graph-based representations before actual analytics are needed. The predictive modeling system generates predictive models in advance, and the graph processing system maintains pre-computed graph structures, enabling real-time querying and analysis without performing complex computations at query time.
Solution Approach 2:
The patent implements dynamic data accumulation and graph processing capabilities that adapt to changing data volumes and processing requirements. The system dynamically scales processing resources and adjusts graph processing parameters based on real-time workload conditions, maintaining high performance across varying operational states.
3Adaptability or versatility
If data silos and relational databases are used for predictive analytics, then processing can be performed, but expensive data management support is required and data variety is limited
Solution Approach 1:
The predictive graph processing system is designed as a universal platform that can process multiple data types (structured, unstructured, semi-structured) from diverse sources through a unified graph-based framework. This multi-functional approach eliminates the need for separate data management systems for different data types, reducing overall system complexity while increasing data variety support.
Data Source
AI summary
A big data analysis system may include a big data repository communicatively coupled to a data accumulation server and a predictive graph processing system. The data accumulation server may be configured to receive information from a plurality of data sources, the information corresponding to user interaction with one or more computing devices associated with an organization via a networked computing system, store the information received from the plurality of sources in the big data repository; and monitor the plurality of data sources to update the data stored in the big data repository. The predictive graph processing system is configured to receive information stored in the big data repository, transform the information received from the big data repository into a predictive graph data set based on a predictive model, and store the predictive graph data set to a visualization data repository.