Aggregated Edge Data Structure for Real-Time Time-Series Graph Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graph database products face challenges in processing massive time-series data, leading to long response times, high hardware costs, and inability to provide real-time query results due to complex calculations and lack of aggregation capabilities based on time-series data.
Innovation Solution
The introduction of an innovative data structure called 'aggregated edge' and 'time-series aggregated edge' for incremental streaming calculations, along with a time-series graph query language that supports basic queries, index calculations, and filtering, reduces calculation complexity and enhances scalability by aggregating data within time windows and storing results in memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If graph database products store all data information as vertices, edges, and attributes in a graph library, then transaction information is complete and easy to display, but the comprehensive data takes time to filter and calculate
Solution Approach 1:
The patent segments the comprehensive graph data by introducing time-window based aggregation that divides data into manageable temporal segments. This allows the system to work with aggregated summaries for most queries while maintaining access to detailed data when needed, thus filtering comprehensive data more efficiently without losing information completeness
Solution Approach 2:
The patent performs preliminary aggregation actions by pre-calculating and storing aggregated statistics (such as transaction counts, sums, averages) for different time windows before queries are executed. This preliminary computation reduces the filtering and calculation time during actual queries while maintaining data completeness through the underlying graph structure
2Ease of operation
If graph database products perform intensive calculations based on graph data structure, then association queries can be executed, but large-scale calculation devices and frameworks are required, leading to high hardware and maintenance costs
Solution Approach 1:
The patent performs preliminary aggregation computations to pre-calculate association statistics (such as transaction frequencies, cumulative amounts, time-based patterns) and stores them in an optimized format. This eliminates the need for intensive real-time calculations during queries, reducing hardware requirements while maintaining full association query capabilities
Solution Approach 2:
The patent segments calculation requirements by separating aggregate computations from detailed graph traversals. Aggregated results are computed and stored separately, allowing simple association queries to use pre-computed results without requiring large-scale computing frameworks, while complex queries can still access the full graph data when needed
3Reliability
If graph database products convert all data into vertices, edges, and attributes for storage, then data is complete and easy to display, but query response time is long for massive data
Solution Approach 1:
The patent performs preliminary aggregation of graph data by time windows, pre-computing statistics such as transaction counts, sums, and patterns for specified time periods. These pre-computed aggregated results are stored and can be quickly returned for queries within those time windows, dramatically reducing response time while maintaining data completeness through the underlying graph structure
Solution Approach 2:
The patent segments the time dimension by dividing data into time windows and performing aggregation at different granularity levels. This allows the system to return aggregated results quickly for broad queries while still providing access to detailed individual records when needed, thus reducing average response time without sacrificing data completeness
Data Source
AI summary
A fast processing method of massive time-series data based on aggregated edges and time-series aggregated edges implements fast and real-time processing based on an association of a graph data structure under a massive data pattern. In terms of incremental flow calculation based on time windows, the present disclosure proposes an innovative data structure of “aggregated edge” and “time-series aggregated edge”, which is suitable for data modeling of real-time dynamic graph. The present disclosure introduces a time-series graph query language, which adds descriptive semantics of time series information, and not only supports basic queries based on vertices, edges, and attributes, but also can implement users' graph queries for index calculation results in a time window. The graph query includes graph matching and graph filtering. The present disclosure is especially suitable for fields such as marketing and real-time risk control based on massive data mining and has good timeliness control and high scalability.