In-Memory Database Pattern Detection for Multi-Source Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current tools are unable to process large amounts of data from multiple database sources to identify common activity patterns efficiently, hindering businesses' ability to understand their customers and improve operations.
Innovation Solution
The system employs an in-memory database management system with an index server, statistics server, preprocessor server, and XS engine to process and analyze data from various sources, using path detection algorithms and data mining techniques to detect patterns and present them in a graphical user interface.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional data processing tools are used to analyze customer activity data, then the system structure is simple, but the system cannot process large amounts of data from multiple database sources to identify common activity patterns efficiently
Solution Approach 1:
The system is divided into multiple specialized servers including index server, statistics server, preprocessor server, and XS engine, each handling specific aspects of data processing. This segmentation allows the system to process large amounts of data from multiple database sources efficiently while maintaining manageable complexity through clear division of labor.
Solution Approach 2:
The database management system is designed to handle multiple types of data sources (ERP, CRM, web logs, social media) and perform various analysis functions (pattern detection, data mining, visualization) within a unified multi-functional platform, improving productivity without requiring separate systems for each function.
2Loss of information
If no pattern detection system is implemented, then the system structure is simple, but companies cannot find common patterns of activities or reasons behind those activities
Solution Approach 1:
The system performs preliminary data processing and pattern detection automatically in the background before users need the information. The index server pre-processes data from multiple sources, and the XS engine detects patterns proactively, so when users access the system, the pattern information is already prepared and available, reducing perceived complexity while preventing information loss.
Solution Approach 2:
The patent introduces an intermediary analysis layer between raw data and user interpretation. The XS engine acts as a mediator that automatically detects patterns and translates complex multi-source data into meaningful activity patterns and insights, preventing information loss without requiring users to directly manage the complexity of analyzing raw data from multiple database sources.
3Loss of time
If manual data analysis is used, then the system complexity is low, but the time required to process large amounts of data and identify patterns is excessive
Solution Approach 1:
The patent replaces manual mechanical data analysis with automated computational systems. The XS engine uses algorithmic pattern detection and data mining techniques to automatically analyze large volumes of data from multiple sources, dramatically reducing analysis time from weeks or months to minutes or hours, while the modular architecture keeps system complexity manageable.
Solution Approach 2:
The system changes the parameters of data processing by using in-memory computing and optimized query execution. The database management system processes data with different performance parameters (speed, concurrency, scalability) compared to traditional systems, enabling rapid pattern detection across large datasets without proportionally increasing perceived system complexity.
4Loss of information
If comprehensive data from multiple sources is collected, then the information completeness is high, but the difficulty of processing and analyzing the data increases
Solution Approach 1:
The system segments data from different sources (ERP, CRM, web logs, social media) and processes each type through specialized servers. The index server handles indexing, the preprocessor server handles data cleaning and transformation, and the XS engine handles pattern detection. This segmentation maintains information completeness while reducing the difficulty of detecting patterns by assigning specific detection tasks to specialized components.
Solution Approach 2:
The patent introduces intermediary processing layers between raw multi-source data and final pattern detection. The preprocessor server acts as an intermediary that standardizes and cleans data from different sources before analysis, and the XS engine acts as an intermediary that translates diverse data types into unified activity patterns. This reduces detection difficulty while preserving information completeness.
Data Source
AI summary
Systems, methods, and apparatuses for activity pattern detection are described herein. Embodiments may process large amounts of data from a plurality of different database sources in order to detect events common to the data of the different database sources. Embodiments further perform data mining operations to detect patterns (e.g., two or more events appearing consecutively or non-consecutively), and present these patterns in a graphical user interface (GUI) to illustrate how a plurality of patterns may comprise a business scenario.