Visual Data Intelligence Platform for Heterogeneous Data Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for cross-source heterogeneous data visual display in business intelligence analysis face challenges such as performance pressure on production databases, real-time data capture issues, and interface transformation costs, while also affecting business systems.
Innovation Solution
A method utilizing a visual data intelligence platform, a distributed in-memory database, and JDBC connectors to create associations between heterogeneous data sources, generating SQL table building statements, and visualizing data without altering production database data or affecting business code.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is extracted from production database to unified library through ETL tool, then cross-source data can be analyzed, but performance pressure is caused on production database and real-time data changes cannot be captured
Solution Approach 1:
The patent introduces a database connector as an intermediary component that enables the visual data intelligence platform to access and analyze data from the production database without extracting it. The connector acts as a mediator that allows cross-source heterogeneous data analysis while keeping the production database intact and performing, thus resolving the contradiction between data analysis capability and database performance.
2Productivity
If data is extracted to external databases, then data can be analyzed, but data security issues arise
Solution Approach 1:
The database connector serves as a secure intermediary that enables data analysis without extracting data to external databases. It maintains data security by keeping all data access operations within the production database environment while still allowing the visual data intelligence platform to perform cross-source heterogeneous data analysis.
3Productivity
If business code is reconstructed to make join association, then data can be searched and analyzed, but business system is directly affected and interface transformation costs are incurred
Solution Approach 1:
The database connector acts as an intermediary layer between the visual data intelligence platform and the production database, eliminating the need to reconstruct business code for data association. It enables join association and cross-source data analysis through the connector without modifying existing business code, thus avoiding interface transformation costs and maintaining business system simplicity.
4Adaptability or versatility
If connectors are created for each data source, then cross-source heterogeneous data can be accessed, but system complexity increases
Solution Approach 1:
The database connector is designed with universal functionality to handle multiple types of data sources (MySQL, PostgreSQL, Oracle, SQL Server, etc.). This multi-functional design allows a single connector type to serve multiple purposes across different data sources, reducing the need for numerous specialized connectors and simplifying connector management while maintaining broad data source compatibility.
Data Source
AI summary
A method and apparatus for intelligent visual analysis and display of cross-source heterogeneous data including: when a visual display command for the target heterogeneous data is detected, creating a connector in the distributed in-memory database corresponding to each target data source where the target heterogeneous data is located via a JDBC driver; in the visual data intelligence platform, the python driver of the distributed in-memory database and the SQLAlchemy framework create the connection between the visual data intelligence platform and the distributed in-memory database, and generating SQL table building statements in the visual data intelligence platform based on the association information corresponding to the target heterogeneous data; generating the target data table with plurality of table associations in the distributed in-memory database based on the SQL table building statements; the target heterogeneous data is visualized and displayed in the visual data intelligence platform based on the target data.