Unified Big Data Analytics Platform for Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for big data analytics face challenges due to the volume, quality, and speed of data from various sources, requiring a unified platform for efficient data analysis and visualization, which is lacking in centralized access, data duplication, and non-standardized data onboarding, leading to increased latency and difficulty in data discovery and usage.
Innovation Solution
A unified analytics platform is introduced, comprising a computer platform, core data platform, and devops platform, enabling unified user experience, scalable data services, and automation of large-scale data analytics, with features like dataset registration, visibility, and integration of notebooks and tools, allowing for real-time data processing and visualization across multiple data sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple systems and applications are used to complete data analytics, then data processing capability is improved, but system complexity and difficulty in unified access increase
Solution Approach 1:
The patent combines multiple distributed data processing systems into a unified data analytics platform that provides centralized access and standardized data onboarding. The platform integrates various data sources and processing applications while maintaining a single point of access, thereby reducing system complexity while preserving enhanced data processing capabilities.
Solution Approach 2:
The unified data analytics platform provides universal access to multiple data sources and processing applications through a single interface. It implements standardized data onboarding procedures that work across different data types and sources, enabling the system to handle diverse data processing tasks through a common platform rather than requiring separate systems for each function.
2Quantity of substance
If data from numerous sources is integrated, then data comprehensiveness is improved, but data discovery difficulty and latency increase
Solution Approach 1:
The platform implements preliminary action by pre-processing and standardizing data from numerous sources during the data onboarding phase. Data is registered, validated, and organized in a unified format before being made available for analysis, which reduces the time required for data discovery and access while maintaining comprehensive data integration.
Solution Approach 2:
The unified data analytics platform acts as an intermediary layer between multiple data sources and the end users or analysis applications. It provides a standardized interface and centralized data registry that simplifies data discovery while maintaining access to comprehensive data from numerous sources, thereby reducing latency without sacrificing data comprehensiveness.
3Ease of operation
If centralized data access is implemented, then data accessibility is improved, but system complexity increases
Solution Approach 1:
The unified data analytics platform implements self-service mechanisms that automatically handle data registration, validation, and routing. The system provides standardized procedures and tools that enable users to access and onboard data without requiring complex manual configuration, thereby improving data accessibility while managing platform complexity through automation.
Data Source
AI summary
Aspects of the present disclosure involve systems, methods, devices, and the like for creating a unified analytics platform for big data analytics, visualization and data store. In one embodiment the unified analytics platform can include at least a computer platform, core data platform, and integration platform. The unified analytics platform is designed to provide visibility into datasets, provide operational metrics, and provide the integration of notebooks and tools for big data analysis.