Unified Data Flow Model for Decoupled Node Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing systems require complex technical skills to implement and modify analysis logic, often necessitating collaboration among multiple parties, leading to increased time, cost, and risk of miscommunication, and limiting user flexibility and understanding of analysis options.
Innovation Solution
A system that models analysis logic as a data flow with a single output table, allowing users to select and switch between logics, with node metadata decoupling node names from output tables, enabling static query operations and improved flexibility and security.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If data processing systems use traditional multi-table output structures, then data can be organized by node, but system complexity and difficulty of modification increase
Solution Approach 1:
The patent merges multiple node-specific output tables into a single unified output table that contains all node data. This consolidation eliminates the complexity of managing multiple separate tables and their relationships, while maintaining the ability to identify and access individual node data through a node identifier column.
Solution Approach 2:
The unified output table serves as a universal structure that can handle data from any number of nodes in the data flow. This single table structure replaces the need for node-specific tables, making the system more adaptable and easier to modify when the number or configuration of nodes changes.
2Ease of operation
If multiple parties collaborate to implement analysis logic, then technical expertise is pooled, but time and cost increase
Solution Approach 1:
The patent enables end users to independently create, modify, and execute their own analysis logic using the unified data flow framework. Users can define custom data flows and select different analysis options without requiring software developers or technical collaborators, thereby reducing implementation time and costs while maintaining ease of operation.
3Reliability
If node names are directly linked to output tables, then data retrieval is straightforward, but security risks and flexibility are reduced
Solution Approach 1:
The patent introduces a node identifier column as an intermediary between node names and the unified output table. This identifier acts as a secure reference key that allows users to access and filter data from specific nodes without directly exposing or linking to the underlying table structures, thereby maintaining security while preserving flexibility.
Solution Approach 2:
The system changes the parameter used for data access from direct table names to standardized node identifiers. This parameter transformation allows the same unified table to serve multiple nodes securely, enabling flexible data retrieval while maintaining consistent security protocols across all node accesses.
Data Source
AI summary
A data flow model format allows for improved data flow model creation and execution. The data flow models can provide a user with greater flexibility in selecting an appropriate data flow model, including providing information about the data flow model and selecting from multiple available data flow models. Tabular or graphical visualizations can aid the user in understanding a data flow model and retrieving desired information. Technical features such as designing a data flow with a single data output, and maintaining node identifiers in node metadata, can reduce coupling between software components and improve security and performance.