Dataflow Graph System for Remote Storage API Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in efficiently interacting with remote data storage systems, particularly for users with large volumes of data, due to compatibility and security issues with HTTP communication, and the complexity of implementing web service APIs, which often requires traditional coding and struggles with maintaining state and securely passing requests through firewalls.
Innovation Solution
A method and system that utilize a dataflow graph with nodes representing data processing components connected by links, providing a graphical user interface to configure and interact with remote data storage systems, transforming input data into request messages, and separating acknowledgments of successful and failed function executions, while encoding session credentials for secure concurrent login requests, leveraging SOAP and HTTP protocols for efficient data access and manipulation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If web service APIs are used to communicate with remote data storage systems, then data access capability is improved, but system complexity increases due to traditional coding requirements and state management issues
Solution Approach 1:
The patent introduces a dataflow graph as an intermediary layer between the user and the web service API. The graph visually represents data processing components and their connections, allowing users to configure data access operations without writing code. The system automatically translates the visual graph into API calls, managing state and authentication credentials behind the scenes, thus reducing system complexity while maintaining data access capability.
Solution Approach 2:
The patent replaces the mechanical coding process with a visual drag-and-drop interface. Instead of requiring users to manually write and manage API code, they simply drag components onto a canvas and connect them visually. The system automatically handles the translation to SOAP/HTTP protocols, credential encoding, and state management, substituting the manual coding mechanism with an automated visual programming approach.
2Adaptability or versatility
If traditional coding methods are used for web service integration, then functionality is achieved, but ease of operation deteriorates due to coding complexity and firewall security issues
Solution Approach 1:
The system provides self-service functionality by automatically handling authentication credentials and firewall security concerns. When users configure a dataflow graph, the system automatically encodes session credentials into subsequent requests and manages the authentication state without requiring users to write security code. This self-service approach maintains full functionality while dramatically improving ease of operation.
Solution Approach 2:
The patent performs preliminary actions by automatically encoding authentication credentials into session tokens before actual data access operations. The system handles login requests, obtains session credentials, and pre-encodes them for use in subsequent API calls. This preliminary authentication setup eliminates the need for users to repeatedly handle security code, making the system easier to operate while maintaining functionality.
3Adaptability or versatility
If data processing operations are performed on large volumes of data, then data manipulation capability is improved, but processing time increases
Solution Approach 1:
The patent segments large data processing operations into smaller, modular components within the dataflow graph. Users can divide data manipulation tasks into discrete steps (filtering, transformation, aggregation) represented by separate nodes in the graph. This segmentation allows for more efficient processing by enabling parallel execution of independent operations and optimizing data flow through the system, thus improving data manipulation capability while reducing overall processing time.
Data Source
AI summary
In some aspects, a method includes connecting over a network to a data storage system, the data storage system storing data objects. A dataflow graph that includes nodes representing data processing components connected by links that represent flows of data access an interface of the data storage system. The interface provides functions for accessing the data objects. At least one of the data processing components performs operations on a received input flow of data that enable the functions provided by the interface to modify one or more stored data objects, and performs operations in response to functions provided by the interface to generate an output flow of data.