Distributed Computational Graph for Data Workflow Automation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Individuals and businesses lack the expertise and resources to create and implement data processing workflows, making it difficult for them to extract valuable information from large datasets without investing time or money into data processing knowledge.
Innovation Solution
A system and method using a distributed computational graph that allows users to create complex data processing workflows by dragging and dropping modules representing data processing steps, with some modules accessing cloud-based services through APIs, enabling processing of large volumes of data without requiring expertise in cloud-based data processing services.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If users attempt to create data processing workflows using traditional methods, then they can access cloud-based data processing services, but they require extensive technical knowledge and time investment
Solution Approach 1:
The patent introduces a visual workflow builder as an intermediary tool that mediates between the user and complex cloud-based data processing services. This workflow builder provides a graphical user interface with drag-and-drop functionality, allowing users to construct data processing workflows through visual assembly of pre-configured modules rather than writing code or configuring complex service parameters directly. The workflow builder translates user-friendly visual definitions into executable workflows that leverage cloud-based services, thereby eliminating the need for users to have extensive technical knowledge while still enabling access to powerful data processing capabilities.
2Productivity
If users hire knowledgeable personnel for data processing, then they can extract valuable information from data, but it becomes cost-prohibitive or unfeasible
Solution Approach 1:
The patent enables self-service data processing by providing a user-friendly visual workflow builder that allows individuals and businesses to create and execute their own data processing workflows without needing to hire specialized data scientists or analysts. The system pre-configures modules for common data processing tasks and provides intuitive controls for defining data sources, transformations, and outputs. Users can independently build, test, and deploy workflows through the graphical interface, eliminating the need to invest in expensive specialized personnel while maintaining productive data processing capabilities.
3Adaptability or versatility
If cloud-based data processing services are used, then scalable computing resources become available, but the complexity of accessing and configuring these services increases
Solution Approach 1:
The patent segments cloud-based data processing services into discrete, pre-configured modules that can be independently selected and combined in the visual workflow builder. Each module represents a specific data processing function (e.g., data ingestion, transformation, analysis, output) and can be configured through simple parameters in the graphical interface. This segmentation allows users to access scalable cloud computing resources by assembling only the specific modules needed for their tasks, rather than having to understand and configure entire cloud service architectures. The modular approach reduces complexity while maintaining adaptability to different data processing needs.
Data Source
AI summary
A system and method for creating and implementing data processing workflows using a distributed computational graph comprising modules that represent various stages within a data processing workflow. Each module represents one or more data processing steps, with some of the modules representing data processing performed by a cloud-based service and containing code for interfacing with the application programming interface (API) of that cloud-based service. A series of modules and their interconnections specify the workflow. Data is processed according to the workflow by implementing the data processing step represented by each module, some of which may access cloud-based data processing services. The result is that users can create complex data processing workflows that utilize cloud-based services to process data without having to know how to access the cloud-based data processing services, or even know that they exist.