Cloud Data Flow Sequencing to Prevent Redundant Processing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In cloud environments, processing multiple instances of the same dataset across multiple servers leads to inconsistencies, computational complexity, latency, and wasteful use of resources due to redundancy and data dependency, resulting in inefficient data processing.

Innovation Solution

A service generates flow identifiers to define a sequence of processing a dataset, indicating predecessor and successor applications, enabling streamlined data processing by maintaining a repository of datasets and configuration files to ensure timely availability of updated datasets.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If multiple instances of the same dataset are processed by multiple servers simultaneously, then processing throughput is improved, but data inconsistencies and computational redundancy increase

Engineering Contradiction:

Improveprocessing throughputVSAvoidcomputational redundancy

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The system performs preliminary actions by generating flow identifiers and establishing processing sequences before actual data processing begins. The service pre-defines which applications should process which datasets in what order, preventing redundant computational work before it occurs.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback mechanisms where the service monitors dataset processing status across multiple servers and dynamically adjusts data distribution. Flow identifiers provide feedback about processing state, enabling the service to optimize data flow and eliminate computational redundancy in real-time.

Inventive Principle:

Principle #23Feedback

2Stability of the object's composition

If datasets are sequenced for processing by multiple servers, then data consistency is improved, but processing latency increases

Engineering Contradiction:

Improvedata consistencyVSAvoidprocessing latency

Core Design Contradiction:

Stability of the object's compositionVSLoss of time

Solution Approach 1:

The system applies dynamics by making the data processing sequence flexible rather than rigid. Flow identifiers enable dynamic adjustment of processing sequences based on current system state, server availability, and data dependencies, allowing the system to maintain consistency while minimizing latency through adaptive scheduling.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The service acts as an intermediary that manages the complex sequencing relationships between multiple servers and datasets. Flow identifiers serve as intermediary mechanisms that encode processing sequences, allowing the service to coordinate data flow without requiring direct communication between all servers, thus reducing latency while maintaining consistency.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Productivity

If multiple applications independently process the same dataset, then processing parallelism is improved, but redundant storage and computational waste increase

Engineering Contradiction:

Improveprocessing parallelismVSAvoidredundant storage

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The system uses copying strategically by creating flow identifier references rather than duplicating actual dataset copies. The service generates flow identifiers that reference the original dataset, allowing multiple applications to process the same data without creating redundant storage copies, thus maintaining parallelism while eliminating storage waste.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

Flow identifiers serve multiple functions simultaneously: they track dataset provenance, define processing sequences, enable parallel execution coordination, and prevent redundant storage. This multi-functionality allows the system to achieve processing parallelism without the overhead of redundant data storage.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS12632301B2Data processing across applications in cloud environments

Publication Date: 2026.05.19 CITIBANK N A

AI summary

Presented herein are system and methods for handling processing of data in cloud environments. A server receives a first dataset generated in response to a function of a first application. A server generates a set of identifiers defining a sequence of processing of the first dataset associated with the function. The identifiers include a first identifier indicating the first application as a predecessor for the first dataset and a second identifier indicating a second application as a successor for the first dataset. The server identifies the second application corresponding to the second identifier as the successor for processing the first dataset. The server communicates at least a portion of the first dataset with a second server hosting the second application to receive a second dataset generated by the second application. The server stores the second dataset in the cloud environment.