Pluggable Data Pipeline Topology for Dynamic Component Integration

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing computing systems face difficulties in dynamically adding, modifying, or removing components in a data pipeline without altering the code of the components or their interfaces, limiting flexibility and efficiency in data processing tasks.

Innovation Solution

A computer-implemented method that determines and configures data input and output properties of data processing components to connect them dynamically using a predetermined data pipeline connection topology, such as diamond, tree, or mesh topologies, allowing components to be seamlessly integrated without code changes, enabling flexible data flow management.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If existing topologies of computing systems are used, then data processing can be performed, but it is difficult to add/modify/remove various components/elements

Engineering Contradiction:

Improveability to add/modify/remove componentsVSAvoidcomplexity of connecting components

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system segments the data pipeline into independent components with standardized interfaces. Each component is a separate unit that can be independently added, modified, or removed without affecting the entire system. The pipeline is divided into discrete processing stages that communicate through defined input and output channels, enabling flexible reconfiguration.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent implements universal connection mechanisms where components can be connected using standardized interfaces and topologies. The same connection framework supports multiple pipeline configurations (series, parallel, time-sliced) and accommodates various component types through a common attachment mechanism, eliminating the need for custom connection code for each component.

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Ease of manufacture

If components are connected in a data pipeline, then data processing tasks can be performed, but code alterations are required to connect components

Engineering Contradiction:

Improveease of connecting componentsVSAvoidcomplexity of code modifications

Core Design Contradiction:

Ease of manufactureVSDevice complexity

Solution Approach 1:

The system introduces an intermediary connection mechanism that mediates between components. Instead of requiring direct code modifications to connect components, the framework provides a standardized attachment interface that automatically establishes connections. The intermediary layer handles the complexity of connection management, allowing components to be linked without altering their source code.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

Components are designed with self-configure capabilities where connection details are automatically discovered and established based on standardized interfaces. The system enables components to self-attach and self-connect without requiring manual code changes or complex configuration scripts, reducing the burden on developers to connect pipeline elements.

Inventive Principle:

Principle #25Self-service

3Adaptability or versatility

If a fixed topology is used for data pipeline, then execution can proceed systematically, but flexibility to handle various topologies is limited

Engineering Contradiction:

Improveflexibility to handle various topologiesVSAvoidsystematic execution guarantee

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent implements a dynamic topology framework where the data pipeline structure can be automatically adjusted based on processing requirements. The system supports multiple topologies (series, parallel, time-sliced, diamond, tree, mesh) and can dynamically reconfigure connections without altering component code. The execution engine adapts to the chosen topology while maintaining systematic execution guarantees through standardized control mechanisms.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS11144373B2Data pipeline using a pluggable topology connecting components without altering code of the components

Publication Date: 2021.10.12 SAP SE

AI summary

A method, a system, and a computer product for connecting computing components are disclosed. One or more data input and output properties of a data processing component in a plurality of data processing components are determined. At least one of the data input properties and data output properties of the data processing component are configured for connection of the data processing component to another data processing component in the plurality of data processing components in the data processing pipeline using a predetermined data pipeline connection topology. Using the predetermined data pipeline connection topology, the data processing component and another data processing component are dynamically connected using the configured properties.