Streaming Data Routing for Schema Evolution Without Node Suspension

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing technologies face challenges in processing continuous queries of network management data without suspending and reconfiguring currently executing queries, especially when schema changes occur in data streams.

Innovation Solution

The system processes different types and subtypes of streaming input data from multiple sources without suspending nodes by casting records of subtypes to super-types and routing them to already-running nodes for processing, allowing continuous query execution without suspension or reconfiguration.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If a node suspends and reconfigures to adapt to schema changes, then adaptability improves, but system uptime and productivity deteriorate

Engineering Contradiction:

Improveschema adaptabilityVSAvoidsystem uptime

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The system segments the node into multiple parallel processing instances (original node and cloned node). The original node continues processing with the old schema while the cloned node processes with the new schema. This segmentation allows schema adaptation without suspending the main processing node, thus maintaining system uptime while improving adaptability.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system performs preliminary actions by creating a cloned node in advance with the new schema configuration before the old node needs to be updated. This cloned node is prepared and ready to take over processing once the schema change is complete, avoiding any suspension of service and maintaining continuous productivity.

Inventive Principle:

Principle #10Preliminary action

2Adaptability or versatility

If a node suspends to process schema changes, then schema evolution capability improves, but data loss risk increases

Engineering Contradiction:

Improveschema evolution capabilityVSAvoiddata loss

Core Design Contradiction:

Adaptability or versatilityVSLoss of information

Solution Approach 1:

The system ensures continuity of useful action by maintaining parallel processing in both the original node (old schema) and cloned node (new schema). Data streams continue flowing through both nodes simultaneously, ensuring no data is lost during the schema transition period. The cloned node processes data with the new schema while the original node continues with the old schema, eliminating any gap in processing.

Inventive Principle:

Principle #20Continuity of useful action

Solution Approach 2:

The system discards the old schema configuration in the original node only after the cloned node is fully operational and verified. The cloning process creates a safe backup that can be activated immediately if needed, allowing the system to recover from any issues during schema transition without data loss.

Inventive Principle:

Principle #34Discarding and recovering

3Adaptability or versatility

If a node reconfigures for schema changes, then adaptability improves, but processing time increases

Engineering Contradiction:

Improveschema adaptabilityVSAvoidprocessing time

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The system segments processing across multiple nodes (original and cloned) operating in parallel. While the cloned node undergoes schema configuration, the original node continues processing without interruption. This segmentation eliminates sequential delays and maintains continuous processing time throughout the schema adaptation process.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system creates a copy (cloned node) of the original processing node with the new schema configuration. This copying approach allows the schema changes to be applied to the clone without affecting the original node's processing speed. Once the clone is verified, it can take over or work in parallel, avoiding any time loss in the overall processing pipeline.

Inventive Principle:

Principle #26Copying

Data Source

PatentEP4231149B1Distributing and processing streams over one or more networks for on-the-fly schema evolution

Publication Date: 2025.02.12 CISCO TECHNOLOGY INC

Figure 1
Figure 2
Figure 3

AI summary

A method performed by one or more computing devices is provided and comprises: receiving (320) a first record (110) comprising one or more first fields (114A) from a first source computer (240); in response to determining (325) that the first record is a first type (112) and a first node (246) is associated (310) with the first type, sending (335) the first record to the first node, which is executed on a first computer (244), to be processed; receiving (320) an updated first record (120) comprising the one or more first fields (114B) and one or more second, new fields (124A) from the first source computer; in response to determining (325) that the updated first record is a second type (122) and a second node (256) is associated (310) with the second type, sending (335) the updated first record to the second node, which is executed on a second computer (254), to be processed; in response to determining (340) that the second type is a first subtype of the first type, sending (350) the second record to the first node to be processed, without suspending and/or reconfiguring the first node.