Configuration Data Migration for Distributed Micro Service Applications
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Managing configuration data for distributed network applications, such as large-scale WAN controllers, is challenging due to the complexity of migrating from flat file-based data repositories to centralized hierarchical models without disrupting service.
Innovation Solution
The method involves storing configuration data in key-value pairs and migrating it to a hierarchical configuration data model using a software upgrade service, which retrieves parameters from legacy repositories and updates the new model, including data transformation operations specified in metadata.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If configuration data is migrated from flat file-based repositories to centralized hierarchical models, then configuration management becomes more efficient and scalable, but service disruption occurs during the migration process
Solution Approach 1:
The system performs preliminary actions by creating a new hierarchical configuration model alongside the existing flat file-based repository before migration. The new model is prepared in advance with all necessary transformations defined, allowing the system to switch to the new model without disrupting ongoing services. This preliminary preparation ensures that the migration can occur without service interruption.
Solution Approach 2:
The patent introduces an intermediary configuration model that acts as a mediator between the legacy flat file-based repository and the target hierarchical model. This intermediate hierarchical model contains external references to the legacy data, allowing the system to gradually transform the configuration data structure while maintaining service continuity. The intermediary model enables incremental migration without forcing a complete switch that would cause service disruption.
2Extent of automation
If automated migration is implemented, then migration complexity is reduced and service continuity is maintained, but data transformation operations become more complex
Solution Approach 1:
The migration process is segmented into distinct transformation operations that can be independently defined and executed. Each data transformation is broken down into specific operations (such as data extraction, conversion, and insertion) that are applied systematically to different configuration parameters. This segmentation allows the complex transformation process to be managed through automated, modular steps rather than requiring monolithic complex processing.
Solution Approach 2:
The system automatically manages data transformation by applying parameter changes to the configuration data during migration. Transformation rules are defined in terms of parameter mappings between the legacy flat file structure and the hierarchical model, allowing automated tools to perform the conversions without manual intervention. The complexity of transformations is encapsulated in configurable parameters and transformation rules rather than hard-coded complex logic.
Data Source
AI summary
Techniques are described for in-service configuration data migration for distributed micro service-based applications. In one example, a network device comprises a plurality of legacy data repositories comprising configuration data in key-value pair format that specifies a plurality of parameters and corresponding values for operation of the network device, and a hierarchical configuration data model having a plurality of nodes arranged in a hierarchical organization having a root node and a plurality of leaf nodes. Each of the nodes of the configuration data model is configured to store a set of configuration data parameters for the network device. One or more of the nodes includes a plurality of external references to respective parameters of the plurality of parameters stored within the plurality of legacy data repositories. Process circuitry is configured to perform a migration of the configuration data from the legacy data repositories to the hierarchical data model.