Dynamic Cloud Pipeline Data Sharing via Key-Value Store
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cloud service management systems face challenges in dynamic and persistent data sharing between infrastructure provisioning and application deployment pipelines, leading to human errors and limited flexibility due to ad-hoc information passing and predefined pipeline relationships.
Innovation Solution
Implementing a dynamic and persistent key-value store that allows direct access from pipeline stages for sharing configuration data, enabling flexible and automated data sharing without requiring a predefined parent-child relationship between pipelines, with versioning and immutability for rollback capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If configuration data is manually copied between pipelines, then information can be passed between provisioning and deployment pipelines, but human error increases and automation is broken
Solution Approach 1:
The patent introduces a shared data store as an intermediary between the provisioning pipeline and deployment pipeline. This data store automatically receives configuration data from the provisioning pipeline and makes it available to the deployment pipeline, eliminating the need for manual copying while maintaining data accuracy and preserving automation throughout the process.
Solution Approach 2:
The patent merges the data storage and sharing functionality into a common infrastructure that both pipelines access. By combining the provisioning and deployment pipelines through a shared data store, the system eliminates the separation that necessitated manual data transfer, thereby preventing human error while maintaining full automation.
2Loss of information
If pipelines are configured to process the same binary artifacts, then artifacts can be shared between pipelines, but flexibility is limited
Solution Approach 1:
The shared data store is designed with a universal interface that can store and retrieve various types of data including configuration data, binary artifacts, and other pipeline information. This universal access mechanism allows different pipeline types (provisioning, deployment, testing) to share data without requiring identical configuration, thereby maintaining flexibility while enabling comprehensive data sharing.
3Loss of information
If a predefined parent-child relationship is established between pipelines, then direct communication is possible, but flexibility and adaptability are reduced
Solution Approach 1:
The patent segments the pipeline architecture by introducing an independent shared data store layer between pipelines. This segmentation allows each pipeline to operate independently while accessing the same data source, eliminating the need for rigid parent-child relationships and enabling flexible, dynamic pipeline configurations that can adapt to different deployment scenarios.
4Loss of information
If manual copying of configuration data is used, then data can be transferred between pipelines, but human error increases
Solution Approach 1:
The shared data store enables pipelines to automatically retrieve the configuration data they need without human intervention. The provisioning pipeline writes configuration data to the data store, and the deployment pipeline automatically reads it, creating a self-service mechanism that eliminates manual copying and associated human errors, thereby improving deployment reliability.
Data Source
AI summary
Apparatus and method for dynamic and persistent data sharing between cloud services. To address limitations in existing systems, resource provisioning and application deployment pipelines are configured with calls to share data using a configuration service. In some implementations, the configuration service includes an API, which is called by one or more stages of the resource provisioning pipeline to write configuration data in key-value pairs (e.g., data related to resource provisioning). One or more stages of the application deployment pipeline are configured with calls to access the configuration data. The stages then use the configuration data for application deployment. The configuration service manages the key-value store using versioning, repeatability, and immutability, to ensures that a configuration generated by any pipeline execution can be automatically re-used by another pipeline, at execution time. Thus, the configuration service is a dynamic, single source of truth for sharing pipeline configuration data.