Microservice Handshaking with Iterative Dataset Slicing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Microservices architectures require large data transfers between services due to their independent technology stacks, lacking a common database, leading to inefficient data sharing and increased network and processing costs.
Innovation Solution
Implementing vertical reducers within microservices to perform intelligent dataset slicing, allowing targeted data transfer based on the needs of recipient services through a vertical reduction process, followed by evaluation and iterative refinement until a desirable result is achieved.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If microservices have their own separate databases and stacks, then independence and deployability are improved, but data transfer volume increases because selective data requests are not possible
Solution Approach 1:
The patent segments the data transfer process into multiple phases: initial metadata exchange, iterative slice generation and evaluation. Each phase transfers only the necessary portion of data, avoiding bulk data transfer. The dataset is divided into slices that are evaluated incrementally, allowing the system to stop transferring data once sufficient information is obtained.
Solution Approach 2:
The patent applies partial action by transferring only the minimum necessary data slices rather than complete datasets. The iterative evaluation mechanism allows the system to determine when enough data has been received, preventing excessive data transfer while maintaining service independence.
2Reliability
If all data is transferred between microservices, then data availability is improved, but network and processing costs increase
Solution Approach 1:
The patent performs preliminary actions by exchanging metadata and dataset descriptions before actual data transfer. This allows both microservices to understand the data structure and requirements in advance, enabling selective slice generation that ensures data availability while minimizing transfer volume and associated costs.
Solution Approach 2:
The evaluation mechanism provides feedback to determine whether received data slices are sufficient. This feedback loop allows the system to adjust data transfer dynamically, ensuring adequate data availability while stopping transfers when sufficiency is achieved, thereby reducing network and processing costs.
3Productivity
If iterative slice evaluation is implemented, then data transfer efficiency is improved, but handshaking complexity increases
Solution Approach 1:
The patent implements a universal handshaking protocol that handles multiple functions: metadata exchange, slice generation requests, evaluation feedback, and termination decisions. This multi-functional protocol reduces the need for separate communication mechanisms for each data transfer stage, managing complexity while improving efficiency.
Data Source
AI summary
A computer-implemented process of intelligent dataset slicing within a network having a plurality of microservices is disclosed. Handshaking between a first microservice and a second microservice is initiated. A vertical reduction of a dataset is required by the second microservice and from the first microservice. A first slice of the dataset generated by the vertical reduction is received by the second microservice and from the first microservice. The sliced dataset is evaluated by the second microservice. The vertical reduction is terminated by the second microservice based upon the evaluating.