Semi-supervised Workload Labeling in SDN Data Centers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Manual labeling of virtual computing instances (VCIs) in large-scale software-defined networking (SDN) data centers is inefficient and prone to errors, especially as the number of VCIs grows, necessitating improved techniques for workload labeling.
Innovation Solution
Implementing semi-supervised learning techniques to reduce manual input by grouping workloads based on feature similarity, using a subset of workloads to generate training data, and training a model to automate label assignment, allowing for efficient and accurate labeling of a large number of workloads with minimal user input.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual labeling is used for all VCIs, then labeling accuracy can be maintained through human review, but the time and effort required increases significantly as the number of VCIs grows
Solution Approach 1:
The patent segments the labeling process into two distinct phases: (1) manual labeling of a representative subset of VCIs to create training data, and (2) automated labeling of remaining VCIs using machine learning models trained on the subset data. This segmentation allows human reviewers to focus on a manageable portion while algorithms handle the bulk, resolving the contradiction between accuracy and time consumption.
Solution Approach 2:
The patent performs preliminary manual labeling on a subset of VCIs before proceeding to automated labeling. This preliminary action creates the training dataset needed for machine learning models, enabling accurate automated labeling subsequent to the initial manual phase. The preliminary work establishes a foundation that accelerates the overall process while maintaining accuracy standards.
2Reliability
If manual labeling is performed on a large number of VCIs, then comprehensive label coverage is achieved, but the process becomes tedious and error-prone
Solution Approach 1:
The patent implements self-service labeling where the system automatically generates labels for VCIs using machine learning models trained on manually labeled data. The automated models serve themselves by continuously improving through feedback loops, reducing the need for ongoing manual intervention while maintaining comprehensive label coverage across all VCIs.
Solution Approach 2:
The patent incorporates feedback mechanisms where manually labeled VCIs are used to train and refine machine learning models. The models generate predictions that can be reviewed and corrected by humans, with corrected labels fed back into the training process. This feedback loop improves model accuracy over time, achieving comprehensive coverage while reducing manual effort and errors.
3Productivity
If automated labeling is used for all VCIs, then processing speed increases, but labeling accuracy decreases without sufficient training data
Solution Approach 1:
The patent applies partial manual labeling to a carefully selected subset of VCIs rather than attempting to manually label all VCIs or using purely automated methods. This partial action provides sufficient training data to achieve high accuracy in automated labeling while maximizing processing throughput for the remaining VCIs.
4Measurement precision
If more VCIs are labeled manually to improve model training, then model accuracy improves, but the initial time investment increases
Solution Approach 1:
The patent changes the parameter of training data quantity from 'all VCIs' to 'representative subset of VCIs'. By carefully selecting a diverse but manageable subset that captures the variability in the VCI population, the system achieves sufficient model accuracy without the time investment required for comprehensive manual labeling of all VCIs.
Data Source
AI summary
The disclosure provides an approach for workload labeling and identification of known or custom applications. Embodiments include determining a plurality of sets of features comprising a respective set of features for each respective workload of a first subset of a plurality of workloads. Embodiments include identifying a group of workloads based on similarities among the plurality of sets of features. Embodiments include receiving label data from a user comprising a label for the group of workloads. Embodiments include associating the label with each workload of the group of workloads to produce a training data set. Embodiments include using the training data set to train a model to output labels for input workloads. Embodiments include determining a label for a given workload of the plurality of workloads by inputting features of the given workload to the model.