Multi-tenancy Machine Learning Data Partitioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network security monitoring systems are inefficiently implemented, particularly in handling multi-tenancy scenarios where they often require duplicating machine learning jobs for each client, leading to increased processing costs and potential dilution of accuracy due to mixed client events.
Innovation Solution
A method and system for multi-tenancy machine-learning that allows a single machine learning job to analyze data from multiple clients simultaneously while maintaining data separation, using a database to partition and analyze client data independently, and triggering alerts based on predefined rules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning jobs are duplicated for each client, then data analysis can be performed independently for each client, but processing costs increase and system complexity increases
Solution Approach 1:
The patent segments client data into separate partitions within a unified machine learning job. Each client's data is processed independently through partitioning, ensuring data isolation and analysis accuracy while avoiding the need to duplicate entire machine learning jobs for each client. This resolves the contradiction by maintaining independent analysis capability without proportionally increasing system complexity.
2Measurement precision
If machine learning jobs are duplicated for each client, then data analysis can be performed independently for each client, but processing costs increase
Solution Approach 1:
The patent merges multiple clients' data into a single machine learning job while maintaining partition separation. This allows the system to process all client data through one unified job rather than creating separate jobs for each client, significantly reducing processing costs while preserving the ability to analyze each client's data independently through the partitioning mechanism.
3Loss of energy
If a single machine learning job processes multiple clients' data, then processing costs are reduced, but data separation and analysis accuracy may be compromised
Solution Approach 1:
The patent implements data partitioning within the single machine learning job, where each client's data is organized into separate partitions. This segmentation ensures that even though all data is processed in one job, each client's data remains isolated and can be analyzed independently, preserving analysis accuracy while achieving cost efficiency through consolidated processing.
4Productivity
If a single machine learning job processes multiple clients' data, then processing efficiency is improved, but system complexity management becomes more challenging
Solution Approach 1:
The patent creates a universal machine learning job structure that can handle multiple clients' data through partitioning. This single job design serves multiple functions by processing all client data in one execution, improving processing efficiency. The partitioning mechanism provides a standardized way to manage different clients' data within this universal structure, making job management more systematic and less complex than coordinating multiple separate jobs.
Data Source
AI summary
Embodiments of the disclosure are related to a method, apparatus, and system for multi-tenancy machine-learning based on collected data from multiple clients, comprising: obtaining client data from multiple clients; sending the client data from the multiple clients to a database; pulling data from the database by a machine learning job based on job parameters; partitioning the data by each client for the machine learning job; analyzing the data from the multiple clients by the machine learning job; sending the results of the analysis of the data from the multiple clients by the machine learning job back to the database; querying the database for data specified by rules; and if rules are met by the queried data for one or more of the multiple clients, transmit an alert to an alerting platform.