Automated Machine Learning Model Generation for Anomalous Event Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high cost and manual effort required to create machine learning models, especially in applications like cybersecurity where large amounts of data exceed human capability for effective analysis, leading to prohibitive costs and potential for human bias in model creation.
Innovation Solution
Automating the generation of machine learning models by selecting significant fields from datasets based on blind field analysis, generating field combinations associated with event types, and deploying these models in a data monitoring pipeline for real-time anomalous event detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning models are created manually, then model accuracy can be optimized, but the cost and time required become prohibitive
Solution Approach 1:
The system performs automatic model generation where the computational system serves itself by autonomously selecting fields, generating combinations, training models, and evaluating performance without human intervention, thereby reducing both time and cost while maintaining accuracy through systematic automation
Solution Approach 2:
The system changes the parameter of model generation from manual configuration to automated configuration by adjusting field selection criteria, combination generation parameters, and model training parameters automatically, enabling rapid iteration and reduced creation time while preserving accuracy through optimized parameter selection
2Manufacturing precision
If manual model creation is used, then model quality can be controlled, but human bias is introduced
Solution Approach 1:
The system eliminates human bias by replacing manual model creation with an automated system that objectively selects fields and generates models based on predetermined criteria and performance metrics, ensuring consistent quality without subjective human influence
Solution Approach 2:
The system incorporates automated feedback loops where model performance is continuously evaluated against objective criteria, and field selections and model generations are adjusted based on measured outcomes, maintaining quality through data-driven rather than human-judgment-driven decisions
3Measurement precision
If comprehensive data analysis is performed, then detection accuracy improves, but the complexity of manual analysis increases
Solution Approach 1:
The system segments the complex analysis task into distinct automated components: field selection, combination generation, model training, and evaluation. Each segment handles a specific aspect of the analysis, reducing overall complexity while maintaining comprehensive analysis capability through systematic division of labor
Solution Approach 2:
The system replaces manual mechanical analysis processes with automated computational processes, substituting human cognitive effort with algorithmic field selection and model generation, thereby reducing analysis complexity while improving detection accuracy through exhaustive computational analysis
4Adaptability or versatility
If multiple machine learning models are generated, then detection coverage improves, but the resource cost increases
Solution Approach 1:
The system generates multiple models using automated field combination methods, creating more models than traditionally necessary. This excessive generation is justified by the automated nature of the process, which can efficiently manage resource consumption while improving detection coverage through increased model diversity
Solution Approach 2:
The system automatically adjusts parameters such as field selection criteria, combination depths, and model training configurations to optimize resource usage while generating multiple models. By dynamically changing these parameters based on performance feedback, the system achieves broad detection coverage without linearly increasing resource consumption
Data Source
AI summary
One or more computing devices, systems, and/or methods for machine learning model generation and/or anomalous event detection are provided. In an example, one or more datasets having first fields are identified. Significance scores associated with the first fields are determined. Second fields are selected from the first fields based upon the significance scores. Field combinations are generated based upon the second fields. Based upon the field combinations, a plurality of machine learning models is generated. The plurality of machine learning models include a first machine learning model associated with a first field combination of the field combinations, and a second machine learning model associated with a second field combination of the field combinations. The plurality of machine learning models is deployed in a data monitoring pipeline. Using the plurality of machine learning models, an anomalous event is detected based upon data passing through the real-time data monitoring pipeline.