ML Model Retraining With Mislabeled File Detection for Malware
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Malware Machine Learning (ML) model development is hindered by mislabeled files, which are difficult to identify manually, leading to increased false negatives and high training costs due to lengthy optimization processes, and traditional approaches lack agility in delivering updated models.
Innovation Solution
A cloud-based system identifies incorrect labels by grouping mislabeled files into sub-groups for human review using multiple ML models trained with supervised and unsupervised learning, and adjusts weights based on prediction errors to improve detection rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual verification of all sandbox malicious files is performed to identify mislabeled files, then label accuracy is improved, but time consumption and resource costs increase significantly
Solution Approach 1:
The patent segments the verification task by dividing all sandbox malicious files into multiple batches, with each batch processed by a different ML model. This segmentation allows parallel processing of different file subsets, reducing overall verification time while maintaining comprehensive coverage through multiple models working on different segments of the data
Solution Approach 2:
The patent introduces ML models as intermediary components between the raw labeled data and final verification results. These models automatically perform preliminary identification of mislabeled files, serving as intermediaries that filter and prioritize candidates for human review, thereby reducing the time burden on manual verification while improving label accuracy
2Measurement precision
If multiple ML models are used to identify mislabeled files through grouping and intelligent sampling, then detection accuracy is improved, but system complexity increases
Solution Approach 1:
The patent segments the detection task across multiple ML models, where each model processes different batches of files. This segmentation enables the system to achieve higher detection accuracy through ensemble reasoning while managing complexity by organizing models in a structured parallel architecture rather than a monolithic complex system
Solution Approach 2:
The patent creates ML models with multi-functionality, where each model serves multiple purposes: detecting mislabeled files, providing confidence scores, and contributing to ensemble decisions. This universality reduces overall system complexity by having each component perform multiple functions rather than requiring separate specialized components for each function
3Measurement precision
If conventional ML model training is performed with lengthy optimization processes to achieve good detection results, then model accuracy is improved, but training time and computational costs increase
Solution Approach 1:
The patent applies preliminary action by using ML models to pre-identify mislabeled files and generate corrected labels before the main training process. This preliminary labeling action provides cleaner, more accurate training data, which reduces the optimization burden during subsequent training phases and enables faster convergence to high accuracy models
Solution Approach 2:
The patent implements feedback mechanisms where ML model predictions and confidence scores feed back into the label correction process. This feedback loop allows the system to iteratively improve label quality and model performance, achieving high accuracy with fewer training iterations by continuously refining both labels and model parameters based on mutual feedback
4Productivity
If mislabeled files are not identified and corrected, then manual review resources are saved, but false negative rates increase artificially
Solution Approach 1:
The patent enables the system to self-correct mislabeled files through automated ML model identification and label correction mechanisms. The system serves itself by detecting and correcting its own training data errors without requiring extensive external manual intervention, thereby maintaining resource efficiency while improving detection reliability through self-improving label quality
Data Source
AI summary
Systems and methods for learning from mistakes to improve detection rates of Machine Learning (ML) models. The systems and methods including receiving data with labels; running the data through a trained ML model for predictions; identifying errors in the predictions based on the labels received with the data; adjusting weights associated with samples in the data based on the identified errors; and retraining the ML model with the adjusted weights.