Byte-Based Transformer Malware Classification With Semi-Supervised Labeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems for analyzing and classifying binary file telemetry data are labor-intensive and economically unfeasible at scale due to the reliance on supervised training and manual labeling, limiting the utility of collected data.
Innovation Solution
A semi-supervised learning approach using a representation-agnostic byte-based AI model, such as a transformer-based model, is employed to generate labels for unlabeled data, allowing automation of feature extraction and enabling supervised training of other models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If supervised training with manual labeling is used for malware classification, then classification accuracy can be achieved, but the process becomes labor-intensive and economically unfeasible at scale
Solution Approach 1:
The system uses the trained AI model to automatically generate labels for unlabeled binary files, enabling the system to self-service the labeling task without human intervention. The model processes files and assigns malware classification labels autonomously, transforming a manual process into an automated one that can scale indefinitely without additional human resources.
Solution Approach 2:
The AI model serves as an intermediary between the labeled training data and the unlabeled production data. It learns patterns from the small labeled subset and applies this knowledge to automatically classify the large unlabeled dataset, bridging the gap between limited annotated examples and comprehensive data coverage.
2Reliability
If manual labeling is used to create training datasets, then supervised learning models can be trained, but the process is too slow and expensive for large-scale deployment
Solution Approach 1:
The system applies partial action by using a small subset of labeled data (only what's necessary for initial model training) rather than requiring complete labeling of all datasets. This partial labeling approach is sufficient to train the model, which then handles the remaining classification tasks automatically, dramatically reducing the time and resources needed for data preparation.
3Extent of automation
If representation-agnostic byte-based models are used, then automation of feature extraction is achieved, but computational complexity increases
Solution Approach 1:
The byte-based transformer model is designed to be representation-agnostic, meaning it can process binary files directly without requiring conversion to specific feature representations. This universal approach allows the same model architecture to handle diverse malware types and binary formats, automating feature extraction across different file types while maintaining a single unified processing pipeline.
Data Source
AI summary
A method of monitoring an endpoint for malicious code includes obtaining a corpus of files collected by an endpoint protection system, selecting a subset of the corpus of files comprising labeled files, wherein the subset of the corpus is representative of the corpus of files, and training a first artificial intelligence (AI) model, using the subset of the corpus of files in byte form, to infer labels for unlabeled data. The method further includes applying the first AI model to unlabeled files of the corpus of files in byte form to generate labels for the unlabeled files, performing supervised training of a second AI model using the corpus of files and the labels generated for the unlabeled data, and deploying the second AI model to the endpoint protection system.