Byte-Level Transformer File Classification for Malware Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional cybersecurity platforms struggle with suboptimal prediction performance due to limitations in tree-based AI models and neural networks, which do not capture a file in its entirety and impose a particular limiting modality or encoding on the data, which represents certain assumptions about the data and thus limits adaptability of the model.
Innovation Solution
A transformer-based AI model that operates directly on file bytes to identify malicious code, automating feature extraction and providing model-driven exploration of the representation space without manual feature engineering, and is adaptable to evolving obfuscation and malware techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If tree-based AI models or neural networks are used for file classification, then the system can process files, but the prediction performance is suboptimal and the model adaptability is limited due to imposing a particular modality or encoding on the data
Solution Approach 1:
The patent applies a transformer-based model that can process multiple file types and representations universally without requiring separate encoding schemes for different data modalities. The model handles binary files, text files, and other formats through a unified attention mechanism, eliminating the need for modality-specific processing pipelines and thereby improving both prediction performance and adaptability across diverse file types.
Solution Approach 2:
The patent changes the fundamental parameter of data representation by operating directly on raw file bytes rather than transforming data into fixed modalities or encodings. This parameter change allows the model to learn optimal representations automatically through the attention mechanism, improving adaptability to evolving malware techniques while maintaining high prediction accuracy through direct byte-level analysis.
2Measurement precision
If conventional AI models are used, then the system can classify files, but manual feature engineering is required which increases complexity and reduces accuracy
Solution Approach 1:
The transformer-based model performs self-service by automatically learning relevant features from raw file bytes through its attention mechanism, eliminating the need for external manual feature engineering. The model identifies and focuses on critical byte patterns, sequences, and structures autonomously, thereby reducing system complexity while improving classification accuracy through data-driven feature discovery.
Solution Approach 2:
The patent replaces the mechanical process of manual feature engineering with an automated attention-based feature extraction mechanism. Instead of manually designing and extracting features through systematic procedures, the model dynamically attends to relevant bytes and patterns, substituting the mechanical feature engineering pipeline with a more flexible and accurate automated representation learning approach.
3Manufacturing precision
If the model operates on entire files with manual feature engineering, then features can be extracted, but the process is time-consuming and less precise
Solution Approach 1:
The transformer-based model segments the file processing task by applying self-attention mechanisms to different byte sequences and positions independently, then aggregating the results. This segmentation allows parallel processing of multiple file regions simultaneously, reducing the overall extraction time while maintaining high precision through the cumulative attention weights that focus on the most relevant segments for classification.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A method of monitoring an endpoint for malicious code includes deploying an artificial intelligence (AI) model to a endpoint protection system, the AI model trained on a plurality of executable code files in byte form, monitoring a target system for execution of a target executable file. The method further includes analyzing, by the AI model, the target executable file in the byte form of the target executable file and determining, based on an output of the AI model, a decision variable for the target executable file.