Automated ML Classification System for Malicious Code Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods for determining sample classification and similarity analysis are costly and human-intensive, lacking scalability and requiring multiple human decisions, which are inefficient and prone to errors, especially when dealing with large datasets.
Innovation Solution
A system utilizing probabilistic machine learning models generated through generative processes, where samples train the system to iteratively improve predictive capabilities, incorporating a directed graph workflow for efficient data processing and resource management, enabling automated classification and similarity analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional human-intensive methods are used for classification and similarity analysis, then decision-making can be performed with human judgment, but the process becomes costly, time-consuming, and lacks scalability
Solution Approach 1:
The patent replaces human decision-making processes with automated machine learning systems. Multiple ML models (random forest, support vector machines, neural networks) are implemented to perform classification and similarity analysis automatically, eliminating the need for human analysts while maintaining high accuracy through ensemble methods and iterative training on large datasets
Solution Approach 2:
The system enables self-service through automated model generation and training. The ML infrastructure automatically trains models on incoming data, performs iterative improvements through feedback loops, and generates predictions without human intervention. The system self-optimizes by continuously learning from new samples and adjusting model parameters
2Reliability
If traditional methods are used for classification and similarity analysis, then human expertise can be applied to complex decisions, but the process becomes costly and non-repeatable
Solution Approach 1:
The patent transforms qualitative human judgment into quantitative machine learning parameters. Features are extracted and converted into numerical representations that ML models can process. The system uses configurable parameters such as classification thresholds, model selection criteria, and similarity metrics that can be adjusted without changing the underlying system architecture, enabling consistent repeatable results
Solution Approach 2:
The patent creates a universal ML platform that handles multiple classification tasks and similarity analysis problems through a common infrastructure. The same system architecture supports different algorithms (random forest, SVM, neural networks), various data types, and multiple business use cases, reducing overall system complexity through consolidation while maintaining versatility
3Productivity
If automated machine learning systems are implemented for classification and similarity analysis, then scalability and repeatability are improved, but the system requires complex infrastructure and model management
Solution Approach 1:
The patent segments the ML system into modular components: data preprocessing module, feature extraction module, model training module, prediction module, and evaluation module. Each component can be independently developed, tested, and deployed. This modular architecture enables parallel processing and distributed computation, improving throughput while making the complex infrastructure more manageable through clear separation of concerns
Solution Approach 2:
The patent introduces an intermediary layer that manages the complexity of ML infrastructure. This includes automated model selection algorithms that choose appropriate algorithms based on data characteristics, hyperparameter optimization systems that automatically tune model parameters, and deployment pipelines that streamline the transition from training to production. These intermediaries shield users from underlying complexity while enabling high productivity
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The invention relates to a method for implementation by one or more data processors forming part of at least one computing system, the method comprising: placing a sample of data comprising files for access or execution by a computing system within a directed acyclic graph represented as a queue of work items for a particular worker class wherein the directed acyclic graph comprises a plurality of hierarchical nodes in which nodes are represented as the individual worker classes, that are used to process the sample of data; scheduling work items according to at least a sample prioritization or a worker rate within the queue for each of a plurality of workers by traversing the nodes of the directed acyclic graph; serving the work items to the workers according to the queue; receiving results from the workers for the work items; and classifying the sample of data based on the received results to indicate whether or not at least one file in the sample of data likely comprises malicious code; wherein: the nodes of the directed graph are traversed based on the received results; each work item comprises one or more tasks to effect extraction from the sample of data; and each worker class represents a set of extracted features and is associated with a specific sample type.