Deep Learning Shell Script Obfuscation Detection

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing techniques for detecting malicious shell scripts are ineffective due to the use of obfuscation tools, which render rule-based methods incapable of identifying malicious code.

Innovation Solution

A system utilizing a deep learning model with a convolutional neural network (CNN) branch and a feedforward neural network (FNN) branch is employed to detect obfuscated shell scripts. The system generates a character frequency feature vector and inputs the shell script text to the CNN branch and the character frequency vector to the FNN branch, determining probability scores for various obfuscation types.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If rule-based detection methods are used to identify malicious shell scripts, then the detection process is simple and fast, but the detection accuracy deteriorates due to obfuscation tools

Engineering Contradiction:
Improvedetection speedVSAvoiddetection accuracy
Core Design Contradiction:
ProductivityVSMeasurement precision

Solution Approach 1:

The patent replaces rule-based detection (mechanical system) with a deep learning-based detection system. The neural network automatically learns patterns and features from training data, substituting manual rule creation and execution with an adaptive machine learning model that can handle obfuscated malicious scripts effectively.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent transforms the shell script into different feature representations (character n-grams, word n-grams, structural features) before feeding them to the neural network. This parameter transformation allows the model to capture patterns at multiple levels of abstraction, improving detection accuracy while maintaining computational efficiency.

Inventive Principle:
Principle #35Parameter changes

2Measurement precision

If deep learning models with multiple branches are used to detect obfuscated shell scripts, then detection accuracy improves, but system complexity increases

Engineering Contradiction:
Improvedetection accuracyVSAvoidmodel complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent divides the detection task into multiple specialized branches: a CNN branch for processing character n-gram sequences, an FNN branch for processing word n-gram features, and another FNN branch for structural features. Each branch specializes in extracting specific types of patterns, and their outputs are combined for final classification. This segmentation improves detection accuracy while keeping each individual branch relatively simple.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The deep learning model is designed to handle multiple types of input features (character n-grams, word n-grams, structural features) through a unified multi-branch architecture. This universal framework can detect various obfuscation techniques and malicious patterns using the same model structure, reducing overall system complexity compared to having separate specialized models for each feature type.

Inventive Principle:
Principle #6Universality (Multi-functionality)

3Measurement precision

If multiple feature types are extracted and processed separately, then detection accuracy improves, but processing time increases

Engineering Contradiction:
Improveclassification accuracyVSAvoidprocessing time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent performs feature extraction (character n-grams, word n-grams, structural features) as preliminary processing steps before feeding data to the neural network. By pre-computing these features and organizing them in advance, the model can focus computational resources on pattern recognition during inference, reducing actual detection time while maintaining high accuracy.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS12299124B2Deep learning based detection of malicious shell scripts
Publication Date: 2025.05.13 CAPITAL ONE SERVICES LLC
  • US12299124B2 patent drawing
  • US12299124B2 patent drawing
  • US12299124B2 patent drawing

AI summary

In some implementations, a system may receive a shell script associated with a computing device. The system may generate a character frequency feature vector based on the shell script. The system may input text of the shell script to a convolutional neural network (CNN) branch of a trained deep learning model. The system may input the character frequency feature vector to a feedforward neural network (FNN) branch of the trained deep learning model. The system may determine using the trained deep learning model, respective probability scores for each of a plurality of obfuscation types for the shell script based on a combined output of the CNN branch and the FNN branch. The system may detect whether the shell script is obfuscated based on respective probability scores for each of the plurality of obfuscation types determined for the shell script.