Runtime Security Agent for Reverse Shell Detection

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current data security systems are limited in detecting and responding to reverse command shell intrusion attacks, often identifying threats at the enterprise network level rather than the target device level, leading to delayed detection and manual, device-level responses that may not effectively mitigate breaches.

Innovation Solution

A data security system with a runtime security agent that injects detour code into user device processes to intercept API calls, allowing for process-level detection and remediation of reverse command shell intrusions, enabling targeted responses without disrupting the entire system.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of time

If network-level monitoring is used to detect reverse command shell intrusions, then detection coverage is provided, but detection timing is delayed and response is manual

Engineering Contradiction:
Improvedetection timingVSAvoidresponse automation
Core Design Contradiction:
Loss of timeVSExtent of automation

Solution Approach 1:

The system segments the monitoring function from the network level to the process level by deploying security agents directly on user devices. Each agent independently monitors API calls for its host process, enabling parallel detection across multiple processes simultaneously. This segmentation eliminates the sequential bottleneck of network-level monitoring and enables automated local response actions.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The system performs preliminary action by pre-installing security agents on user devices that continuously monitor API calls. When a reverse command shell intrusion is detected, the system immediately executes automated response actions such as terminating the malicious process or blocking network connections, without waiting for manual analysis. This preliminary automation of detection and response significantly reduces the time loss compared to manual network-level monitoring.

Inventive Principle:
Principle #10Preliminary action

2Measurement precision

If process-level detection is implemented, then detection precision is improved, but system complexity increases

Engineering Contradiction:
Improvedetection precisionVSAvoidsystem complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The system divides the complex task of process-level detection into manageable segments by deploying independent security agents on each user device. Each agent focuses solely on monitoring API calls for its host process, rather than attempting to analyze all system processes centrally. This segmentation reduces the complexity burden on any single component while maintaining high detection precision through localized, focused monitoring.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The security agent acts as an intermediary between the process and the detection system. Instead of directly analyzing complex process behaviors, the agent intercepts and filters API calls, presenting simplified information to the detection engine. This intermediary layer reduces system complexity by abstracting away the complexity of direct process analysis while preserving detection precision through careful monitoring of critical system interfaces.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Difficulty of detecting and measuring

If detour code is injected into processes, then API call interception is enabled, but process stability may be affected

Engineering Contradiction:
ImproveAPI call interception capabilityVSAvoidprocess stability
Core Design Contradiction:
Difficulty of detecting and measuringVSReliability

Solution Approach 1:

The detour code serves as an intermediary layer between the original API calls and the operating system. Rather than directly modifying or blocking API calls, the detour code captures them for analysis and then passes them through to the original handlers. This intermediary approach enables comprehensive API call interception while maintaining process stability, as the original process logic remains intact and continues to execute normally after detour inspection.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The detour code is designed to be homogeneous with the original process code in terms of execution context and memory space. By injecting code that matches the existing process architecture and calling conventions, the system enables API interception without introducing foreign elements that could destabilize the process. This homogeneity ensures that the detoured process maintains its original stability characteristics while gaining enhanced monitoring capabilities.

Inventive Principle:
Principle #33Homogeneity

Data Source

PatentEP3430557B1System and method for reverse command shell detection
Publication Date: 2021.04.21 CARBON BLACK
  • EP3430557B1 patent drawingFigure 1
  • EP3430557B1 patent drawingFigure 2
  • EP3430557B1 patent drawingFigure 3

AI summary

A system and method for detecting reverse command shell intrusions at a process-level on a user device is disclosed. In one embodiment, the system detects each process starting on an operating system of the user device, such as a mobile phone or laptop computer, and monitors Application Programming Interface (API) calls between each process and the operating system. The system then determines whether each process is associated with a reverse command shell intrusion based on information associated with each process and/or the API calls, and executes security policies against the processes associated with the reverse command shell intrusion to remediate the processes. In another embodiment, the system determines whether processes starting on a user device are associated with a reverse command shell intrusion by monitoring and analyzing information associated with the parent process of each process and/or API calls between each parent process and the operating system.