Module-Granular Shadow Stack Enforcement for External Modules
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing shadow stack violation enforcement systems terminate an entire thread or process when a violation occurs, regardless of the specific code that triggered it, which can be impractical due to interactions with external modules or environments that tamper with return addresses.
Innovation Solution
Enforce shadow stack violations at module granularity by determining if the violation occurred during execution of a called module and enabling or disabling enforcement based on module-specific settings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If shadow stack violation enforcement is enabled for an application, then security against stack buffer overflow attacks is improved, but the application cannot interact with external modules or environments that tamper with return addresses
Solution Approach 1:
The patent segments shadow stack enforcement at the module level rather than applying it uniformly to the entire thread. Each module can have independent shadow stack enforcement settings, allowing the application to enable enforcement for trusted internal modules while disabling it for external modules that may legitimately tamper with return addresses. This granular segmentation resolves the contradiction by providing both security where needed and compatibility where required.
Solution Approach 2:
The patent implements local quality by allowing different shadow stack enforcement policies to apply to different modules within the same thread. The enforcement status is stored per-module in the module data structure, enabling the system to enforce shadow stack violations for security-critical modules while permitting violations in external modules that require flexibility. This local differentiation simultaneously achieves security improvement and adaptability.
2Reliability
If shadow stack violations are enforced at thread level, then security is simplified to enforce, but entire threads or processes are terminated even when only external modules cause violations
Solution Approach 1:
The patent divides shadow stack enforcement into module-level segments rather than thread-level enforcement. When a shadow stack violation occurs, the system identifies which specific module caused the violation and only terminates or penalizes that module, allowing the rest of the thread and other modules to continue executing. This segmentation prevents unnecessary termination of entire applications and maintains productivity while preserving security enforcement.
Solution Approach 2:
The patent introduces an intermediary mechanism that sits between the shadow stack violation detection and the thread termination action. The module-level enforcement status acts as an intermediary that filters violations, allowing the system to distinguish between malicious violations (from untrusted modules) and benign violations (from external modules). This intermediary prevents premature thread termination and maintains application continuity.
3Reliability
If shadow stack enforcement is enabled, then protection against malicious code execution is improved, but applications must disable enforcement to work with legacy or third-party modules
Solution Approach 1:
The patent segments shadow stack enforcement by module, allowing applications to enable enforcement for newly compiled modules while disabling it for legacy or third-party modules that haven't been updated. This granular approach eliminates the all-or-nothing choice and allows simultaneous coexistence of enforced and non-enforced modules, improving both protection and compatibility.
Solution Approach 2:
The patent implements dynamic enforcement where the shadow stack enforcement status can be adjusted on a per-module basis. Applications can dynamically enable or disable enforcement for specific modules based on their trust level and compatibility requirements. This dynamic flexibility allows applications to optimize security for critical modules while maintaining compatibility with legacy modules, resolving the contradiction between protection and adaptability.
Data Source
Figure 1A
Figure 1B
Figure 2A
AI summary
Enforcing shadow stack violations at module granularity, rather than at thread or process granularity. An exception is processed during execution of a thread based on code of an application binary, which is enabled for shadow stack enforcement, that calls an external module. The exception results from a mismatch between a return address popped from the thread's call stack and a return address popped from the thread's shadow stack. Processing the exception includes determining that the exception resulted from execution of an instruction in the external module, and determining whether or not the external module is enabled for shadow stack enforcement. Based at least on these determinations, execution of the thread is terminated when the external module is enabled for shadow stack enforcement, or the thread is permitted to continue executing when the external module is not enabled for shadow stack enforcement.