Circular Shadow Stack Audit Mode Overflow Prevention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Certain software programs, especially those not aware of shadow stack functionality, can cause shadow stack overflow by continuously performing function calls without corresponding returns, leading to fatal system errors due to unmanaged stack growth, as traditional control protection exceptions are not triggered.
Innovation Solution
Enabling at least a portion of the shadow stack to function as a circular stack in audit mode, where contents are overwritten when usage reaches a defined threshold, preventing overflow and allowing data collection on non-compliant application binaries without causing fatal errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If shadow stack enforcement is enabled to protect return addresses from malicious modification, then system security is improved, but software compatibility deteriorates because non-compliant programs cause fatal system errors
Solution Approach 1:
The shadow stack is implemented with dynamic characteristics by allowing it to be configured in different modes (enforcement mode and audit mode). The stack can transition between strict enforcement that terminates non-compliant programs and a more lenient audit mode that collects compatibility data, enabling the system to adapt its security behavior based on operational context.
Solution Approach 2:
The audit mode acts as an intermediary between full shadow stack enforcement and complete disablement. It provides a intermediate state where the shadow stack operates but allows non-compliant programs to continue running while collecting telemetry data, serving as a bridge to achieve both security and compatibility goals.
2Reliability
If shadow stack enforcement is enabled to detect return address mismatches, then protection against buffer overflow attacks is improved, but system stability deteriorates due to fatal errors from non-compliant software
Solution Approach 1:
The system dynamically switches between enforcement mode and audit mode. In audit mode, the shadow stack continues to operate and detect mismatches but does not terminate programs, allowing the system to maintain stability while still gathering data on compatibility issues that affect long-term reliability.
Solution Approach 2:
The audit mode serves as a cushioning mechanism that prevents fatal system errors before they occur. By collecting telemetry data on non-compliant programs in advance, the system can prepare compatibility fixes or workarounds that will prevent stability issues when full enforcement is eventually implemented.
3Loss of information
If audit mode collects telemetry data on non-compliant programs, then software compatibility information is improved, but system complexity increases due to additional monitoring and reporting mechanisms
Solution Approach 1:
The audit mode implementation leverages existing shadow stack infrastructure and telemetry frameworks already present in the system. Rather than building completely new monitoring mechanisms, the solution reuses existing components to collect compatibility data, reducing the additional complexity introduced by the audit functionality.
4Reliability
If shadow stack usage threshold is set low to prevent overflow, then memory safety is improved, but productivity deteriorates due to frequent overwriting of valid return addresses
Solution Approach 1:
The system allows the usage threshold parameter to be configured and adjusted. By optimizing this parameter based on empirical data collected in audit mode, the system can find the optimal balance point that ensures memory safety while minimizing disruptions to program execution and maintaining high productivity.
Data Source
AI summary
Performing shadow stack functionality for a thread in an audit mode includes initiating execution of a thread at the processor. Execution of the thread includes initiating execution of executable code of an application binary as part of the thread and enabling shadow stack functionality for the thread in an audit mode. Based at least on the execution of the thread in the audit mode, at least a portion of the shadow stack is enabled to be a circular stack. In response to determining that usage of the shadow stack has reached the defined threshold, one or more currently used entries of the shadow stack are overwritten, preventing the shadow stack from overflowing.