Hook Re-entry Prevention via State Flag Isolation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hook layer configurations face challenges in preventing hook re-entry during dynamic API hooking, particularly in managing and configuring hook layers to avoid recursive execution of original functions, which complicates programming and update management.
Innovation Solution
A hook re-entry prevention device and method that utilizes a state information management module to set and release a state information flag, determining whether a hook function is called from within or outside the hook layer, and a re-entry prevention module to execute the original program request based on this flag, thereby preventing recursive hook execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the hook layer manages addresses used to call hook target API original functions and uses auxiliary methods to prevent hook influence, then the hook layer can support multiple functions and provide filtering processing, but the programming becomes complicated and update management becomes difficult
Solution Approach 1:
The patent introduces a hook layer management module as an intermediary between the hook layer and the target API original functions. This module automatically manages the addresses of hook target API original functions and handles the calling relationships, eliminating the need for complex manual address management and auxiliary methods. The intermediary module abstracts the complexity while maintaining the ability to support multiple functions.
2Reliability
If module-based exception handling is performed to prevent hook re-entry, then hook re-entry can be prevented, but runtime modules may be incorrectly identified and configuration management becomes magnified
Solution Approach 1:
The patent implements a self-service mechanism where the hook layer management module automatically identifies and manages runtime modules that belong to the hook layer. The module uses automatic detection algorithms to determine which modules require hook re-entry prevention, eliminating the need for manual configuration and extension of functionality. This self-identifying approach prevents hook re-entry while avoiding the complexity of manual configuration management.
3Reliability
If all runtime modules in the hook layer are configured to know addresses of original functions and call appropriate addresses, then hook re-entry can be prevented, but the program becomes complicated in proportion to the number of hook target APIs
Solution Approach 1:
The patent creates a universal hook layer management module that handles all hook target API original functions through a single, unified interface. Instead of requiring each runtime module to individually manage addresses for multiple APIs, the universal module provides centralized address management and calling functionality that works across all hook target APIs, reducing program complexity while maintaining reliable hook re-entry prevention.
4Stability of the object's composition
If the hook layer does not use the hook target API to avoid hook influence, then the hook layer remains independent, but it is not easy to prevent the hook layer from using the hook target API and requires emulator implementation
Solution Approach 1:
The patent introduces a hook layer management module as an intermediary that allows the hook layer to use hook target APIs while maintaining independence. The intermediary module intercepts and manages API calls, ensuring that the hook layer can functionality while preventing hook re-entry. This approach maintains hook layer independence and stability while avoiding the need for complex emulator implementation.
Data Source
AI summary
Disclosed are a hook re-entry prevention device and a recording medium, in which a program for executing the method thereof in a computer is recorded thereon. A state information management module sets or releases a state information flag which indicates whether a hook function is called. If the state information flag is set when the hook function is called, a re-entry prevention module determines that the calling of the hook function has been conducted from the inside of a hook layer and returns a value to a system layer. According to the present invention, an entry/breakaway layer which is independent from the hook layer functions in such a manner that the hook layer is safely isolated from the hook, thereby preventing hook re-entry caused by the callback of an original function of a lower function while a hook routine within the hook layer is performed during a dynamic API hooking.