A Frida-based dynamic instrumentation and coverage collection method for multi-threaded applications
Through the dynamic instrumentation and coverage collection method of multi-threaded applications based on Frida, the problems of low execution speed and platform dependence in traditional technology are solved, and efficient dynamics for multi-threaded applications are achieved without leaking source code. Analyzing and testing ensures the accuracy and reliability of coverage data.
Patent Information
- Application Number
- CN202410685436.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-30
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-05-30
AI Technical Summary
Traditional binary dynamic instrumentation technology is low in execution speed and platform dependent in multi-threaded applications, making it difficult to achieve efficient fuzz testing.
Using the dynamic instrumentation and coverage collection method of multi-threaded applications based on Frida, the child processes or subthreads of multi-threaded applications are monitored and organized in real time by initializing the Frida module, Interceptor interceptor, Observer observer and Hook entry function pointer, and statistics Coverage information and use Ptrace to listen to system calls to ensure thread-safe coverage data collection.
Without leaking the source code, efficient dynamic analysis and testing of multi-threaded applications is achieved, race conditions are avoided, and the accuracy and reliability of coverage data are ensured.
Smart Images

Figure CN118394658B_ABST
Abstract
Description
Technical Field
[0001] The embodiment of the present invention relates to the field of fuzzy testing technology, and in particular to a Frida-based multi-threaded application dynamic instrumentation and coverage collection method. Background Art
[0002] With the continuous advancement of fuzz testing technology, more and more application developers who prioritize software confidentiality have begun to pay attention to code security. They seek to implement efficient fuzz testing without disclosing the source code, which requires binary dynamic instrumentation technology. Although traditional binary dynamic instrumentation technologies, such as QEMU-based simulation execution and Intel Processor Trace-based methods, can achieve dynamic instrumentation, these technologies often face the challenge of significantly reduced execution speed, which is only one-tenth to one-half of the original program, and these technologies are platform-dependent.
[0003] Therefore, it is necessary to provide a Frida-based multi-threaded application dynamic instrumentation and coverage collection method to solve the above problems. Summary of the invention
[0004] The present invention provides a Frida-based multi-thread application dynamic plugging and coverage collection method, which can perform efficient and dynamic analysis and testing on multi-thread application programs without leaking source code.
[0005] The embodiment of the present invention provides a Frida-based multi-threaded application dynamic instrumentation and coverage collection method, including:
[0006] Obtaining a target program and startup parameters and environment variables when the target program is started;
[0007] Initialize the Frida module, Interceptor interceptor, Observer observer, log stream module and Hook entry function pointer;
[0008] Create a global Hash table, and when a new thread is created, add the new thread ID number to the global Hash table;
[0009] Create a first thread to start the target program, and add the entry address of the first thread to the Frida module;
[0010] Monitor in real time whether the target program starts a subprocess or a subthread, and when the target program starts a subprocess or a subthread, obtain the ID number and the thread group number of the subprocess or the subthread, and add the ID number to the global hash table and the queue;
[0011] The subprocess or subthread of the ID number newly added in the queue is binary plugged, and when the basic block of the subprocess or subthread is executed, the Frida module synchronously executes the stub function and counts the coverage information.
[0012] Preferably, the initializing the Frida module includes removing the own code address from the Frida module to prevent the coverage error caused by tracking the own code.
[0013] Preferably, the Interceptor is used to intercept the function being executed by the target program.
[0014] Preferably, a pipeline is used to flow the coverage information from each sub-process or sub-thread to the main collector, and the coverage information is aggregated into a global coverage array.
[0015] Preferably, the real-time monitoring of whether the target program starts a subprocess or a subthread includes:
[0016] Ptrace is used to monitor whether the target program calls fork, vfork, or pthread_create. When the target program calls fork, vfork, or pthread_create, Ptrace returns the ID number and thread group number of the child process or child thread.
[0017] Preferably, the real-time monitoring of whether the target program starts a subprocess or a subthread includes:
[0018] Monitor whether the target program has created a child process or child thread. When the target program starts, read the ID numbers and thread group numbers of all processes or threads, loop the ID numbers of all processes or threads, and determine whether their thread group numbers are the same as the thread group number of the target program. If they are the same, it means that they are created by the target program. The Frida module saves the thread IDs with the same thread group number in the global hash table.
[0019] Preferably, whenever a new process or thread is generated, it is determined whether it is created by the target program. If it is created by the target program, the Frida module adds the thread ID number of the target program to the global Hash variable, and the Frida module adds the ID number of the process or thread to the queue.
[0020] Preferably, when starting the target program, it is determined whether the entry function address is set. If the entry function address is not set, a new thread is started and the target program is started in an exec manner. If the entry function address is set, it is determined whether the entry function address is available. If the entry function address is available, the entry function address is converted into a function pointer, and relevant parameters are passed into the function pointer to start the target program. If the entry function address is not available, the main function is used to start the target program.
[0021] Preferably, it is determined whether the ID number is a process or a thread. If it is a process, the starting code address of the process is obtained, and the starting code address and the ID number are added to the Frida module together. If it is a thread, the ID number is directly added to the Frida module.
[0022] Preferably, the Frida module obtains the initial code address and the end code address of its basic block and then runs the stub function, the stub function performs an XOR operation on the end code address and the previous code address of the end code address to obtain a unique coverage ID, and performs a plus 1 operation on the coverage array through the unique coverage ID index.
[0023] Compared with the prior art, the technical solution of the embodiment of the present invention has the following beneficial effects:
[0024] The Frida-based multi-threaded application dynamic stub and coverage collection method of the embodiment of the present invention includes: obtaining a target program and startup parameters and environment variables when the target program is started; initializing a Frida module, an Interceptor interceptor, an Observer observer, a log stream module and a Hook entry function pointer; creating a global Hash table, and when a new thread is created, adding the new thread ID number to the global Hash table; creating a first thread to start the target program, and adding the entry address of the first thread to the Frida module; real-time monitoring of whether the target program starts a subprocess or a subthread, when the target program starts a subprocess or a subthread, obtaining the ID number and thread group number of the subprocess or subthread, and adding the ID number to the global Hash table and a queue; performing binary stub on the subprocess or subthread of the ID number newly added in the queue, and when the basic block of the subprocess or subthread is executed, the Frida module synchronously executes the stub function and counts the coverage information, so that the multi-threaded application can be efficiently and dynamically analyzed and tested without leaking the source code;
[0025] Further, Ptrace is used to monitor whether the target program calls fork, vfork, or pthread_create. When the target program calls fork, vfork, or pthread_create, Ptrace returns the ID number and thread group number of the child process or child thread, or monitors whether the target program creates a child process or child thread. When the target program starts, the ID numbers and thread group numbers of all processes or threads are read, and the ID numbers of all processes or threads are looped to determine whether their thread group numbers are the same as the thread group number of the target program. If they are the same, it means that they are created by the target program. The Frida module saves the thread ID number with the same thread group number in the global hash table, thereby ensuring that the process or thread created by the target program will be intercepted by the Frida module.
[0026] Furthermore, the Frida module obtains the initial code address and the end code address of its basic block and then runs the stub function, the stub function performs an XOR operation on the end code address and the previous code address of the end code address to obtain a unique coverage ID, and performs a plus 1 operation on the coverage array through the unique coverage ID index, and the coverage information of each thread uses a thread-safe coverage array to store the coverage information, thereby avoiding race conditions when multiple threads access and modify shared data. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are some embodiments of the present invention, not all embodiments. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.
[0028] Figure 1 A schematic flow chart of a Frida-based multi-threaded application dynamic instrumentation and coverage collection method provided in accordance with an embodiment of the present invention;
[0029] Figure 2 A schematic flow chart of a Frida-based multi-threaded application dynamic instrumentation and coverage collection method provided in accordance with another embodiment of the present invention;
[0030] Figure 3 A schematic flow chart of a Frida-based multi-threaded application dynamic instrumentation and coverage collection method provided in accordance with another embodiment of the present invention. DETAILED DESCRIPTION
[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0032] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0033] Based on the problems existing in the prior art, an embodiment of the present invention provides a Frida-based multi-threaded application dynamic instrumentation and coverage collection method, which can perform efficient and dynamic analysis and testing on multi-threaded applications without leaking source code.
[0034] Figure 1 A flowchart of a Frida-based multi-threaded application dynamic instrumentation and coverage collection method provided by an embodiment of the present invention. Figure 1 The embodiment of the present invention provides a Frida-based multi-threaded application dynamic instrumentation and coverage collection method, including:
[0035] Step S101: Acquire a target program and startup parameters and environment variables when the target program is started;
[0036] Step S102: Initialize the Frida module, Interceptor, Observer, log stream module and Hook entry function pointer;
[0037] Step S103: Create a global Hash table, and when a new thread is created, add the new thread ID number to the global Hash table;
[0038] Step S104: creating a first thread to start the target program, and adding the entry address of the first thread to the Frida module;
[0039] Step S105: monitoring in real time whether the target program starts a subprocess or a subthread, and when the target program starts a subprocess or a subthread, obtaining the ID number and the thread group number of the subprocess or the subthread, and adding the ID number to the global hash table and the queue;
[0040] Step S106: binary stub is performed on the child process or child thread of the ID number newly added in the queue. When the basic block of the child process or child thread is executed, the Frida module synchronously executes the stub function and counts the coverage information.
[0041] Specifically, in step S101, the Frida module runs in advance before the target program is started through the LD_PRELOAD environment variable preloading mechanism to intercept the actual program. Before the actual program is started, the Frida module will perform an initialization step.
[0042] In step S105, the multithreaded tracker is used to handle the plugging operation of multithreading and multiprocess, and mainly uses two methods to track threads and processes. Ptrace is used to monitor whether the target program calls fork, vfork, and pthread_create. When the target program calls fork, vfork, and pthread_create, Ptrace returns the ID number and thread group number of the child process or child thread, or monitors whether the target program creates a child process or child thread. When the target program starts, the ID number and thread group number of all processes or threads are read, and the ID number of all processes or threads is looped to determine whether its thread group number is the same as the thread group number of the target program. If the same, it means that it is created by the target program. The Frida module saves the thread ID number with the same thread group number in the global hash table, thereby ensuring that the process or thread created by the target program will be intercepted by the Frida module.
[0043] In step S106, the thread stub (GumStalker) stubs the ID numbers collected in the multi-thread tracker. First, a GumStalker instance is created, and the Observer observer is set to process events related to the thread. When the thread crashes, the Observer observer is recycled. Then, the thread stub GumStalker instance is used to perform process analysis on the thread ID number, and the relevant stub code is executed after each basic block is executed.
[0044] In a specific implementation, the initialization of the Frida module includes removing the own code address from the Frida module to prevent the tracking of the own code from causing false coverage.
[0045] In a specific implementation, the Interceptor is used to intercept the function being executed by the target program.
[0046] In a specific implementation, a pipeline is used to flow the coverage information from each sub-process or sub-thread to the main collector, and the coverage information is aggregated into a global coverage array. The main thread is used to sum up the coverage information of all sub-threads so that the fuzz testing tool can obtain relevant coverage information.
[0047] In a specific implementation, whenever a new process or thread is generated, it is determined whether it is created by the target program. If it is created by the target program, the Frida module adds the thread ID number of the target program to the global variable, and the Frida module adds the ID number of the process or thread to the queue.
[0048] In a specific implementation, when starting the target program, it is determined whether the entry function address is set. If the entry function address is not set, a new thread is started and the target program is started in an exec manner. If the entry function address is set, it is determined whether the entry function address is available. If the entry function address is available, the entry function address is converted into a function pointer, and relevant parameters are passed into the function pointer to start the target program. If the entry function address is not available, the main function is used to start the target program.
[0049] In a specific implementation, it is determined whether the ID number is a process or a thread. If it is a process, the starting code address of the process is obtained, and the starting code address and the ID number are added to the Frida module together. If it is a thread, the ID number is directly added to the Frida module.
[0050] In a specific implementation, the Frida module obtains the initial code address and the end code address of its basic block and then runs a pile function, and the pile function performs an XOR operation on the end code address and the previous code address of the end code address to obtain a unique coverage ID, and performs a plus 1 operation on the coverage array through the unique coverage ID index. The coverage information of each thread uses a thread-safe coverage array to store coverage information to avoid race conditions when multi-threaded access and modification of shared data.
[0051] After the stub function is finished running, the coverage array is passed to the multi-threaded coverage collector through the pipeline, and the multi-threaded coverage collector will put the coverage information into the total coverage array.
[0052] Figure 2 A flowchart of a Frida-based multi-threaded application dynamic instrumentation and coverage collection method is provided for another embodiment of the present invention. Figure 2 The embodiment of the present invention provides a Frida-based multi-threaded application dynamic instrumentation and coverage collection method, including:
[0053] Step S201: Acquire a target program and startup parameters and environment variables when the target program is started;
[0054] Step S202: Initialize the Frida module, Interceptor, Observer, log stream module and Hook entry function pointer;
[0055] Step S203: Create a global Hash table, and when a new thread is created, add the new thread ID number to the global Hash table;
[0056] Step S204: creating a first thread to start the target program, and adding the entry address of the first thread to the Frida module;
[0057] Step S205: monitoring in real time whether the target program starts a subprocess or a subthread, and when the target program starts a subprocess or a subthread, obtaining the ID number and the thread group number of the subprocess or the subthread, and adding the ID number to the global hash table and the queue;
[0058] Step S206: using Ptrace to monitor whether the target program calls fork, vfork, or pthread_create. When the target program calls fork, vfork, or pthread_create, Ptrace returns the ID number and thread group number of the child process or child thread.
[0059] Step S207: binary stub is performed on the child process or child thread of the ID number newly added in the queue. When the basic block of the child process or child thread is executed, the Frida module synchronously executes the stub function and counts the coverage information.
[0060] Figure 3 A flowchart of a Frida-based multi-threaded application dynamic instrumentation and coverage collection method is provided for another embodiment of the present invention. Figure 3 The embodiment of the present invention provides a Frida-based multi-threaded application dynamic instrumentation and coverage collection method, including:
[0061] Step S301: Acquire a target program and startup parameters and environment variables when the target program is started;
[0062] Step S302: Initialize the Frida module, Interceptor, Observer, log stream module and Hook entry function pointer;
[0063] Step S303: Create a global Hash table. When a new thread is created, add the new thread ID number to the global Hash table.
[0064] Step S304: Create a first thread to start the target program, and add the entry address of the first thread to the Frida module;
[0065] Step S305: monitoring in real time whether the target program starts a subprocess or a subthread, and when the target program starts a subprocess or a subthread, obtaining the ID number and the thread group number of the subprocess or the subthread, and adding the ID number to the global hash table and the queue;
[0066] Step S306: monitoring whether the target program has created a child process or child thread. When the target program is started, reading the ID numbers and thread group numbers of all processes or threads, looping through the ID numbers of all processes or threads, and determining whether the thread group number is the same as the thread group number of the target program. If they are the same, it means that the thread is created by the target program. The Frida module saves the thread ID number with the same thread group number in the global hash table.
[0067] Step S307: binary stub is performed on the child process or child thread of the ID number newly added in the queue. When the basic block of the child process or child thread is executed, the Frida module synchronously executes the stub function and counts the coverage information.
[0068] In summary, the Frida-based multi-threaded application dynamic plugging and coverage collection method of the embodiment of the present invention includes: obtaining a target program and startup parameters and environment variables when the target program is started; initializing the Frida module, Interceptor interceptor, Observer observer, log stream module and Hook entry function pointer; creating a global Hash table, and when a new thread is created, adding the new thread ID number to the global Hash table; creating a first thread to start the target program, and adding the entry address of the first thread to the Frida module; real-time monitoring of whether the target program starts a subprocess or subthread, when the target program starts a subprocess or subthread, obtaining the ID number and thread group number of the subprocess or subthread, and adding the ID number to the global Hash table and queue; binary plugging of the subprocess or subthread of the ID number newly added in the queue, when the basic block of the subprocess or subthread is executed, the Frida module synchronously executes the stub function, counts the coverage information, and can perform efficient and dynamic analysis and testing on multi-threaded applications without leaking the source code;
[0069] Further, Ptrace is used to monitor whether the target program calls fork, vfork, or pthread_create. When the target program calls fork, vfork, or pthread_create, Ptrace returns the ID number and thread group number of the child process or child thread, or monitors whether the target program creates a child process or child thread. When the target program starts, the ID numbers and thread group numbers of all processes or threads are read, and the ID numbers of all processes or threads are looped to determine whether their thread group numbers are the same as the thread group number of the target program. If they are the same, it means that they are created by the target program. The Frida module saves the thread ID number with the same thread group number in the global hash table, thereby ensuring that the process or thread created by the target program will be intercepted by the Frida module.
[0070] Furthermore, the Frida module obtains the initial code address and the end code address of its basic block and then runs the stub function, the stub function performs an XOR operation on the end code address and the previous code address of the end code address to obtain a unique coverage ID, and performs a plus 1 operation on the coverage array through the unique coverage ID index, and the coverage information of each thread uses a thread-safe coverage array to store the coverage information, thereby avoiding race conditions when multiple threads access and modify shared data.
[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A Frida-based multi-threaded application dynamic instrumentation and coverage collection method, characterized in that: include: Obtaining a target program and startup parameters and environment variables when the target program is started; Initialize the Frida module, Interceptor interceptor, Observer observer, log stream module and Hook entry function pointer; Create a global Hash table, and when a new thread is created, add the new thread ID number to the global Hash table; Create a first thread to start the target program, and add the entry address of the first thread to the Frida module; Monitor in real time whether the target program starts a subprocess or a subthread, and when the target program starts a subprocess or a subthread, obtain the ID number and the thread group number of the subprocess or the subthread, and add the ID number to the global hash table and the queue; Perform binary stub on the subprocess or subthread of the ID number newly added in the queue. When the basic block of the subprocess or subthread is executed, the Frida module synchronously executes the stub function and counts the coverage information. The real-time monitoring of whether the target program starts a subprocess or a subthread includes: Monitor whether the target program has created a child process or child thread. When the target program starts, read the ID numbers and thread group numbers of all processes or threads, loop the ID numbers of all processes or threads, and determine whether their thread group numbers are the same as the thread group number of the target program. If they are the same, it means that they are created by the target program. The Frida module saves the thread ID numbers with the same thread group number in the global hash table.
2. The Frida-based multi-threaded application dynamic instrumentation and coverage collection method according to claim 1, characterized in that: The initialization of the Frida module includes removing the own code address from the Frida module to prevent the coverage error caused by tracking the own code.
3. The Frida-based multi-threaded application dynamic instrumentation and coverage collection method according to claim 1, characterized in that: The Interceptor is used to intercept the function being executed by the target program.
4. The Frida-based multi-threaded application dynamic instrumentation and coverage collection method according to claim 1, characterized in that: The coverage information is flowed from each sub-process or sub-thread to the main collector using a pipeline, and the coverage information is aggregated into a global coverage array.
5. The Frida-based multi-threaded application dynamic instrumentation and coverage collection method according to claim 1, characterized in that: The real-time monitoring of whether the target program starts a subprocess or a subthread includes: Ptrace is used to monitor whether the target program calls fork, vfork, or pthread_create. When the target program calls fork, vfork, or pthread_create, Ptrace returns the ID number and thread group number of the child process or child thread.
6. The Frida-based multi-threaded application dynamic instrumentation and coverage collection method according to claim 1, characterized in that: Whenever a new process or thread is generated, it is determined whether it is created by the target program. If it is created by the target program, the Frida module adds the thread ID number of the target program to the global variable, and the Frida module adds the ID number of the process or thread to the queue.
7. The Frida-based multi-threaded application dynamic instrumentation and coverage collection method according to claim 1, characterized in that: When starting the target program, determine whether the entry function address is set. If the entry function address is not set, start a new thread and use exec to start the target program. If the entry function address is set, determine whether the entry function address is available. If the entry function address is available, convert the entry function address into a function pointer, pass relevant parameters into the function pointer to start the target program. If the entry function address is not available, use the main function to start the target program.
8. The Frida-based multi-threaded application dynamic instrumentation and coverage collection method according to claim 1, characterized in that: Determine whether the ID number is a process or a thread. If it is a process, obtain the starting code address of the process, add the starting code address and the ID number together to the Frida module. If it is a thread, directly add the ID number to the Frida module.
9. The Frida-based multi-threaded application dynamic instrumentation and coverage collection method according to claim 8, characterized in that: The Frida module obtains the initial code address and the end code address of its basic block and runs the stub function. The stub function performs an XOR operation on the end code address and the previous code address of the end code address to obtain a unique coverage ID, and performs a plus 1 operation on the coverage array through the unique coverage ID index.
