Method for dynamic analysis of memory safety of multithreaded programs
Patent Information
- Application Number
- CN202310934654.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-27
- Publication Date
- 2026-08-18
- Estimated Expiration
- 2043-07-27
AI Technical Summary
[0006]但是，现有的动态分析工具都是针对单线程程序而设计，由于多线程程序线程调度的不确定性，使得这些工具容易产生误报和漏报，无法准确检测多线程程序中的内存错误
[0040]本发明与现有技术相比，其显著效果如下：
Smart Images
Figure CN117009977B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer software, and more particularly to a dynamic analysis method for the memory safety of multithreaded programs. Background Technology
[0002] C's low-level control over memory layout, explicit memory management, and access to low-level hardware make it a primary programming language for implementing system and embedded software. However, this low-level control also frequently leads to memory errors. Recent CVE announcements indicate that memory errors remain among the most dangerous software bugs and can cause program crashes and security vulnerabilities.
[0003] OpenMP is a multithreaded program compilation and processing scheme for shared-memory parallel systems. OpenMP uses a fork-join execution model, initially with only one main thread. When parallel computation is needed, several branch threads are spawned to execute parallel tasks. After the parallel tasks are completed, the branch threads merge and hand over control to the main thread. However, contention for memory access in multithreaded programs leads to more memory errors.
[0004] Dynamic analysis is a lightweight and efficient verification method that can detect potential memory safety vulnerabilities during program execution. Existing dynamic analysis tools can insert code snippets for runtime monitoring into the program code through compile-time code instrumentation, and use pointer metadata to record the upper and lower bounds and state information of the memory objects pointed to by each pointer, thereby achieving automatic detection of memory errors at runtime.
[0005] These dynamic analysis tools primarily work by traversing the program's abstract syntax tree and inserting statements that implement pointer metadata updates and memory error detection into specific locations in the source code during object creation, object assignment, function calls, object release, and memory read / write operations. Finally, by running the instrumented program, the location of memory errors in the source code can be precisely located at runtime.
[0006] However, existing dynamic analysis tools are designed for single-threaded programs. Due to the uncertainty of thread scheduling in multi-threaded programs, these tools are prone to false alarms and false negatives, and cannot accurately detect memory errors in multi-threaded programs. Summary of the Invention
[0007] Purpose of the invention: The purpose of this invention is to provide a dynamic memory safety analysis method that can accurately and automatically detect memory errors in multithreaded programs and improve the program execution speed in dynamic analysis.
[0008] Technical solution: The present invention provides a dynamic analysis method for memory safety of multi-threaded programs, comprising the following steps:
[0009] S1. Manually create or automatically generate a new source code file, and insert the source code for type definitions and function definitions for dynamic analysis into the new source code file; the type definitions include type definitions for pointer metadata and type definitions for function metadata, and the source code for function definitions includes code for CAS operation functions and code for hash table operation functions; the code for hash table operation functions is used to store and maintain pointer metadata for different pointer variables and function metadata for different functions, while the hash table uses the partition key in the pointer metadata to sort the lock-free ordered linked list;
[0010] S2 uses the compiler to preprocess the source code of the multithreaded program, including macro substitution, header file import, and conditional compilation; then the compiler parses the source code of the multithreaded program, including lexical analysis, syntax analysis, and semantic analysis, to generate an abstract syntax tree.
[0011] S3 traverses all nodes in the abstract syntax tree, instrumenting the definitions and initialization statements of pointer variables and structure variables containing pointer variables to create and initialize pointer metadata at runtime; instrumenting the assignment statements of pointer variables and structure variables containing pointer variables to update pointer metadata at runtime; instrumenting function call statements to pass pointer metadata for parameters and return values at runtime; instrumenting pointer dereferencing, array subscript access, and structure member access statements to check for errors in these memory accesses at runtime based on pointer metadata; and instrumenting clauses in OpenMP multithreaded programs.
[0012] S4 compiles the instrumented project directory or files using a compiler to generate an executable file; by running the executable file, the program is dynamically analyzed, and the source code location of any discovered memory errors is identified.
[0013] Furthermore, in step S1, the structure type of the pointer metadata includes: the starting address, ending address, status node address, pointer variable address, partition key, pointer to the next pointer metadata, and thread ID where the pointer variable resides. The status node stores the valid status of the memory space pointed to by the pointer, and its values include invalid, function, global, static, heap, stack, and system function. The value of the partition key is related to the binary reversal of the pointer variable address.
[0014] The structure type of the function metadata includes: pointer metadata and number of pointer type variables in function parameters and return values, function address, partition key, pointer to the next function metadata, and thread ID of the function call;
[0015] The CAS operation function takes the form of CAS(V,O,N), where V represents the variable to be updated, O represents the expected old value, and N represents the new value.
[0016] The value of V will be set to N if and only if the value of V is equal to the value of 0; if the value of V is different from the value of 0, it means that other threads have updated the value of V, and the current thread will not perform any operation on the variable V.
[0017] Furthermore, the CAS operation function includes:
[0018] Atomic addition operation function: Adds the value to the value pointed to by the pointer ptr, and returns the original value in the memory pointed to by ptr;
[0019] Atomic subtraction operation function: Subtracts the value from the value in the memory pointed to by the pointer ptr, and returns the original value in the memory pointed to by ptr;
[0020] Atomic comparison and swap function: Compares the value in the memory pointed to by pointer ptr with the value of oldval. If they are equal, the value of newval is written to the memory pointed to by ptr and true is returned; if they are not equal, no writing is performed and false is returned.
[0021] Furthermore, in step S1, when operating on the pointer metadata and function metadata in the hash table, each thread defines and uses private variables prev, cur, and next to access the lock-free ordered linked list in the hash table, and uses CAS atomic operations to ensure that only one thread inserts, modifies, or deletes pointer metadata at any given time. At the same time, each thread's private variables are stored in a dangerous array, and are removed from the dangerous array only after the thread's access operation is completed. When each thread releases memory, it needs to check whether the node to be released exists in the dangerous array. If it exists, it cannot be released directly; if it does not exist in the dangerous array, it means that the node has not been accessed by other threads, and the memory is released directly.
[0022] The hash table operation functions include:
[0023] A) Inserting pointer metadata: Based on the pointer metadata pmd to be inserted, search the hash table to determine if there exists pointer metadata with the same pointer variable address and delimiter key as the pointer metadata pmd to be inserted.
[0024] If it exists, the variable cur points to the pointer metadata, and the variable prev points to the address member of the next node of the previous node of the pointer metadata in the linked list. Therefore, the value of the next node address in the pointer metadata pmd to be inserted is set to the value of the next node address in the pointer metadata pointed to by cur. Then, the value of the variable pointed to by prev is set to the address of the pointer metadata pmd to be inserted. The pointer metadata pointed to by cur is deleted. Finally, the value of the variable cur is set to the address of the pointer metadata pmd to be inserted.
[0025] If it does not exist, the value of the partition key of the pointer metadata pointed to by the variable cur is the smallest value in the linked list that is greater than the partition key value of the pointer metadata pmd to be inserted. Therefore, the value of the next node address in the pointer metadata pmd to be inserted is set to cur, and then the value of the variable pointed to by the variable prev is set to the address of the pointer metadata pmd to be inserted through a CAS operation. If the CAS operation fails, the insertion operation is re-executed.
[0026] B) Modify pointer metadata: Based on the new pointer metadata pmd, search the hash table to determine if there exists pointer metadata with the same pointer variable address and delimiter key as the new pointer metadata pmd.
[0027] If it exists, the variable cur points to the pointer metadata, so the value of the member variable of the pointer metadata is modified to the value of the member variable in the new pointer metadata pmd through a CAS operation;
[0028] If it does not exist, no modifications will be made;
[0029] C) Delete pointer metadata: Based on the given pointer variable address and partition key, search the hash table to determine if pointer metadata exists that is equal to the given pointer variable address and partition key.
[0030] If it does not exist, then no pointer metadata needs to be deleted;
[0031] If it exists, the variable `cur` points to the pointer metadata, and the variable `prev` points to the address of the next node of the pointer metadata in the linked list. Therefore, a CAS operation is used to set the least significant bit of the binary representation of the next node address of the pointer metadata to 1, indicating that the node is marked as deleted. If the operation fails, the deletion operation is re-executed. If the operation succeeds, the value of the variable pointed to by `prev` is set to the address of the next node of the pointer metadata pointed to by `cur`, and the pointer metadata pointed to by `cur` is deleted. If the value of the variable pointed to by `prev` is not equal to the value of the variable `cur`, it means that another thread has modified the node, so the deletion operation is re-executed.
[0032] Furthermore, in step S1, the value of the segmentation key is obtained by converting the value of the pointer variable address. Specifically, there are two cases: when the pointer metadata is a bucket element, the hash value of the pointer variable address is calculated first, and then the binary is reversed to obtain the value of the segmentation key.
[0033] When the pointer metadata is an item element, first calculate the hash value of the pointer variable address, then perform binary reversal and set the least significant bit to 1 to obtain the value of the split key.
[0034] Furthermore, in step S3, instrumenting the clauses of the OpenMP multithreaded program includes:
[0035] S31. For each pointer variable specified in the private clause, insert the pointer metadata variable name of the pointer variable in the private clause, insert the assignment statement of the pointer metadata in the omp block, and modify the value of the pointer variable address member in the pointer metadata.
[0036] For each pointer variable specified in the firstprivate clause, insert the pointer metadata variable name of that pointer variable in the firstprivate clause, and insert the assignment statement of the pointer metadata in the omp block. Only the value of the pointer variable address member in the pointer metadata needs to be modified.
[0037] For each pointer variable specified in the lastprivate clause, insert the pointer metadata variable name of that pointer variable in the lastprivate clause, insert an assignment statement for that pointer metadata in the omp block, and modify the value of the pointer variable address member in that pointer metadata;
[0038] S32, For each pointer variable specified in the shared clause, insert the pointer metadata variable name of that pointer variable into the shared clause;
[0039] S33. For each function called in each omp block, if the parameters or return value contain pointers, then insert a corresponding wrapper function to pass pointer metadata. This wrapper function creates different function metadata for different threads based on the thread ID, distinguishes them using the thread ID member in the function metadata, and verifies both the hash value and the thread ID when looking up the hash table.
[0040] Compared with the prior art, the significant advantages of this invention are as follows:
[0041] 1. This invention inserts pointer metadata variable names of pointer variables into the clauses of OpenMP multithreaded programs, enabling each thread to create its own independent copy of the pointer metadata. Assignment statements of the pointer metadata are then inserted into the omp block, thereby instrumenting the OpenMP multithreaded program and performing dynamic memory safety analysis, thus improving the memory safety of multithreaded programs.
[0042] 2. This invention uses a hash table to store pointer metadata, employs a lock-free ordered linked list to handle hash collisions of pointer metadata, and uses atomic operations to access pointer metadata without the need for locking operations, thereby improving the program execution speed in dynamic analysis.
[0043] 3. This invention passes pointer metadata by inserting a corresponding wrapper function for each function called in each OMP block. The wrapper function creates different function metadata for different threads based on the thread ID, avoiding contention for accessing and erroneously modifying function metadata, and improving the accuracy of detecting memory errors in multithreaded programs. Attached Figure Description
[0044] Figure 1 This is a flowchart of the present invention;
[0045] Figure 2 This is a diagram of the data structure used in this invention. Detailed Implementation
[0046] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0047] To address the weaknesses of existing dynamic analysis tools in detecting memory errors in multithreaded programs, this invention utilizes pointer metadata-based dynamic analysis techniques, combined with multi-core multithreading technology, lock-free data structure technology, and source code instrumentation to achieve dynamic analysis of memory safety in multithreaded programs. Because multithreaded programs have more complex language structures and their syntax differs significantly from single-threaded programs, and they also suffer from data race conditions, this invention primarily handles the relevant language structures of multithreaded programs specially to avoid contentious access to pointer metadata. The main problems addressed by this invention are as follows:
[0048] (1) Handling of OpenMP special language structures
[0049] OpenMP statements contain different clauses that handle data differently. For example, the `private` clause causes each thread to copy the variables in the clause, so each thread also needs to copy the pointer metadata. The `firstprivate` clause causes each thread to copy the variables in the clause and assign them the values from before the parallel block, so each thread also needs to copy the pointer metadata from before the parallel block and ensure that the values of its member variables are passed. When the `default` clause is written as `default(none)`, all variables in the parallel block need to be declared as private or public, so the pointer metadata used in the parallel block also needs to be declared as private or public.
[0050] (2) Contentious access to metadata
[0051] Because dynamic analysis requires inserting, modifying, or deleting pointer metadata at runtime, it can lead to data contention among multiple threads for pointer metadata. Furthermore, the state node records the type of memory it points to and the number of pointers pointing to that memory. Since multiple pointer metadata entries may share a single state node, concurrent access to the state node and operations such as adding or subtracting pointers can also cause data contention. Therefore, it is necessary to isolate multi-threaded access to pointer metadata.
[0052] (3) Handling of wrapper functions
[0053] Since dynamic analysis requires inserting wrapper functions for functions whose parameters or return values contain pointers to pass and update function metadata, and multiple threads in a multithreaded program may call the wrapper functions simultaneously, leading to data contention for function metadata, it is necessary to isolate multithreaded access to function metadata.
[0054] like Figure 1 As shown, the multi-threaded program memory safety dynamic analysis method of the present invention includes the following steps:
[0055] Step 1: Manually or automatically create a new source code file, and insert the source code for type definitions and function definitions used for dynamic analysis into the new source code file, including the following steps:
[0056] Step 11, insert the type definition of pointer metadata.
[0057] The pointer metadata structure type includes: the starting address and ending address of the memory space pointed to by the pointer variable, the address of the state node, the address of the pointer variable, the partition key, a pointer to the next pointer metadata, and the thread ID of the pointer variable. The state node stores the valid state of the memory space pointed to by the pointer, with values including invalid, function, global, static, heap, stack, and system function. The partition key is related to the binary reversal of the pointer variable's address. Due to the uncertainty of multi-threaded scheduling and potential data races, the data structure design needs to ensure the correctness of pointer metadata insertion, modification, and deletion by different threads.
[0058] In this embodiment, the code defining the structure type of the pointer metadata is as follows: where PRFstat_node is the status node type, PRFptr_addr is the pointer address type, and PRFso_key_t is the split key type:
[0059]
[0060] Step 12, insert the type definition of function metadata. The structure type of function metadata includes: pointer metadata and number of pointer type variables in function parameters and return values, function address, separator key, pointer to the next function metadata, and the thread ID of the function call.
[0061] In this embodiment, the code defining the structure type of the function metadata is as follows, where PRFfunc_addr is the function address type:
[0062]
[0063]
[0064] Step 13: Insert the code for the CAS (Compare And Swap) operation function. The CAS operation function has the form CAS(V,O,N), which includes three parameters: V represents the variable to be updated, O represents the expected old value, and N represents the new value.
[0065] The value of V will be set to N if and only if the value of V is equal to 0.
[0066] If the value of V is different from the value of O, it means that other threads have updated the value of V, so the current thread will no longer perform any operation on the variable V.
[0067] CAS operation functions include:
[0068] A1) Atomic addition operation function: Adds the value to the value pointed to by the pointer ptr, and returns the original value in the memory pointed to by ptr. For example, a declaration is as follows:
[0069] type__sync_fetch_and_add(type*ptr,type value);
[0070] A2) Atomic subtraction operation function: Subtracts the value from the value pointed to by the pointer ptr, and returns the original value in the memory pointed to by ptr. For example, a declaration is as follows:
[0071] type__sync_fetch_and_sub(type*ptr,type value);
[0072] A3) Atomic comparison and swap functions: Compare the value in the memory pointed to by pointer ptr with the value oldval. If they are equal, write the value newval to the memory pointed to by ptr and return true; if they are not equal, do not write, and return false. For example, one declaration is as follows:
[0073] bool__sync_bool_compare_and_swap(type*ptr,type oldval,type newval);
[0074] Step 14: Insert the code for hash table operation functions to store and maintain pointer metadata for different pointer variables and function metadata for different functions, while using a lock-free ordered linked list to resolve hash collisions.
[0075] like Figure 2 As shown, the data structure in this invention uses a hash table. The hash table utilizes the partition key in the pointer metadata to sort the lock-free ordered linked list, thus resolving hash collisions. Specifically, the partition key is obtained by converting the value of the pointer variable's address, specifically in two cases: when the pointer metadata is a bucket element, the hash value of the pointer variable's address is first calculated, and then the binary representation is reversed to obtain the partition key; when the pointer metadata is an item element, the hash value of the pointer variable's address is first calculated, then the binary representation is reversed, and the least significant bit is set to 1 to obtain the partition key. This operation distinguishes between bucket elements and item elements, while simultaneously sorting the nodes in the hash table.
[0076] Inserting bucket elements into a hash table and labeling them can achieve data separation, ensure the correctness of data access in the hash table by multiple threads, and avoid problems such as dirty reads, phantom reads, and duplicate deletions when searching, modifying, and deleting pointer metadata.
[0077] When operating on pointer metadata and function metadata in the hash table, each thread defines and uses private variables `prev`, `cur`, and `next` to access the lock-free ordered linked list in the hash table, employing CAS atomic operations to ensure that only one thread inserts, modifies, or deletes pointer metadata at any given time. Simultaneously, each thread's private variables are stored in a dangerous array, and are removed from the dangerous array only after the thread's access operation is complete. When releasing memory, each thread needs to check if the node to be released exists in the dangerous array. If it does, it cannot be released directly; if it does not exist in the dangerous array, it means that the node has not been accessed by other threads, and the memory can be released directly.
[0078] Hash table operation functions include:
[0079] B1) Insert pointer metadata: Search the hash table based on the pointer metadata pmd to be inserted, and determine whether there is pointer metadata that is equal to the pointer variable address and delimiter key in the pointer metadata pmd to be inserted.
[0080] If such pointer metadata exists, the variable `cur` points to that pointer metadata, and the variable `prev` points to the address of the next node of the previous node in the linked list of that pointer metadata. Therefore, the value of the next node address in the pointer metadata `pmd` to be inserted is set to the value of the next node address in the pointer metadata pointed to by `cur`. Then, the value of the variable pointed to by `prev` is set to the address of the pointer metadata `pmd` to be inserted. The pointer metadata pointed to by `cur` is deleted. Finally, the value of the variable `cur` is set to the address of the pointer metadata `pmd` to be inserted.
[0081] If such pointer metadata does not exist, the partition key of the pointer metadata pointed to by the variable `cur` is the smallest partition key in the linked list greater than the partition key of the pointer metadata `pmd` to be inserted. Therefore, the address of the next node in the pointer metadata `pmd` to be inserted is set to `cur`, and then the value of the variable pointed to by the variable `prev` is set to the address of the pointer metadata `pmd` to be inserted using a CAS operation. If the CAS operation fails, the insertion operation is re-executed.
[0082] In this embodiment, the code is as follows: Wherein, PRFpmd is a pointer metadata type, the ptra member of PRFpmd is the address of the pointer variable, the so_key member of PRFpmd is the delimiter key, the parameter head is the address of the hash table, the function Find is used to search for the pointer metadata to be inserted in the hash table, and the function PRFpmd_free is used to delete the pointer metadata.
[0083]
[0084]
[0085] B2) Modify pointer metadata: Search the hash table based on the new pointer metadata pmd to determine if there is pointer metadata that has the same pointer variable address and delimiter key as the new pointer metadata pmd.
[0086] If such pointer metadata exists, and the variable cur points to that pointer metadata, then the value of the member variable of that pointer metadata can be modified to the value of the member variable in the new pointer metadata pmd through a CAS operation.
[0087] If such pointer metadata does not exist, no modifications are made.
[0088] B3) Delete pointer metadata: Search the hash table based on the given pointer variable address and partition key to determine if there is pointer metadata that is equal to the given pointer variable address and partition key.
[0089] If such pointer metadata does not exist, then there is no need to delete any pointer metadata.
[0090] If such pointer metadata exists, the variable `cur` points to this pointer metadata, and the variable `prev` points to the address of the next node of the node preceding this pointer metadata in the linked list. Therefore, a CAS operation is used to set the least significant bit of the binary representation of the address of the next node of this pointer metadata to 1, indicating that the node is marked as deleted. If the operation fails, the deletion operation is re-executed. If the operation succeeds, the value of the variable pointed to by `prev` is set to the address of the next node of the pointer metadata pointed to by `cur`, and the pointer metadata pointed to by `cur` is deleted. If the value of the variable pointed to by `prev` is not equal to the value of the variable `cur`, it means that another thread has modified the node, so the deletion operation is re-executed.
[0091] In this embodiment, the code is as follows, where PRFpmd is a pointer metadata type, the ptra member of PRFpmd is the address of the pointer variable, the so_key member of PRFpmd is the delimiter key, the head parameter is the address of the hash table, the Find function is used to search for the pointer metadata to be deleted in the hash table, the PRFmark_ptr function sets the least significant bit of the variable to 1, and the PRFpmd_free function is used to delete the pointer metadata.
[0092]
[0093] Step 2: Use the compiler to preprocess the source code of the multithreaded program, including macro substitution, header file import, and conditional compilation. Then, use the compiler to parse the source code of the multithreaded program, including lexical analysis, syntax analysis, and semantic analysis, to generate an abstract syntax tree.
[0094] Step 3: Traverse all nodes in the abstract syntax tree, instrumenting the definitions and initialization statements of pointer variables and structure variables containing pointer variables to create and initialize pointer metadata at runtime; instrumenting the assignment statements of pointer variables and structure variables containing pointer variables to update pointer metadata at runtime; instrumenting function call statements to pass pointer metadata for parameters and return values at runtime; instrumenting pointer dereferencing, array subscript access, and structure member access statements to check for errors in these memory accesses at runtime based on pointer metadata; and instrumenting the clauses of OpenMP multithreaded programs as follows:
[0095] Step 31: For each pointer variable specified in the private, firstprivate, and lastprivate clauses, each thread will create an independent copy of that pointer variable. Therefore, insert the pointer metadata variable name of the pointer variable in the clause so that each thread creates its own independent copy of the pointer metadata. Insert the assignment statement of the pointer metadata in the omp block and modify the value of the pointer variable address member in the pointer metadata.
[0096] (1) For each pointer variable specified in the private clause, insert the pointer metadata variable name of the pointer variable in the private clause, insert the assignment statement of the pointer metadata in the omp block, and modify the value of the pointer variable address member in the pointer metadata.
[0097] In this embodiment, the function code to be instrumented is as follows:
[0098]
[0099]
[0100] The function code after instrumentation is as follows:
[0101]
[0102] Here, `PRFpmd` is the pointer metadata type, `PRFpmd_ch_94649638998352` is the pointer metadata variable for pointer `ch`, `PRFpmd_init_val` is the initial value of the pointer metadata, the function `PRFpmd_set_ret` is used to set the value of the pointer metadata variable, `PRFglobal_sa` is the state node of the global memory object, and `PRFinvalid` is the value indicating an invalid memory state. The `omp` block copies a pointer metadata variable with the same name for each thread, but does not perform initialization. Therefore, an assignment statement for this pointer metadata variable is inserted in the `omp` block. Since the address of the variable `ch` is different for each thread, the value of the `ptra` member of this pointer metadata variable is modified in the `omp` block to the address of the variable `ch` of the current thread.
[0103] (2) For each pointer variable specified in the firstprivate clause, insert the pointer metadata variable name of the pointer variable in the firstprivate clause, and insert the assignment statement of the pointer metadata in the omp block. Only the value of the pointer variable address member in the pointer metadata needs to be modified.
[0104] In this embodiment, the function code to be instrumented is as follows:
[0105]
[0106] The function code after instrumentation is as follows:
[0107]
[0108] The omp block copies a pointer metadata variable with the same name for each thread and performs initialization. Therefore, in the omp block, you only need to modify the value of the ptra member of the pointer metadata variable to the address of the variable ch of the current thread.
[0109] (3) For each pointer variable specified in the lastprivate clause, insert the pointer metadata variable name of the pointer variable in the lastprivate clause, insert the assignment statement of the pointer metadata in the omp block, and modify the value of the pointer variable address member in the pointer metadata.
[0110] In this embodiment, the function code to be instrumented is as follows:
[0111]
[0112]
[0113] The function code after instrumentation is as follows:
[0114]
[0115] Step 32: For each pointer variable specified in the shared clause, insert the pointer metadata variable name of that pointer variable into the shared clause.
[0116] In this embodiment, the function code to be instrumented is as follows:
[0117]
[0118] The function code after instrumentation is as follows:
[0119]
[0120] Step 33: For each function called within each OMP block, if the parameters or return value contain pointers, a corresponding wrapper function is inserted to pass the pointer metadata. This wrapper function creates different function metadata for different threads based on their thread IDs, distinguishing them using the thread ID member in the function metadata. When looking up the hash table, both the hash value and the thread ID are verified simultaneously, thus avoiding contention for access to function metadata and erroneous modification.
[0121] In this embodiment, the program includes the following functions: gridInfo and test, where the omp block calls the function gridInfo, which takes a pointer parameter:
[0122]
[0123] During instrumentation, a wrapper function `PRFgridInfo` is generated for the function `gridInfo`. This wrapper function searches the function metadata table and updates the function metadata based on the pointer metadata of the actual arguments. Since function calls are not thread-isolated, when creating function metadata, different function metadata for functions with the same name need to be created based on the thread ID to avoid data races when updating pointer metadata. The instrumented code is as follows:
[0124]
[0125] Specifically, the function PRFfmd_tbl_create creates function metadata containing four pointer metadata for the function gridInfo; the function PRFfmd_tbl_update_pmd updates the pointer metadata in the function metadata; the function PRFfmd_tbl_remove deletes the function metadata of the function gridInfo; the function PRFpmd_free_null_ptr deletes the pointer metadata; the function PRFpmd_create creates a pointer metadata; PRFstack_sa is the state node of the stack memory object; and PRFstack is the value representing the state of the stack memory.
[0126] Step 4: Compile the instrumented project directory or files using a compiler to generate an executable file. Run the executable file to perform dynamic analysis of the program and locate the source code of any discovered memory errors.
[0127] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0128] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application.
Claims
1. A method for dynamic analysis of memory safety in multithreaded programs, characterized in that, Includes the following steps: S1, manually or automatically create a new source code file, and insert the source code for type definitions and function definitions for dynamic analysis into the new source code file; The type definition includes the type definition of pointer metadata and the type definition of function metadata. The source code of the function definition includes the code of the CAS operation function and the code of the hash table operation function. The code of the hash table operation function is used to store and maintain pointer metadata of different pointer variables and function metadata of different functions. At the same time, the hash table uses the partition key in the pointer metadata to sort the lock-free ordered linked list. S2 uses the compiler to preprocess the source code of multithreaded programs, including macro substitution, header file import, and conditional compilation. Then, the compiler is used to parse the source code of the multithreaded program, including lexical analysis, syntax analysis, and semantic analysis, to generate an abstract syntax tree; S3 traverses all nodes in the abstract syntax tree and instrumentes the definitions and initialization statements of pointer variables and structure variables containing pointer variables, which is used to create pointer metadata and initialize pointer metadata at runtime. Instrumentation is performed on assignment statements of pointer variables and structure variables containing pointer variables to update pointer metadata at runtime; instrumentation is performed on function call statements to pass pointer metadata for parameters and return values at runtime; instrumentation is performed on pointer dereferencing, array subscript access, and structure member access statements to check for errors in these memory accesses at runtime based on pointer metadata; and instrumentation is performed on clauses of OpenMP multithreaded programs. S4 compiles the instrumented project directory or files using a compiler to generate an executable file; by running the executable file, the program is dynamically analyzed, and the source code location of any discovered memory errors is identified.
2. The method for dynamic analysis of memory safety in multi-threaded programs according to claim 1, characterized in that, In step S1, the structure type of the pointer metadata includes: the starting address, ending address, status node address, pointer variable address, partition key, pointer to the next pointer metadata, and thread ID where the pointer variable is located. The status node stores the valid status of the memory space pointed to by the pointer, and the values include invalid, function, global, static, heap, stack, and system function. The value of the partition key is related to the binary reversal of the pointer variable address. The structure type of the function metadata includes: pointer metadata and number of pointer type variables in function parameters and return values, function address, partition key, pointer to the next function metadata, and thread ID of the function call; The CAS operation function takes the form of CAS(V,O,N), where V represents the variable to be updated, O represents the expected old value, and N represents the new value. The value of V will be set to N only if the value of V is equal to the value of 0; if the value of V is different from the value of 0, it means that other threads have updated the value of V, and the current thread will not perform any operation on the variable V.
3. The method for dynamic analysis of memory safety in multi-threaded programs according to claim 2, characterized in that, The CAS operation functions include: Atomic addition operation function: Adds the value to the value pointed to by the pointer ptr, and returns the original value in the memory pointed to by ptr; Atomic subtraction operation function: Subtracts the value from the value in the memory pointed to by the pointer ptr, and returns the original value in the memory pointed to by ptr; Atomic comparison and swap function: Compares the value in the memory pointed to by pointer ptr with the value of oldval. If they are equal, the value of newval is written to the memory pointed to by ptr and true is returned; if they are not equal, no writing is performed and false is returned.
4. The method for dynamic analysis of memory safety in multi-threaded programs according to claim 1, characterized in that, In step S1, when operating on pointer metadata and function metadata in the hash table, each thread defines and uses private variables prev, cur, and next to access the lock-free ordered linked list in the hash table, and uses CAS atomic operations to ensure that only one thread inserts, modifies, or deletes pointer metadata at any given time. At the same time, each thread's private variables are stored in a dangerous array, and are removed from the dangerous array only after the thread's access operation is completed. When each thread releases memory, it needs to check whether the node to be released exists in the dangerous array. If it does, it cannot be released directly. If the node is not in the dangerous array, it means that the node has not been accessed by other threads, and the memory is released directly. The hash table operation functions include: A) Inserting pointer metadata: Based on the pointer metadata pmd to be inserted, search the hash table to determine if there exists pointer metadata with the same pointer variable address and delimiter key as the pointer metadata pmd to be inserted. If it exists, the variable cur points to the pointer metadata, and the variable prev points to the address member of the next node of the previous node of the pointer metadata in the linked list. Therefore, the value of the next node address in the pointer metadata pmd to be inserted is set to the value of the next node address in the pointer metadata pointed to by cur. Then, the value of the variable pointed to by prev is set to the address of the pointer metadata pmd to be inserted. The pointer metadata pointed to by cur is deleted. Finally, the value of the variable cur is set to the address of the pointer metadata pmd to be inserted. If it does not exist, the value of the partition key of the pointer metadata pointed to by the variable cur is the smallest value in the linked list that is greater than the partition key value of the pointer metadata pmd to be inserted. Therefore, the value of the next node address in the pointer metadata pmd to be inserted is set to cur, and then the value of the variable pointed to by the variable prev is set to the address of the pointer metadata pmd to be inserted through a CAS operation. If the CAS operation fails, the insertion operation is re-executed. B) Modify pointer metadata: Based on the new pointer metadata pmd, search the hash table to determine if there exists pointer metadata with the same pointer variable address and delimiter key as the new pointer metadata pmd. If it exists, the variable cur points to the pointer metadata, so the value of the member variable of the pointer metadata is modified to the value of the member variable in the new pointer metadata pmd through a CAS operation; If it does not exist, no modifications will be made; C) Delete pointer metadata: Based on the given pointer variable address and partition key, search the hash table to determine if pointer metadata exists that is equal to the given pointer variable address and partition key. If it does not exist, then no pointer metadata needs to be deleted; If it exists, the variable `cur` points to the pointer metadata, and the variable `prev` points to the address of the next node of the pointer metadata in the linked list. Therefore, a CAS operation is used to set the least significant bit of the address of the next node of the pointer metadata to 1, indicating that the node is marked as deleted. If the operation fails, the deletion operation is re-executed. If the operation succeeds, the value of the variable pointed to by `prev` is set to the address of the next node of the pointer metadata pointed to by `cur`, and the pointer metadata pointed to by `cur` is deleted. If the value of the variable pointed to by `prev` is not equal to the value of the variable `cur`, the deletion operation is re-executed.
5. The method for dynamic analysis of memory safety in multi-threaded programs according to claim 1, characterized in that, In step S1, the value of the partition key is obtained by converting the value of the pointer variable address. Specifically, there are two cases: when the pointer metadata is a bucket element, the hash value of the pointer variable address is calculated first, and then the binary is reversed to obtain the value of the partition key. When the pointer metadata is an item element, first calculate the hash value of the pointer variable address, then perform binary reversal and set the least significant bit to 1 to obtain the value of the split key.
6. The method for dynamic analysis of memory safety in multi-threaded programs according to claim 1, characterized in that, Step S3, instrumenting the clauses of the OpenMP multithreaded program includes: S31. For each pointer variable specified in the private clause, insert the pointer metadata variable name of the pointer variable in the private clause, insert the assignment statement of the pointer metadata in the omp block, and modify the value of the pointer variable address member in the pointer metadata. For each pointer variable specified in the firstprivate clause, insert the pointer metadata variable name of that pointer variable in the firstprivate clause, and insert the assignment statement of the pointer metadata in the omp block. Only the value of the pointer variable address member in the pointer metadata needs to be modified. For each pointer variable specified in the lastprivate clause, insert the pointer metadata variable name of that pointer variable in the lastprivate clause, insert an assignment statement for that pointer metadata in the omp block, and modify the value of the pointer variable address member in that pointer metadata; S32, For each pointer variable specified in the shared clause, insert the pointer metadata variable name of that pointer variable into the shared clause; S33. For each function called in each omp block, if the parameters or return value contain pointers, then insert a corresponding wrapper function to pass pointer metadata. This wrapper function creates different function metadata for different threads based on the thread ID, distinguishes them using the thread ID member in the function metadata, and verifies both the hash value and the thread ID when looking up the hash table.