A code defect detection method for customizing a memory management function

Abstract

The application provides a code defect detection method of self-defined memory management function, collects memory management function prototypes as positive samples, and collects non-memory management function prototypes as negative samples; uses a training sample data set to train a twin neural network model, and classifies a to-be-tested memory management function through the trained model; when the to-be-tested memory management function is classified as a memory allocation function or a memory release function, checks whether the to-be-tested memory management function is associated with a standard memory management function; captures feature information of the to-be-tested memory management function, records memory object structure information of the to-be-tested memory management function, and detects code defects through the memory object structure information. The application uses a trained twin neural network model to classify functions to determine whether the functions are self-defined memory management functions, captures feature information to record memory object structure information of the functions, and detects code defects through the memory object structure information, thereby greatly improving accuracy and making the application suitable for detection of large programs.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of program code security detection, and particularly relates to a code defect detection method of self-defined memory management function. BACKGROUND

[0002] Software security problems and security attack events caused by code defects occur frequently. Although developers invest a lot of effort to formulate various security programming specifications, code defects still exist widely. In reality, memory destruction problems in software developed by using C / C++ language, such as memory leakage, double release and use after release, are often the most dangerous defects. Memory destruction type code defects can be used by attackers to cause denial of service, information leakage, code execution or privilege escalation, and finally destroy the security of the software system.

[0003] In the past few decades, researchers have proposed many automatic detection technologies to scan memory destruction type defects in code. These technologies mainly fall into two categories: one is program analysis based code defect detection, and the other is data driven code defect detection.

[0004] Program analysis based code defect detection detects abnormal behaviors in code by reasoning about behaviors in code rather than actual execution. The idea is to parse the code, extract data flow / control flow information in the code, then abstractly represent variables in the code, use abstract symbols instead of accurate values of variables at runtime, and approximate estimate code defect behaviors according to symbol values on each feasible path of the code, and detect possible code defects according to the approximate results.

[0005] Data driven code defect detection treats code as text, learns the syntax representation, semantic representation or graph structure representation of code, and detects code defects using these representations. This type of detection method usually tokenizes code into many word sequences, and infers on these word sequences. Some methods will transform the code in each function into a graph form according to the control flow, and learn the structure representation of the entire graph through the code statements in each node to infer.

[0006] Although the above two types of technologies can find potential defects in the code to some extent, they both have inherent defects: the code defect detection method based on program analysis is difficult to apply to large programs. With the increase of the amount of code in the program, the path depth in the program is constantly increasing, while the number of feasible paths also expands exponentially. The huge time and resource overhead makes it impossible to analyze the entire program. The code defect detection based on data driving ignores various constraints in the code and lacks precise inference of the program running logic, which cannot give a deterministic defect detection result, resulting in a large number of false positives in the reported code fragments, although they seem to have code defects, but the path does not actually exist, and finally a large number of false positives are produced.

[0007] The inherent defects of the existing automated memory destruction type code defect detection technology are essentially that they use a simple memory management model, which believes that the memory objects in the code are only managed by the system standard memory management function API, such as malloc and free, and are always used in pairs, such as the execution of an allocation function always corresponds to the execution of a release function. However, in actual software code projects, developers will customize more complex memory management models, especially when using customized memory management functions for memory object management, which do not completely follow the above simple memory management model, resulting in that the existing detection technology cannot achieve satisfactory results in accuracy and scalability. SUMMARY

[0008] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a code defect detection method for custom memory management functions, which solves the problem of poor accuracy of code defect detection and difficulty in applying to large programs in the prior art.

[0009] To solve the above technical problems, the present application is implemented as follows: a code defect detection method for custom memory management functions, the method comprising the following steps:

[0010] Step 1, collect memory management function prototypes as positive samples, and collect non-memory management function prototypes as negative samples;

[0011] Step 2, construct a training sample data set for training a twin neural network model through the positive samples and the negative samples;

[0012] Step 3, train a twin neural network model using the training sample data set, and classify the to-be-tested memory management function through the trained model;

[0013] Step 4, when classified as a memory allocation function or a memory release function, check whether it has an association relationship with a standard memory management function; if there is an association relationship, it is determined as a custom memory management function;

[0014] Step 5, capturing the feature information of the memory management function to be tested, recording the memory object structure information, and performing code defect detection through the memory object structure information.

[0015] In order to further solve the technical problems to be solved by the present application, in the code defect detection method for customizing memory management functions provided by the present application, step 2 includes processing the function name and parameter name of the function prototype in the positive sample and the negative sample by word segmentation, and normalizing the return type and parameter type of the function prototype according to the pointer type.

[0016] In order to further solve the technical problems to be solved by the present application, in the code defect detection method for customizing memory management functions provided by the present application, in step 3, training the twin neural network model using the training sample data set includes the following steps:

[0017] Step 3.1, using a convolutional neural network as two branches;

[0018] Step 3.2, mapping the function prototype in the positive sample and the negative sample into a digital vector;

[0019] Step 3.3, calculating the average value of the digital vector corresponding to the function prototype in the positive sample and the negative sample to obtain a reference vector;

[0020] Step 3.4, classifying the to-be-classified sample by cosine similarity with the reference vector;

[0021] Step 3.5, inputting two different to-be-classified samples into the upper and lower branches each time, and feeding back the classification results for training.

[0022] In order to further solve the technical problems to be solved by the present application, in the code defect detection method for customizing memory management functions provided by the present application, the reference vector includes a memory allocation function reference vector, a memory release function reference vector, and a non-memory function reference vector.

[0023] In order to further solve the technical problems to be solved by the present application, in the code defect detection method for customizing memory management functions provided by the present application, step 4 includes generating a candidate function classified as a memory allocation function or a memory release function after classification, and performing data flow analysis on the internal implementation of the candidate function; if the sub-branch of the candidate function calls at least one standard memory management function, and the memory object allocated or released by the standard memory management function is returned to the parameter or return value of the candidate function through the sub-branch, then the candidate function is determined as a custom memory management function.

[0024] In order to further solve the technical problems to be solved by the present application, the code defect detection method for self-defined memory management function provided by the present application, wherein the feature information comprises function name, function attribute, memory object and object type.

[0025] In order to further solve the technical problems to be solved by the present application, the code defect detection method for self-defined memory management function provided by the present application, wherein the feature information comprises function name, function attribute, memory object and object type.

[0026] In order to further solve the technical problems to be solved by the present application, the code defect detection method for self-defined memory management function provided by the present application, wherein the feature information comprises function name, function attribute, memory object and object type.

[0027] In order to further solve the technical problems to be solved by the present application, the code defect detection method for self-defined memory management function provided by the present application, wherein the feature information comprises function name, function attribute, memory object and object type.

[0028] In order to further solve the technical problems to be solved by the present application, the code defect detection method for self-defined memory management function provided by the present application, wherein the feature information comprises function name, function attribute, memory object and object type.

[0029] As described above, the code defect detection method for self-defined memory management function of the present application, through ingenious design, uses the trained twin neural network model to classify the function to determine whether it is a self-defined memory management function, captures the feature information to record the memory object structure information, and detects the code defects through the memory object structure information, which greatly improves the accuracy and makes it suitable for large program detection. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 The flowchart shown is a method for detecting code defects in a custom memory management function according to an embodiment of the present invention.

[0031] Figure 2 The diagram shows a training schematic of a Siamese neural network model for a code defect detection method for a custom memory management function in an embodiment of the present invention.

[0032] Figure 3 The diagram shown illustrates the generation of a feature list for a code defect detection method for a custom memory management function according to an embodiment of the present invention. Detailed Implementation

[0033] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0034] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0035] Please see Figure 1 This invention provides a method for detecting code defects in custom memory management functions, the method comprising the following steps:

[0036] Step 1: Collect prototypes of memory management functions as positive samples and prototypes of non-memory management functions as negative samples. The collection process is done manually. For the positive samples, collect common memory management function interface prototypes from typical code projects; the number is unlimited, but approximately 100 is preferred. For the negative samples, collect common non-memory management function interface prototypes from typical code projects, such as encryption functions and logging functions; the number is also unlimited, but approximately 100 is preferred.

[0037] Step 2, the function prototype includes return type, function name, parameter type and parameter name, etc. First, the function name and parameter name of the function prototype in the positive sample and the negative sample are segmented, and the return type and parameter type of the function prototype are normalized according to the pointer type, such as int and char in C language. The training sample data set for training the twin neural network model is constructed by using the positive sample and the negative sample.

[0038] Step 3, the twin neural network model is trained using the training sample data set, and the to-be-tested memory management function is classified by the trained model.

[0039] As shown in Figure 2 , using the twin neural network training model, a specific Transformer Encoder is trained to map a function prototype into a numerical vector. Training the twin neural network model using the training sample data set includes the following steps:

[0040] Step 3.1, the convolutional neural network is used as two branches.

[0041] Step 3.2, the function prototype in the positive sample and the negative sample is mapped into a numerical vector, which is realized by the Embedding function of Keras.

[0042] Step 3.3, the numerical vectors corresponding to the function prototypes in the positive sample and the negative sample are averaged, specifically, the numerical vectors corresponding to the memory allocation function, the memory release function in the positive sample and all function prototypes in the negative sample are averaged to obtain a reference vector, which includes a memory allocation function reference vector, a memory release function reference vector and a non-memory function reference vector. For example, for the memory allocation function, if there are n groups of m-dimensional vectors W, the sum of each dimension W i is calculated, and then divided by n to obtain the average value of each dimension, and then an average vector is formed.

[0043] Step 3.4, the to-be-classified sample is classified by the cosine similarity with the reference vector, and the cosine similarity with the reference vector is calculated to classify whether it is a memory allocation function, a memory release function or a non-memory management function.

[0044] Step 3.5, two different samples to be classified are input into the upper and lower branches each time, and feedback training is performed according to the classification results. Step 4, when being classified as a memory allocation function or a memory release function, it is checked whether there is an association with a standard memory management function. If there is an association, it is determined to be a custom memory management function. After classification, candidate functions classified as memory allocation functions or memory release functions are generated, and data flow analysis of the internal implementation of the candidate functions is performed; if the sub-branch of the candidate function calls at least one standard memory management function, and the memory object allocated or released by the standard memory management function is returned to the parameter or return value of the candidate function through the sub-branch, the candidate function is determined to be a custom memory management function.

[0045] Step 5, when being determined as a custom memory management function, the characteristic information of the memory management function to be tested is captured, the memory object structure information is recorded, and code defect detection is performed through the memory object structure information. The characteristic information includes function name, function attribute, memory object and object type.

[0046] As shown in Figure 3 , a characteristic list of the custom memory management function is generated, including function name, function attribute, memory object and object type. The characteristic list generation process of the function dma_page is as follows:

[0047] 1) Identify the memory function (kmalloc, dma_alloc_coherent) and its properties (allocation or release).

[0048] 2) Perform data flow analysis on the identified memory function to determine the allocated or released memory object (page, page->vaddr).

[0049] 3) Determine the function parameters and return values affected by the allocated or released memory object.

[0050] 4) Generate a characteristic list including four parts: main function name, main attribute, memory object list, and object type.

[0051] When generating the characteristic list, data flow analysis is performed on the custom memory management function, and the memory object allocated or released by the standard memory management function is tracked; if the standard memory management function performs memory allocation operation, all data flows from the standard memory management function to the return value and parameter (data flow from the standard memory management function to the return value and parameter) are merged; if the standard memory management function performs memory release operation, all data flows from the function parameter to the standard memory management function (data flow from the function parameter to the standard memory management function) are merged, so that the complete structure information of the memory object allocated or released by the custom memory management function can be recorded completely.

[0052] In the detection, the entry of the code project to be detected (the main function of the code by default) is selected to start traversing the entire code, and in the traversal process, once a function is found to be a custom memory management function, code defect detection is performed by establishing or destroying an abstract memory object. If the memory management function to be tested is a custom memory allocation function, a symbolic expression for referring to the memory object is generated according to the dynamically managed memory object, and is added to the current memory object list and passed to subsequent analysis; if the memory management function to be tested is a custom memory release function, the symbolic expression specified by the input parameter is updated to the release state, and is removed from the current memory object list and destroyed; if a symbolic expression marked as the release state is updated to the release state again, a double release defect is detected; and if a symbolic expression marked as the release state is accessed by subsequent code reading and writing, a release-after-use defect is detected.

[0053] The present application has been tested on Linux Kernel, FreeBSD Kernel, OpenSSL, Redis and other C / C++ code libraries of different sizes (code size ranging from 100,000 lines to 300 million lines). The results show that more than 10,000 custom memory management functions can be identified, the entire analysis process (including memory management function identification and memory destruction code defect detection) of any project can be completed within 10 hours using a dual Intel Xeon CPU and 256G memory, greatly improving the analysis efficiency (compared with traditional analysis tools that require more than 100 hours of running time and cannot complete complete analysis of large projects), and more than 80 previously unreported code defects have been found, verifying the protection effect of the present application.

[0054] In summary, the code defect detection method for custom memory management functions of the present application uses a trained twin neural network model to classify functions to determine whether they are custom memory management functions, captures feature information to record memory object structure information, and performs code defect detection based on the memory object structure information, greatly improving the accuracy and making it suitable for large program detection. Therefore, the present application effectively overcomes the various shortcomings of the prior art and achieves better practical effects.

[0055] The above embodiments only exemplarily illustrate the principles and effects of the present application, and are not intended to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and technical idea disclosed by the present application should be covered by the claims of the present application.

Claims

1. A method for detecting code defects in custom memory management functions, characterized in that, The method comprises the following steps: Step 1, collect memory management function prototypes as positive samples, and collect non-memory management function prototypes as negative samples; Step 2, construct a training sample data set for training a twin neural network model through the positive samples and the negative samples; Step 3, train the twin neural network model using the training sample data set, and classify the to-be-tested memory management function through the trained model; Step 4, when being classified as a memory allocation function or a memory release function, check whether there is an association relationship with a standard memory management function; if there is an association relationship, it is determined as a custom memory management function; Step 5, capture feature information of the to-be-tested memory management function, record memory object structure information, and perform code defect detection through the memory object structure information; In step 3, training the twin neural network model using the training sample data set comprises the following steps: Step 3.1, use a convolutional neural network as two branches; Step 3.2, map the function prototypes in the positive samples and the negative samples into digital vectors; Step 3.3, calculate the average value of the digital vectors corresponding to the function prototypes in the positive samples and the negative samples to obtain a reference vector; Step 3.4, classify the to-be-classified samples through the cosine similarity with the reference vector; Step 3.5, select two different to-be-classified samples each time and input them into the upper and lower branches, and perform feedback training according to the classification results. Step 5 comprises code defect detection by establishing or destroying an abstract memory object, if the to-be-tested memory management function is a custom memory allocation function, generating a symbolic expression for referring to the memory object according to the dynamically managed memory object, and adding it to the current memory object list and passing it to subsequent analysis; if the to-be-tested memory management function is a custom memory release function, update its state to release according to the symbolic expression specified by the input parameter, and remove it from the current memory object list and destroy it.

2. The method for detecting code defects of self-defined memory management function according to claim 1, characterized in that: Step 2 comprises word segmentation processing of the function names and parameter names of the function prototypes in the positive samples and the negative samples, and normalizing the return types and parameter types of the function prototypes according to the pointer types.

3. The method for detecting code defects of a self-defined memory management function according to claim 1, characterized in that: The reference vector comprises a memory allocation function reference vector, a memory release function reference vector, and a non-memory function reference vector.

4. The method of claim 1, wherein the self-defined memory management function is a function of the operating system. Step 4 comprises generating a candidate function classified as a memory allocation function or a memory release function after classification, performing data flow analysis on the internal implementation of the candidate function; if at least one standard memory management function is called by a sub-branch of the candidate function, and the memory object allocated or released by the standard memory management function is returned to the parameter or return value of the candidate function through the sub-branch, the candidate function is determined as a custom memory management function.

5. The method of claim 1, wherein the self-defined memory management function is a function of the operating system. The feature information comprises function name, function attribute, memory object, and object type.

6. The method of claim 5, wherein the self-defined memory management function is a function of the operating system. A feature list of the custom memory management function is generated, which contains function name, function attribute, memory object, and object type. A feature list of the custom memory management function is generated, which contains function name, function attribute, memory object, and object type.

7. The method of claim 6, wherein the self-defined memory management function is a function of the operating system. When generating the feature list, the custom memory management function is subjected to data flow analysis, and memory objects allocated or released by the standard memory management function are tracked; if the standard memory management function performs a memory allocation operation, all data flows from the standard memory management function to return values and parameters are merged; if the standard memory management function performs a memory release operation, all data flows from the function parameters to the standard memory management function are merged.

8. The method of claim 1, wherein the self-defined memory management function is a function of the operating system. If a symbolic expression marked as a release state is updated to the release state again, a double release defect is detected; If a symbolic expression marked as a release state is accessed by read-write subsequent code, a use-after-free defect is detected.

Images