A method for determining the optimal neural network input vector length

Abstract

The present invention discloses a method for determining the optimal neural network input vector length, belonging to the field of software testing technology. The method comprises the following steps: step 1, sorting the number of symbols of all code gadgets from small to large; step 2, approximating the number of symbols of all code gadgets to obtain an approximate value; step 3, using the idea of ​​approximate derivative to obtain the approximate slope of each data point; step 4, according to the approximate slope, the change trend of the five points before and after is obtained to obtain the average slope; step 5, determining the optimal input vector length of the neural network. Aiming at the characteristics of software source code vulnerability detection based on neural network, the present invention uses deep learning technology to perform vulnerability detection based on source code, provides a method for determining the optimal neural network input vector length, and effectively determines the optimal vector length of the data set for training the neural network model.

Description

Technical Field

[0001] The invention relates to software testing technology, and in particular to a method for determining an optimal neural network input vector length. Background Art

[0002] With the development of software development technology, more and more commercial software vendors are purchasing and using open source software, including some security-critical software such as medical, banking and other industry software. In the past 30 years, research has mainly focused on static testing based on source code, trying to discover vulnerabilities in program development and reuse. Vulnerabilities in source code are an issue that cannot be ignored in the current open source software development, especially the vulnerability propagation problem caused by the reuse of open source software. Due to the wide dissemination of source code, open source software vulnerabilities can be spread between different systems, which brings difficulties and challenges to software quality and software security. Software vulnerability detection methods based on deep learning can effectively solve this problem.

[0003] Deep learning-based methods do not require experts to manually define features and can automatically generate vulnerability patterns. Related research in vulnerability detection is just getting started. Lin et al. used deep learning models to detect vulnerabilities at the function level for cross-project situations. Xu et al. used a neural network method to perform binary vulnerability detection based on code similarity at the function level. Rajpal et al. used a neural network to learn patterns from past fuzzy detection input files to guide future fuzzy detection in the process of fuzz testing to find vulnerabilities. Russell et al. developed a large-scale function-level vulnerability detection system for C / C++ open source software code, learned deep feature representations for source code after lexical parsing, and built an open source software dataset using the results of three different static analysis detection tools. Harer et al. used machine learning methods for data-driven vulnerability detection for C / C++ programs, built an open source function dataset based on the results of a static analyzer, and compared the effects of applying to source code and compiled code. However, the current vulnerability detection methods based on deep learning have many shortcomings, which are mainly manifested in the following four aspects: 1) It is impossible to accurately locate various types of vulnerabilities. The vulnerability detection granularity of current work is basically at the function level, which is too coarse to locate the specific location of the vulnerability; 2) There is a lack of large-scale annotated datasets covering various types of vulnerabilities; 3) The deep learning models used in existing work for vulnerability detection are limited, and it is not clear which deep learning model is more suitable for detecting vulnerabilities, or which type of vulnerabilities is more suitable for detecting; 4) Existing work can only detect whether there is a vulnerability, and cannot provide more comprehensive vulnerability information. Summary of the invention

[0004] The purpose of the present invention is to determine the optimal neural network input vector length. Since deep learning requires input of a fixed-length vector form, and different program codes are converted into vector representations of different lengths, the present invention proposes a method for determining the optimal neural network input vector length to solve the above problem.

[0005] The technical solution adopted by the present invention is: a method for determining the optimal neural network input vector length, comprising the following steps:

[0006] Step 1 sorts the number of symbols of all code gadgets from small to large;

[0007] Step 2 approximates the number of symbols of all code gadgets to obtain an approximate value;

[0008] Step 3 uses the idea of ​​approximate derivative to obtain the approximate slope of each data point;

[0009] Step 4: Calculate the change trend of five adjacent data points according to the approximate slope to obtain the average slope;

[0010] Step 5 determines the optimal input vector length of the neural network.

[0011] Further, the step 1 specifically includes:

[0012] Processing each code gadget after the program code conversion, calculating the number of symbols corresponding to each code gadget and converting the corresponding symbols into vectors, and performing interception or filling operations according to the length of the vector;

[0013] Use a sorting algorithm to sort the trend of the number of symbols from small to large, and draw a trend change graph after sorting.

[0014] Further, the step 2 specifically includes:

[0015] The number of symbols corresponding to each code gadget is different. The number of symbols is approximated and an area where the trend of the number of symbols changes from slow to fast is found;

[0016] The average number of symbols corresponding to every hundred data points is taken as the number of symbols corresponding to the new data subset; where the data point refers to each code gadget;

[0017] A bar graph of the number of symbols corresponding to the approximated code gadgets is drawn, where the horizontal axis refers to each code gadget, and the vertical axis refers to the number of symbols corresponding to each code gadget.

[0018] Further, the step 3 specifically includes:

[0019] Use the idea of ​​approximate derivative to find the approximate derivative, which is the change in y value divided by the change in x value, where y is the change in the number of symbols and x is the subscript of each data point;

[0020] Calculate the approximate derivative for each data point in turn and plot the results in a graph.

[0021] Further, the step 4 specifically includes:

[0022] Calculate the average slope of 5 adjacent data points and simply fit the changes in data; add up the average slopes to calculate the corresponding change trend. A positive change trend is a gradual increase in the slope change, and a negative change trend is a gradual decrease in the slope change. The change trends of the five adjacent data points can offset each other.

[0023] The curve changes of the average slope of the fit are plotted in the graph.

[0024] Further, the step 5 specifically includes:

[0025] Calculate the data points where the average slope change trend is greater than 2;

[0026] Count the average slopes of 10 adjacent data points, find the positions where there are more than two overall change trends greater than 2, determine the first occurrence position, and use the number of symbols corresponding to this position as the optimal input vector length of the neural network.

[0027] The beneficial effects of the present invention are:

[0028] 1. Sort the symbols corresponding to the code gadgets after the program code conversion and approximate them. This method can approximate the number of symbols, and taking the average value can reduce the impact of errors.

[0029] 2. The idea of ​​approximate derivative and average slope can effectively find the position where the trend of the number of symbols changes from slow to fast. Calculate the point where the average slope (i.e., the trend) of the five adjacent points is greater than two, and use the corresponding number of symbols as the optimal vector length for the data set.

[0030] 3. A method for determining the optimal neural network input vector length. First, sort the number of symbols of the code gadget corresponding to the program code; then take the average of the sorted vector lengths and use it as the approximate result; use the idea of ​​approximate derivatives to calculate the change trend corresponding to each data point; calculate the average slope of five adjacent points, and find the first point within ten points that has two change trends greater than two as the optimal number of symbols determined by the method proposed by the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1It is a line graph of the number of symbols corresponding to the code gadget after the sorting program code is converted;

[0032] Figure 2 It is a bar chart that approximates the number of symbols corresponding to the code gadget after the program code is converted;

[0033] Figure 3 It is a line chart of the results of calculating the approximate derivative for each approximated data point;

[0034] Figure 4 It is a line graph that calculates the average slope after using the approximate derivative results;

[0035] Figure 5 is a flow chart of the present invention;

[0036] Figure 6 It is a schematic diagram for comparing experimental results of the optimal vector length determined by the method proposed in the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described below in conjunction with the accompanying drawings and embodiments. It should be noted that the described embodiments are only intended to facilitate the understanding of the present invention and do not have any limiting effect on the present invention.

[0038] The purpose of the present invention is to provide a method for determining the optimal neural network input vector length based on the characteristics of software source code vulnerability detection based on neural network, effectively determine the optimal vector length of the data set, and conduct sufficient experiments to prove the feasibility and effectiveness of the method.

[0039] First, several concepts involved in the examples of the present invention are defined as follows:

[0040] Definition 1: The number of symbols corresponding to the code gadget after the sorting program code is converted, such as Figure 1 As shown;

[0041] Definition 2: Approximate the number of symbols corresponding to the code gadget after the program code conversion. The number of symbols corresponding to each data point is different, and in order to remove the influence of outliers, approximate it. Therefore, we take the average number of symbols corresponding to every hundred data points and take it as the result of the new subset, such as Figure 2 As shown;

[0042]

[0043] Where y_mean[i] is the average number of symbols, y is the original number of symbols, j is the subscript of the original data, and i is the subscript after approximation;

[0044] Definition 3 calculates the trend of the number of symbols corresponding to the code gadget after each program code conversion. We initially tried to fit all data points with a high-order function, and find the point where the derivative of the quadratic equation is zero, and use the corresponding number of symbols as the result. However, the fitting degree of the high-order function is greatly affected by the degree of the function, and it is also impossible to determine the optimal vector length due to the influence of abnormal points. Therefore, we calculate the approximate derivative of each data point as the trend of the data point, such as Figure 3 As shown;

[0045]

[0046] Among them, y_slope[i] is the approximate derivative, x i+1 is the next position of the current position, x i is the current position, y[x i+1 ] is the number of symbols corresponding to the next position of the current position, y[x i ] is the number of symbols corresponding to the current position;

[0047] Definition 4 calculates the average slope of five adjacent points, and then adds them up to calculate the corresponding trend. The trend may be positive or negative. A positive trend is a gradual increase in the slope change, and a negative trend is a gradual decrease in the slope change. The change trends of five adjacent data can offset each other, that is, the positive trend and the negative trend offset each other. Therefore, we take the number of symbols corresponding to the data points with a cumulative change trend greater than two as the optimal vector length. However, due to the existence of outliers, this data point may not be the final result. In order to eliminate this influence, we observe whether there is another data point with a change slope greater than two after this data point, such as Figure 4 As shown;

[0048]

[0049] Among them, y_trend[i] is the average trend, x i+k is the kth position after the current position, x i+k-1 is the k-1th position after the current position, x i+4 is the fourth position after the current position, y[x i+k ] is the number of symbols corresponding to the kth position after the current position, y[x i+k-1 ] is the number of symbols corresponding to the k-1th position after the current position, y[x i+4 ] is the number of symbols corresponding to the 4th position after the current position;

[0050] like Figure 5 As shown, the method proposed by the present invention determines the optimal input vector length of the neural network, including:

[0051] Step 201 sorts the symbol quantities of all code gadgets from small to large.

[0052] The present invention uses deep learning technology to detect vulnerabilities based on source code to detect whether there are vulnerabilities in the program code. Deep learning technology mainly uses neural networks to automatically learn vulnerability rules and uses trained neural network models to detect potential vulnerabilities. However, neural networks need to input vector representations of fixed lengths, which is also the problem solved by the present invention, that is, finding an optimal vector length for training neural network models.

[0053] Step 202 approximates the number of symbols of all code gadgets to obtain an approximate value.

[0054] Since different code gadgets correspond to different numbers of symbols and are subject to the influence of the data set, we take the approximate value of one hundred data as the result of the new subset and use it as the approximated result. Approximating the vector length can avoid the influence of outliers, because the outliers are offset by the average result, and when making approximate derivatives and average slopes in the next step, it can also roughly reflect the trend of changes and reduce the number of calculations.

[0055] Step 203 uses the idea of ​​approximate derivative to obtain the approximate slope of each data point;

[0056] Step 2031 fits the high-order function to calculate the data point where the second-order derivative is zero. The first-order derivative of the function is the trend of the slope. Our idea is to find a position where the slope changes from slow to fast, and use the corresponding number of symbols as the final result. To this end, we need to differentiate the first-order derivative, obtain the trend of the first-order derivative, and then find the position where the trend changes from slow to fast. Although we sort the number of symbols of all code gadgets in step 201, the trend of the first-order derivative corresponding to each code gadget is not necessarily positive. That is to say, the growth of the number of symbols is fast and slow, which is also positive and negative in the second-order derivative. Therefore, we try to use the point where the second-order derivative is zero as the final position, and use the corresponding number of symbols as the optimal vector length. However, this method is affected by the fitting function, that is, the degree of function fitting is greatly restricted to the result, and the fitting function cannot fully show the trend of the number of symbols. To this end, we use the idea of ​​approximate derivatives to calculate the changes of each data point.

[0057] In step 2032, we simply subtract the number of symbols of the next point from the number of symbols of the current point, and use the result as the change trend of the point to obtain the change of its approximate derivative. And plot its change trend. Among them, since the number of symbols is sorted in step 201, the approximation in step 202 will not change it. Therefore, in this step, the change of the approximate derivative is greater than or equal to zero, that is, it is either a positive change or no change.

[0058] Step 204 calculates the change trend of the five points before and after according to the approximate slope to obtain an average slope;

[0059] We calculated the change trend corresponding to each data point in step 2032, and the results are all positive or equal to zero. To this end, we need to find the position where the change trend changes from slow to fast.

[0060] Step 2041 In this step, we calculate the change trend of the five adjacent data points, some of which are positive and some are negative. The positive one is that the change of the approximate derivative is positive, and the negative one is that the value of the approximate derivative of the point is less than the next point, so the change is negative. The change trend is slow and fast, but most of the abnormal points have been excluded due to the reasons of steps 201 and 202, and most of the remaining change trends are between 0 and 2, except for the area where the change trend is very obvious later. Therefore, in this step, if half of the change trends of the five adjacent data are calculated to be greater than or equal to 1, and half of the trends are less than or equal to 1, that is, the positive trend and the negative trend cancel each other out, that is, these trends cancel each other out internally and do not need to be considered. The trend of the remaining data points is positive. Therefore, the overall change trend of the area is positive. Therefore, we take the number of symbols corresponding to this position as the final result.

[0061] Step 2042 In step 2041, we calculated the position where the adjacent change trend is positive, and then took its first point, but this point may be just an abnormal point, just because the change trend obtained at this position is positive, and then the overall change trend is not positive. For this reason, we consider calculating the average slope of each data point in step 2042. Then count the average slopes of ten similar data points respectively, find the position where there are more than two overall change trends greater than two, and select the first appearing position as the final result.

[0062] Step 205 determines the optimal input vector length of the neural network by the method proposed by the present invention.

[0063] This step summarizes step 201 to step 204. First, the number of symbols in the data set is sorted, and then its average value is calculated. The approximate derivative of each data point after approximation is obtained, and the position where the change trend changes from slow to fast is found through the average slope. The corresponding number of symbols is used as the optimal vector length of the data set.

[0064] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0065] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A method for determining an optimal neural network input vector length, characterized in that The following steps are involved: Step 1: Sort the number of symbols of all code gadgets from small to large; Step 2: approximate the number of symbols of all code gadgets and obtain an approximate value; Step 3: Use the idea of ​​approximate derivative to obtain the approximate slope of each data point; Step 4: Calculate the change trend of five adjacent data points according to the approximate slope to obtain the average slope; Step 5: Determine the optimal input vector length of the neural network; The step 1 specifically includes: Processing each code gadget after the program code conversion, calculating the number of symbols corresponding to each code gadget and converting the corresponding symbols into vectors, and performing interception or filling operations according to the length of the vector; Use a sorting algorithm to sort the trend of the number of symbols from small to large, and draw a trend change graph after sorting; The step 2 specifically includes: The number of symbols corresponding to each code gadget is different. The number of symbols is approximated and an area where the trend of the number of symbols changes from slow to fast is found; The average number of symbols corresponding to every hundred data points is taken as the number of symbols corresponding to the new data subset; where the data point refers to each code gadget; Draw a bar graph of the number of symbols corresponding to the approximated code gadgets, where the horizontal axis refers to each code gadget and the vertical axis refers to the number of symbols corresponding to each code gadget; The step 5 specifically includes: Calculate the data points where the average slope change trend is greater than 2; Count the average slopes of 10 adjacent data points, find the positions where there are more than two overall change trends greater than 2, determine the first occurrence, and use the number of symbols corresponding to the first occurrence of the overall change trend greater than 2 as the optimal input vector length of the neural network.

2. The method for determining an optimal neural network input vector length according to claim 1, characterized in that: The step 3 specifically includes: Use the idea of ​​approximate derivative to find the approximate derivative, which is the change in y value divided by the change in x value, where y is the change in the number of symbols and x is the subscript of each data point; Calculate the approximate derivative for each data point in turn and plot the results in a graph.

3. The method for determining an optimal neural network input vector length as claimed in claim 1, characterized in that: The step 4 specifically includes: Calculate the average slope of 5 adjacent data points and simply fit the changes in the data; add up the average slopes to calculate the corresponding change trend. A positive change trend is a gradual increase in the slope change, and a negative change trend is a gradual decrease in the slope change. The change trends of the five adjacent data points cancel each other out. The curve changes of the average slope of the fit are plotted in the graph.

Images