A source code classification method based on neural network

Abstract

This invention provides a source code classification method based on neural networks. The method includes: converting the source code to be classified into an Abstract Syntax Tree (AST); defining and initializing the vector matrix corresponding to the leaf nodes of the entire AST, denoted as randomly selecting n paths {p1,…,p...} from the AST. n}; For any extracted path p i It can be converted into a binary tuple. <x s ,x e >; Query the current path p from value_vocab respectively i The vectors of the starting and ending nodes are used to obtain the current path p. i vector c i , denoted as: c i As input to a neuron in a fully connected neural network, n corresponding output paths {p1,…,p} are obtained. n The corresponding n vectors {c1,…,c} n After inputting into a fully connected neural network with n neurons, n outputs are obtained. All outputs are aggregated into a code embedding v. Define a category label matrix, denoted as the predicted distribution of code embedding v obtained by performing operations on the code embedding v with the embeddings of each training function label in tags_vocab.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of code analysis, and particularly relates to a source code classification method based on a neural network. BACKGROUND

[0002] There are many application scenarios for analyzing and researching codes, such as code plagiarism detection, software vulnerability search, and the like. The existing code analysis methods mainly include the following three kinds:

[0003] The first code analysis method: since there is a natural similarity between program languages and natural languages, both are composed of some words and can be represented by a syntax tree, therefore some scholars borrow some techniques of natural languages to analyze codes. For example, codes are represented as a sequence and applied in vulnerability detection, code cloning and code classification tasks. Although there is some commonality with natural languages, codes themselves also have some unique properties, such as self-defined identifiers, stronger structural features, and the like. It is inevitable that a large amount of information will be lost by only using the processing method of natural languages to process codes.

[0004] The second code analysis method: in order to establish an algorithm model more suitable for code analysis, some scholars use their own expert field knowledge to formulate a series of rules to analyze and research codes. However, the traditional method of relying too much on expert field knowledge faces the following problems: (1) the model performs better when the data set is small, but when the data set is too large, the formulation of rules becomes more complex and it is difficult to adapt to massive and complex code data sets. (2) the extracted features are limited in number and not necessarily suitable for the current task by relying on the expert knowledge of researchers to extract features. (3) the rules developed with a lot of time and effort are usually only for the current task and have poor transferability.

[0005] The third code analysis method: In order to reduce the reliance on expert domain knowledge, many scholars have devoted themselves to applying deep learning to code analysis research in recent years. These models mostly use the control flow graph or abstract syntax tree (AST) of the code to represent the code, and extract information from the code by analyzing these intermediate representations. Among them, Alon et al. proposed a method called code2vec. The analysis process of this method is as follows: First, the code is converted into the corresponding AST, and then the AST is represented as a set of paths. Each path in the set of paths can be represented by a triple. The embedding of the corresponding code can be generated by a neural network and an attention mechanism. Compared with other algorithms (such as CNN and LSTM), this method has the advantages of simple structure and fast training speed. However, there are still two problems: (1) For the intermediate path representation, the algorithm only performs a hash calculation on each letter of the path to use as the identifier of the current path. This will result in two basically identical paths having completely different identifiers, which not only loses key information, but also greatly increases the time cost of training the model. (2) When aggregating paths, an attention parameter needs to be trained, which increases the complexity of the model. Summary of the Invention

[0006] To reduce reliance on expert domain knowledge and simultaneously decrease the complexity and training time of code classification, this invention provides a source code classification method based on neural networks.

[0007] The source code classification method based on neural networks provided by this invention includes:

[0008] Step 1: Convert the source code to be categorized into an AST;

[0009] Step 2: Define and initialize the vector matrix corresponding to the entire AST, denoted as Where X is the number of all terminal nodes in the AST; d is the dimension of the vector corresponding to the terminal node;

[0010] Step 3: Randomly select n paths {p1, ..., p2} from the AST. n}; For any extracted path p i Convert it into a binary tuple <x s x e >;where, x s For the current path p i The value corresponding to the starting node, x e For the current path p i The value corresponding to the upper end node;

[0011] Step 4: Query the current path p from value_vocab. iThe vector of the start node and the end node is denoted as c i The vector c i is denoted as: Wherein, value_vocab s represents the vector of the start node, and value_vocab e represents the vector of the end node.

[0012] Step 5: input the n vectors corresponding to the n extracted paths into the full connection neural network containing n neurons respectively to obtain n corresponding outputs All the outputs are aggregated into a code embedding v.

[0013] Step 6: define a category label matrix, denoted as Wherein, Y is the number of all training function labels in the training set.

[0014] Step 7: the prediction distribution of the code embedding v is obtained after the code embedding v is operated with the embedding of each training function label in tags_vocab.

[0015] Further, in step 5, the vector c i The output of the full connection layer is denoted as: Wherein, W represents a learning weight matrix,

[0016] Further, in step 5, all the outputs are aggregated into a code embedding v, and the specific process is as follows: N represents the number of paths.

[0017] Further, when training the full connection neural network, a cross-entropy loss function is used as a loss function.

[0018] The beneficial effects of the present application are as follows:

[0019] The idea of the present application is to represent the code segment as a code vector, and then predict the function of the code function according to the code function body. The whole process does not need any expert field knowledge to participate in the training of the model, only needs to train the model by inputting the preprocessed code data.

[0020] Moreover, the present application proposes a more simple binary tuple representation method to represent the path, avoids using a hash algorithm to generate an identifier for the AST path, reduces the parameters required for model training. Moreover, the proposed neural network model does not need to train an additional attention mechanism, so that the model structure is more simple. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1A flowchart illustrating the source code classification method based on neural networks provided in this embodiment of the invention;

[0022] Figure 2 This is a schematic diagram of the structure of a fully connected neural network model provided in an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] like Figure 1 As shown, this embodiment of the invention provides a source code classification method based on neural networks, including the following steps:

[0025] S101: Convert the source code to be classified into AST;

[0026] S102: Define and initialize the vector matrix corresponding to the entire AST, denoted as Where X is the number of all terminal nodes in the AST; d is the dimension of the vector corresponding to the terminal node;

[0027] Specifically, each element in `value_vocab` represents a vector corresponding to a terminal node. `||` represents a set. During training, there may be multiple training sets. For example, if there are three training sets with terminal nodes x1, x2, and x3 respectively, then `|X|` represents {x1, x2, x3}. A terminal node is a node without child nodes, also called a leaf node. When extracting paths, each path will have two terminal nodes: a start point and an end point. These two points have no child nodes.

[0028] S103: Randomly select n paths {p1,…,p from the AST. n}; For any extracted path p i Convert it into a binary tuple <x s x e >;where, x s For the current path p i The value corresponding to the starting node, x e For the current path p i The value corresponding to the upper end node;

[0029] S104: Query the current path p from value_vocab. iThe vector of the start node and the end node of the path p i The vector c i is denoted as: where value_vocab s represents the vector of the start node, and value_vocab e represents the vector of the end node.

[0030] S105: input the n vectors corresponding to the n extracted paths into a fully connected neural network containing n neurons respectively to obtain n outputs corresponding thereto Aggregate all the outputs into a code embedding v.

[0031] Specifically, for the vector c i , the output of the fully connected layer is denoted as: where W represents a learning weight matrix, Aggregate all the outputs into a code embedding v, specifically: n represents the number of paths.

[0032] S106: define a category label matrix, denoted as where Y is the number of all training function labels in the training set, and here, d is also the dimension of the embedding corresponding to the training function label.

[0033] Specifically, each row element of tags_vocab represents an embedding of a training function label. tag i represents the i-th row element in tags_vocab.

[0034] S107: obtain the prediction distribution of the code embedding v after performing operation on the code embedding v and the embedding of each training function label in tags_vocab, that is:

[0035] Specifically, the structure diagram of the fully connected neural network model is shown in Figure 2 . When training the fully connected neural network model, a cross-entropy loss function is used as the loss function, which is in the form as follows: the expression of the cross-entropy loss function is as follows: where q is the prediction distribution of the sample, p is the true distribution of the sample, and y true represents the true label of the sample. That is, the loss is the negative logarithm of q(y true ), and the closer q(y true ) is to 1, the closer the loss is to 0. Therefore, minimizing this loss is equivalent to maximizing the possibility of the model assigning y true to the real label.

[0036] In the embodiment of the present application, a simpler binary tuple representation method is proposed to represent the path, avoiding using a hash algorithm to generate an identifier for the AST path, thereby reducing the parameters required for model training. Moreover, the proposed neural network model does not need to additionally train an attention mechanism, so that the model structure is simpler.

[0037] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A neural network-based source code classification method, characterized by, Comprising: Step 1: convert the source code to be classified into AST; Step 2: Define and initialize the vector matrix corresponding to the whole AST, denoted as where X is the number of all terminal nodes in the AST; d is the dimension of the vector corresponding to the terminal node. Step 3: Randomly sample n paths {p1,..., pn} from AST; for any path pi sampled from AST, convert it into a pair <x n , x i > where x s is the value of the start node on the path pi and x e is the value of the end node on the path pi s . i e i ​​​ Step 4: Query the current path p from value_vocab respectively i The vector of the start node and the end node, and then the vector of the current path p i The vector c of the current path p is recorded as: i Wherein, value_vocab s represents the vector of the start node, and value_vocab e represents the vector of the end node;​ Step 5: input the n vectors corresponding to the n extracted paths into a fully connected neural network containing n neurons respectively to obtain n corresponding outputs All outputs are aggregated into a code embedding v, and all outputs are aggregated into a code embedding v, specifically: n represents the number of paths; Step 6: Define the class label matrix, denoted as Y where Y is the number of all training function labels in the training set. Step 7: obtain the prediction distribution of the code embedding v by operating the code embedding v with the embedding of each training function label in tags_vocab.

2. The neural network-based source code classification method of claim 1, wherein, In step 5, the vector c is computed for each of the input vectors x i the output of the fully connected layer where W denotes a matrix of learned weights, 3. The neural network-based source code classification method of claim 1, wherein, In training the fully connected neural network, a cross-entropy loss function is used as the loss function.

Images