Code classification and clustering method based on convolution and recurrent neural network

Abstract

The invention discloses a code classification and clustering method based on convolution and a recurrent neural network. An experimental data set comprises 104 types of codes. The specific steps of code classification are as follows: generating AST of codes by using srcml,generating a word vector of the node by using Word2vec according to the expanded sequence of the AST, cutting the AST into a subtree sequence composed of a cycle and a condition AST, wherein each sub-tree uses a tree convolutional neural network TBCNN for encoding, inputting a vector obtained by encoding into a bidirectionalLSTM, putting a code vector generated in each time step into a matrix, compressing the matrix by using maximum pooling to obtain a vector, mapping the vector by using a layer of fully connected neuralnetwork to obtain a 104-dimensional code vector, generating a marker vector by using a onehot method, wherein cross entropy is used as a loss function, carrying out dimension reduction by using a TSNE method, and clustering on the codes by using a Kmeans clustering method. The classification accuracy can reach 97.8%, and the clustering accuracy can reach 83.6%.

Description

technical field [0001] The invention belongs to the field of software engineering, mainly relates to deep learning, program static analysis, code big data and other technologies, and can mainly realize automatic classification of marked codes and automatic clustering of unmarked codes. [0002] technical background [0003] Code classification and clustering is a long-term research hotspot in the field of software engineering, and this work can promote the development of multiple sub-fields, such as program understanding, concept location, code plagiarism detection, vulnerability classification, and malware detection. To a certain extent, the label of code classification is the expression of the internal semantics of the code, which can assist the subsequent code modularization and refactoring, and reduce the cost of software maintenance. [0004] Different from natural language, code language has a strong logical structure. It is difficult to understand the specific function...

AI Technical Summary

Problems solved by technology

At present, there are also many models designed for program AST. These models can efficiently extract the structural information of the code. However, when the size of the tree is very large, directly extracting features from the entire AST will inevitably make it difficult to extract some code features. In the final generated code vector

Images