Java vulnerability classification method based on natural language processing and deep forest
By processing Java code with an improved ASTNN encoding layer and a deep forest model, the problems of high false positive rate and poor generalization in code vulnerability mining are solved, achieving higher accuracy and recall, and is suitable for Java source code vulnerability classification.
Patent Information
- Application Number
- CN202211288161.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-20
- Publication Date
- 2026-02-10
- Estimated Expiration
- 2042-10-20
AI Technical Summary
Existing code vulnerability discovery methods suffer from high false positive rates and poor generalization, especially in static analysis. Traditional static detection methods have high false positive rates and lack generalization, while machine learning-based methods have low model accuracy due to insufficient code feature representation.
We employ a method based on natural language processing and deep forests. By improving the encoding layer processing of the abstract syntax tree, we use the ASTNN model to divide the abstract syntax tree into expression subtrees and perform multiple traversals. We combine the Word2Vec model for node encoding and use multi-granularity scanning and cascaded forests for training to generate a Java code vulnerability classification model.
The improved model enhances the accuracy and recall of the vulnerability classification model, reduces training time, and preserves the semantic information of the abstract syntax tree. The improved model demonstrates better generalization ability and classification performance on different datasets.
Smart Images
Figure CN115630368B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application discloses a Java source code vulnerability classification method, and particularly relates to a Java source code vulnerability classification method based on natural language processing and deep forest, and belongs to the technical field of source code vulnerability mining and classification. BACKGROUND
[0002] In recent years, with the expansion of software size and the continuous improvement of code complexity, the number of software vulnerabilities is also growing. According to the data released by the vulnerability knowledge base website CVE details, 20169 vulnerabilities were submitted to the Common Vulnerabilities & Exposures (CVE) in 2021, an increase of 1844 from 18325 in 2020 and a growing trend year by year. In addition to the increase in the number, the morphology of software vulnerabilities also presents complexity and diversity, which poses an increasing threat to the normal and safe operation of computer systems.
[0003] Traditional vulnerability mining methods mainly include static analysis and dynamic analysis. Dynamic analysis methods such as symbolic execution have problems such as path explosion, constraint solving difficulty, memory modeling and parallel processing complexity, so dynamic analysis methods are not suitable for projects with huge code size and batch vulnerability mining. The matching rules of the static analysis technology for the source code depend on the vulnerability patterns constructed by expert knowledge, and because they are hard-coded into the analysis tool, they do not have generalizability, so the static detection method usually has a high false positive rate.
[0004] Unlike traditional static analysis, the vulnerability mining method based on machine learning learns the information hidden in historical vulnerability data, mines the features in the vulnerability data and uses them for vulnerability mining problems, so the number and effectiveness of vulnerability sample code representation determine the final performance of the model. Because the code metrics and token sequence feature representation methods contain less source code syntax structure and semantic information, the model trained on their basis has a low accuracy. Therefore, some research attempts to use code intermediate representation containing code syntax, semantics and context information as features to train vulnerability mining models. Experiments have shown that machine learning models based on abstract syntax trees can better balance model efficiency and performance. Existing code representation methods based on abstract syntax trees can be divided into path information representation and node information representation according to the different feature extraction methods. The two representation methods have similar performance, but because the number of abstract syntax tree paths is large, the model training time based on path information representation is relatively long. SUMMARY
[0005] (1) Technical problems to be solved
[0006] The technical problem to be solved by this invention is to address the issues of high false positive rate and poor generalization in existing static analysis of code vulnerability mining.
[0007] (II) Technical Solution
[0008] To address the aforementioned technical problems, this invention provides a Java vulnerability classification method based on natural language processing and deep forest, comprising the following steps:
[0009] S1. Obtain or select a vulnerability dataset and divide the dataset into a training dataset and a test dataset.
[0010] S2. Perform data preprocessing on the dataset;
[0011] S3. Parse the source code of the training dataset into an abstract syntax tree, and create a corresponding test label set for the test dataset;
[0012] S4. The ASTNN model is used to process the abstract syntax tree. An abstract syntax tree is first divided into a list containing several expression subtrees.
[0013] S5. Perform a preorder traversal and a postorder traversal on each expression subtree in the expression subtree list to obtain two sets of node label sets as a corpus for training the Word2Vec model.
[0014] S6. Perform a second preorder and postorder traversal on each expression subtree in the expression subtree list, while pruning to remove irrelevant nodes to obtain a node set. Use the Word2Vec model trained in S5 to convert each node into a d-dimensional vector v. n The d-dimensional vector of each expression subtree is recursively updated using Equation 2 to obtain a vector representation of an abstract syntax tree.
[0015]
[0016] in, Let n be the weight matrix, n represent a non-leaf node of the expression subtree, T be the matrix transpose operation, d be the dimension of the word embedding vector, k be the encoding dimension, C be the number of child nodes corresponding to node n, and b be the weight matrix. n Let σ be the bias, h be the activation function, and h be the activation function. i The updated vector for the child node i of node n;
[0017] S7. Use a deep forest for training, and test the generated deep forest using a test label set to obtain a Java code vulnerability classification model.
[0018] In S6, the pruning method is to first determine whether the root node of the expression subtree is an irrelevant node such as a comment declaration or package declaration during the traversal process. If it is such a node, it is discarded, and only the necessary semantic information is retained for encoding.
[0019] Furthermore, the deep forest in S7 consists of two parts: multi-granularity scanning and cascaded forest. In the multi-granularity scanning, a preprocessing stage of matrix transpose and max pooling is introduced to handle variable-length inputs and obtain the final vector representation of the expression subtree.
[0020] Specifically, the final vector representation of the obtained expression subtree is used to represent an abstract syntax tree. The sliding window and step size of the multi-granularity scan are controlled to 1, that is, the local features of the abstract syntax tree are scanned in units of expression subtrees and fed into the cascaded forest to generate class vectors.
[0021] More specifically, the first layer of the cascaded forest takes the output of the multi-granularity scan as input, and in each subsequent layer's decision, it concatenates the output of the previous cascaded forest as the input of the next layer. During training, each time the cascaded forest adds a layer, it uses samples from the test label set to test the already generated deep forest. If the accuracy is less than the accuracy of the previous layer, the number of cascaded forest layers is no longer increased; otherwise, the cascaded layers continue to be added until the accuracy of the test set is less than the accuracy of the previous layer. After the deep forest stops growing, the last layer of the cascaded forest takes the average of the probability vectors of all outputs, and outputs the label category with the highest probability as the final vulnerability classification decision.
[0022] (III) Beneficial Effects
[0023] The above-described technical solution of the present invention has the following advantages:
[0024] This invention improves the source code representation of the ASTNN encoding layer by pruning irrelevant information contained in class-level Java files and performing both preorder and postorder traversals on the abstract syntax tree to reduce information loss. It also improves multi-granularity scanning to address the characteristics of the abstract syntax tree by adding matrix transpose and max pooling preprocessing stages to handle variable-length input sequences, and changing the fixed-length sliding window to scanning according to the expression subtree sequence, thereby reducing training time while ensuring feature extraction effectiveness. Finally, a control experiment is designed to test the method provided by this invention using a vulnerability code dataset, verifying the model's excellent accuracy and effectiveness.
[0025] This invention uses an improved ASTNN encoding layer algorithm to encode an abstract syntax tree (ASTNN). The output of the ASTNN encoding layer is then used to convert the ASTNN into vectors using an improved multi-granularity scan. Finally, a cascaded forest is used for training to obtain a Java code vulnerability classification model. This invention improves the source code representation of the ASTNN encoding layer and combines it with deep forests, applying it to the Java source code vulnerability classification problem. Experiments demonstrate the feasibility of using a deep learning model based on NLP and deep forests for Java vulnerability discovery, representing a meaningful supplement to the field of source code vulnerability discovery. The classification model of this invention exhibits superior accuracy and effectiveness.
[0026] This invention improves model efficiency while better preserving the semantic information contained in the abstract syntax tree. The final experimental results show that the vulnerability mining method based on the improved ASTNN network has higher accuracy and recall than the traditional vulnerability mining method.
[0027] In addition to the technical problems solved by the present invention, the technical features of the technical solutions constituted by the present invention, and the advantages brought about by the technical features of these technical solutions as described above, other technical features of the present invention and the advantages brought about by these technical features will be further explained in conjunction with the accompanying drawings. Attached Figure Description
[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0029] Figure 1 This is a schematic diagram of the steps of the present invention.
[0030] Figure 2 This is a schematic diagram of the classification process of the present invention. Detailed Implementation
[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0032] A Java vulnerability classification method based on natural language processing and deep forest is proposed, mainly consisting of three stages: parsing the abstract syntax tree, extracting features from the ASTNN encoding layer, and classifying using deep forest. For example... Figure 1 As shown: It includes the following steps:
[0033] S1. Construct a vulnerability dataset;
[0034] S2. Perform data preprocessing on the dataset;
[0035] S3. Parse the source code into an abstract syntax tree;
[0036] S4. The ASTNN model is used to process the abstract syntax tree. An abstract syntax tree is first divided into a list containing several expression subtrees.
[0037] S5. Perform a preorder traversal and a postorder traversal on each expression subtree in the expression subtree list to obtain two sets of node label sets as a corpus for training the Word2Vec model.
[0038] S6. Perform a second preorder and postorder traversal on each expression subtree in the expression subtree list, while pruning to remove irrelevant nodes to obtain a node set. Use the Word2Vec model trained in S5 to convert each node into a d-dimensional vector v. n The d-dimensional vector of each expression subtree is recursively updated using Equation 2 to obtain a vector representation of an abstract syntax tree.
[0039]
[0040] in, Let n be the weight matrix, n represent a non-leaf node of the expression subtree, T be the matrix transpose operation, d be the dimension of the word embedding vector, k be the encoding dimension, C be the number of child nodes corresponding to node n, and b be the weight matrix. n Let σ be the bias, h be the activation function, and h be the activation function. i The updated vector for the child node i of node n;
[0041] S7. Use deep forest for training and add data preprocessing in the multi-granularity scanning stage: First, transpose the input matrix, then perform max pooling row by row, and use the max pooling result as the input for multi-granularity scanning. At the same time, control the sliding window and step size to 1 with the abstract syntax tree node as the unit. Use the test label set to test the generated deep forest to obtain the Java code vulnerability classification model.
[0042] Example 1
[0043] 1. Model Construction
[0044] This embodiment mainly includes the following three tasks:
[0045] (1) Improve the source code representation of the ASTNN encoding layer: trim irrelevant information contained in the ASTNN encoding layer when processing class-level Java files, and perform both preorder and postorder traversals on the abstract syntax tree to reduce information loss.
[0046] (2) Improve multi-granularity scanning based on the characteristics of abstract syntax tree, add matrix transpose and max pooling preprocessing stages to handle variable-length sequence input, change the fixed-length sliding window to scan according to the expression subtree sequence, and reduce training time while ensuring feature extraction effect.
[0047] (3) Design a control experiment and use the vulnerability code dataset to conduct experiments on the method in this paper to verify the accuracy and effectiveness of the model.
[0048] The vulnerability classification method based on natural language processing and deep forest mainly consists of three stages: parsing the source code into an abstract syntax tree, improving the ASTNN encoding algorithm to process the abstract syntax tree to obtain a statement sequence, and deep forest classification.
[0049] 1.1 Source code is parsed into an abstract syntax tree
[0050] Since deep forest models cannot understand source code in text form, a suitable method is needed to first convert the source code into vector data containing code syntax and semantic structure before inputting it into the deep forest model for learning. An Abstract Syntax Tree (AST) is an intermediate representation in the code compilation process. AST nodes store the syntactic structure information of the code, effectively reflecting the structure and semantic information of the source code. In this embodiment, an open-source Python library called javalang is used to perform the task of parsing source code into an AST. This Python library can be installed directly using pip and can parse an entire Java source code file or code snippet into the corresponding AST.
[0051] 1.2 Converting an Abstract Syntax Tree to a Vector
[0052] The abstract syntax tree obtained from source code parsing cannot be directly used as input to a deep forest model. Therefore, it is necessary to encode the abstract syntax tree into a text sequence and convert it into vector form.
[0053] 1.2.1 ASTNN Coding Layer
[0054] To address the problem of converting an Abstract Syntax Tree (ASTNN) into vectors, the ASTNN model is employed. This model uses the ASTNN as its processing object, dividing it into expression subtrees and capturing the information from each node of the expression subtrees, encoding it into a sequence of statements as input to the Word2Vec model. An expression subtree is a subtree within the ASTNN rooted at the expression declaration node. This subtree contains all nodes of an expression, operating at the statement level, and is the smallest unit containing semantics.
[0055] 1.2.2 Improvements to the ASTNN coding layer
[0056] The ASTNN encoding process described above can effectively preserve the semantic structure information of the abstract syntax tree, but it also has some shortcomings: for example, it retains irrelevant information in the tree during traversal, and a single traversal method still results in the loss of some semantic information. To address these two issues, further improvements are made to the ASTNN encoding method.
[0057] (1) Syntax tree pruning
[0058] The aforementioned ASTNN encoding layer is suitable for code snippets at the function level. However, when processing Java source code at the compiler unit level, all nodes of the resulting expression subtrees are encoded. This leads to irrelevant nodes, such as class annotations and their contents, being encoded as features. Encoding these as features interferes with training, causing overfitting and increasing model training time. To remove redundant and interfering information, this embodiment prunes the expression subtrees. During traversal, it first determines whether the root node of the expression subtree is an irrelevant node such as a comment or package declaration. If so, it is discarded, and only the necessary semantic information is retained for encoding.
[0059] (2) Feature preservation
[0060] In binary tree traversal, the node sequence obtained by a single traversal method cannot uniquely determine a binary tree, and the traversal of expression subtrees also suffers from this problem. In practice, it has been found that different abstract syntax trees (ASTNNs) can be transformed into the same sequence of expression subtrees. Therefore, this transformation method loses some semantic and structural information of the ASTNN. To address the feature loss issue, this embodiment performs a post-order traversal of the expression subtree while simultaneously performing a pre-order traversal. The results of the two traversals are concatenated to form the expression subtree sequence, which represents an ASTNN as input to the encoding layer. The pseudocode below illustrates the further improved ASTNN encoding layer algorithm.
[0061] Pseudocode 1 Improved ASTNN Coding Algorithm
[0062] Algorithm input: a preorder traversal node set `prenodes`, a postorder traversal node set `lstnodes`, and a node index list `indexs`.
[0063] Algorithm output: The set of vectors h corresponding to the set of nodes
[0064]
[0065] In the above encoding process, when encoding an abstract syntax tree, an abstract syntax tree is first divided into a list [T1, T2, ..., T] containing several expression subtrees. N Then, for each expression subtree T in the list... N A preorder and postorder traversal is performed to obtain two sets of node labels, which are used as the corpus to train the Word2Vec model and obtain the node index list `indexs`. Then, a second preorder and postorder traversal is performed on each expression subtree in the list, while pruning and removing irrelevant nodes to obtain the node sets `prenodes` and `lstnodes`. The expression subtree `T` is then processed using the Word2Vec model trained during the first traversal. N Each node in the vector is converted into a d-dimensional vector v. n Specifically, given an expression subtree T, where n represents a non-leaf node of T, we first perform word embedding on node n using Equation 1 to obtain the vector representation v of node n. n , where x n It is the one-hot encoding of node n, T is the matrix transpose operation, and W e T ∈P |V|×d Let V be the weight matrix, V be the vocabulary size, and d be the vector dimension of the word embeddings.
[0066] W e T x n Formula 1
[0067] Then for vector v n Equation 2 is used for recursive updates, thereby including child nodes and hierarchical structure information to obtain a vector representation h of an expression subtree. The above operation is performed on each expression subtree to obtain a vector representation [h1, h2, ..., h] of an abstract syntax tree. n In Equation 2 Here is the weight matrix, where C is the number of child nodes corresponding to node n, k is the encoding dimension, and b n Let σ be the bias, σ be the activation function (here, the tanh function is used), and h be the activation function. i This is the updated vector for the child node i of node n.
[0068]
[0069] 1.3 Vulnerability Code Classification
[0070] After the ASTNN encoding layer has extracted code features from the Java source code, the next step is to classify it using multi-granularity scanning and cascaded forest.
[0071] 1.3.1 Improved multi-granularity scanning
[0072] After extracting the list of vectors containing the features of the abstract syntax tree, further feature extraction is needed on the list of vectors and they are converted into vectors of fixed dimensions as input to the deep forest.
[0073] The original deep forest consists of two parts: multi-granularity scanning and cascaded forest. It can only handle fixed-length inputs, while the ASTNN encoding layer produces a vector list [h1, h2, ..., h...]. n The number of subtrees obtained from parsing different source codes varies, resulting in a variable number of vectors in each list, making it unsuitable as direct input for multi-granularity scans. To address this issue, a preprocessing stage involving matrix transpose and max pooling is introduced into multi-granularity scans to handle variable-length inputs.
[0074] To obtain the most important features contained in a node, in the preprocessing stage of multi-granularity scanning, as shown in Equation 3, the vector list of an abstract syntax tree is first transposed, transforming the n×d vector list into a d×n matrix, where d is the dimension of the vectors and n is the number of expression subtrees. Then, the vector h in each row of the matrix is... i Max pooling using Equation 4 yields the final vector representation e of an expression subtree. i .
[0075] Next, use the vector list [e1,...,e] corresponding to the expression subtree. i ,...e d ], i∈[1,d] represents an abstract syntax tree. The sliding window and step size of the multi-granularity scan are controlled to 1, that is, the local features of the abstract syntax tree are scanned in units of expression subtrees and fed into random forest and fully random forest to generate class vectors.
[0076]
[0077] e i =[max(h i,1 ),...,max(h i,n Equation 4, i∈[1,d]
[0078] 1.3.2 Generation of Cascaded Forests
[0079] Cascaded forests are an integrated forest structure proposed in existing technologies, consisting of an equal number of random forests and fully random forests.
[0080] Random forests are composed of a set of decision trees {T(X,θ)} k An ensemble classifier consisting of {θ, k=1, K}, where K represents the number of decision trees in the random forest. k} are independent and identically distributed random vectors. The steps to generate a random forest are as follows:
[0081] (1) Use the bootstrap method to randomly select K sample sets from the training set, and then use the randomly selected samples to construct K decision trees.
[0082] (2) Given n features, randomly select m (m≤n) features at each node of each tree, and calculate the Gini coefficient Gini(D,a) for each feature using Equation 4. Select the feature with the smallest Gini coefficient to split the tree node. In Equation 4, |y| and p k Let be the number of classes in dataset D and the proportion of each class in the dataset; 'a' be the feature to be divided; |D| be the total number of samples; |D V | represents the total number of samples in category V.
[0083]
[0084] (3) Growth stops when there is only one class in a node of each tree or the number of samples in a node is less than the minimum splitting level.
[0085] (4) The generated K decision trees are combined into a random forest, and the random forest is used for classification. The voting results of all decision trees in the random forest are used as the classification results.
[0086] Similar to random forest, in step 2 of the above process, fully random forest randomly selects a feature from each tree as the split node of the tree, and the other steps are consistent with the random forest generation process.
[0087] 1.3.3 Classification Process
[0088] like Figure 2 As shown: The first layer of the cascaded forest takes the output of the multi-granularity scan as input, and in each subsequent layer's decision, the output of the previous cascaded forest is concatenated as the input of the next layer. During training, each time a cascaded forest is added, it is tested against samples in the test set. If the accuracy is lower than that of the previous layer, the number of cascaded forest layers is not increased; otherwise, cascaded layers are added until the accuracy on the test set is lower than that of the previous layer.
[0089] After the deep forest stops growing, the cascaded forest in the last layer takes the average of the probability vectors of all outputs, and outputs the label category with the highest probability as the final decision result for vulnerability classification.
[0090] 2. Experiment and Results Analysis
[0091] 2.1 Experimental Environment and Dataset
[0092] This is a source code vulnerability classification system based on NLP and deep forest. It runs on an Intel® Core™ i9-11980HK octa-core processor, with 64GB of RAM and a 64-bit Windows 10 operating system. Python 3.6 is used for source code parsing into Abstract Syntax Trees (ASTNN), encoding, and deep forest classification.
[0093] The source code vulnerability data used in this article comes from the OWASP Benchmark vulnerability dataset v1.1, which contains 21,041 runnable test cases across 11 vulnerability types, including command injection, weak encryption algorithms, SQL injection, and directory traversal. Each category contains positive samples with vulnerabilities and negative samples without vulnerabilities, with a ratio of approximately 1.28:1. Basic information is shown in Table 1.
[0094] When using data for training, 80% of it is randomly selected as the training set and the remaining 20% is used as the test set.
[0095] Table 1 Vulnerability Dataset
[0096]
[0097] 2.2 Evaluation Criteria
[0098] This paper uses accuracy, recall, and F1 score as evaluation metrics, and the calculation formulas are as follows:
[0099]
[0100]
[0101]
[0102] Where TP represents the number of samples correctly classified as vulnerable, FP represents the number of samples falsely reported as vulnerable, FN represents the number of samples not reported, and TN represents the number of samples correctly classified as not vulnerable.
[0103] 2.3 Analysis of Experimental Results
[0104] In the experiment, 80% of the OWASP dataset was randomly selected as the training set and 20% as the test set. To effectively evaluate the feasibility and efficiency of the model in the Java vulnerability classification problem, three sets of comparative experiments were designed.
[0105] The first set of experiments used Token sequences and MCDF methods at the encoding layer, respectively.
[20] The accuracy of the classifiers based on the improved ASTNN coding layer model was compared using cascaded forests. The experimental results are shown in Table 2, verifying the effectiveness of the improved ASTNN coding layer method in vulnerability classification.
[0106] Table 2 Comparison of Source Code Representation Methods
[0107]
[0108] The second set of experiments compared the model's generalization ability. In this set of experiments, the model trained on the OWASP dataset in the first set of experiments was used to test the SARD vulnerability dataset. The experimental results are shown in Table 3, indicating that the model performs well on different datasets and has good generalization ability.
[0109] Table 3. Experiment on model generalization ability
[0110]
[0111] The third set of experiments compared the performance of the proposed model with FindSecBugs, a commonly used Java static code analysis tool. Since traditional static code analysis tools lack a training phase, the entire OWASP dataset was used to evaluate the accuracy of the static code analysis tools. The experimental results are shown in Table 4. This set of comparative experiments demonstrates that the NLP-based and deep forest-based model significantly outperforms traditional static code analysis tools in terms of accuracy, false positive rate, and false negative rate.
[0112] Table 4 Comparison with common static code analysis tools
[0113]
[0114] This embodiment improves the source code representation of the ASTNN encoding layer and combines it with deep forest, applying it to the Java source code vulnerability classification problem. Experiments demonstrate the feasibility of using a deep learning model based on NLP and deep forest for Java vulnerability mining, which is a meaningful supplement to the field of source code vulnerability mining.
[0115] The specific embodiments of the present invention have been described in detail above. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.
Claims
1. A Java vulnerability classification method based on natural language processing and deep forest, characterized by: Includes the following steps: S1. Obtain or select a vulnerability dataset; S2. Perform data preprocessing on the dataset; S3. Parse the source code into an abstract syntax tree; S4. The ASTNN model is used to process the abstract syntax tree. An abstract syntax tree is first divided into a list containing several expression subtrees. S5. Perform a preorder traversal and a postorder traversal on each expression subtree in the expression subtree list to obtain two sets of node label sets as a corpus for training the Word2Vec model. S6. Perform a second preorder and postorder traversal on each expression subtree in the expression subtree list, while pruning to remove irrelevant nodes to obtain a node set. Then, use the Word2Vec model trained in S5 to transform each node into... dimensional vector For each expression subtree The dimensional vector is recursively updated using Equation 2 to obtain a vector representation of an abstract syntax tree; Formula 2 in, This is the weight matrix. This represents a non-leaf node in the expression subtree. This is a matrix transpose operation. Let be the vector dimension of the word embedding. For the encoding dimension, For nodes The corresponding number of child nodes, For deviation, For activation function, For nodes child nodes The updated vector; S7. Use a deep forest for training, and test the generated deep forest using a test label set to obtain a Java code vulnerability classification model.
2. The Java vulnerability classification method based on natural language processing and deep forest as described in claim 1, characterized in that: In S6, the pruning method is to first determine whether the root node of the expression subtree is an irrelevant node of the comment declaration or package declaration during the traversal process. If it is such a node, it is discarded, and only the necessary semantic information is retained for encoding.
3. The Java vulnerability classification method based on natural language processing and deep forest according to claim 1 or 2, characterized in that: In S7, the deep forest consists of two parts: multi-granularity scanning and cascaded forest. In the multi-granularity scanning, a preprocessing stage of matrix transpose and max pooling is introduced to handle variable-length inputs and obtain the final vector representation of the expression subtree.
4. The Java vulnerability classification method based on natural language processing and deep forest as described in claim 3, characterized in that: The resulting expression subtree vector represents an abstract syntax tree. The sliding window and step size of the multi-granularity scan are controlled to 1, that is, the local features of the abstract syntax tree are scanned in units of expression subtrees and fed into the cascaded forest to generate class vectors.
5. The Java vulnerability classification method based on natural language processing and deep forest as described in claim 4, characterized in that: The first layer of the cascaded forest takes the output of the multi-granularity scan as input, and in each subsequent layer's decision, it concatenates the output of the previous cascaded forest as the input of the next layer. During training, each time the cascaded forest adds a layer, it uses samples from the test label set to test the already generated deep forest. If the accuracy is less than the accuracy of the previous layer, the number of cascaded forest layers is no longer increased; otherwise, the cascaded layers continue to be added until the accuracy of the test set is less than the accuracy of the previous layer. After the deep forest stops growing, the last layer of the cascaded forest takes the average of the probability vectors of all outputs, and outputs the label category with the highest probability as the final vulnerability classification decision result.