An effective machine learning approach to software requirement traceability link recovery

Abstract

The application discloses an effective machine learning method for software requirement traceability link recovery, comprising the following steps: classifying and extracting code features and annotation features of a code file to be determined according to rules, and fusing and labeling; combining code corpora according to the labeling dimension; respectively preprocessing different code corpora and requirement corpora, and extracting features of the data based on an information retrieval model; inputting different feature data to train, predict and evaluate a machine learning model, and selecting a code corpus combination with the best effect; and combining machine learning models by using an integrated algorithm, and training a new classifier by using the feature data with the best effect to perform prediction. The method solves the problem of insufficient code corpus utilization and negative impact, uses an integrated algorithm to obtain an effective prediction model, guarantees the accuracy of software requirement traceability link recovery, reduces the recovery cost, does not require manual participation, and improves the automation degree.

Description

Technical Field

[0001] This invention belongs to the field of requirements tracing in computer technology and software engineering, and particularly relates to an effective machine learning method for restoring traceability links in software requirements. Background Technology

[0002] Software systems typically consist of a variety of software artifacts, such as requirements specifications, design documents, use cases, code, bug reports, and test cases. Software requirements traceability attempts to discover the relationships between these artifacts. Establishing traceability links between different artifacts, enabling the mapping from requirements documents to implemented software artifacts, plays a crucial role. Use cases and source code are common and important software artifacts in the software development process, and engineers typically focus on the traceability links between them. In a standard software development process, traceability links need to be created and maintained throughout the system lifecycle. Unfortunately, maintaining traceability links is usually manual, which is both time-consuming and error-prone. Therefore, due to time pressures in software development, such traceability links are often lacking in practice. Although extensive research has been conducted on automating requirements traceability, automation techniques are often affected by noise, generating numerous false alarms and resulting in low accuracy.

[0003] In recent years, researchers have proposed various solutions for automating the creation and maintenance of tracking links, including information retrieval and machine learning. Information retrieval methods calculate the similarity between requirement text and code text, relying on pre-set thresholds to filter tracking links for software requirements, significantly saving labor costs; however, performance often falls short of expectations. Machine learning methods can more effectively predict tracking links between software requirements and code files by extracting and constructing textual features, and by training effective classifiers, achieving better performance than information retrieval methods. However, these methods all rely on rich and high-quality textual information from the software files to achieve better results.

[0004] Currently, the majority of code information used in code files consists of class names, method names, API sequences, and structural relationships between code blocks. Some technologies also utilize comments, but the use of comment features is mostly limited to simple keyword extraction, without in-depth mining and utilization of comment information. How to effectively utilize comment information and incorporate more comment data into code corpora for model training has not been considered. How to more effectively utilize comments and enrich the code corpus by combining them with other code features is urgently needed to improve model prediction accuracy. Furthermore, while machine learning models used in the demand tracking field can achieve higher accuracy and improved generalization ability than existing single-algorithm models through algorithm optimization and parameter tuning, the relatively complex data in software code files limits the effectiveness of single prediction algorithms, resulting in weak generalization. Therefore, addressing these issues has become a key research focus in order to improve the accuracy of demand tracking link recovery. Summary of the Invention

[0005] In view of this, the present invention provides an effective machine learning method for recovering traceable links in software requirements. The purpose is to improve the utilization rate of code file information, reduce the negative impact caused by insufficient utilization of code corpus, improve the accuracy of recovering traceable links in software requirements, reduce recovery costs, reduce manual labor, and improve the degree of automation and project work efficiency.

[0006] To achieve the above objectives, this invention provides an effective machine learning method for recovering traceability links in software requirements, comprising the following steps:

[0007] Step 1: Classify, extract, label, and fuse the code features and annotation features of the code files whose tracking relationships are to be determined according to the rules; then combine the descriptive code corpus according to the labeling dimensions.

[0008] Step 2: Preprocess the different descriptive code corpora and the demand corpus respectively, and extract features from the preprocessed data based on several models most frequently used in information retrieval.

[0009] Step 3: Input different feature data to train, predict and evaluate several of the most commonly used models in the field of machine learning demand tracking, and select the combination of descriptive code corpora with the best performance.

[0010] Step 4: Use the Stacking ensemble algorithm to combine machine learning models and train a new classifier using the best-performing feature data for prediction.

[0011] Furthermore, the specific implementation method of step 1 is as follows;

[0012] Step 11: Use the Javalang library of Python tools to parse Java source code into an abstract syntax tree, and use the walk method to traverse and obtain information on the nodes of the abstract syntax tree. At the code class level, package name, variable name, class name, method name, method parameters, method return value, and API sequence are extracted as code features.

[0013] Step 12: Use the Javalang library of Python tools to parse Java source code into an abstract syntax tree, and use the walk method to traverse and obtain the comment information on the nodes of the abstract syntax tree. Then, classify the extracted comment features into five categories: functional comments, usage comments, structural comments, development comments, and exception comments.

[0014] Step 13: Generate multidimensional code annotations based on the functional attributes of the code features obtained in Step 11 and the annotation features obtained in Step 12;

[0015] Step 14: Combine the corpus formed by multi-dimensional code annotation and fusion based on step 13 according to the annotation dimensions.

[0016] Furthermore, the multidimensional code annotation in step 13 describes the code from four aspects: functional annotation, usage annotation, method body annotation, and development annotation. The content of the multidimensional code annotation is a fusion of code features and comment features. Functional annotation includes package name, variable name, method name, and functional comments. Usage annotation includes method parameters, method return value, and usage comments. Method body annotation includes call sequence, exception comments, and structural comments. Development annotation includes development comments.

[0017] Furthermore, the specific implementation method of preprocessing in step 2 is as follows;

[0018] a) Character Removal: Perform tokenization on all requirements and source code files, removing non-alphabetic characters, punctuation marks, stop words, and meaningless terms; b) Tokenization: Extract meaningful words from identifiers in requirements and source code files. Identifiers are compound words connecting two or more words, abbreviations, or any other delimiters. Use camelCase to split compound words; c) Translation: Translate the text of requirements and source code into English when they contain non-English words. Then, use information retrieval models VSM, LSI, LDA, and JS to index the software documents into the document space by extracting term occurrence information from the corpus of requirements and code documents. The extracted information is stored in an m×n matrix called the term-by-document matrix, where m is the number of all terms appearing in the document, n is the number of documents in the repository, and the elements of this matrix are a ijThe weight of the i-th term in the j-th document is represented by the relevance. Finally, the feature value is obtained by calculating the text similarity between the two software document vectors in the term-by-document matrix.

[0019] Furthermore, the specific implementation method of step 3 is as follows;

[0020] The feature data of different corpus combinations obtained in step 2 are input into three machine learning models most commonly used in the field of machine learning demand tracking: Random Forest (RF), Decision Tree (DR), and Gaussian Naive Bayes (GNB). The prediction results are evaluated using three evaluation metrics: Precision, Recall, and F1-score. If the prediction performance of a certain corpus combination is better than that of other corpus combinations, then this code corpus combination will be retained.

[0021] Furthermore, the specific implementation method of step 4 is as follows;

[0022] Step 41: Obtain the feature dataset with the best performance from Step 3;

[0023] Step 42: Select the three most commonly used machine learning models in the field of demand tracking based on machine learning: Decision Tree, RandomForest, and Gaussian Naive Bayes, as primary learners, and select the optimal parameters of individual learners using the grid search method in the sklearn library.

[0024] Step 43: Divide the feature dataset from Step 41 into a training set Y and a test set X according to a certain ratio, and then divide the processed training set into 5 equal parts Y. i , i∈{1,2,3,4,5};

[0025] Step 44: In the first-layer prediction model, the three primary learners are trained using 5-fold cross-validation.

[0026] Step 45: In order to better obtain effective information from the primary learners, different weights are set for each learner in the first layer based on the different prediction results of the models in the primary learners.

[0027] Step 46: After setting weights for the three learners in the first layer based on step 45, the three primary learners are then trained and predicted. After completion, two new matrix datasets (Y1, Y2, Y3) and (X1, X2, X3) are obtained. In the second layer, the secondary learner Multiple Linear Regression (MLR) is trained based on matrix Y as the training set and matrix X as the test set to obtain the final prediction result, i.e. the demand tracking link recovery result. 0 indicates that there is no tracking link and 1 indicates that there is a tracking link.

[0028] Furthermore, in step 43, for the Decision Tree, each cross-validation involves two processes: first, four portions of the dataset are used as the training set for the Decision Tree, and the remaining portion is used as the test set for the Decision Tree model's predictions, resulting in the prediction dataset y. i Let i ∈ {1,2,3,4,5}, and combine them into one column, denoted as Y1. Then, use the Decision Tree model trained on the four datasets in the previous step to predict the test set X, thus obtaining the prediction dataset x. i Then, consider the results of the 5 predictions x. i The average of the sums of the rows is used to obtain dataset X1.

[0029] Furthermore, the specific formula for calculating the weight value in step 45 is as follows:

[0030]

[0031] Among them, D j The sum of squared errors of the j-th model is given a higher weight, i.e., models with smaller sums of squared errors are assigned higher weights, where J is the number of models;

[0032]

[0033] Where N is the total number of predicted tracking links, x t Represents the true value. This represents the predicted value.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. This invention classifies, extracts, and annotates code features and annotation features, then combines code corpora according to annotation dimensions, trains models on different combinations of corpora, and obtains an effective code corpus by comparing the results, thereby reducing the negative impact caused by insufficient utilization of code corpora.

[0036] 2. To improve the generalization ability of machine learning models and fully leverage the advantages of individual learners, this invention employs the Stacking ensemble method. It nonlinearly combines three heterogeneous learners most commonly used in the field of machine learning-based requirement tracking: Random Forest, Decision Tree, and Gaussian Naive Bayes. This constructs a two-layer Stacking ensemble learning model, which effectively improves the accuracy of software requirement traceability link recovery, reduces recovery costs, minimizes manual labor, and enhances automation and project efficiency.

[0037] 3. In constructing a two-layer Stacking ensemble learning model, since the performance of the models in the primary learners varies, a weight value w is set to reduce the impact of this difference on the prediction results. j This reduces the impact and thus improves the accuracy of predictions. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart of an effective machine learning method for recovering software requirement traceability links according to an embodiment of the present invention;

[0040] Figure 2 This is a fusion diagram of code feature and comment feature annotations according to an embodiment of the present invention;

[0041] Figure 3 This is a code corpus combination diagram according to an embodiment of the present invention;

[0042] Figure 4 This is a flowchart of a two-layer Stacking ensemble learning model according to an embodiment of the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] like Figure 1As shown, this embodiment of the invention provides an effective machine learning method for recovering traceable links in software requirements, characterized by comprising the following steps:

[0045] Step 1: Classify, extract, label, and fuse the code features and annotation features of the code files whose tracking relationships need to be determined according to the rules, and then combine the descriptive code corpus according to the annotation dimensions;

[0046] In one embodiment of the present invention, such as Figure 2 As shown, through code feature extraction, annotation feature extraction, classification, and annotation fusion, then as follows... Figure 3 As shown, the descriptive code corpora are combined according to the labeled dimensions to obtain different code corpora; the specific steps are as follows:

[0047] Step 11: Use the Javalang library of Python tools to parse Java source code into an abstract syntax tree, and use the walk method to traverse and obtain information on the nodes of the abstract syntax tree. At the code class level, package name, variable name, class name, method name, method parameters, method return value, and API sequence are extracted as code features.

[0048] Step 12: Use the Javalang library of Python tools to parse the Java source code into an abstract syntax tree, and use the walk method to traverse and obtain the comment information on the nodes of the abstract syntax tree. Then, classify the extracted comment features into five categories: functional comments, usage comments, structural comments, development comments, and exception comments.

[0049] Step 13: Generate multi-dimensional code annotations based on the functional attributes of the code features obtained in Step 11 and the comment features obtained in Step 12. Multi-dimensional code annotations describe the code from four aspects: functional annotations, usage annotations, method body annotations, and development annotations. The content of multi-dimensional code annotations is a fusion of code features and comment features. Functional annotations mainly include package name (code feature), variable name (code feature), method name (code feature), and functional features (comment features). Usage annotations mainly include parameters (code feature), return value (code feature), and usage features (comment features). Method body annotations mainly include call sequence (code feature), exception features (comment features), and structural features (comment features). Development annotations mainly include development features (comment features), see details... Figure 2 .

[0050] Step 14: Combine the corpus with multi-dimensional code annotations and fusion obtained from Step 13 according to the annotation dimensions. There are 10 possible combinations, see details below. Figure 3 .

[0051] Step 2: Preprocess the different descriptive code corpora of the combination with the demand corpus, and extract features from the data based on the four most commonly used models in information retrieval;

[0052] Furthermore, the different corpora formed by the combination are preprocessed separately with the required corpus, and features are extracted from the data based on the most commonly used models in information retrieval: Vector Space Model (VSM), Latent Semantic Indexing (LSI), Latent Dirichlet Allocation (LDA), and Jensen-Shannon Model (JSM). The specific steps are as follows:

[0053] Based on the different code corpora formed in step 14, preprocessing is performed on the requirement corpora: a) Character removal: Tokenization is performed on all texts (requirement files and source code files), removing non-alphabetic characters, punctuation marks, stop words, and meaningless terms; b) Tokenization: Meaningful words are extracted from identifiers in the requirement files and source code files. Identifiers are compound words connecting two or more words (or abbreviations) or any other separator (such as underscores), and camel spelling is used to split compound words; c) Translation: When the requirements and source code contain non-English words, the text content of the requirements and source code is translated into English; then, information retrieval models VSM, LSI, LDA, and JS are used to index the software documents into the document space by extracting term occurrence information from the corpora of requirement and code documents. The extracted information is usually stored in an m×n matrix (called the term-by-document matrix), where m is the number of all terms appearing in the document, and n is the number of documents in the repository. The elements of this matrix a ij The weight (i.e., relevance) of the i-th term in the j-th document is represented by the term-by-document matrix. Finally, the feature value is obtained by calculating the text similarity between the two software document vectors in the term-by-document matrix.

[0054] Step 3: Input different feature data to train, predict, and evaluate the three most commonly used models in the field of machine learning-based demand tracking, and select the code corpus combination with the best performance;

[0055] Furthermore, the feature data of different corpus combinations obtained in step 3 are input into the machine learning model for training and prediction, and the code corpus combination with the best effect is selected, specifically through the following steps:

[0056] The feature data of different corpus combinations obtained in step 2 are input into three machine learning models most commonly used in the field of machine learning demand tracking: Random Forest (RF), Decision Tree (DR), and Gaussian Naive Bayes (GNB). The prediction results are evaluated using three evaluation metrics: Precision, Recall, and F1-score. If the prediction performance of a certain corpus combination is better than that of other corpus combinations, then this code corpus combination will be retained.

[0057] Step 4: Use the Stacking ensemble algorithm to combine the machine learning models and input the feature data with the best performance obtained in Step 3 into the new classifier for training and prediction;

[0058] like Figure 4 As shown, the Stacking ensemble algorithm combines machine learning models and trains a new classifier for prediction using the best-performing feature data. Specifically, this includes the following steps:

[0059] Step 41: Obtain the feature dataset with the best performance from Step 3;

[0060] Step 42: Select the three most commonly used machine learning models in the field of demand tracking based on machine learning—Decision Tree, RandomForest, and Gaussian Naive Bayes—as primary learners, and select the optimal parameters of individual learners using the grid search method in the sklearn library.

[0061] Step 43: Divide the feature dataset from Step 41 into a training set Y and a test set X according to a certain ratio, and then divide the processed training set into 5 equal parts Y. i , i∈{1,2,3,4,5};

[0062] Step 44: In the first-layer prediction model, train the three primary learners using 5-fold cross-validation. Taking Decision Tree as an example, each cross-validation involves two processes. First, use four portions of the dataset as the training set for Decision Tree training, and use the remaining portion as the test set for the Decision Tree model's predictions, thus obtaining the prediction dataset y. i (i∈{1,2,3,4,5}), these are combined into one column and denoted as Y1. Then, the Decision Tree model trained on the four datasets in the previous step is used to predict the test set X, resulting in the prediction dataset x. i(i∈{1,2,3,4,5}), and then consider the results of the 5 predictions x i (i∈{1,2,3,4,5}) sum the rows and take the mean to get the dataset X1;

[0063] Step 45: At the same time, in order to better obtain effective information from the primary learners, different weights are set for each learner in the first layer based on the different prediction results of the models in the primary learners.

[0064] Step 46: After setting weights for the three learners in the first layer based on Step 45, train and predict the three primary learners. This will result in two new matrix datasets: (Y1, Y2, Y3) and (X1, X2, X3). In the second layer, the secondary learner, Multiple Linear Regression (MLR), is trained using matrix Y as the training set and matrix X as the test set to obtain the final prediction result, i.e., the demand tracking link recovery result (0 for no tracking link, 1 for a tracking link).

[0065] Furthermore, the stacking ensemble algorithm is used to combine machine learning models for training and prediction. Since the performance of the models in the primary learners varies, weight values ​​are set to reduce the impact of these differences on the prediction results and improve prediction accuracy. The specific formula for calculating the weight values ​​is as follows:

[0066]

[0067] Among them, D j Let J be the sum of squared errors of the j-th model. This method assigns higher weights to models with smaller sums of squared errors, where J is the number of models.

[0068]

[0069] Where N is the total number of predicted tracking links, x t Represents the true value. This represents the predicted value.

[0070] The foregoing description of specific exemplary embodiments of the invention is for illustrative and explanatory purposes. These descriptions are not intended to limit the invention to the precise forms disclosed, and it will be apparent that many changes and variations can be made in accordance with the foregoing teachings. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application, thereby enabling those skilled in the art to implement and utilize various different exemplary embodiments of the invention, as well as various different choices and variations. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. An effective machine learning method for restoring traceability links in software requirements, characterized in that, Includes the following steps: Step 1: Classify, extract, label, and fuse the code features and annotation features of the code files whose tracking relationships are to be determined according to the rules; then combine the descriptive code corpus according to the labeling dimensions. Step 2: Preprocess the different descriptive code corpora and the demand corpus respectively, and extract features from the preprocessed data based on several models most frequently used in information retrieval. Step 3: Input different feature data to train, predict and evaluate several of the most commonly used models in the field of machine learning demand tracking, and select the combination of descriptive code corpora with the best performance. Step 4: Use the Stacking ensemble algorithm to combine machine learning models and train a new classifier using the feature data with the best performance for prediction. The specific implementation method of step 4 is as follows; Step 41: Obtain the feature dataset with the best performance from Step 3; Step 42: Select the three most commonly used machine learning models in the field of demand tracking based on machine learning: Decision Tree, RandomForest, and Gaussian Naive Bayes, as primary learners, and select the optimal parameters of individual learners using the grid search method in the sklearn library. Step 43: Divide the feature dataset from Step 41 into a training set Y and a test set X according to a certain ratio, and then divide the processed training set into 5 equal parts. ; Step 44: In the first-layer prediction model, the three primary learners are trained using 5-fold cross-validation. Step 45: In order to better obtain effective information from the primary learners, different weights are set for each learner in the first layer based on the different prediction results of the models in the primary learners. The specific formula for calculating the weight value in step 45 is as follows: ； in, For the first j The sum of squared errors of each model, i.e., assigning higher weights to models with smaller sums of squared errors, where J is the number of models; ； Where N is the total number of predicted tracking links, Represents the true value. Indicates the predicted value; Step 46: After setting weights for the three learners in the first layer based on step 45, train and predict the three primary learners. This will result in two new matrix datasets. )and( In the second layer, the secondary learner performs multiple linear regression using a matrix... As a training set, the matrix As a test set, it is used as a basis for training to obtain the final prediction result, namely the demand tracking link recovery result. 0 indicates that there is no tracking link and 1 indicates that there is a tracking link.

2. The effective machine learning method for restoring traceable links in software requirements as described in claim 1, characterized in that: The specific implementation method of step 1 is as follows; Step 11: Use the Javalang library of Python tools to parse Java source code into an abstract syntax tree, and use the walk method to traverse and obtain information on the nodes of the abstract syntax tree. At the code class level, package name, variable name, class name, method name, method parameters, method return value and API sequence are extracted as code features. Step 12: Use the Javalang library of Python tools to parse Java source code into an abstract syntax tree, and use the walk method to traverse and obtain the comment information on the nodes of the abstract syntax tree. Then, classify the extracted comment features into five categories: functional comments, usage comments, structural comments, development comments, and exception comments. Step 13: Generate multidimensional code annotations based on the functional attributes of the code features obtained in Step 11 and the annotation features obtained in Step 12; Step 14: Combine the corpus formed by multi-dimensional code annotation and fusion based on step 13 according to the annotation dimensions.

3. The effective machine learning method for recovering traceable links in software requirements as described in claim 2, characterized in that: The multidimensional code annotation in step 13 describes the code from four aspects: functional annotation, usage annotation, method body annotation, and development annotation. The content of the multidimensional code annotation is a fusion of code features and comment features. Functional annotation includes package name, variable name, method name, and functional comments. Usage annotation includes method parameters, method return value, and usage comments. Method body annotation includes API sequence, exception comments, and structural comments. Development annotation includes development comments.

4. The effective machine learning method for recovering traceable links in software requirements as described in claim 1, characterized in that: The specific implementation method of preprocessing in step 2 is as follows; a) Character removal: Perform tokenization on all requirements and source code files, removing non-alphabetic characters, punctuation marks, stop words, and meaningless terms; b) Word segmentation: Extract meaningful words from identifiers in the requirements and source code files. Identifiers are compound words that connect two or more words, abbreviations, or any other separators. Compound words are split using camelCase. c) Translation: When the requirements and source code contain non-English words, translate the text content of the requirements and source code into English. Then, information retrieval models VSM, LSI, LDA, and JS are used to index the software documents into the document space by extracting term occurrence information from the corpus of requirements and code documents. The extracted information is stored in an m×n matrix called the term-by-document matrix, where m is the number of all terms appearing in the document, n is the number of documents in the repository, and the elements of this matrix... The weight of the i-th term in the j-th document is represented by the relevance. Finally, the feature value is obtained by calculating the text similarity between the two software document vectors in the term-by-document matrix.

5. The effective machine learning method for recovering traceable links in software requirements as described in claim 1, characterized in that: The specific implementation method of step 3 is as follows; The feature data of different corpus combinations obtained in step 2 are input into three machine learning models most commonly used in the field of machine learning demand tracking: Random Forest, Decision Tree, and Gaussian Naive Bayes. The prediction results are evaluated using three evaluation metrics: Precision, Recall, and F1-score. If the prediction performance of a certain corpus combination is better than that of other corpus combinations, then this code corpus combination will be retained.

6. The effective machine learning method for recovering traceable links in software requirements as described in claim 1, characterized in that: In step 43, for the Decision Tree, each cross-validation involves two processes: first, four portions of the dataset are used as the training set for the Decision Tree, and the remaining portion is used as the test set for the Decision Tree model's predictions, thus obtaining the prediction dataset. Combine them into 1 column and denote it as Then, the Decision Tree model trained on the four datasets in the previous step is used to predict the test set X, resulting in the prediction dataset. Then analyze the results of the 5 predictions. The dataset is obtained by summing the values ​​of each row. .

Images