A complex class code smell detection method

Abstract

The application discloses a complex code smell detection method, comprising the following steps: obtaining code metrics in a source code file through a CK tool and corresponding to a label to form a code metric and smell label dataset, and then preprocessing; performing three times of feature selection on the preprocessed code metric and smell label dataset to obtain feature importance corresponding to three kinds of feature selection of all features, accumulating the three kinds of feature importance, and calculating the average importance of each feature; then arranging in descending order, removing the last 20% features, and selecting three kinds of machine learning algorithms for training on the remaining 80% features, and integrating the three kinds of models through a stacking method to obtain a final detection method. The complex code smell detection method has the advantages of high accuracy, strong detection ability and generalization ability, and the like.

Description

Technical Field

[0001] This invention relates to the field of code smell detection for complex classes, and more particularly to a method for detecting code smells in complex classes. Background Technology

[0002] In software development, source code quality remains a constant concern in the field of software engineering. Typically, software development is conducted within a defined code architecture. However, with prolonged software maintenance, code often deviates from this architecture, becoming difficult to read, hindering maintenance, increasing code modification frequency, and leading to serious code quality and maintainability issues. In software engineering, poorly formatted code snippets (classes, methods, parameters) are commonly referred to as code smells. The presence of code smells significantly complicates software maintenance, severely impacting code maintainability and reusability. Therefore, regularly detecting code smells and promptly refactoring the code are crucial for maintaining code maintainability and reusability.

[0003] Developers often discover code smells during software development, which manifest in different ways and require different refactoring methods. To better identify code smells, Fowler defines 22 types, including Long Method, Feature Envy, and Duplicated Code. Each code smell has its own characteristics, and the methods for identifying them differ, providing a strong foundation for further research on code smells.

[0004] Early code smell detection was typically done manually. However, manual detection is time-consuming and labor-intensive, and the accuracy of code smell identification varies with the developer's experience. Therefore, researchers have successively proposed code smell detection methods based on threshold metrics and machine learning. The former involves developers judging code smells based on a combination of thresholds for a series of code metrics, while the latter entrusts the determination of thresholds and the identification of code smells to a machine learning model, demonstrating stronger detection capabilities.

[0005] In terms of machine learning models, previous research has focused on improving single machine learning models, ignoring the lack of generalization of single machine learning models in code smell detection, i.e., no single model can detect all code smells.

[0006] In terms of feature selection, since the code contains a large number of features, too many features will reduce the generalization of machine learning models and significantly increase the running time. However, existing feature selection techniques rely solely on a single indicator to judge the importance of features, lacking comprehensive consideration of multiple feature selection opinions. Summary of the Invention

[0007] In view of the above-mentioned shortcomings of the prior art, the technical problem to be solved by the present invention is that existing methods for detecting complex class code smells have many features and that single machine learning models have low generalization ability. The present invention provides a method for detecting complex class code smells, based on a detection method that combines feature selection and stacking heterogeneity, to detect complex classes in code smells, and has advantages such as high accuracy, strong detection ability and strong generalization ability.

[0008] To achieve the above objectives, the present invention provides a method for detecting code odors in complex classes, comprising the following steps:

[0009] The CK tool is used to obtain code metrics from source code files and correspond them one-to-one with labels to form a code metric and odor label dataset.

[0010] Data preprocessing is performed on the obtained code metrics and odor label datasets;

[0011] The preprocessed code metric and odor label datasets were subjected to first, second and third feature selections to obtain the feature importance of each feature corresponding to the three feature selections. The three feature importances were summed and the average importance of each feature was calculated.

[0012] The average importance of each feature is sorted in descending order, the bottom 20% of features are removed, and the dataset is modified accordingly to include only the remaining 80% of features.

[0013] For the remaining 80% of features, three machine learning algorithms were selected to train the dataset, and the three models were integrated using the Stacking method to obtain the final detection method.

[0014] Furthermore, the code metrics in the source code files are obtained using the CK tool and matched one-to-one with the labels to form a code metric and odor label dataset. In a given Java open source software system, there exists a set of Java open source software system version files. The CK tool is used to parse the code metrics in each source code file in the Java open source software system version file set and combine them with the corresponding labels to form a code metric and odor label dataset.

[0015] Furthermore, the obtained code metric and odor label datasets are preprocessed, specifically by using median imputation to fill in features with missing values ​​to ensure the integrity of the dataset; and then using Min-Max to standardize the code metric and odor label datasets.

[0016] Furthermore, Mutual Information, RFE, and LightGBM were used to perform initial feature selection on the preprocessed code metric and odor label datasets to obtain the feature importance of all features corresponding to the three feature selections. The three feature importances were summed, and the average importance of each feature was calculated.

[0017] Furthermore, Mutual Information is used to perform the first feature selection on the code metric set to obtain the mutual information between each feature and label.

[0018] Furthermore, RFE is used to perform a second feature selection on the code metric set to obtain the Gini index of each feature.

[0019] Furthermore, LightGBM is used to perform a third feature selection on the code metric set to obtain the information gain ratio of each feature.

[0020] Furthermore, weights are fused based on mutual information content, Gini index, and information gain ratio. The fused weights are then sorted in descending order, and the last 20% of the features are removed. The remaining features are used as features for the code metric and odor label datasets to obtain a new dataset.

[0021] Furthermore, the new dataset was divided into a 70% training set and a 30% test set, and stratified sampling was used in the process to make the ratio of positive and negative samples the same as that of the original code metric and odor label dataset.

[0022] Furthermore, Random Forest, SVM, and LightGBM were selected as base classifiers for the Stacking heterogeneous algorithm. Random Forest, SVM, and LightGBM were trained sequentially on the training set, and the optimal parameters were found to obtain a base classifier with the best parameters.

[0023] Technical effect

[0024] This invention discloses a method for detecting complex code smells based on feature weight fusion and stacking heterogeneous approaches. Feature weight fusion integrates feature selection methods from different underlying principles to obtain the average importance of each feature, allowing for more reasonable feature selection and improving model generalization. Stacking heterogeneous approaches combine multiple different types of machine learning models, making the model's boundaries more stable and avoiding the problems of poor prediction performance and generalization of single homogeneous models. By combining these two methods, a better detection and generalization capability can be achieved in detecting complex code smells.

[0025] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating a preferred embodiment of a method for detecting odors in complex class code according to the present invention. Detailed Implementation

[0027] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0028] In the following description, specific details, such as particular internal procedures and techniques, are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will appreciate that the invention may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of the invention with unnecessary detail.

[0029] like Figure 1 As shown, this invention provides a method for detecting code odors in complex classes, comprising the following steps:

[0030] Step 100: Obtain code metrics from the source code files using the CK tool and map them one-to-one with the labels to form a code metric and anomaly label dataset; specifically, in a given Java open-source software system, there exists a set F = {F_m × n Java open-source software system version files}. 1,1 ,F 1,2 ,…,F i,j ,…,F m,n}, where m represents the number of Java open-source software systems, n represents the number of versions, and F i,j Let F represent the j-th version file of the i-th Java open-source software system. Use the CK tool to parse each source code file F. i,j The code metrics in the dataset are combined with the corresponding labels to form a code metric and odor label dataset, represented as D. i,j =<ClassName,Label,CodeMetric> Where ClassName represents the source code file version F i,j The class name is specified in the code. The Label indicates whether the class is a complex class (1 for complex, 0 for non-complex). CodeMetric indicates the source code version. i,jThe code metrics set C. The CK tool open-source project address is https: / / github.com / mauricioaniche / ck.

[0031] Code metrics set C = <C CBO C NOC ,…,C LOC C LCOM There are a total of 47 class-level code metrics. Due to the large number of features, they are not listed individually, among which C... CBO Indicates the degree of coupling between classes, C NOC Represents the number of children, ..., C LOC Indicates the number of lines of code, C LCOM The degree of cohesion in the representation method.

[0032] Step 200: Preprocess the obtained code metrics and odor label datasets; use median imputation to handle features C with missing values. LOCM C TCC C LCC Perform padding to ensure that dataset D i,j The completeness of this facilitates subsequent feature selection and model training processes, where C TCC C represents the cohesion tightness within a class. LCC Indicates the degree of cohesion within a class.

[0033] Using Min-Max on dataset D i,j Standardize it, as in formula (1):

[0034]

[0035] Where X represents the sample data, X * This represents standardized data, where min represents the minimum value of the sample data and max represents the maximum value of the sample data.

[0036] Step 300 involves performing a first, second, and third feature selection on the preprocessed code metric and odor label datasets, respectively, to obtain the feature importance of each feature according to the three feature selection methods. The reason for performing three feature selections on the code metric and odor label datasets is that the three feature selection methods—filtering, wrapping, and embedding—are respectively employed, obtaining different feature importances through different underlying principles, thus providing a more objective assessment of the importance of each feature. Specifically, the filtering method calculates the correlation between all features and the classification label; the wrapping method evaluates the importance of the selected feature subset based on the predictive performance of the classification algorithm; and the embedding method combines both, embedding the feature selection process into the classification algorithm to select the optimal feature subset.

[0037] The importance of the three features is then summed, and the average importance of each feature is calculated. Mutual Information, RFE, and LightGBM are used to perform initial feature selection on the preprocessed code metric and odor label datasets to obtain the feature importance of all features corresponding to the three feature selections. The importance of the three features is then summed, and the average importance of each feature is calculated. Specifically, in the first feature selection, the Mutual Information method is used, as shown in formula (2):

[0038]

[0039] Where I(X;Y) represents the mutual information between X and Y, p(x,y) represents the joint probability distribution function of X and Y, p(x) represents the marginal probability distribution function of X, and p(y) represents the marginal probability distribution function of Y. After performing the first feature selection on the code metric set C using Mutual Information, the mutual information I between each feature and the label is obtained. CBO ,I NOC ,…,I LOC ,I LCOM >, where I X This represents the mutual information between feature X and label. The greater the mutual information, the stronger the effect of the feature on label.

[0040] In the second feature selection, the RFE (Recursive Feature Elimination) method is adopted. RFE uses the current classifier to search for a subset of features starting from all features in the training dataset and calculates the importance of each feature. Here, the random forest classifier is used as the current classifier, and its feature importance is calculated as shown in formula (3):

[0041]

[0042] Among them Gini(D) A () indicates that dataset D is divided into sample sets D based on feature A. A The Gini index after that, K represents the number of categories, |C k | represents the number of samples belonging to category k, |D A | represents the number of data sets after being partitioned by feature A. After performing a second feature selection on the code metric set C using RFE, the Gini index G for each feature is obtained. <G CBO G NOC ,…,G LOC G LCOM >, where G X ​The Gini index represents the degree of uncertainty of feature X. The larger the Gini index, the stronger the classification effect of the feature on the label.

[0043] In the third feature selection, the LightGBM method is adopted. LightGBM uses the information gain ratio of features to perform subset partitioning, and the information gain ratio is calculated as shown in formula (4):

[0044]

[0045] Where RatioGain(D) A Gain(D) A () represents the dataset D being partitioned into sample sets D based on feature A. A Information gain ratio and information gain, IV(A) represents the information entropy of feature A, K represents the number of categories, |D A | represents the number of datasets after being partitioned by feature A, and |D| represents the number of datasets. After performing a third feature selection on the code metric set C using LightGBM, the information gain ratio R of each feature is obtained. <R CBO ,R NOC ,…,R LOC ,R LCOM >, where R X The information gain ratio indicates the degree of uncertainty of feature X. The higher the information gain ratio, the stronger the classification effect of the feature on the label.

[0046] Step 400: Sort the average importance of each feature in descending order, remove the bottom 20% of features, and make corresponding changes to the dataset, including only the remaining 80% of features; perform weight fusion based on mutual information, Gini index, and information gain ratio, and sort the fused weights in descending order, removing the bottom 20% of features from the weighted dataset. The remaining features will be used as features for the code metric and odor label datasets, resulting in a new dataset. Specifically,

[0047] The weights of I, G, and R are combined as shown in formula (5):

[0048]

[0049] The fusion weight W is obtained. <W CBO W NOC ,…,W LOC W LCOM > where n represents the number of feature selection methods used, FSM iThis represents the feature importance obtained by the i-th feature selection method. The fusion weights W are sorted in descending order, and the bottom 20% of features with the lowest weights are removed. The remaining features will be used as the features of dataset D, resulting in a new dataset D' =<ClassName,Label,CodeMetric’> Where CodeMetric' is a proper subset of CodeMetric, and its code metric set C' = <C CBO C NOC A total of 36 code metrics features, ,…>.

[0050] The new dataset D' is divided into a 70% training set ξ and a 30% test set η, and stratified sampling is used in the process to make the ratio of positive to negative samples the same as that of dataset D.

[0051] Step 500: For the remaining 80% of features, three machine learning algorithms are selected and trained on the new dataset. The three models are then integrated using the Stacking method to obtain the final detection method. Random Forest, SVM, and LightGBM are selected as base classifiers for the Stacking heterogeneous algorithm. Random Forest, SVM, and LightGBM are trained sequentially on the training set, and the optimal parameters are found to obtain the base classifier with the best parameters. The process of selecting the above three base classifiers follows three criteria:

[0052] (1) The selected base classifier should be a recognized classification technique, such as random forest or SVM;

[0053] (2) The base classifier should cover different classification clusters, such as those based on geometric margin, voting, and boosting;

[0054] (3) Select representative classification algorithms from the classification clusters. Random Forest, SVM, and LightGBM meet the above criteria and can therefore be used as base classifiers. During training, the training set ξ is divided into 10 parts, with one part used as the test set η' and the remaining 9 parts used as the training set ξ', to obtain the prediction results λ' of each base classifier on the test set η'.

[0055] The above process is repeated 10 times for each of Random Forest, SVM, and LightGBM. Finally, the prediction results λ' obtained from each iteration are combined to obtain a base dataset λ = <λ' with the same length as dataset D' but a different number of features. rf ,λ svm ,λ lgb >, where λ x This represents the prediction result of the base classifier x on the training set ξ using a 10-fold test.

[0056] The meta-classifier of the Stacking algorithm should be as simple as possible. Therefore, a decision tree is chosen as the meta-classifier for the Stacking heterogeneous algorithm. The base dataset λ is used as the input to the decision tree meta-classifier. That is, the prediction results of Random Forest, SVM, and LightGBM on the training set ξ are used as new features and trained by the decision tree. Finally, the prediction results of the Stacking heterogeneous model are obtained by testing on the test set η, and the performance is evaluated by precision, recall, F-measure, and AUC.

[0057] To illustrate the effectiveness of the present invention, under the same conditions and with the same experimental data, the present invention was compared with those without feature fusion but using Stacking heterogeneous and those with feature fusion but using Stacking homogeneous. The comparison results were quantified by precision, recall, F-measure, and AUC.

[0058] As shown in Tables 1 and 2, the method for detecting anomalies in complex code according to the present invention has better precision, recall, F-measure, and AUC compared with methods without feature fusion and without stacking heterogeneity, thus verifying the effectiveness of the present invention.

[0059] Table 1 Comparison of the present invention with single feature selection technology

[0060]

[0061] Table 2 Comparison of the present invention with Stacking isomorphism

[0062]

[0063] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for detecting code odors in complex classes, characterized in that, Includes the following steps: The CK tool is used to obtain code metrics from source code files and correspond them one-to-one with labels to form a code metric and odor label dataset. Data preprocessing is performed on the obtained code metrics and odor label datasets; The preprocessed code metric and odor label datasets were subjected to first, second and third feature selections to obtain the feature importance of each feature corresponding to the three feature selections. The three feature importances were summed and the average importance of each feature was calculated. The average importance of each feature is sorted in descending order, the bottom 20% of features are removed, and the dataset is modified accordingly to include only the remaining 80% of features. For the remaining 80% of features, three machine learning algorithms were selected to train the dataset, and the three models were integrated using the Stacking method to obtain the final detection method.

2. The method for detecting code odors in complex classes as described in claim 1, characterized in that, The CK tool is used to obtain code metrics from source code files and match them one-to-one with labels to form a code metric and odor label dataset. In a given Java open source software system, there exists a set of Java open source software system version files. The CK tool is used to parse the code metrics in each source code file in the Java open source software system version file set and combine them with the corresponding labels to form a code metric and odor label dataset.

3. The method for detecting code odors in complex classes as described in claim 1, characterized in that, The obtained code metrics and odor label datasets were preprocessed, specifically by using median imputation to fill in features with missing values ​​to ensure the integrity of the dataset. Then, Min-Max was used to standardize the code metrics and odor label datasets.

4. The method for detecting code odors in complex classes as described in claim 1, characterized in that, MutualInformation, RFE, and LightGBM were used to perform initial feature selection on the preprocessed code metrics and odor label datasets to obtain the feature importance of all features corresponding to the three feature selections. The three feature importances were summed and the average importance of each feature was calculated.

5. The method for detecting code odors in complex classes as described in claim 4, characterized in that, MutualInformation is used to perform the first feature selection on the code metric set to obtain the mutual information between each feature and label.

6. The method for detecting code odors in complex classes as described in claim 5, characterized in that, A second feature selection was performed on the code metric set using RFE to obtain the Gini index for each feature.

7. The method for detecting code odors in complex classes as described in claim 6, characterized in that, LightGBM is used to perform a third feature selection on the code metric set to obtain the information gain ratio of each feature.

8. The method for detecting code odors in complex classes as described in claim 7, characterized in that, The weights are fused based on mutual information, Gini index, and information gain ratio. The fused weights are then sorted in descending order, and the bottom 20% of features with the lowest weights are removed. The remaining features are used as features for the code metric and odor label datasets to obtain a new dataset.

9. The method for detecting code odors in complex classes as described in claim 8, characterized in that, The new dataset is divided into a 70% training set and a 30% test set. Stratified sampling is used in this process to ensure that the ratio of positive to negative samples is the same as that of the original code metric and odor label datasets.

10. The method for detecting code odors in complex classes as described in claim 9, characterized in that, Random Forest, SVM, and LightGBM were selected as base classifiers for the Stacking heterogeneous algorithm. Random Forest, SVM, and LightGBM were trained sequentially on the training set, and the optimal parameters were found to obtain a base classifier with the best parameters.

Images