Android code smell coexistence detection method integrating static program analysis and machine learning

Abstract

The present invention relates to an Android code smell coexistence detection method that integrates static program analysis and machine learning. The purpose of the present invention is to solve the problem of low accuracy of existing methods in detecting Android code smell coexistence. The process is as follows: 1. Obtain a sample set as a labeled data set: download Android applications from an open source code library as a code corpus; parse the Android application source code to generate an abstract syntax tree of the code; use a static program analysis method to detect MC smells; based on the MC smell detection results, use a static program analysis method to detect the coexistence of MIM and MC smells; use two nested hash tables to store the smell detection results; generate a sample set based on automatically generated positive and negative samples; 2. Obtain a processed data set; 3. Obtain a trained deep neural network classifier; 4. Detect whether code smell coexistence exists in the Android code smell to be tested. The present invention is used in the field of code smell coexistence detection.

Description

Technical Field

[0001] The present invention relates to a method for detecting coexistence of Android code odors. Background Art

[0002] Code smells are poorly structured code segments in software systems. In 1999, Martin Fowler coined the term "code smell" to describe code structures in object-oriented programs that fail to meet specific quality requirements. This type of smell is also known as object-oriented code smell. Code smells reduce the maintainability and understandability of applications, hindering software maintenance and evolution. In recent years, the rapid development of mobile application technology has led to an increasing reliance on mobile applications. According to statistics, Android holds the largest market share of all mobile operating systems, exceeding 85%. Research has shown that code smells exist not only in traditional desktop applications but also in Android applications. Due to the differences in hardware and software between traditional desktop applications and Android applications, code smells in Android applications differ from those in object-oriented applications and are more complex. Android applications present not only traditional object-oriented code smells but also Android-specific code smells. In 2014, researchers such as Reimann proposed the concept of Android-specific code smells and provided a list of these smells. This odor has many negative impacts on Android applications, such as energy consumption, security, stability, memory and startup time, which reduces the user experience and is not conducive to the maintenance and evolution of Android applications.

[0003] Code smell coexistence occurs when two or more code smells coexist within a single method or class. Research shows that coexistence is more detrimental to applications than a single smell. Studies have shown that coexistence increases the difficulty for developers to understand a program, complicates maintenance, and increases the likelihood of introducing bugs. Researchers Hamdi et al. conducted an in-depth study of code smells in Android applications. Their results showed that approximately 51% of classes containing code smells were affected by two or more code smells, indicating that code smell coexistence is ubiquitous in programs. Researchers Manman et al. also showed that object-oriented code smells in Android applications can be detected using traditional code smell detection tools. However, traditional tools cannot detect Android-specific code smells. Code smell coexistence in Android applications falls into three categories: coexistence of traditional object-oriented code smells, coexistence of Android-specific code smells, and coexistence of both object-oriented and Android-specific code smells. As a preliminary study, this paper focuses on the coexistence of object-oriented and Android-specific code smells. Research by Hamdi et al. shows that four pairs of object-oriented code smells coexist with Android-specific code smells in Android applications. As a preliminary exploration, this paper investigates one pair of these smells, which coexists more frequently. These code smells are the over-coupled message chains (MC) smell and the member ignoring method (MIM) smell. MC is a traditional object-oriented code smell, while MIM is an Android-specific code smell. The presence of this pair of smells significantly impacts the energy consumption of Android applications, reducing program understandability and maintainability. Currently, the best tool for detecting Android-specific code smells is DAAP (Detection of Android Application), which has an average accuracy rate of over 90% for detecting MIM smells. DAAP utilizes static program analysis techniques commonly used by traditional detection tools, combining detection rules with code metrics to identify smells by traversing the abstract syntax tree. However, DAAP can only detect a single type of Android-specific code smell and cannot detect object-oriented code smells or the coexistence of smells.

[0004] Research has shown that a major limitation of static program analysis is its heavy reliance on thresholds and / or rule combinations. Application developers are subjective in setting rules and choosing metrics, leading to discrepancies in detection results even for the same code smell. To alleviate these limitations, machine learning has attracted interest among researchers.

[0005] In recent years, with breakthroughs and widespread application in machine learning, researchers have applied machine learning algorithms to the detection of traditional object-oriented code smells, achieving promising results. Despite their numerous advantages, one of the main limitations of current machine learning-based approaches is their heavy reliance on manual data annotation. Training an effective machine learning model requires a large amount of labeled data. In the field of code smell detection, there is currently a lack of publicly available, standardized datasets for researchers to utilize. Furthermore, existing research has primarily focused on traditional object-oriented code smells, with limited research on code smells in Android applications.

[0006] Existing methods have studied the detection methods of Android-specific code smells, but they only detect a single type of code smell, without considering the coexistence of smells or object-oriented code smells. Summary of the Invention

[0007] The purpose of this invention is to address the problem that existing methods for detecting Android-specific code smells only detect a single type of code smell, do not consider the coexistence of code smells, and do not consider object-oriented code smells, resulting in low accuracy in detecting the coexistence of Android code smells. Instead, this invention proposes an Android code smell coexistence detection method that integrates static program analysis and machine learning.

[0008] The specific process of the Android code smell coexistence detection method that integrates static program analysis and machine learning is as follows:

[0009] Step 1: Obtain a sample set as a label dataset; the specific process is:

[0010] Step 1. Download the Android application developed in Java language from the open source code library GitHub as the code corpus;

[0011] Step 1 and 2: Use JavaParser to parse the Android application source code and generate the abstract syntax tree of the code;

[0012] Step 13: Use static program analysis methods to detect MC odors; Based on the MC odor detection results, use static program analysis methods to detect the coexistence of MIM and MC odors;

[0013] Step 14: Use two nested hash tables to store the odor detection results;

[0014] Step 15: Automatically generate positive and negative samples based on step 14, and generate a sample set based on the positive and negative samples as a label data set;

[0015] Step 2: Process the sample set to obtain a processed data set;

[0016] Step 3: Build a deep neural network classifier and obtain a trained deep neural network classifier based on the processed data set;

[0017] Step 4: Input the Android code smell to be tested into the trained deep neural network classifier, and output whether MIM and MC code smells coexist in the Android code smell to be tested.

[0018] The beneficial effects of the present invention are:

[0019] Compared to a single code smell, the coexistence of code smells is more harmful to a program. Existing research on Android-specific code smells primarily focuses on detecting a single type of smell, ignoring the negative impact of coexisting smells on Android applications. To identify coexisting Android-specific code smells, this paper proposes a coexistence detection method for Android code smells that combines static program analysis with machine learning. As a preliminary exploration, the coexistence of the overly coupled message chain smell and the method smell of ignoring class member variables is detected. First, an algorithm for detecting the overly coupled message chain smell based on static program analysis is proposed. Building on this, a method for detecting the coexistence of the overly coupled message chain smell and the method smell of ignoring class member variables is proposed, also based on static program analysis. Furthermore, to provide abundant training samples for subsequent machine learning models, a method for automatically generating positive and negative samples is proposed and implemented as an ASSD tool. Then, nine machine learning models are used to identify coexisting code smells: six traditional machine learning models and three improved deep learning models. Experimental results show that the random forest model performs best, achieving a 9.1% improvement in F1 score compared to manual detection.

[0020] This paper combines static program analysis with machine learning to detect the coexistence of object-oriented code smells and Android-specific code smells in Android applications. The main contributions of this paper are as follows:

[0021] (1) A method for detecting over-coupled message chain smell based on static program analysis is proposed.

[0022] (2) A method for identifying coexistence of Android code smells based on static program analysis is proposed.

[0023] (3) A method for automatic sample generation is proposed. (2) is combined with (3) to achieve automatic completion from odor coexistence detection to sample generation and implement the tool ASSD (Android-Specific Smell Detector).

[0024] (4) A coexistence detection method for Android code smells that combines static program analysis and machine learning is proposed.

[0025] Compared to a single code smell, the coexistence of code smells is more harmful to a program. Existing research on Android-specific code smells primarily focuses on detecting a single type of smell, ignoring the negative impact of coexisting smells on Android applications. To identify coexisting Android-specific code smells, this paper proposes a coexistence detection method for Android code smells that combines static program analysis with machine learning. As a preliminary exploration, the coexistence of the overly coupled message chain smell and the method smell of ignoring class member variables is detected. First, an algorithm for detecting the overly coupled message chain smell based on static program analysis is proposed. Building on this, a method for detecting the coexistence of the overly coupled message chain smell and the method smell of ignoring class member variables is proposed, also based on static program analysis. Furthermore, to provide abundant training samples for subsequent machine learning models, a method for automatically generating positive and negative samples is proposed and implemented as an ASSD tool. Then, nine machine learning models are used to identify coexisting code smells: six traditional machine learning models and three improved deep learning models. Experimental results show that the random forest model performs best, achieving a 9.1% improvement in F1 score compared to manual detection.

[0026] This paper uses a method based on static program analysis combined with machine learning to detect the coexistence of object-oriented code smells and Android-specific code smells in Android applications, exploring the feasibility of using machine learning methods to identify the coexistence of Android-specific code smells. As a preliminary exploration, the coexistence of the over-coupled message chain smell and the method smell of ignoring class member variables was identified. First, a static program analysis-based MC smell detection method was proposed. Based on this, a static program analysis-based method for detecting the coexistence of Android code smells and a method for automatically generating positive and negative samples were proposed, along with a tool (ASSD). The tool's output provides a rich set of training samples for subsequent machine learning models. Nine machine learning classifiers were then used to detect coexistence smells, including six traditional machine learning models and three improved deep learning models. Experimental results show that among the nine machine learning models, the random forest model performed best in detection, outperforming the deep learning model. Furthermore, compared to manual detection methods, the proposed method achieved a 9.1% improvement in the F1 score. Therefore, machine learning methods can be used to detect the coexistence of Android-specific code smells. Among the nine machine learning models, the RF model demonstrated superior detection accuracy and time. Therefore, it is recommended to use the RF model when detecting the coexistence of MC and MIM smells in Android applications.

[0027] The proposed method for detecting coexisting code smells provides a new approach to detecting coexisting code smells. In subsequent work, we will investigate optimization methods for coexisting code smells, thereby refactoring coexisting smells to improve the quality of mobile software. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Flowchart of the present invention;

[0029] Figure 2 An overview of the MC and MIM coexistence detection method that combines static program analysis with machine learning;

[0030] Figure 3 Flowchart for odor coexistence detection and automatic generation of positive and negative samples;

[0031] Figure 4 Tokenizer markup code example diagram;

[0032] Figure 5 It is a code text information data format diagram;

[0033] Figure 6 This is the structure diagram of the neural network classifier based on CNN;

[0034] Figure 7 This is the structure diagram of the deep neural network classifier based on RNN;

[0035] Figure 8 This is the structure diagram of the deep neural network classifier of CNN-LSTM. DETAILED DESCRIPTION

[0036] Specific implementation method 1: This implementation method integrates static program analysis and machine learning to detect the coexistence of Android code odors. The specific process is as follows:

[0037] 1. Related Work

[0038] 1.1 Object-oriented code smell

[0039] In 1999, Martin Fowler proposed the concept of "code smell" in order to explain the design problems of code in traditional desktop applications. Code smells are not bugs, but they indicate possible problems with the code or areas that need improvement. Code smells increase the burden of program maintenance and hinder the evolution of the program. Fowler listed 24 code smells in the second edition of his classic book "Refactoring: Improving the Design of Existing Code". For example, too long method smells, over-coupled message chain smells and duplicate code smells. These code smells were later referred to as object-oriented (OO) code smells. Object-oriented code smells are usually caused by developers' excessive pursuit or improper use of certain principles or programming paradigms. For example, excessive use of design-style, inheritance and generic methods may reduce the readability and maintainability of the code. The MC smell studied in this invention is object-oriented code smell.

[0040] 1.2. Android-specific code smells

[0041] In recent years, with the advancement of mobile application technology, mobile applications have evolved from simple applications to rapidly evolving complex systems. For example, in 2021, the number of mobile applications in various app stores exceeded 5 million, and the number of app downloads exceeded 23 billion. To meet this growing demand, mobile applications are developed at a rapid pace and continuously evolve to meet user needs. However, rapid development within a limited timeframe to address bugs or add missing features can lead to poor design or implementation choices, thereby increasing technical debt. Android-specific code smells, also known as Android code smells, are code segments in Android applications that violate code quality requirements. Unlike traditional object-oriented code smells, Android-specific code smells do not affect the functionality of Android applications, but they increase system memory consumption, reduce application performance, and increase system energy consumption. In 2014, researchers Reimann et al. linked the concepts of smell, quality, and refactoring to propose the concept of "quality smells," representing code structures in Android applications that affect software quality and provide a list of these smells. These quality smells have subsequently been referred to as Android-specific code smells. This code smell has many negative impacts on Android applications, such as energy consumption, security, stability, memory usage, and startup time, which is detrimental to the maintenance and evolution of Android applications. Therefore, detecting and refactoring code smells in the early stages of design is very important to improve the quality and performance of the code and the entire system.

[0042] Manman et al. found that traditional tools for detecting object-oriented code smells are incapable of detecting Android-specific code smells. Therefore, researchers have proposed various methods for detecting Android-specific code smells. Palomba et al. proposed a method that combines code metrics with simple text comparison to detect 15 Android code smells and implemented the tool aDoctor. aDoctor can detect a single type of Android-specific code smell or a combination of Android-specific code smells. Although aDoctor is free software, its source code is not publicly available. Iannone et al. extended aDoctor's progress by refactoring five Android-specific code smells related to energy consumption. To detect a wider range of Android code smells, Rasool et al. defined heuristic detection rules, combined them with code metrics, and implemented the tool DAAP. DAAP outperforms aDoctor in both the variety of smells detected and accuracy. The source code is publicly available to facilitate subsequent use and functional expansion by researchers. Recently, researchers such as Prestat proposed a method for dynamically detecting Android-specific code smells. However, this method focuses on bytecode rather than source code. Therefore, when a smell is detected, it cannot pinpoint its location. The MIM smells studied in this paper are Android-specific code smells.

[0043] 1.3. Coexistence of code smells

[0044] In software systems, when a class or method is simultaneously affected by two or more code smells, this phenomenon is called code smell coexistence. Research indicates that coexisting code smells are more detrimental to software systems than a single code smell. Research by Palomba et al. shows that classes affected by multiple code smells are 350% more likely to change and 400% more likely to malfunction than classes affected by a single code smell. Furthermore, code smell coexistence complicates developers' understanding of source code. In 2017, Palomba et al. used association rule learning to study 13 object-oriented code smells and found that six pairs of smells coexisted frequently. A year later, they conducted in-depth research on the frequency, symptoms, and causes of code smell coexistence. In 2021, researchers Hamdi et al., through extensive experimental research, confirmed the prevalence of code smell coexistence in Android applications. The results showed that over 50% of classes are affected by two or more code smells. Among existing methods, only aDoctor can detect the coexistence of Android-specific code smells. This method uses traditional static program analysis methods to identify coexisting smells, but it cannot detect object-oriented code smells.

[0045] 1.4. Odors studied in this invention

[0046] Research has shown that four pairs of object-oriented code smells coexist in Android applications, along with Android-specific code smells. As a preliminary study, this paper focuses on one pair of these smells, which coexists most frequently. These two pairs are MC and MIM. MC is an object-oriented code smell, while MIM is an Android-specific code smell.

[0047] 1.4.1. Over-coupled message chain smell MC

[0048] The overly coupled message chain smell is an object-oriented code smell that indicates a long chain of method calls is required to implement a class's functionality. In this scenario, a method call on one object triggers a method call on another object, forming a "message chain." The presence of the MC smell can lead to decreased code readability, increased coupling, and maintenance difficulties. Below is a code snippet that exhibits the MC smell.

[0049]

[0050]

[0051] In line 10 of the code, the getB() method in class A returns an instance of class B. In line 16 of the code, the getC() method in class B returns an instance of class C. In the main() method (line 4), these methods are called through a.getB().getC().getValue() to form a message chain.

[0052] 1.4.2. Method Smell MIM Ignoring Class Member Variables

[0053] Ignoring class member variables is a unique Android code smell. A MIM is a method of a class in an Android application that is neither empty nor static, and does not include methods where a subclass overrides a superclass method, but does not use any properties of its class. Research has shown that MIM is the most frequently occurring code smell in Android applications, and has therefore attracted considerable attention from the academic community. Tian Yingchen et al. proposed a criticality quantification method to quantify the severity of code smells. Using this method, MIM is the most critical of the many Android-specific code smells. A more critical smell indicates a greater impact on the application and should be prioritized during refactoring. Furthermore, the MIM smell affects the energy consumption and maintainability of Android applications. Below is a code snippet containing the MIM smell.

[0054]

[0055] The calculateDiscount() method on line 8 does not use the username or age properties defined in its UserProfile class, so this method exhibits a MIM smell. This method should not be part of the UserProfile class because it is not related to the user's profile. This method could be refactored to a more appropriate location, such as a utility class or as a static method.

[0056] 1.4.3 Coexistence of MC and MIM Odors

[0057] The following is a code example of the coexistence of MIM and MC.

[0058]

[0059] On line 4, the processUser() method fails to use the Address and username variables defined in the UserInfo class, generating a MIM code smell. Line 5 calls these methods using the user.getAresses().stream().filter().map().orElse() method, forming a coupled message chain. Although this short code segment exhibits both code smells.

[0060] 1.5 Code Smell Detection Method Based on Static Program Analysis

[0061] Program static analysis is a method for analyzing source code without executing the program. It examines the structure, syntax, and semantics of the code itself to identify potential code issues, errors, or optimization opportunities. In the study of program static analysis, researchers have proposed many different static analysis techniques and methods, such as abstract syntax trees, data flow analysis, symbolic execution, model checking, and type inference. These techniques can be applied to different programming languages ​​and software systems and can be customized and extended according to needs. Static program analysis is a commonly used method for detecting code smells. Existing tools for detecting Android code smells, DAAP and aDoctor, both adopt this approach. While DAAP performs well in detecting a single type of smell, it cannot detect the coexistence of smells. aDoctor can detect the coexistence of smells, but only between Android-specific code smells and cannot detect traditional object-oriented code smells.

[0062] Static analysis-based methods primarily employ two approaches to code smell detection. One approach involves detecting code smells using a series of thresholds that rely on structural parameters (i.e., software metrics) to categorize and describe the symptoms of specific code smells. The other approach involves detecting code smells through predefined rules or heuristics. The main limitation of these methods is that their performance relies heavily on thresholds and / or combined rules. Until now, experts have not reached a consensus on standard values ​​and rules. To alleviate these limitations, machine learning methods have attracted the interest of researchers in recent years.

[0063] 1.6 Code Smell Detection Method Based on Machine Learning

[0064] In recent years, machine learning research has made tremendous progress. Software engineering researchers have also attempted to use this technology to solve software engineering problems such as software testing, source code representation, source code quality analysis, refactoring, and code smell detection, achieving excellent results. Khomh et al. used a Bayesian belief network model to detect code smells. Abdou et al. used support vector machines to detect three code smells: blob, attachment complexes, and spaghetti code. Fontana et al. used 16 traditional machine learning models to detect four object-oriented code smells, comparing the effectiveness of different algorithms. Amorin et al. combined decision trees with genetic algorithms to detect code smells. Kim et al. used a multilayer perceptron to detect six code smells (God classes, long methods, attachment complexes, pure data classes, redundant classes, and parallel inheritance) with high accuracy. Kuar et al. combined the SVM algorithm with k-means clustering to detect four object-oriented code smells and provide refactoring strategies. These studies show that while machine learning methods are helpful for code smell detection and have high accuracy, these methods focus on a single type of object-oriented (OO) code smell. Therefore, it remains unknown whether machine learning algorithms can detect the coexistence of object-oriented and Android-specific code smells in Android applications. Despite their many advantages, one of the main limitations of current machine learning-based methods is their heavy reliance on manual data annotation. Training an effective machine learning model requires a large amount of labeled data. In the field of code smell detection, there is currently a lack of publicly available, standardized datasets for researchers to use, and manually labeling data is both expensive and infeasible.

[0065] In summary, static program analysis or machine learning methods alone have their own limitations when detecting the coexistence of code smells in Android applications. Therefore, this paper proposes a method for detecting the coexistence of code smells in Android applications that combines static program analysis with machine learning. First, a method for detecting MC smells using static program analysis is proposed. Then, based on this method, a method for detecting the coexistence of MC and MIM smells using static program analysis is proposed. Furthermore, to quickly and accurately provide training samples for subsequent machine learning models, a method for automatically generating positive and negative samples is proposed, automating the process from code smell coexistence detection to sample generation, and implementing the ASSD tool. Then, nine machine learning models are used to identify coexisting code smells. These include six traditional machine learning models and three improved deep learning models. The six traditional machine learning models are Naive Bayes (NB), Decision Tree (DT), Random Tree (RT), Random Forest (RF), Rule-based Induction (JRIP), and Bayesian Network (BN). The three improved deep learning models are an improved convolutional neural network model, an improved recurrent neural network model, and an improved CNN-LSTM network model. Machine learning methods can automatically extract and learn high-level features by learning features from large amounts of data, which are not easily captured by traditional static program analysis methods. This method provides a new approach for detecting the coexistence of object-oriented code smells and Android-specific code smells, preparing for subsequent refactoring and ultimately improving the quality of mobile software.

[0066] Step 1: Obtain a sample set as a label dataset; the specific process is:

[0067] Step 1. Download the Android application developed in Java language from the open source code library GitHub as the code corpus;

[0068] Step 1 and 2: Use JavaParser to parse the Android application source code and generate the abstract syntax tree of the code;

[0069] Step 13: Use static program analysis methods to detect MC odors; Based on the MC odor detection results, use static program analysis methods to detect the coexistence of MIM and MC odors;

[0070] Step 14: Use two nested hash tables to store the odor detection results;

[0071] Step 15: Automatically generate positive and negative samples based on step 14, and generate a sample set based on the positive and negative samples as a label data set;

[0072] Step 2: Process the sample set to obtain a processed data set;

[0073] Step 3: Build a deep neural network classifier and obtain a trained deep neural network classifier based on the processed data set;

[0074] Step 4: Input the Android code smell to be tested into the trained deep neural network classifier, and output whether MIM and MC code smells coexist in the Android code smell to be tested.

[0075] Specific embodiment 2: This embodiment differs from specific embodiment 1 in that in step 1, an Android application developed in Java is downloaded from the open source code library GitHub as a code corpus; the specific process is as follows:

[0076] The code corpus was screened from GitHub according to the following criteria.

[0077] Step 1: The application in the code corpus is at least 1000KB;

[0078] Step 112: The number of stars of the application must be greater than 500;

[0079] The number of stars indicates the attention and popularity of the application, which indirectly reflects the quality of the application.

[0080] Step 113: The last update date of the Android application should be later than April 1, 2023.

[0081] Other steps and parameters are the same as those in the first embodiment.

[0082] Specific embodiment three: This embodiment differs from specific embodiment one or two in that, in steps one and three, a static program analysis method is used to detect MC odor; based on the MC odor detection results, a static program analysis method is used to detect the coexistence of MIM and MC odors;

[0083] The specific process is:

[0084] 1) Use static program analysis method to detect MC odor; the specific process is:

[0085] Use the Visitor Pattern to traverse all method calls in the Abstract Syntax Tree (AST);

[0086] Recursively trace the call chain of each method call until it reaches the starting point of the chain;

[0087] If the length of a call chain exceeds the set threshold (the default is 3 layers), the call chain is considered to be an over-coupled message chain smell and is marked as an MC smell;

[0088] The MC smell detection method based on static program analysis is shown in Algorithm 1.

[0089] 2) Based on the MC odor detection results, a static program analysis method is used to detect the coexistence of MIM and MC odors.

[0090] Other steps and parameters are the same as those in the first or second embodiment.

[0091] Specific embodiment 4: This embodiment differs from any one of specific embodiments 1 to 3 in that two nested hash tables are used in step 14 to store the odor detection results;

[0092] The specific process is:

[0093] Record the name of each code smell and the number of times each code smell appears in the current class in a hash table called map2;

[0094] Then, the classpath of each class is combined with map2 to form a more comprehensive hash table map1.

[0095] The other steps and parameters are the same as those in the first to third embodiments.

[0096] Specific embodiment 5: This embodiment differs from any one of specific embodiments 1 to 4 in that in step 15, positive and negative samples are automatically generated based on step 14, and a sample set is generated based on the positive and negative samples as a label data set;

[0097] The specific process is:

[0098] (1) Create the file directory smellyFile and the file directory nonsmellyFile respectively (corresponding to lines 2-5 and 12-15 in Algorithm 3);

[0099] The file directory smellyFile is used to store code segments containing MIM and MC coexisting smells;

[0100] The file directory nonsmellySets is used to store code segments that do not contain MIM and MC smells;

[0101] Create a file directory smellyFile for storing positive samples and a file directory nonsmellyFile for storing negative samples respectively;

[0102] (2) Traverse smellySets and write the code segments where MIM and MC smells coexist into a Java file to generate a positive sample set (corresponding to lines 6-11 in Algorithm 3);

[0103] (3) Traverse nonsmellySets and write the code segments where MIM and MC smells do not coexist into a Java file to generate a negative sample set (corresponding to lines 16-20 in Algorithm 3);

[0104] Generate a sample set based on positive samples and negative samples as a label dataset;

[0105] MC is an over-coupled message chain smell;

[0106] MIM is the method smell of ignoring class member variables.

[0107] The other steps and parameters are the same as those in the first to fourth embodiments.

[0108] In map1, not only the number of code smell types existing in each class path is recorded, but also the specific number of occurrences of each smell is listed in detail.

[0109] In this way, using map1 and map2 can not only detect a single code smell, but also further identify and detect the coexistence of code smells, as shown in Algorithm 2.

[0110] Two nested hash tables are used to store smell detection results. The hash tables store the following: classpath, the names of Android-specific code smells, and the number of occurrences of traditional object-oriented code smells. Because there is a one-to-many relationship between classpath and code smell name, and a one-to-one correspondence between code smell name and number, a single hash table can be used to record multiple code smells and their numbers. This hash table can then be associated with the corresponding classpath and stored in another hash table.

[0111] To detect the coexistence of MIM and MC smells, we first proposed using program static analysis methods to detect MC smells. Then, based on this, we proposed using static program analysis methods to detect the coexistence of MIM and MC smells, and implemented the tool ASSD. This DAAP expands its functionality to enable it to detect both MIM and MC smells individually, as well as the coexistence of the two. After implementing MC smell detection, to detect the coexistence of MC and MIM smells, we use two nested hash tables to store the detection results (without changing the original DAAP MIM smell detection rules). To distinguish it from the original DAAP tool, this improved tool is referred to as ASSD. Figure 3The figure shows the coexistence detection of Android code smells and the process of generating positive and negative samples based on static program analysis. The dotted part in the figure shows the original DAAP tool, and the functions outside the dotted line are the extended functions of the present invention.

[0112] Specific embodiment 6: This embodiment differs from specific embodiments 1 to 5 in that the data set is preprocessed in step 2 to obtain a preprocessed data set; the specific process is as follows:

[0113] Use the Tokenizer tool to convert positive and negative samples into integers and use digital vectors to represent program text information;

[0114] Remove duplicate samples in program text information;

[0115] Normalize the program text information represented by the digital vector;

[0116] The program text information represented by the normalized digital vector is used as the input data of the deep neural network classifier.

[0117] The other steps and parameters are the same as those in the first to fifth embodiments.

[0118] Specific embodiment seven: This embodiment is different from any one of specific embodiments one to six in that the deep neural network classifier in step three is any one of naive Bayes (NB), decision tree (DT), random tree (RT), random forest (RF), rule-based induction algorithm (JRip), Bayesian network (BN), improved convolutional neural network model, improved recurrent neural network model, and improved CNN-LSTM network model.

[0119] Naive Bayes (NB), decision tree (DT), random tree (RT), random forest (RF), rule-based induction algorithm (JRip), and Bayesian network (BN) are traditional machine learning models.

[0120] The improved convolutional neural network model, the improved recurrent neural network model, and the improved CNN-LSTM network model are deep learning models.

[0121] The other steps and parameters are the same as those in the first to sixth embodiments.

[0122] Specific embodiment eight: This embodiment is different from any one of specific embodiments one to seven in that the improved convolutional neural network model includes an embedding layer, a first convolutional layer, a first maximum pooling layer, a second convolutional layer, a second maximum pooling layer, a third convolutional layer, a third maximum pooling layer, a Dropout layer, a Flatten layer, a first fully connected layer, a second fully connected layer, and an output layer in sequence;

[0123] The mask_zero parameter is set to True in the embedding layer;

[0124] The dropout rate of the Dropout layer is set to 0.5;

[0125] The activation function of the first fully connected layer is the relu activation function;

[0126] The activation function of the second fully connected layer is a sigmoid function;

[0127] The improved convolutional neural network model CNN loss function is the binary_crossentropy function, and the optimizer is RMSProp;

[0128] The data set preprocessed in step 2 is input into the improved convolutional neural network model until the loss function converges to obtain a trained improved convolutional neural network model.

[0129] The other steps and parameters are the same as those in the first to seventh embodiments.

[0130] Specific embodiment nine: This embodiment differs from any one of specific embodiments one to eight in that the improved recurrent neural network model includes an embedding layer, a long short-term memory network LSTM, a flatten layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, and an output layer in sequence;

[0131] The number of neurons in the first fully connected layer is 64;

[0132] The number of neurons in the second fully connected layer is 32;

[0133] The units digit of the neurons in the third fully connected layer is 1;

[0134] The activation functions of the first fully connected layer and the second fully connected layer are relu functions;

[0135] The activation function of the third fully connected layer is a sigmoid function;

[0136] The output dimension of the long short-term memory network LSTM is set to 128;

[0137] The output of the third fully connected layer is a one-dimensional vector;

[0138] The loss function of the improved RNN model is binary_crossentropy function, and the optimizer is RMSProp;

[0139] The data set preprocessed in step 2 is input into the improved recurrent neural network model until the loss function converges to obtain a trained improved recurrent neural network model.

[0140] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0141] Specific embodiment ten: This embodiment differs from any one of specific embodiments one to nine in that the improved CNN-LSTM network model sequentially includes an embedding layer, a first convolutional layer, a first maximum pooling layer, a long short-term memory network LSTM, a Dropout layer, a Flatten layer, a first fully connected layer, and an output layer;

[0142] The improved CNN-LSTM network model uses early stopping (epoch 50 and patience 10) and model checkpoint callback;

[0143] To prevent overfitting and ensure optimal model performance, early stopping is used. This method allows the model to terminate training early if the validation loss does not decrease for 10 consecutive iterations before reaching the preset maximum number of iterations. This parameter is called patience in early stopping and is set to 10.

[0144] The number of filters in the first convolutional layer is 32, and the kernel size is 3;

[0145] The window size of the first maximum pooling layer is 2;

[0146] The output dimension of the LSTM layer is set to 128;

[0147] The dropout rate of the Dropout layer is set to 0.5;

[0148] The activation functions of the first convolutional layer and the first maximum pooling layer are relu activation functions;

[0149] The activation function of the LSTM layer is the sigmoid activation function;

[0150] The loss function of the improved CNN-LSTM network model is the binary_crossentropy function, and the optimizer is RMSProp;

[0151] The data set preprocessed in step 2 is input into the improved CNN-LSTM network model until the loss function converges to obtain a trained improved CNN-LSTM network model.

[0152] The number of neurons in the first fully connected layer is {1, 32, 64} respectively.

[0153] The other steps and parameters are the same as those in Specific Embodiments 1 to 9.

[0154] 2. Code Smell Coexistence Detection Methods:

[0155] The Android code odor coexistence detection method proposed in this paper combines static program analysis with machine learning. Figure 2 shown.

[0156] 2.2 Dataset Construction

[0157] 2.2.1 Acquisition of experimental subjects

[0158] The code corpus required for the experiments in this paper was obtained from GitHub. GitHub is a code hosting platform for version control and collaboration. Repositories are the basic unit for storing and managing code projects on GitHub. GitHub provides repositories as containers for hosting code projects, and users can perform various operations and interactions within these repositories through GitHub. Existing Android code smell detection tools can only detect applications developed in Java. Therefore, this paper selected Android applications developed in Java as the code corpus. The code corpus was screened from GitHub according to the following criteria. First, the application code base must be at least 1000KB. Second, the application must have more than 500 stars. The number of stars indicates the level of attention and popularity of the application, which indirectly reflects the quality of the application. Finally, the last update date of the Android application should be after April 1, 2023. The update time indicates the activity level and maintenance frequency of the application. Generally speaking, a frequently updated project indicates an active developer community and continuous improvement efforts. Therefore, for many users, update time is an important consideration when deciding whether to use the application. A search on GitHub based on the above criteria resulted in 70 Android projects1. Table 1 shows the information of seven of these applications. All application information has been uploaded to: https: / / github.com / biboyu / MC-IMDetection.

[0159] Table 1 Some Android application information

[0160]

[0161]

[0162] Although code smells can be identified using machine learning methods, one of the main limitations of current machine learning-based methods is that they rely heavily on manual labeling of data, and training an efficient machine learning model requires a large amount of labeled data. However, in the field of code smell detection, there is currently a lack of open standard datasets for researchers to use, and manually labeling data is both expensive and infeasible. In traditional application areas of machine learning, such as image processing and speech recognition, there are a large number of labeled datasets for researchers to use, but in the field of code smell detection, there are no similar resources for researchers to use. In order to overcome this difficulty, Liu Hui's team and Sharma's team first used existing code smell detection tools to find the smell of the source program, and then used the output results of the tool as the training set for the subsequent machine learning model. The present invention draws on this method and uses the tool ASSD independently developed by the research team of the present invention to identify the coexistence of MC and MIM smells and automatically generate positive and negative sample sets as input for the subsequent machine learning model.

[0163] 2.2.2 Detecting Coexisting Android Code Smells Using Static Program Analysis

[0164] Among the existing Android code smell detection tools, DAAP is capable of detecting Android-specific code smells. DAAP is an open source tool that users can expand upon based on their needs. Therefore, the present invention selects DAAP as the basic tool. However, DAAP can only detect a single type of Android-specific code smell, and cannot detect traditional object-oriented code smells, nor can it detect the coexistence of smells. In order to detect the coexistence of MIM and MC smells, we first propose to use a program static analysis method to detect MC smells, and then, based on this, we propose to use a static program analysis method to detect the coexistence of MIM and MC smells, and implement the tool ASSD. Based on this DAAP, we expand its functionality so that it can detect both MIM and MC smells individually, as well as the coexistence of the two smells. The MC smell detection method based on static program analysis is shown in Algorithm 1.

[0165] As shown in Algorithm 1, line 2 defines the maximum allowed call chain length, which defaults to 3. This means that when detecting message chains in the code, the developer sets a threshold, indicating the number of object method call chains allowed within the code. In this algorithm, the default maximum length is 3, meaning that if a call chain exceeds three levels, it will be marked as an "overly coupled message chain." Lines 4-7 check the command line input parameters to ensure that a Java source file path is passed. If the correct path is not passed, the program outputs a usage prompt and exits. Lines 8-10 use FileInputStream to read the Java source file from the input parameter filePath. JavaParse.parse uses a parsing library (such as JavaParser) to parse the Java source file into an abstract syntax tree (AST), which is stored in a CompilationUnit for subsequent traversal and analysis. Line 11 uses the MethodCallVisitor class to traverse all method call expressions in the AST tree. This visitor pattern automatically traverses each node in the Java code, searching for method calls. Line 16 of the code calls the get.MethodCallChain method to obtain the chain of the current method call (i.e., consecutive get(), set(), and other method calls). Line 17 of the code, If(chain.size()>MESSAGE_CHAIN_THRESHOLD): Determine whether the length of the call chain exceeds the threshold (3 layers). If it exceeds, it is marked as an over-coupled message chain. Lines 18-22 of the code find the chain call by checking the Scope of the method call (the preamble of the method call). For example, in a.getB().getC().getD(), a.getB() is the preamble of getC(), and a.getB().getC() is the preamble of getD(). Use a while loop to trace the entire call chain forward until the starting point of the chain (i.e., there is no more preamble call);

[0166] To summarize, the algorithm's workflow is as follows:

[0167] First, read the Java source file from the input parameter and parse it into an abstract syntax tree (AST);

[0168] Then, use the Visitor Pattern to traverse all method calls in the Abstract Syntax Tree (AST);

[0169] Next, recursively trace the call chain of each method call until the starting point of the chain;

[0170] If the length of a call chain exceeds a set threshold (the default is 3 layers), the call chain is considered to have an over-coupled message chain smell and is marked. After implementing MC smell detection, to achieve coexistence detection of MC and MIM smells, two nested hash tables are used to store the detection results (without changing the original DAAP MIM smell detection rules). To distinguish it from the original DAAP tool, this improved tool is called ASSD. Figure 3 The figure shows the coexistence detection of Android code smells and the process of generating positive and negative samples based on static program analysis. The dotted part in the figure shows the original DAAP tool, and the functions outside the dotted line are the extended functions of the present invention.

[0171]

[0172] like Figure 3 As shown in the figure, first, the input of the tool is Java source code. The source code is converted into an abstract syntax tree by JavaParser. Then, the syntax tree is traversed according to the smell detection rules and code metrics to obtain the detection results. Next, two nested hash tables are used to store the smell detection results. The following content is stored in the hash table: class path, the names of Android-specific code smells and traditional object-oriented code smells, and their occurrence counts. Since the class path and the code smell name are in a one-to-many relationship, and the code smell name and the number are in a one-to-one correspondence, a hash table can be used to record multiple code smells and their numbers, and then this hash table can be associated with the corresponding class path and stored in another hash table.

[0173] During the detection process, the name of each code smell and its occurrence count within the current class are recorded in a hash table called map2, which serves as a statistical basis. Subsequently, each class's classpath is combined with map2 to form a more comprehensive hash table, map1. Map1 not only records the number of code smell types present in each classpath but also lists the specific occurrence counts for each smell. This approach allows map1 and map2 to not only detect individual code smells but also identify and detect the coexistence of code smells. This is shown in Algorithm 2.

[0174]

[0175] Manually inputting the output of odor coexistence detection into subsequent machine learning models is time-consuming and prone to errors. Therefore, to quickly and accurately provide a rich set of training samples for subsequent machine learning models, we propose an automatic positive and negative sample generation method and integrate it into the ASSD tool. This allows ASSD to automatically complete the entire process from odor coexistence detection to sample generation. Algorithm 3 shows the automatic positive and negative sample generation process, which primarily generates positive and negative sample sets from detection results through file operations.

[0176]

[0177] The steps to automatically generate positive and negative samples are as follows:

[0178] (1) Create the file directory smellyFile for storing positive samples and the file directory nonsmellyFile for storing negative samples (corresponding to lines 2-5 and 12-15 in Algorithm 3).

[0179] (2) Traverse smellySets and write each code segment containing MIM and each code segment containing MC into a .java file to generate a positive sample set (corresponding to lines 6-11 in Algorithm 3).

[0180] (3) Traverse nonsmellySets and write each code segment that does not contain MIM and each code segment that does not contain MC into a .java file to generate a negative sample set (corresponding to lines 16-20 in Algorithm 3).

[0181] 2.3 Generating digital vectors

[0182] The positive and negative samples output by the smell detection tool ASSD are program codes, while the input of the machine learning model is a digital vector. Therefore, it is necessary to convert the program code into a digital vector. This paper draws on the research of scholars such as Sharma and uses the tool Tokenizer to convert the source code into integers. These digital vectors contain all the information of the program and are input into the machine learning model as a feature set. Tokenizer is an open source tool that realizes the function of converting source code into integers. Different code information maps to different integers. Figure 4 The following is an example of using Tokenizer to convert Java source code into a digital vector.

[0183] The input and output of Tokenizer are shown below.

[0184] Tokenizer code ={ele(m1),…,ele(m n )}

[0185] input = {Tokenizercode}

[0186] Where ele(m) is the converted digital vector. The digital vector is used as the feature dataset of the traditional machine learning model input. The dataset format is as follows: Figure 5 shown.

[0187] After Tokenizer processing, the resulting feature set may contain some information that is not helpful for the classification task. These irrelevant features may even have a negative impact on the accuracy and efficiency of the classification. In addition, the large number of features increases the difficulty and time of model training, resulting in an increase in model complexity. In order to optimize the detection effect of the model, the present invention filters the text features generated by the Tokenizer and removes those feature items that are irrelevant to the learning task. At the same time, the present invention also unifies the number of features of the model input samples and sets it to 96 (78% of the samples have no more than 88 features). Samples with more than 96 features will not be considered; for those samples with less than 96 features, they are processed by padding to 96 features with zero vectors to improve the performance of the model and the accuracy of detection. In addition, we trimmed the sample data set and found that 99% of the feature values ​​did not exceed 9999. Therefore, the present invention discards "abnormal samples" with feature values ​​exceeding 9999 to avoid the influence of a single sample on the overall model. Compared to code smells, there is an imbalanced data distribution in practice. In actual software projects, code smells are relatively rare. Therefore, the number of positive samples collected is smaller than the number of negative samples, and the difference between positive and negative samples where code smells coexist is even greater. To ensure a balance between positive and negative samples in the training set, this paper uses undersampling in traditional machine learning training sets. This involves randomly selecting negative samples with a ratio of approximately 1:1 to positive samples.

[0188] 2.4 Machine Learning Classifier

[0189] The present invention uses nine machine learning classifiers to detect the coexistence of Android code smells, including six traditional machine learning models and three improved deep neural network models.

[0190] 2.4.1 Traditional Machine Learning Classifiers

[0191] The present invention selects 6 traditional machine learning algorithms for odor coexistence detection, namely Naive Bayes (NB), Decision Tree (DT), Random Tree (RT), Random Forest (RF), Rule-based Induction Algorithm (JRip), and Bayesian Network (BN). Among them, the first 5 algorithms refer to the research results of scholars such as Fontana, and the 5 learning algorithms with the best performance are selected, namely NB, DT, RT, RF, and JRip. Compared with Naive Bayes, Bayesian Network can more flexibly represent the dependency relationship between variables, provide probabilistic classification prediction by modeling the dependency relationship between variables, and can adapt to different data conditions and feature independence assumptions. It often achieves excellent results in some classification problems. Therefore, the present invention selects Bayesian Network as the 6th machine learning model.

[0192] 2.4.2 Deep Neural Network Classifier

[0193] The three deep neural network models used in this paper are an improved convolutional neural network model, an improved recurrent neural network model, and an improved long short-term memory network. These three models are classic deep learning models and are also the most frequently used deep learning models in the field of code smell detection. The structures of the three deep learning models used in this paper are improved based on the research of Sharma's team and Liu Hui's team.

[0194] 2.4.2.1 Improved Convolutional Neural Network Model

[0195] Convolutional Neural Networks (CNN) is a deep feedforward neural network with local connections and weight sharing. Its most important feature is the convolution operation, which is one of the representative algorithms of deep learning. The convolutional neural network model structure used in the present invention to detect odor is as follows: Figure 6 As shown in Figure 2. The model consists of a feature extraction part and a classification part. Figure 6 As shown, the feature extraction part consists of convolutional layers and maximum pooling layers. These layers constitute the hidden layers of the architecture. The convolutional layer performs convolution operations based on the specified filter and kernel parameters and calculates the network weights for the next layer, while the maximum pooling layer can reduce the number of parameters and computational complexity while retaining important numerical features. The hidden layer consisting of the convolutional layer and the pooling layer is repeated three times. Repeated convolutional layers can help the network learn more complex features because each layer can further abstract features based on the previous layer. The output of the pooling layer is then connected to the Dropout layer. The Dropout layer is a regularization technique used to prevent neural networks from overfitting. During training, it "drops" (i.e., temporarily removes) a portion of neurons with a probability of 0.5, forcing the network to learn more robust features.

[0196] The output of the last Dropout layer is fed into a densely connected classifier network consisting of two stacked fully connected layers. These classifiers process one-dimensional vectors, while the output from the last hidden layer is a three-dimensional tensor. This tensor corresponds to the height, width, and number of channels of the input sample, where the number of channels is one. Therefore, a Flatten layer is first used to convert the data into the appropriate format before it is fed into the first fully connected layer with 32 units and a ReLU activation function. This is followed by a second fully connected layer with a single unit and a sigmoid activation function. The final fully connected layer forms the output layer and contains a single neuron that predicts whether a given instance belongs to the positive or negative class of the odor being studied. This layer uses a sigmoid activation function to produce a result in the range of 0 to 1.

[0197] Table 2 lists the parameters of each layer selected in the improved CNN model. In the embedding layer, the input dataset contains 2239 different characters, each character is encoded as an 8-dimensional vector, and the maximum length of the input data is set to 243. The number of filters determines the number of convolution kernels in the convolution layer, and the size of the convolution kernel determines the size of the convolution window. The window size of the pooling layer determines the window size of the downsampling process during pooling. The number of model iterations, i.e., the epoch value, is the maximum number of iterations during model training. To prevent overfitting and ensure that the model achieves optimal performance, we use early stopping. This method allows the model to terminate training early if the verification loss does not decrease in 10 consecutive iterations before reaching the preset maximum number of iterations. This parameter is called patience in the early stopping method and is set to 10.

[0198] Table 2 CNN model layer parameter values

[0199]

[0200] 2.4.2.2 Improved Recurrent Neural Network Model

[0201] Recurrent Neural Networks (RNN) is a classic deep learning algorithm suitable for processing time series data and is widely used in speech processing and natural language processing. Figure 7 The following diagram shows the architecture of an RNN model for detecting odor coexistence. This architecture is based on research conducted by the Sharma and Liu Hui teams. The model consists of an embedding layer, followed by a feature learning layer consisting of a hidden LSTM layer and a Flatten layer. Finally, the classification layer consists of three fully connected layers.

[0202] Similar to the CNN model, first, the converted text information is used as input through the embedding layer and enters the LSTM layer. During training, the original RNN is prone to gradient explosion or gradient vanishing problems as the training time increases and the number of network layers increases, resulting in the inability to process longer sequence data and thus the inability to obtain information about long-distance data. In order to solve these problems, the present invention uses a long short-term memory network. One of the advantages of LSTM networks is that they can effectively process sequences of different lengths. The output dimension of the LSTM layer is set to 128. After the Flatten layer, the data enters three fully connected layers, the activation function of the first two layers is the relu function, and the activation function of the final output layer is the sigmoid function, outputting a one-dimensional vector. Similar to the CNN model, the early shopping method (epoch is 50 and patience is 10) and the model checkpoint (ModelCheckpoint) callback are used here. In addition, the model loss function is the binary_crossentropy function and the optimizer is RMSProp.

[0203] Table 3 shows the model hyperparameters. The parameters of the embedding layer remain consistent with those used in the CNN model. In the LSTM layer, the dimension of each hidden unit is 128. For the fully connected layers, the parameters represent the dimension of each layer's output features. Based on the optimal parameter combination and after multiple iterations, the model is retrained to achieve peak performance.

[0204] Table 3. Parameter values ​​of RNN model layer numbers

[0205]

[0206] 2.4.2.3 Improved CNN-LSTM Network Model

[0207] As a feedforward neural network, CNN has demonstrated excellent performance in image and natural language processing tasks. LSTM networks are particularly suitable for training text data because of their internal storage mechanism, which can capture and memorize the historical characteristics of data. In the field of code smell detection, the present invention adopts a method that combines local CNN and LSTM networks to simultaneously process spatial features and short-term temporal dependencies. This combined architecture was originally called a long-term recurrent convolutional network (Long-term Recurrent Convolutional Network), and in the present invention, it is named a CNN-LSTM network model, which represents a model structure that integrates the CNN hidden layer with the LSTM network. Figure 8 The figure shows the structure of the CNN-LSTM network model for detecting odor coexistence. This structure is improved based on the research of Sharma's team and Liu Hui's team.

[0208] like Figure 8 As shown, the hidden layers of a CNN consist of multiple stacked convolutional and pooling layers, a process repeated once in the experiment. A ReLU activation function is used between each convolutional and pooling layer to accelerate model convergence. Data is first fed into the convolutional and pooling layers of the CNN via an embedding layer. After processing in these layers, features are locally aggregated and extracted. These local features are then passed to an LSTM network, where a Sigmoid activation function is used to further capture long-term dependencies. After processing through an LSTM layer, a Dropout layer (with a dropout rate of 0.5), and a Flatten layer, the result is finally output in a fully connected layer. This model uses an early shopping method (with an epoch of 50 and a patience of 10) and a model checkpoint callback. The model loss function is the binary_crossentropy function, and the optimizer is RMSProp. Table 4 shows the parameter values ​​for each layer of the model architecture.

[0209] Table 4 CNN-LSTM model layer parameter values

[0210]

[0211] The following examples are used to verify the beneficial effects of the present invention:

[0212] Example 1:

[0213] Research Questions: This paper evaluates the effectiveness of the proposed method by answering the following five research questions (RQ).

[0214] RQ1: What is the detection effect of the MC odor detection method based on static program analysis proposed in this paper?

[0215] RQ2: Can traditional machine learning models identify the coexistence of object-oriented code smells and Android-specific code smells? If so, which model has the best detection effect?

[0216] RO3: Is the detection effect of traditional machine learning models better than that of deep learning models?

[0217] RQ4: Compared with manual detection of odor coexistence, is the detection effect of the method of the present invention better than the manual detection method?

[0218] RQ5: How much time does the proposed method take to construct a sample dataset, train a traditional machine learning model, and use the trained model for prediction?

[0219] Experimental design

[0220] Regarding RQ1: Existing Android-specific code smell detection tools cannot detect traditional object-oriented code smells or the coexistence of code smells. To detect the coexistence of MIM and MC code smells, this paper proposes using program static analysis to detect MC code smells. Based on this, it then proposes using static program analysis to detect the coexistence of MIM and MC code smells and implements the tool ASSD. The accuracy of the MC code smell detection method is the basis for the subsequent use of machine learning models to detect the coexistence of code smells. Therefore, to verify the effectiveness of the MC code smell detection method proposed in this paper, the tool ASSD developed by this paper was used to detect MC code smells in six open source applications. The detection results were then manually analyzed. Table 5 shows the detailed information of the six applications.

[0221] Table 5 Six Android applications used in the test

[0222]

[0223]

[0224] Regarding RQ2: This paper uses code text information as the feature set and inputs six traditional machine learning models. The models were tested using ten-fold cross-validation, and the recognition performance of the six methods was compared using precision, recall, and F1 scores. The six traditional machine learning algorithms selected in this paper are NB, DT, RT, RF, JRip, and BN. The first five algorithms were selected based on the research results of scholars such as Fontana, and the five best-performing learning algorithms, namely NB, DT, RT, RF, and JRip, were selected. Compared to naive Bayesian, Bayesian networks can more flexibly represent the dependencies between variables. By modeling these dependencies, they provide probabilistic classification predictions and can adapt to different data conditions and feature independence assumptions. They often achieve excellent results in some classification problems. Therefore, this paper selected Bayesian networks as the sixth machine learning model. The training set for the model is the 70 Android applications listed in Section 2.2.1. The test set is composed of eight open source applications. These eight open source projects were selected using the criteria in Section 3.2.1 and do not overlap with the 70 applications in the training set. The detailed information of the 8 applications used in the experiment is shown in Table 8. Table 8 includes a total of 11 applications, of which the first 8 applications are the 8 applications used to answer RQ2.

[0225] Regarding RO3: With the widespread application of deep learning in detecting code smells, more and more software engineering researchers are using deep learning to detect code smells and have achieved good results. However, research by researchers such as Sharma shows that not all code smells are suitable for using deep learning as a detection method. In order to try to use deep learning to detect the feasibility of coexistence of Android code smells and compare it with the detection effect of traditional machine learning, the present invention selects the three most frequently used deep learning models in existing code smell detection research as the basic models, improves them, and detects the coexistence of odors. The training set used in the experiment is the 70 applications shown in Section 2.2.1. 11 open source applications are used as the test set, and these 11 applications do not overlap with the 70 applications in the training set. Table 6 shows the detailed information of the 11 applications.

[0226] Table 6. 11 Android applications used in the test set

[0227]

[0228]

[0229] Regarding RQ4: Since existing tools cannot detect the two smells studied in this invention, for comparative research, this invention adopts a comparison method with manual detection. Manual detection here means first using the tool DAAP to detect the source code to be analyzed and identify the MIM smell therein. Then, in the code segment containing the MIM smell, the MC smell is manually identified. After detecting the coexisting smells, the tool Tokenizer is used to convert the code into a digital vector, which is input into the subsequent machine learning model, and the evaluation index is obtained according to the model output. This invention selects the 8 open source applications shown in Table 6 as the test set. Table 6 includes a total of 11 applications, of which the first 8 applications are the 8 applications used to answer RQ4. The manual analysis process is as follows:

[0230] 1. Evaluator Selection: Two computer science graduate students with experience in software engineering were selected to perform manual testing (the first and third authors of this invention). Both students were very familiar with both smells and had used the detection tool to analyze the output of a single smell.

[0231] 2. Manual Inspection: Each student independently analyzed the source code of four of the eight applications to be analyzed. The evaluators first used the DAAP tool to inspect the source code. Then, they manually identified MC smells in the code segments containing MIM smells and recorded the detection results.

[0232] 3. Discussion and Negotiation: After the test was completed, the two students exchanged test results and performed a consistency check to ensure that their assessments were reliable and consistent. When disagreements arose, the two students discussed and negotiated the issues to understand each other's perspectives and reach a consensus. For samples where consensus could not be reached and disagreements arose, statistical analysis was performed to determine the causes and impact of the disagreement. The second author of this paper conducted an independent assessment to help resolve disagreements.

[0233] 4. Statistical analysis: After manual inspection, the code is converted into a digital vector using the Tokenizer tool and input into the subsequent machine learning model. Evaluation indicators are obtained based on the model output, namely precision, recall, and F1 value.

[0234] Regarding RQ5: Research shows that model complexity is an important factor to consider when selecting a model, because complex models may require additional resource consumption (such as storage and computing time costs). Deep learning models are more complex than traditional machine learning models. Therefore, time complexity is an important metric when evaluating deep learning models. This research question quantitatively reveals the training and testing time of the proposed method. In order to evaluate the time efficiency of the method of the present invention. The present invention runs the entire process of the experiment on a computer (8GB RAM, Intel Core CPU i7-7700), and records the time consumption (in minutes) of generating the dataset and the training and testing process of the classifier to analyze the time performance of the proposed method.

[0235] Model evaluation metrics

[0236] This paper uses three classic evaluation metrics: precision, recall, and F1-score to evaluate the performance of different classifiers. The relevant formulas are as follows:

[0237]

[0238]

[0239]

[0240] Among them, TP (True Positive) means that the model has detected actual co-existing code smells. FN (False Negative) means that the model predicted an actual segment with co-existing code smells as a segment without co-existing code smells, resulting in a missed detection. FP (False Positive) means that the model mistakenly identified a normal code segment as a segment with code smells, resulting in a false positive. Precision measures the proportion of samples that the model determines to have co-existing code smells that actually have co-existing code smells. Recall measures the proportion of all samples that actually have co-existing code smells that are correctly identified by the model. The F1 score is a balance between precision and recall. Using the calculation results of these evaluation metrics, the performance of different classifiers in identifying the co-existence of code smells can be fairly evaluated.

[0241] Experimental results and analysis

[0242] RQ1: What is the detection effect of the MC odor detection method based on static program analysis proposed in this paper?

[0243] Table 7 shows the precision, recall, and F1 values ​​for the six tested applications.

[0244] Table 7. Detection results of MC code smells of 6 apps using ASSD

[0245]

[0246] As shown in Table 7, the maximum precision is 1, the minimum is 0.729, and the average is 0.859; the maximum recall is 1, the minimum is 0.5, and the average is 0.892; the maximum F1 value is 0.945, the minimum is 0.666, and the average is 0.86. Therefore, the MC odor detection method based on static program analysis proposed in this paper is effective and can detect MC odors in programs. This lays a good foundation for odor coexistence detection, thereby providing high-quality input samples for subsequent machine learning models.

[0247] RQ2: Can traditional machine learning models identify the coexistence of object-oriented code smells and Android-specific code smells? If so, which model has the best detection effect?

[0248] Table 8 shows the detection results of six traditional machine learning models. All six models achieved high precision, recall, and F1 scores, with the random forest model (RF) performing the best. This demonstrates that machine learning is feasible for detecting the coexistence of MIM and MC odors. The F1 score order for the six classifiers is: RF > RT > JRip > DT > NB > BN. Therefore, the RF model performs best when detecting the coexistence of MC and MIM odors. The detection results are analyzed as follows: First, RF is an ensemble learning-based classifier that constructs multiple decision trees for classification, while RT uses a single decision tree. Furthermore, RF effectively handles feature correlation and generalizes well when processing large amounts of data. Second, RT outperforms JRip. This is because while JRip typically classifies based on the rules in the dataset, it may be limited by these rules and perform poorly when processing large amounts of data. Finally, JRip outperforms DT. This is because, compared to DT, JRip better captures patterns and regularities in the data and performs better in complex situations.

[0249] Table 8 Odor coexistence detection results of different classifiers

[0250]

[0251] The Fontana team used 16 traditional machine learning models to detect four types of object-oriented code smells. However, they only considered a single type of object-oriented code smell, without considering the coexistence of smells, let alone Android code smells. The present invention selected the five models with the best detection effects in their research, namely DT, RF, JRip, NB and BN for comparison. The detection results of the Fontana team were: DT>RF>JRip>NB>BN. This shows that when using machine learning models to detect code smells, the detection effect of the model will vary depending on the type of smell and the way the smell exists. This result is consistent with the conclusion of researchers such as Sharma. That is, the detection effect of the machine learning model varies depending on the type of smell. This shows that in actual program development and maintenance, the appropriate machine learning model should be selected according to the actual situation of the smell. No machine learning model can achieve good results in all code smell identification.

[0252] In summary, when dealing with the coexistence of MC and MIM odors, random forest is the best choice, followed by random tree.

[0253] RQ3: Is the detection effect of traditional machine learning models due to the deep learning model?

[0254] Table 9 shows the detection results using three improved deep learning models. The best detection model is the improved LSTM, with an F1 score of 89.7%. Combining Tables 10 and 11, the top six machine learning classifiers with the best detection performance are: RF > LSTM > RT > CNN > RNN > JRip. Therefore, RF performs best when detecting code smells where MC and MIM coexist in Android applications. While the three improved deep neural network models perform equally well, the RF model performs even better.

[0255] Table 9 Odor coexistence detection results based on deep learning

[0256]

[0257] The three performance indicators of the three improved deep neural network models change over time on the training and validation sets. The curves show that as the number of iterations increases, the three indicators of the CNN, RNN, and CNN-LSTM models are all higher in both the training and validation sets, with similar differences. This indicates that these three models accurately capture the patterns in the data and are capable of effectively detecting the coexistence of code smells.

[0258] RQ4: Compared to manual detection of odor coexistence, does the method of the present invention outperform manual detection? The first eight Android applications shown in Table 8 were tested, and Table 10 lists the comparison results between six machine learning models and manual detection. The detection results of the six machine learning models used in the present invention all outperformed manual detection results, with RF performing the best. Compared to manual detection, the F1 value increased by an average of 9.1% (90.1%-81%). Furthermore, the manual detection process took approximately 120 hours.

[0259] Research has shown that due to the scale and complexity of mobile software applications, manually analyzing code smells from source code is a very difficult and challenging task, both time-consuming and prone to errors. Furthermore, because manual detection of code smells relies heavily on the developer's experience and judgment, different people may produce inconsistent results for the same code snippet.

[0260] RQ5: How much time does the proposed method take to construct a sample dataset, train a traditional machine learning model, and use the trained model for prediction?

[0261] Due to space limitations, Table 11 lists the time (in minutes) taken to generate the training set for 8 of the Android applications. Detailed information on the time consumed by all 70 applications is available at https: / / github.com / biboyu / MC-MIMDetection.

[0262] Table 10 Performance comparison of different classifiers

[0263]

[0264] As shown in Table 11, collecting sample data is quite time-consuming. Extracting training data from 70 Android applications took 1540 minutes (approximately 25.7 hours), an average of 22 minutes per project. The most time-consuming program was a2dpvolume-master (1007 classes, 1183 methods, 89,051 lines of code, and a total of 151 minutes), while the least time-consuming program was Dali-master (33 classes, 201 methods, 1548 lines of code, and a total of 9 minutes).

[0265] Table 11 Time consumption for generating training set data

[0266]

[0267]

[0268] Table 12 Time consumption of the training and testing processes of the six machine learning models (in seconds)

[0269]

[0270] As shown in Table 12, compared to the over 25 hours required to generate the training set, the machine learning classifiers took an average of 0.1 seconds to train on the test set. The Random Forest classifier, the most effective detection model, took 1.77 seconds to train. During testing, all six classifiers demonstrated excellent time performance.

[0271] Due to the complexity of its models, deep learning typically requires more computing resources and training time. Table 13 shows the training and testing times (in seconds) for three improved deep learning models. Compared to the over 25 hours required to generate the training set, the average training time for deep learning on the test set was 171 seconds. The LSTM model, the most effective detection model, took 3600 seconds to train, which is not as time-efficient as the other two deep learning models. Combining the results in Tables 12 and 13, we can see that traditional machine learning models have an advantage over deep learning models in terms of detection time.

[0272] Table 13 Detection time of different methods

[0273]

[0274] In summary, among the nine machine learning models, the RF model has advantages in both detection accuracy and detection time. Therefore, the RF model is recommended for detecting the coexistence of MC and MIM odors in Android applications.

[0275] The present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for detecting Android code smells that combines static program analysis with machine learning, characterized by: The specific process of the method is: Step 1: Obtain a sample set as a label dataset; the specific process is: Step 1. Download the Android application developed in Java language from the open source code library GitHub as the code corpus; Step 1 and 2: Use JavaParser to parse the Android application source code and generate the abstract syntax tree of the code; Step 13: Use static program analysis methods to detect MC odors; Based on the MC odor detection results, a static program analysis method is used to detect the coexistence of MIM and MC odors; Step 14: Use two nested hash tables to store the odor detection results; Step 15: Automatically generate positive and negative samples based on step 14, and generate a sample set based on the positive and negative samples as a label data set; Step 2: Process the sample set to obtain a processed data set; Step 3: Build a deep neural network classifier and obtain a trained deep neural network classifier based on the processed data set; Step 4: Input the tested Android code smell into the trained deep neural network classifier, and output whether MIM and MC code smells coexist in the tested Android code smell; The deep neural network classifier in step 3 is any one of naive Bayes, decision tree, random tree, random forest, rule-based induction algorithm, Bayesian network, improved convolutional neural network model, improved recurrent neural network model, and improved CNN-LSTM network model; The improved convolutional neural network model includes an embedding layer, a first convolutional layer, a first maximum pooling layer, a second convolutional layer, a second maximum pooling layer, a third convolutional layer, a third maximum pooling layer, a Dropout layer, a Flatten layer, a first fully connected layer, a second fully connected layer, and an output layer in sequence; The mask_zero parameter is set to True in the embedding layer; The dropout rate of the Dropout layer is set to 0.5; The activation function of the first fully connected layer is the relu activation function; The activation function of the second fully connected layer is a sigmoid function; The loss function is binary_crossentropy function, and the optimizer is RMSProp; Input the data set preprocessed in step 2 into the improved convolutional neural network model until the loss function converges to obtain a trained improved convolutional neural network model; The improved recurrent neural network model includes an embedding layer, a long short-term memory network LSTM, a flatten layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, and an output layer in sequence; The number of neurons in the first fully connected layer is 64; The number of neurons in the second fully connected layer is 32; The units digit of the neurons in the third fully connected layer is 1; The activation functions of the first fully connected layer and the second fully connected layer are relu functions; The activation function of the third fully connected layer is a sigmoid function; The output dimension of the long short-term memory network LSTM is set to 128; The output of the third fully connected layer is a one-dimensional vector; The loss function is binary_crossentropy function, and the optimizer is RMSProp; The data set preprocessed in step 2 is input into the improved recurrent neural network model until the loss function converges to obtain a trained improved recurrent neural network model; The improved CNN-LSTM network model includes an embedding layer, a first convolutional layer, a first maximum pooling layer, a long short-term memory network LSTM, a Dropout layer, a Flatten layer, a first fully connected layer, and an output layer in sequence; The improved CNN-LSTM network model uses early stopping and model checkpoint callback; The number of filters in the first convolutional layer is 32, and the kernel size is 3; The window size of the first maximum pooling layer is 2; The output dimension of the long short-term memory network LSTM layer is set to 128; The dropout rate of the Dropout layer is set to 0.5; The activation functions of the first convolutional layer and the first maximum pooling layer are relu activation functions; The activation function of the long short-term memory network LSTM layer is a sigmoid activation function; The loss function of the improved CNN-LSTM network model is the binary_crossentropy function, and the optimizer is RMSProp; The data set preprocessed in step 2 is input into the improved CNN-LSTM network model until the loss function converges to obtain a trained improved CNN-LSTM network model.

2. The Android code smell coexistence detection method integrating static program analysis and machine learning according to claim 1 is characterized by: In the step 1, an Android application developed in Java is downloaded from the open source code library GitHub as a code corpus; The specific process is: Step 1: The application in the code corpus is at least 1000KB; Step 112: The number of stars of the application must be greater than 500; Step 113: The last update date of the Android application should be later than April 1, 2023.

3. The Android code smell coexistence detection method integrating static program analysis and machine learning according to claim 2 is characterized in that: In the steps 1 and 3, a static program analysis method is used to detect the MC odor; based on the MC odor detection results, a static program analysis method is used to detect the coexistence of MIM and MC odors; the specific process is: 1) Use static program analysis method to detect MC odor; the specific process is: Use the visitor pattern to traverse all method calls in the abstract syntax tree; Recursively trace the call chain of each method call until it reaches the starting point of the chain; If the length of a call chain exceeds the set threshold, the call chain is considered to be an over-coupled message chain smell and is marked as an MC smell; 2) Based on the MC odor detection results, a static program analysis method is used to detect the coexistence of MIM and MC odors.

4. The Android code smell coexistence detection method integrating static program analysis and machine learning according to claim 3 is characterized by: In step 14, two nested hash tables are used to store the odor detection results; the specific process is as follows: Record the name of each code smell and the number of times each code smell appears in the current class in a hash table called map2; Then, the classpath of each class is combined with map2 to form a more comprehensive hash table map1.

5. The Android code smell coexistence detection method integrating static program analysis and machine learning according to claim 4 is characterized in that: In step 15, positive and negative samples are automatically generated based on step 14, and a sample set is generated based on the positive and negative samples as a label data set; the specific process is: (1) Create the file directory smellyFile and the file directory nonsmellyFile respectively; The file directory smellyFile is used to store code segments containing MIM and MC coexisting smells; The file directory nonsmellySets is used to store code segments that do not contain MIM and MC smells; Create a file directory smellyFile for storing positive samples and a file directory nonsmellyFile for storing negative samples respectively; (2) Traverse smellySets and write the code segments where MIM and MC smells coexist into a Java file to generate a positive sample set; (3) Traverse nonsmellySets and write the code segments where MIM and MC smells do not coexist into a Java file to generate a negative sample set; Generate a sample set based on positive samples and negative samples as a label dataset; MC is an over-coupled message chain smell; MIM is the method smell of ignoring class member variables.

6. The Android code smell coexistence detection method integrating static program analysis and machine learning according to claim 5 is characterized in that: In step 2, the data set is preprocessed to obtain a preprocessed data set; the specific process is: Use the Tokenizer tool to convert positive and negative samples into integers and use digital vectors to represent program text information; Remove duplicate samples in program text information; Normalize the program text information represented by the digital vector; The program text information represented by the normalized digital vector is used as the input data of the deep neural network classifier.

Images