Android code peculiar smell coexistence detection method fusing static program analysis and machine learning

Abstract

The invention discloses an Android code peculiar smell coexistence detection method fusing static program analysis and machine learning, and relates to a code peculiar smell coexistence detection method. The objective of the invention is to solve the problem of low detection accuracy of Android code peculiar smell coexistence by an existing method. The method comprises the following steps of: 1, acquiring a sample set as a label data set: downloading an Android application program from an open source code library as a code corpus; analyzing the Android application program source code to generate an abstract syntax tree of the code; detecting MC peculiar smell by using a static program analysis method; detecting MIM and MC peculiar smell coexistence by adopting a static program analysis method based on an MC peculiar smell detection result; two nested hash tables are used for storing peculiar smell detection results; generating a sample set based on automatically generated positive and negative samples; 2, obtaining a processed data set; 3, obtaining a trained deep neural network classifier; and 4, detecting whether code peculiar smell coexistence exists in the to-be-detected Android code peculiar smell or not. The method is applied to the field of code odor coexistence detection.

Description

Technical Field

[0001] The invention relates to an Android code odor coexistence detection method. Background Art

[0002] Code smell is a poorly structured code segment in a software system. In 1999, Martin Fowler used "code smell" to describe the code structure in an object-oriented program that does not meet specific quality requirements. This smell is also called object-oriented code smell. Code smell reduces the maintainability and understandability of applications, which is not conducive to software maintenance and version evolution. In recent years, the rapid development of mobile application technology has made people's dependence on mobile applications increasingly obvious. According to statistics, among all mobile operating systems, Android operating system has the largest market share, reaching more than 85%. Studies have shown that code smell exists not only in traditional desktop applications, but also in Android applications. Due to the differences between traditional desktop applications and Android applications in hardware and software, code smells in Android applications are different from code smells in object-oriented applications, and the former is more complex than the latter. In Android applications, there are not only traditional object-oriented code smells, but also Android-specific code smells. In 2014, researchers such as Reimann proposed Android-specific code smells and gave a list of smells. This odor has many negative impacts on Android applications, such as energy consumption, security, stability, memory and activation time, which reduces the user experience and is not conducive to the maintenance and evolution of Android applications.

[0003] Code smell coexistence is the presence of two or more code smells in a single method or class. Studies have shown that code smell coexistence is more harmful to applications than one code smell. Studies have shown that code smell coexistence increases the difficulty for developers to understand programs, makes maintenance more complicated, and is more likely to introduce bugs. Researchers such as Hamdi conducted in-depth research on code smells in Android applications. The results showed that among the classes containing code smells, about 51% of the classes were jointly affected by two or more code smells, indicating that code smell coexistence is ubiquitous in programs. The results of researchers such as Manman showed that object-oriented code smells in Android applications can be detected using traditional code smell detection tools. However, traditional tools cannot detect Android-specific code smells. Code smell coexistence in Android applications includes three types: traditional object-oriented code smell coexistence, Android-specific code smell coexistence, and object-oriented and Android-specific code smell coexistence. As a preliminary study, the present invention focuses on the coexistence of object-oriented code smells and Android-specific code smells. Research by Hamdi and other scholars shows that in Android applications, there are 4 pairs of object-oriented code smells and Android-specific code smells coexisting. As a preliminary exploration, the present invention conducts research on one pair of smells with a higher coexistence frequency. This pair of code smells is the over-coupled message chain (MC) smell and the member ignoring method (MIM) smell. MC is a traditional object-oriented code smell, and MIM is an Android-specific code smell. The existence of this pair of smells has a great impact on the energy consumption of Android applications, reducing the comprehensibility and maintainability of the program. At present, the best tool for detecting Android-specific code smells is DAAP (Detection of Android Application), which has an average accuracy rate of more than 90% in detecting MIM smells. DAAP adopts the static program analysis technology commonly used by traditional detection tools, combines detection rules with code metrics, and realizes the identification of smells by traversing the abstract syntax tree. However, DAAP can only detect a single type of Android-specific code smell, cannot detect object-oriented code smells, and cannot detect the coexistence of smells.

[0004] Research shows that the main limitation of static program analysis is that its performance is heavily dependent on thresholds and / or combination rules. Application developers are subjective in setting rules and choosing metrics, which leads to differences in detection results even for the same code smell. To alleviate these limitations, machine learning has attracted the interest of researchers.

[0005] In recent years, with the breakthroughs and widespread applications in the field of machine learning, researchers have applied machine learning algorithms to the detection of traditional object-oriented code smells, with good results. Although machine learning methods have many advantages, one of the main limitations of current machine learning-based methods is that they rely heavily on manual annotation of data, and training an efficient machine learning model requires a large amount of labeled data. In the field of code smell detection, there is currently a lack of open standard datasets for researchers to use. In addition, existing studies have mainly focused on traditional object-oriented code smells, and there is little research on code smells in Android applications.

[0006] Existing methods study the detection methods of Android-specific code smells, but only detect a single type of code smell, without considering the coexistence of smells or object-oriented code smells. Summary of the invention

[0007] The purpose of the present invention is to solve the problem that the existing methods for detecting Android-specific code smells only detect a single type of code smell, do not consider the coexistence of smells, and do not consider object-oriented code smells, resulting in low detection accuracy of Android code smell coexistence, and propose an Android code smell coexistence detection method that integrates static program analysis and machine learning.

[0008] The specific process of the Android code smell coexistence detection method integrating static program analysis and machine learning is as follows:

[0009] Step 1: Get a sample set as a label data set. The specific process is as follows:

[0010] Step 1. Download the Android application developed in Java language from the open source code library GitHub as the code corpus;

[0011] Step 1 and 2: Use JavaParser to parse the Android application source code to generate an abstract syntax tree of the code;

[0012] Step 13: Use static program analysis method to detect MC odor; Based on the MC odor detection results, use static program analysis method to detect the coexistence of MIM and MC odors;

[0013] Step 14: Use two nested hash tables to store the odor detection results;

[0014] Step 15: Automatically generate positive and negative samples based on step 14, and generate a sample set based on the positive and negative samples as a label data set;

[0015] Step 2: Process the sample set to obtain a processed data set;

[0016] Step 3: Build a deep neural network classifier and obtain a trained deep neural network classifier based on the processed data set;

[0017] Step 4: Input the Android code smell to be tested into the trained deep neural network classifier, and output whether MIM and MC code smells coexist in the Android code smell to be tested.

[0018] The beneficial effects of the present invention are:

[0019] Compared with a single code smell, the coexistence of code smells is more harmful to the program. For Android-specific code smells, existing studies mainly focus on the detection of a single type of smell, ignoring the negative impact of the coexistence of smells on Android applications. In order to identify the coexistence of Android-specific code smells, a coexistence detection method for Android code smells combining static program analysis and machine learning is proposed. As a preliminary exploration, the coexistence of over-coupled message chain smells and method smells that ignore class member variables is identified. First, an over-coupled message chain smell detection algorithm based on static program analysis is proposed, and on this basis, a coexistence detection method for over-coupled message chain smells and method smells that ignore class member variables based on static program analysis is proposed. In addition, in order to provide rich training samples for subsequent machine learning models, a positive and negative sample automatic generation method is proposed and the tool ASSD is implemented. Then, 9 machine learning models are used to identify coexisting code smells. Among them, 6 traditional machine learning models and 3 improved deep learning models. Experimental results show that the random forest model has the best detection effect. Compared with manual detection, the F1 value of the proposed method is improved by 9.1%.

[0020] This paper combines program static analysis with machine learning to detect the coexistence of object-oriented code smells and Android-specific code smells in Android applications. The main contributions of this paper are as follows:

[0021] (1) A method for detecting over-coupled message chain smell based on static program analysis is proposed.

[0022] (2) A method for identifying coexistence of Android code smells based on static program analysis is proposed.

[0023] (3) A method for automatic sample generation is proposed. (2) is combined with (3) to achieve automatic completion from odor coexistence detection to sample generation and implement the tool ASSD (Android-Specific Smell Detector).

[0024] (4) A method for detecting Android code smells that combines static program analysis with machine learning is proposed.

[0025] Compared with a single code smell, the coexistence of code smells is more harmful to the program. For Android-specific code smells, existing studies mainly focus on the detection of a single type of smell, ignoring the negative impact of the coexistence of smells on Android applications. In order to identify the coexistence of Android-specific code smells, a coexistence detection method for Android code smells combining static program analysis and machine learning is proposed. As a preliminary exploration, the coexistence of over-coupled message chain smells and method smells that ignore class member variables is identified. First, an over-coupled message chain smell detection algorithm based on static program analysis is proposed, and on this basis, a coexistence detection method for over-coupled message chain smells and method smells that ignore class member variables based on static program analysis is proposed. In addition, in order to provide rich training samples for subsequent machine learning models, a positive and negative sample automatic generation method is proposed and the tool ASSD is implemented. Then, 9 machine learning models are used to identify coexisting code smells. Among them, 6 traditional machine learning models and 3 improved deep learning models. Experimental results show that the random forest model has the best detection effect. Compared with manual detection, the F1 value of the proposed method is improved by 9.1%.

[0026] The present invention adopts a method based on static program analysis combined with machine learning to detect the coexistence of object-oriented code smells and Android-specific code smells in Android applications, so as to explore the possibility of using machine learning methods to identify the coexistence of Android-specific code smells. As a preliminary exploration, the coexistence of over-coupled message chain smells and method smells that ignore class member variables is identified. First, an MC smell detection method based on static program analysis is proposed, and on this basis, an Android code smell coexistence detection method based on static program analysis and a positive and negative sample automatic generation method are proposed and the tool ASSD is implemented. The output of the tool provides rich training samples for subsequent machine learning models. Then, 9 machine learning classifiers are used to detect coexistence smells, including 6 traditional machine learning models and 3 improved deep learning models. The experimental results show that among the 9 machine learning models, the random forest model performs best in detection effect, and its performance is better than the deep learning model. In addition, compared with the manual detection method, the F1 value of the method of the present invention is improved by 9.1%. Therefore, the coexistence of Android-specific code smells can be detected using machine learning methods. Among the 9 machine learning models, the RF model has advantages in detection accuracy and detection time. Therefore, it is recommended to use the RF model when detecting the coexistence of MC and MIM smells in Android applications.

[0027] The method for detecting the coexistence of code smells proposed in this invention provides a new idea for detecting the coexistence of code smells. In subsequent work, the optimization method for the coexistence of code smells will be studied so as to reconstruct the coexisting smells, with the aim of improving the quality of mobile software. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flow chart of the present invention;

[0029] Figure 2 An overview of the MC and MIM coexistence detection method that combines static program analysis with machine learning;

[0030] Figure 3 Flowchart for odor coexistence detection and automatic generation of positive and negative samples;

[0031] Figure 4 Tokenizer code example diagram;

[0032] Figure 5 It is a code text information data format diagram;

[0033] Figure 6 This is a structural diagram of a neural network classifier based on CNN;

[0034] Figure 7 This is a structural diagram of a deep neural network classifier based on RNN;

[0035] Figure 8 This is the structure diagram of the deep neural network classifier of CNN-LSTM. DETAILED DESCRIPTION

[0036] Specific implementation method 1: The specific process of the Android code odor coexistence detection method integrating static program analysis and machine learning in this implementation method is as follows:

[0037] 1. Related Work

[0038] 1.1 Object-oriented code smell

[0039] In 1999, Martin Fowler proposed the concept of "code smell" in order to explain the design problems of code in traditional desktop applications. Code smells are not bugs, but they indicate possible problems with the code or areas that need improvement. Code smells increase the burden of program maintenance and hinder the evolution of the program. Fowler listed 24 code smells in the second edition of his classic book "Refactoring: Improving the Design of Existing Code". For example, too long method smells, over-coupled message chain smells, and duplicate code smells. These code smells are called object-oriented (OO) code smells by subsequent studies. Object-oriented code smells are usually caused by developers' excessive pursuit or improper use of certain principles or programming paradigms. For example, excessive use of design-based, inheritance, and generic methods may reduce the readability and maintainability of the code. The MC smell studied in this invention is an object-oriented code smell.

[0040] 1.2. Android-specific code smells

[0041] In recent years, with the development of mobile application technology, mobile applications have evolved from simple applications to rapidly evolving complex systems. For example, in 2021, the number of mobile applications in various app stores has exceeded 5 million, and the number of application downloads has exceeded 23 billion times. In order to meet the growing demand, mobile applications are developed very quickly and continue to evolve to meet user needs. However, rapid development in a limited time to solve errors or add missing features can lead to improper design or implementation choices, thereby increasing technical debt. Android-specific code smells, also known as Android code bad smells, are code segments in Android applications that violate code quality requirements. Unlike traditional object-oriented code smells, Android-specific code smells do not affect the functionality of Android applications, but they increase system memory consumption, reduce application performance, and increase system energy consumption. In 2014, researchers such as Reimann linked concepts such as smell, quality, and refactoring, proposed the concept of "quality smell", represented the code structure in Android applications that affects software quality, and gave a list of smells. These quality smells were referred to as Android-specific code smells by subsequent studies. This smell has many negative effects on Android applications, such as energy consumption, security, stability, memory and startup time, which is not conducive to the maintenance and evolution of Android applications. Therefore, detecting and refactoring code smells in the early stages of design is very important to improve the quality and performance of the code and the entire system.

[0042] Manman and other scholars found that the tools for detecting traditional object-oriented code smells cannot detect Android-specific code smells. Therefore, scholars have proposed different methods to detect Android-specific code smells. Palomba et al. proposed a method that combines code metrics with simple text comparison to detect 15 types of Android code smells, and implemented the tool aDoctor. aDoctor can detect a single type of Android-specific code smell, and can also detect the coexistence of Android-specific code smells. Although aDoctor is free software, its source code is not public. Iannone and other researchers have extended the progress of aDoctor and reconstructed five types of Android-specific code smells related to energy consumption. In order to detect more types of Android code smells, Rasool et al. defined heuristic detection rules, combined code metrics to detect 25 types of smells, and implemented the tool DAAP. DAAP is superior to aDoctor in terms of the types and accuracy of detecting smells, and the source code is open to facilitate the use and function expansion of subsequent researchers. Recently, researchers such as Prestat proposed a method for dynamically detecting Android-specific code smells, but the method studies bytecodes rather than source codes, so when a smell is found, the method cannot locate the smell. The MIM smell studied by the present invention is an Android-specific code smell.

[0043] 1.3. Code smells coexist

[0044] In a software system, when a class or method is disturbed by two or more code smells at the same time, this phenomenon is called code smell coexistence. Studies have shown that coexisting code smells are more harmful to software systems than single code smells. Research by Palomba et al. shows that classes affected by multiple code smells are 350% more likely to change and 400% more likely to fail than classes affected by a single type of smell. In addition, code smell coexistence makes it more difficult for developers to understand the source code. In 2017, Palomba et al. used association rule learning methods to study 13 object-oriented code smells and found that six pairs of smells coexisted frequently. A year later, they conducted in-depth research on the frequency of code smell coexistence, the manifestations of smell coexistence, and the causes of smell coexistence. In 2021, Hamdi et al. confirmed the prevalence of code smell coexistence in Android applications through a large number of experimental studies. The results show that more than 50% of classes are disturbed by two or more code smells. Among the existing methods, only aDoctor can detect the coexistence of Android-specific code smells. This method uses traditional static program analysis methods to identify coexisting smells, but it cannot detect object-oriented code smells.

[0045] 1.4. Peculiar smell studied by the present invention

[0046] Research shows that there are four pairs of object-oriented code smells and Android-specific code smells coexisting in Android applications. As a preliminary study, the present invention studies one pair of code smells with a higher coexistence frequency. This pair of code smells is MC and MIM. MC is an object-oriented code smell, and MIM is an Android-specific code smell.

[0047] 1.4.1. Over-coupled message chain smell MC

[0048] Over-coupled message chain smell is an object-oriented code smell, which means that a long chain of methods needs to be called to implement the function of a class. In this case, the method call of one object triggers the method call of another object, thus forming a "message chain". The existence of MC smell leads to problems such as decreased code readability, increased coupling, and difficulty in maintenance. The following is a code snippet containing MC smell.

[0049]

[0050]

[0051] In line 10 of the code, the getB() method in class A returns an instance of class B, and in line 16 of the code, the getC() method in class B returns an instance of class C. In the main() method (line 4), these methods are called in the form of a.getB().getC().getValue() to form a message chain.

[0052] 1.4.2. Method smell MIM that ignores class member variables

[0053] Ignoring class member variables is an Android-specific code smell. MIM is a method of a class in an Android application. This method is neither an empty method nor a static method. It does not include a method where a subclass overrides a parent class, but the method does not use any attributes of the class to which it belongs. Studies have shown that MIM is the most frequently occurring code smell in Android applications, and therefore has attracted much attention from the academic community. Tian Yingchen et al. proposed a criticality quantification method to quantify the severity of code smells. According to this method, MIM is the most critical code smell among many Android-specific code smells. The higher the criticality of the smell, the greater its impact on the application, and it should be given a higher priority during the refactoring process. In addition, the MIM smell has an impact on the energy consumption and maintainability of Android applications. The following is a code snippet containing the MIM smell.

[0054]

[0055] The calculateDiscount() method in line 8 does not use the username or age attributes defined in the UserProfile class, so this method has a MIM smell. This method should not be part of the UserProfile class because it has nothing to do with the user's profile. This method can be refactored to another more appropriate place, such as a utility class or as a static method.

[0056] 1.4.3 MC and MIM odor coexist

[0057] The following is a code example of the coexistence of MIM and MC.

[0058]

[0059] In line 4 of the code, the processUser() method does not use the Address variable and username variable defined in the UserInfo class, which produces a MIM code smell. In line 5 of the code, these methods are called in the user.getAresses().stream().filter().map().orElse() manner, forming a coupled message chain. Although the code segment is not long, it is infected with two smells at the same time.

[0060] 1.5 Code smell detection method based on static program analysis

[0061] Program static analysis is a method of analyzing source code without executing the program. It discovers potential code problems and errors or optimization opportunities by checking the structure, syntax, semantics and other aspects of the code itself. In the study of program static analysis, researchers have proposed many different static analysis techniques and methods, such as abstract syntax tree, data flow analysis, symbolic execution, model detection, type inference, etc. These technologies can be applied to different programming languages ​​and software systems, and can be customized and extended according to needs. Static program analysis is a common method for code smell detection. The existing tools for detecting Android code smells, DAAP and aDoctor, both adopt this method. Although DAAP performs well in detecting a single type of smell, it cannot detect the coexistence of smells. aDoctor can detect the coexistence of smells, but it is limited to the coexistence between Android-specific code smells and cannot detect traditional object-oriented code smells.

[0062] Static analysis-based methods mainly use two ways to detect code smells. One is to detect code smells using a series of thresholds that depend on structural parameters (i.e., software metrics) to classify and describe the symptoms of specific code smells. The other is to detect code smells through predefined rules or heuristic methods. The main limitation of these methods is that their performance is heavily dependent on thresholds and / or combined rules. Until now, experts have not reached a consensus on standard values ​​and rules. To alleviate these limitations, machine learning methods have attracted the interest of researchers in recent years.

[0063] 1.6 Code smell detection method based on machine learning

[0064] In recent years, the research of machine learning has made great progress. Researchers in the field of software engineering have also tried to use this technology to solve problems in the field of software engineering, such as software testing, source code representation, source code quality analysis, refactoring, code smell detection, etc., and have achieved good results. Khomh et al. used the Bayesian belief network model to detect code smells. Abdou et al. used support vector machines to detect three code smells: Blob, attachment complex, and spaghetti code. Fontana et al. used 16 traditional machine learning models to detect four object-oriented code smells and compared the effects of different algorithms in detecting code smells. Amorin et al. combined decision trees with genetic algorithms to detect code smells. Kim et al. used multi-layer perceptrons to detect six code smells (God class, long method, attachment complex, pure data class, redundant class, and parallel inheritance) and achieved a high accuracy rate. Kuar et al. combined the SVM algorithm with k-means clustering to detect four object-oriented code smells and gave a refactoring strategy. The above studies show that although machine learning methods are helpful for code smell detection and have high accuracy, these methods focus on a single type of object-oriented (OO) code smell. Therefore, it is still unknown whether machine learning algorithms can detect the coexistence of object-oriented and Android-specific code smells in Android applications. Despite the many advantages of machine learning methods, one of the main limitations of current machine learning-based methods is that they heavily rely on manual annotation of data, and training an efficient machine learning model requires a large amount of labeled data. In the field of code smell detection, there is currently a lack of open standard datasets for researchers to use, and manually annotating data is both expensive and infeasible.

[0065] In summary, for the detection problem of code smell coexistence in Android applications, the use of static program analysis methods or machine learning methods alone has their own limitations. Therefore, the present invention proposes an Android code smell coexistence detection method that combines static program analysis with machine learning. First, a method for detecting MC smell using static program analysis is proposed, and then, on this basis, a method for detecting the coexistence of MC and MIM smells using static program analysis is proposed. In addition, in order to quickly and accurately provide training samples for subsequent machine learning models, a positive and negative sample automatic generation method is proposed to achieve automatic completion from smell coexistence detection to sample generation, and implement the tool ASSD. Then, 9 machine learning models are used to identify coexisting code smells. Among them, there are 6 traditional machine learning models and 3 improved deep learning models. The 6 traditional machine learning models are Naive Bayes (NaiveBayes, referred to as NB), Decision Tree (DecisionTree, referred to as DT), Random Tree (RandomTree, referred to as RT), Random Forest (RandomForest, referred to as RF), Rule-based Induction Algorithm (JRip) and Bayesian Network (BayesNet, referred to as BN). The three improved deep learning models are respectively an improved convolutional neural network model, an improved recurrent neural network model and an improved CNN-LSTM network model. The machine learning method can automatically extract and learn high-level features by learning features from a large amount of data, which are not easily captured by traditional static program analysis methods. The method of the present invention provides a new method for detecting the coexistence of object-oriented code smells and Android-specific code smells, preparing for subsequent reconstruction, and the ultimate goal is to improve the quality of mobile software.

[0066] Step 1: Get a sample set as a label data set. The specific process is as follows:

[0067] Step 1. Download the Android application developed in Java language from the open source code library GitHub as the code corpus;

[0068] Step 1 and 2: Use JavaParser to parse the Android application source code to generate an abstract syntax tree of the code;

[0069] Step 13: Use static program analysis method to detect MC odor; Based on the MC odor detection results, use static program analysis method to detect the coexistence of MIM and MC odors;

[0070] Step 14: Use two nested hash tables to store the odor detection results;

[0071] Step 15: Automatically generate positive and negative samples based on step 14, and generate a sample set based on the positive and negative samples as a label data set;

[0072] Step 2: Process the sample set to obtain a processed data set;

[0073] Step 3: Build a deep neural network classifier and obtain a trained deep neural network classifier based on the processed data set;

[0074] Step 4: Input the Android code smell to be tested into the trained deep neural network classifier, and output whether MIM and MC code smells coexist in the Android code smell to be tested.

[0075] Specific implementation method 2: This implementation method is different from the specific implementation method 1 in that in step 1, an Android application developed in Java language is downloaded from the open source code library GitHub as a code corpus; the specific process is:

[0076] The code corpus was screened from GitHub according to the following criteria.

[0077] Step 1: The application in the code corpus is at least 1000KB;

[0078] Step 112: The number of stars of the application must be greater than 500;

[0079] The number of stars indicates the attention and popularity of the application, which indirectly reflects the quality of the application.

[0080] Step 113. The last update date of the Android application should be later than April 1, 2023.

[0081] The other steps and parameters are the same as those in the first embodiment.

[0082] Specific implementation method three: This implementation method is different from specific implementation method one or two in that, in the steps one and three, a static program analysis method is used to detect MC odor; based on the MC odor detection result, a static program analysis method is used to detect the coexistence of MIM and MC odors;

[0083] The specific process is:

[0084] 1) Use static program analysis method to detect MC odor; the specific process is:

[0085] Use the Visitor Pattern to traverse all method calls in the Abstract Syntax Tree (AST);

[0086] Recursively trace the call chain of each method call until the starting point of the chain;

[0087] If the length of a call chain exceeds the set threshold (the default is 3 layers), the call chain is considered to be an over-coupled message chain smell and is marked as an MC smell;

[0088] The MC smell detection method based on static program analysis is shown in Algorithm 1.

[0089] 2) Based on the MC odor detection results, a static program analysis method is used to detect the coexistence of MIM and MC odors.

[0090] The other steps and parameters are the same as those in the first or second embodiment.

[0091] Specific implementation method 4: This implementation method is different from any one of specific implementation methods 1 to 3 in that two nested hash tables are used in step 14 to store the odor detection results;

[0092] The specific process is:

[0093] Record the name of each code smell and the number of times each code smell appears in the current class in a hash table called map2;

[0094] Then, the classpath of each class is combined with map2 to form a more comprehensive hash table map1.

[0095] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0096] Specific implementation mode 5: This implementation mode is different from any one of specific implementation modes 1 to 4 in that in step 15, positive and negative samples are automatically generated based on step 14, and a sample set is generated based on the positive and negative samples as a label data set;

[0097] The specific process is:

[0098] (1) Create the file directory smellyFile and the file directory nonsmellyFile respectively (corresponding to lines 2-5 and 12-15 in Algorithm 3);

[0099] The file directory smellyFile is used to store code segments containing the coexistence smell of MIM and MC;

[0100] The file directory nonsmellySets is used to store code segments that do not contain MIM and MC odors;

[0101] Create a file directory smellyFile to store positive samples and a file directory nonsmellyFile to store negative samples respectively;

[0102] (2) Traverse smellySets and write the code segments where MIM and MC smells coexist into a java file to generate a positive sample set (corresponding to lines 6-11 in Algorithm 3);

[0103] (3) Traverse nonsmellySets and write the code segments where MIM and MC smells do not coexist into a java file to generate a negative sample set (corresponding to lines 16-20 in Algorithm 3);

[0104] Generate a sample set based on positive samples and negative samples as a label dataset;

[0105] MC is the smell of over-coupled message chains;

[0106] MIM is the method smell of ignoring class member variables.

[0107] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

[0108] In map1, not only the number of code smell types existing in each class path is recorded, but also the specific number of occurrences of each smell is listed in detail.

[0109] In this way, map1 and map2 can not only detect a single code smell, but also further identify and detect the coexistence of code smells, as shown in Algorithm 2.

[0110] Two nested hash tables are used to store the smell detection results. The following contents are stored in the hash table: class path, name of Android-specific code smells and traditional object-oriented code smells and their occurrence counts. Since the class path and the code smell name are in a one-to-many relationship, and the code smell name and the number are in a one-to-one correspondence, a hash table can be used to record multiple code smells and their numbers, and then this hash table can be associated with the corresponding class path and stored in another hash table.

[0111] In order to detect the coexistence of MIM and MC smells, it is first proposed to use a program static analysis method to detect MC smells, and then on this basis, it is proposed to use a static program analysis method to detect the coexistence of MIM and MC smells, and implement the tool ASSD. On the basis of this DAAP, its function is expanded so that it can detect MIM and MC smells separately, and can also detect the coexistence of the two smells. After implementing MC smell detection, in order to realize the coexistence detection of MC and MIM smells, two nested hash tables are used to store the detection results (without changing the detection rules of MIM smell in the original DAAP). To distinguish it from the original DAAP tool, the present invention calls the improved tool ASSD. Figure 3The figure shows the Android code smell coexistence detection and positive and negative sample generation process based on static program analysis. The dotted line part in the figure shows the original DAAP tool, and the functions outside the dotted line are the extended functions of the present invention.

[0112] Specific implementation method 6: This implementation method is different from the specific implementation methods 1 to 5 in that in step 2, the data set is preprocessed to obtain a preprocessed data set; the specific process is:

[0113] Use the Tokenizer tool to convert positive and negative samples into integers and use digital vectors to represent program text information;

[0114] Remove duplicate samples in program text information;

[0115] Normalize the program text information represented by the digital vector;

[0116] The program text information represented by the normalized digital vector is used as the input data of the deep neural network classifier.

[0117] The other steps and parameters are the same as those in Specific Implementation Methods 1 to 5.

[0118] Specific implementation method seven: This implementation method is different from any one of specific implementation methods one to six in that the deep neural network classifier in step three is any one of naive Bayes (NaiveBayes, NB for short), decision tree (DecisionTree, DT for short), random tree (RandomTree, RT for short), random forest (RandomForest, RF for short), rule-based induction algorithm (JRip), Bayesian network (BayesNet, BN for short), improved convolutional neural network model, improved recurrent neural network model, and improved CNN-LSTM network model.

[0119] Naive Bayes (NB), decision tree (DT), random tree (RT), random forest (RF), rule-based induction algorithm (JRip), and Bayesian network (BN) are traditional machine learning models.

[0120] The improved convolutional neural network model, the improved recurrent neural network model, and the improved CNN-LSTM network model are deep learning models.

[0121] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0122] Specific implementation eight: This implementation is different from any one of specific implementations one to seven in that the improved convolutional neural network model sequentially includes an embedding layer, a first convolutional layer, a first maximum pooling layer, a second convolutional layer, a second maximum pooling layer, a third convolutional layer, a third maximum pooling layer, a Dropout layer, a Flatten layer, a first fully connected layer, a second fully connected layer, and an output layer;

[0123] The mask_zero parameter is set to True in the embedding layer;

[0124] The dropout rate of the Dropout layer is set to 0.5;

[0125] The activation function of the first fully connected layer is a relu activation function;

[0126] The activation function of the second fully connected layer is a sigmoid function;

[0127] The improved convolutional neural network model CNN loss function is the binary_crossentropy function, and the optimizer is RMSProp;

[0128] The data set preprocessed in step 2 is input into the improved convolutional neural network model until the loss function converges to obtain a trained improved convolutional neural network model.

[0129] The other steps and parameters are the same as those in Specific Embodiments 1 to 7.

[0130] Specific implementation nine: This implementation is different from any one of specific implementations one to eight in that the improved recurrent neural network model includes an embedding layer, a long short-term memory network LSTM, a Flatten layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, and an output layer in sequence;

[0131] The number of neurons in the first fully connected layer is 64;

[0132] The number of neurons in the second fully connected layer is 32;

[0133] The unit digit of the neurons in the third fully connected layer is 1;

[0134] The activation functions of the first fully connected layer and the second fully connected layer are relu functions;

[0135] The activation function of the third fully connected layer is a sigmoid function;

[0136] The output dimension of the long short-term memory network LSTM is set to 128;

[0137] The output of the third fully connected layer is a one-dimensional vector;

[0138] The loss function of the improved loop neural network model RNN is binary_crossentropy function, and the optimizer is RMSProp;

[0139] The data set preprocessed in step 2 is input into the improved recurrent neural network model until the loss function converges to obtain a trained improved recurrent neural network model.

[0140] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0141] Specific implementation ten: This implementation is different from any one of specific implementations one to nine in that the improved CNN-LSTM network model includes an embedding layer, a first convolutional layer, a first maximum pooling layer, a long short-term memory network LSTM, a Dropout layer, a Flatten layer, a first fully connected layer, and an output layer in sequence;

[0142] The improved CNN-LSTM network model uses early stopping (epoch 50 and patience 10) and model checkpoint callback;

[0143] In order to prevent overfitting and ensure that the model achieves optimal performance, early stopping is used. This method allows the model to terminate training early if the validation loss does not decrease in 10 consecutive iterations before reaching the preset maximum number of iterations. This parameter is called patience in the early stopping method and is set to 10.

[0144] The number of filters in the first convolutional layer is 32, and the kernel size is 3;

[0145] The first maximum pooling layer window size is 2;

[0146] The output dimension of the LSTM layer is set to 128;

[0147] The dropout rate of the Dropout layer is set to 0.5;

[0148] The activation functions of the first convolutional layer and the first maximum pooling layer are relu activation functions;

[0149] The activation function of the LSTM layer is a sigmoid activation function;

[0150] The loss function of the improved CNN-LSTM network model is the binary_crossentropy function, and the optimizer is RMSProp;

[0151] The data set preprocessed in step 2 is input into the improved CNN-LSTM network model until the loss function converges to obtain a trained improved CNN-LSTM network model.

[0152] The number of neurons in the first fully connected layer is {1, 32, 64} respectively.

[0153] The other steps and parameters are the same as those in Specific Embodiments 1 to 9.

[0154] 2. Code smell coexistence detection method:

[0155] The Android code odor coexistence detection method proposed in this invention combines static program analysis with machine learning. Figure 2 shown.

[0156] 2.2 Dataset Construction

[0157] 2.2.1 Acquisition of experimental subjects

[0158] The code corpus required for the experiment of the present invention is taken from GitHub. GitHub is a code hosting platform for version control and collaboration. Repository is the basic unit for storing and managing code projects on GitHub. GitHub provides containers such as repositories to host code projects, and users can perform various operations and interactions in these repositories through GitHub. The existing Android code smell detection tools can only detect applications developed in Java language, so the present invention selects Android applications developed in Java language as code corpus. The code corpus is screened from GitHub according to the following standards. First, the application code library is at least 1000KB. Secondly, the number of stars of the application must be greater than 500. The number of stars indicates the degree of attention and popularity of the application, which indirectly reflects the quality of the program. Finally, the last update date of the Android application should be later than April 1, 2023. The update time indicates the activity and maintenance frequency of the application. Generally speaking, a frequently updated project represents an active developer community and continuous improvement work. Therefore, for many users, the update time is an important consideration for choosing whether to use the program. According to the above standards, GitHub was searched and 70 Android projects were obtained1. Table 1 shows the information of seven of these applications. All program information has been uploaded to: https: / / github.com / biboyu / MC-IMDetection.

[0159] Table 1 Some Android application information

[0160]

[0161]

[0162] Although code smells can be identified using machine learning methods, one of the main limitations of current machine learning-based methods is that they rely heavily on manual annotation of data, and training an efficient machine learning model requires a large amount of labeled data. However, in the field of code smell detection, there is currently a lack of open standard datasets for researchers to use, and manually labeling data is both expensive and infeasible. In traditional application areas of machine learning, such as image processing, speech recognition, etc., there are a large number of labeled datasets for researchers to use, but in the field of code smell detection, there are no similar resources for researchers to use. In order to overcome this difficulty, Liu Hui's team and Sharma's team first used existing code smell detection tools to find the smell of the source program, and then used the output of the tool as the training set for the subsequent machine learning model. The present invention draws on this method and uses the tool ASSD independently developed by the research team of the present invention to identify the coexistence of MC and MIM smells and automatically generate positive and negative sample sets as input for subsequent machine learning models.

[0163] 2.2.2 Detecting Coexisting Android Code Smells Using Static Program Analysis

[0164] Among the existing Android code smell detection tools, DAAP can detect Android-specific code smells. DAAP is an open source tool, and users can expand on it according to their needs. Therefore, the present invention selects DAAP as the basic tool. However, DAAP can only detect a single type of Android-specific code smell, and cannot detect traditional object-oriented code smells, nor can it detect the coexistence of smells. In order to detect the coexistence of MIM and MC smells, it is first proposed to use a program static analysis method to detect MC smells, and then on this basis, it is proposed to use a static program analysis method to detect the coexistence of MIM and MC smells, and implement the tool ASSD. On the basis of this DAAP, its functions are expanded so that it can detect MIM and MC smells separately, and can also detect the coexistence of the two smells. The MC smell detection method based on static program analysis is shown in Algorithm 1.

[0165] As shown in Algorithm 1, line 2 of the code defines the maximum allowed call chain length, which defaults to 3. That is, when detecting the message chain in the code, the developer sets a threshold to indicate how many levels of object method call chains are allowed in the code. In this algorithm, the default maximum length is 3, which means that if the call chain exceeds 3 levels, it will be marked as an "over-coupled message chain". Lines 4-7 of the code check the parameters input from the command line to ensure that a Java source file path is passed in. If the correct path is not passed, the program will output a usage prompt and exit. Lines 8-10 of the code, FileInputStream: Read the Java source file from the input parameter filePath. JavaParse.parse: Use a parsing library (such as JavaParser) to parse the Java source file into an abstract syntax tree (AST), which is stored in the CompilationUnit for subsequent traversal analysis. Line 11 of the code uses the MethodCallVisitor class to traverse all method call expressions in the AST tree. This visitor pattern automatically traverses each node in the Java code to find method calls. Line 16 of the code calls the get.MethodCallChain method to obtain the chain of the current method call (i.e., consecutive get(), set(), and other method calls). Line 17 of the code, If(chain.size()>MESSAGE_CHAIN_THRESHOLD): Determine whether the length of the call chain exceeds the threshold (3 layers). If it does, it is marked as an overly coupled message chain. Lines 18-22 of the code find the chain call by checking the Scope of the method call (the preamble to the method call). For example, in a.getB().getC().getD(), a.getB() is the preamble to getC(), and a.getB().getC() is the preamble to getD(). Use a while loop to trace the entire call chain forward until the starting point of the chain (i.e., there are no more preamble calls);

[0166] To summarize, the algorithm’s workflow is as follows:

[0167] First, read the Java source file from the input parameter and parse it into an abstract syntax tree (AST);

[0168] Then, use the Visitor Pattern to traverse all method calls in the Abstract Syntax Tree (AST);

[0169] Next, recursively trace the call chain of each method call until the starting point of the chain;

[0170] If the length of a call chain exceeds the set threshold (the default is 3 layers), the call chain is considered to be an over-coupled message chain smell and is marked; after implementing MC smell detection, in order to achieve coexistence detection of MC and MIM smells, two nested hash tables are used to store the detection results (without changing the detection rules of MIM smell in the original DAAP). To distinguish it from the original DAAP tool, the present invention calls the improved tool ASSD. Figure 3 The figure shows the Android code smell coexistence detection and positive and negative sample generation process based on static program analysis. The dotted line part in the figure shows the original DAAP tool, and the functions outside the dotted line are the extended functions of the present invention.

[0171]

[0172] like Figure 3 As shown, first, the input of the tool is Java source code. The source code is converted into an abstract syntax tree by JavaParser. Then, the syntax tree is traversed according to the smell detection rules and code metrics to obtain the detection results. Next, two nested hash tables are used to store the smell detection results. The following contents are stored in the hash table: class path, Android-specific code smells, and the names of traditional object-oriented code smells and their number of occurrences. Since the class path and the code smell name are in a one-to-many relationship, and the code smell name and the number are in a one-to-one correspondence, a hash table can be used to record multiple code smells and their numbers, and then this hash table can be associated with the corresponding class path and stored in another hash table.

[0173] During the detection process, the name of each code smell and the number of times it appears in the current class are recorded in a hash table called map2 as a statistical basis. Subsequently, the class path of each class is combined with map2 to form a more comprehensive hash table map1. In map1, not only the number of code smell types existing in each class path is recorded, but also the specific number of occurrences of each smell is listed in detail. In this way, map1 and map2 can not only detect a single code smell, but also further identify and detect the coexistence of code smells. As shown in Algorithm 2.

[0174]

[0175] If the output results of odor coexistence detection are manually input into the subsequent machine learning model, it is time-consuming and prone to errors. Therefore, in order to quickly and accurately provide abundant training samples for the subsequent machine learning model, we propose a method for automatically generating positive and negative samples and integrate it into the tool ASSD, so that ASSD can automatically complete the process from odor coexistence detection to sample generation. As shown in Algorithm 3, this is the process of automatically generating positive and negative samples, which mainly generates positive and negative sample sets from detection results through file operations.

[0176]

[0177] The steps to automatically generate positive and negative samples are as follows:

[0178] (1) Create a file directory smellyFile for storing positive samples and a file directory nonsmellyFile for storing negative samples (corresponding to lines 2-5 and 12-15 in Algorithm 3).

[0179] (2) Traverse smellySets and write each code segment containing MIM and MC into a .java file to generate a positive sample set (corresponding to lines 6-11 in Algorithm 3).

[0180] (3) Traverse nonsmellySets and write each code segment that does not contain MIM and each code segment that does not contain MC into a .java file to generate a negative sample set (corresponding to lines 16-20 in Algorithm 3).

[0181] 2.3 Generating digital vectors

[0182] The positive and negative samples output by the smell detection tool ASSD are program codes, while the input of the machine learning model is a digital vector. Therefore, the program code needs to be converted into a digital vector. This invention draws on the research of scholars such as Sharma and uses the tool Tokenizer to convert the source code into integers. These digital vectors contain all the information of the program and are input into the machine learning model as feature sets. Tokenizer is an open source tool that realizes the function of converting source code into integers. Different code information maps different integers. Figure 4 The following is an example of using Tokenizer to convert Java source code into a digital vector.

[0183] The input and output of Tokenizer are shown below.

[0184] Tokenizer code ={ele(m 1 ),…,ele(m n )}

[0185] input = {Tokenizer code}

[0186] Where ele(m) is the converted digital vector. The digital vector is used as the feature data set of the traditional machine learning model input. The data set format is as follows: Figure 5 shown.

[0187] After being processed by Tokenizer, the obtained feature set may contain some information that is not helpful for the classification task, and these irrelevant features may even have a negative impact on the accuracy and efficiency of the classification. In addition, a large number of features increases the difficulty and time of model training, resulting in an increase in the complexity of the model. In order to optimize the detection effect of the model, the present invention screens the text features generated by Tokenizer and removes those feature items that are irrelevant to the learning task. At the same time, the present invention also unifies the number of features of the model input samples, which is set to 96 (78% of the sample features do not exceed 88). Samples with more than 96 features will not be considered; and for those samples with less than 96 features, they are processed by filling 96 features with zero vectors to improve the performance of the model and the accuracy of detection. In addition, we trimmed the sample data set and found that 99% of the feature values ​​did not exceed 9999. Therefore, the present invention discards "abnormal samples" with feature values ​​exceeding 9999 to avoid the influence of a single sample on the overall model. Compared with code smell, there is an unbalanced data distribution in practice. In actual software projects, the frequency of code smell is relatively low. Therefore, the number of positive samples collected is less than that of negative samples, and the difference between positive and negative samples with coexisting code smell is greater. In order to ensure the balance of positive and negative samples in the training set, the present invention uses an undersampling method on the traditional machine learning training set, that is, randomly selects approximately 1:1 number of positive samples from the negative samples.

[0188] 2.4 Machine Learning Classifier

[0189] The present invention uses 9 machine learning classifiers to detect the coexistence of Android code smells, including 6 traditional machine learning models and 3 improved deep neural network models.

[0190] 2.4.1 Traditional Machine Learning Classifier

[0191] The present invention selects 6 traditional machine learning algorithms for odor coexistence detection, namely Naive Bayes (NB), Decision Tree (DT), Random Tree (RT), Random Forest (RF), Rule-based Induction Algorithm (JRip), and Bayesian Network (BN). Among them, the first 5 algorithms refer to the research results of scholars such as Fontana, and select the 5 learning algorithms with the best performance, namely NB, DT, RT, RF, and JRip. Compared with Naive Bayes, Bayesian Network can more flexibly represent the dependency relationship between variables, provide probabilistic classification prediction by modeling the dependency relationship between variables, and can adapt to different data conditions and feature independence assumptions, and often achieve excellent results in some classification problems. Therefore, the present invention selects Bayesian Network as the sixth machine learning model.

[0192] 2.4.2 Deep Neural Network Classifier

[0193] The three deep neural network models used in the present invention are respectively an improved convolutional neural network model, an improved recurrent neural network model and an improved long short-term memory network. These three models are classic models of deep learning and are also the most frequently used deep learning models in the field of code smell detection. The structures of the three deep learning models used in the present invention are improved based on the research of Sharma team and Liu Hui team.

[0194] 2.4.2.1 Improved Convolutional Neural Network Model

[0195] Convolutional Neural Networks (CNN) is a deep feedforward neural network with local connections and weight sharing. Its most important feature is the convolution operation, and it is one of the representative algorithms of deep learning. The convolutional neural network model structure used in the present invention for detecting odor is as follows: Figure 6 As shown in Figure 2, the model consists of a feature extraction part and a classification part. Figure 6 As shown, the feature extraction part consists of convolutional layers and maximum pooling layers. These layers constitute the hidden layers of the architecture. The convolutional layer performs convolution operations according to the specified filter and kernel parameters and calculates the network weights to the next layer, while the maximum pooling layer can reduce the number of parameters and computational complexity while retaining important numerical features. The hidden layer consisting of convolutional layers and pooling layers is repeated three times. Repeated convolutional layers can help the network learn more complex features because each layer can further abstract features based on the previous layer. The output of the pooling layer is then connected to the Dropout layer. The Dropout layer is a regularization technique used to prevent neural networks from overfitting. During training, it "drops" (i.e. temporarily removes) a portion of neurons with a probability of 0.5, forcing the network to learn more robust features.

[0196] The output of the last Dropout layer is fed into a densely connected classifier network consisting of two stacked fully connected layers. These classifiers process one-dimensional vectors, while the output from the last hidden layer is a three-dimensional tensor. This tensor corresponds to the height, width and channel of the input sample, where the number of channels is one. Therefore, a Flatten layer is first used, which converts the data into the appropriate format before it is fed into the first fully connected layer with 32 units and the ReLU activation function. This is followed by a second fully connected layer with one unit and the sigmoid activation function. The last fully connected layer constitutes the output layer and contains a single neuron that predicts whether a given instance belongs to the positive or negative class of the odor being studied. This layer uses the sigmoid activation function to produce a result in the range of 0 to 1.

[0197] Table 2 lists the parameters of each layer selected in the improved CNN model. In the embedding layer, the input dataset contains 2239 different characters, each character is encoded as an 8-dimensional vector, and the maximum length of the input data is set to 243. The number of filters determines the number of convolution kernels in the convolution layer, and the size of the convolution kernel determines the size of the convolution window. The window size of the pooling layer determines the window size of the downsampling process in the pooling process. The number of iterations of the model, that is, the epoch value, is the maximum number of iterations during the model training process. In order to prevent overfitting and ensure that the model achieves optimal performance, we use the early stopping method. This method allows the model to terminate the training early if the verification loss does not decrease in 10 consecutive iterations before reaching the preset maximum number of iterations. This parameter is called patience in the early stopping method and is set to 10.

[0198] Table 2 CNN model layer parameter values

[0199]

[0200] 2.4.2.2 Improved Recurrent Neural Network Model

[0201] Recurrent Neural Networks (RNN) is a classic deep learning algorithm that is suitable for processing time series data and is widely used in speech processing and natural language processing. Figure 7 The figure shows the structure of the RNN model for detecting odor coexistence, which is improved based on the research of Sharma team and Liu Hui team. The model consists of an embedding layer, followed by the feature learning part - a hidden LSTM layer and Flatten layer. Finally, there is the classification part (consisting of three fully connected layers).

[0202] Similar to the CNN model, first, the converted text information is used as input to the LSTM layer through the embedding layer. In the original RNN training, as the training time increases and the number of network layers increases, the problem of gradient explosion or gradient disappearance is prone to occur, resulting in the inability to process longer sequence data, thereby failing to obtain information about long-distance data. In order to solve these problems, the present invention uses a long short-term memory network. One of the advantages of LSTM networks is that they can effectively process sequences of different lengths. The output dimension of the LSTM layer is set to 128. After the Flatten layer, the data enters three fully connected layers, the activation function of the first two layers is the relu function, and the activation function of the final output layer is the sigmoid function, outputting a one-dimensional vector. Similar to the CNN model, the early shopping method (epoch is 50 and patience is 10) and the model checkpoint (ModelCheckpoint) callback are used here. In addition, the model loss function is the binary_crossentropy function, and the optimizer is RMSProp.

[0203] Table 3 shows the hyperparameters of the model. The parameters of the embedding layer are consistent with the settings in the CNN model. In the LSTM layer, the dimension of each LSTM hidden unit is 128. For the fully connected layer, the parameters represent the dimension size of the output features of each layer. Based on the best performing parameter combination and after multiple iterations, the model is retrained to capture its best performance state.

[0204] Table 3 RNN model layer parameter values

[0205]

[0206] 2.4.2.3 Improved CNN-LSTM network model

[0207] As a feedforward neural network, CNN has shown excellent performance in image and natural language processing tasks. LSTM network is particularly suitable for training tasks of text data because of its internal storage mechanism that can capture and remember the historical characteristics of data. In the field of code smell detection, the present invention adopts a method that combines local CNN and LSTM networks to simultaneously process spatial features and short-term temporal dependencies. This combined architecture was originally called Long-term Recurrent Convolutional Network, and in the present invention, it is named CNN-LSTM network model, which represents the model structure that integrates the CNN hidden layer with the LSTM network. Figure 8 The figure shows the structure of the CNN-LSTM network model for detecting the coexistence of odors. This structure is improved based on the research of Sharma's team and Liu Hui's team.

[0208] like Figure 8 As shown in Figure 1, the hidden layer of CNN is composed of multiple convolutional layers and pooling layers stacked together, and this process is repeated once in the experiment. Between each convolutional layer and pooling layer, the ReLU activation function is used to speed up the convergence of the model. The data is first sent to the convolutional layer and pooling layer of CNN through the embedding layer. After being processed by these layers, the features are locally aggregated and extracted. Subsequently, these local features are passed to the LSTM network, and the Sigmoid activation function is used to further capture the long-term dependencies of these features. After being processed by the LSTM layer, the Dropout layer (the dropout rate is set to 0.5) and the Flatten layer, the result is finally output in the fully connected layer. The model uses the early shopping method (epoch is 50 and patience is 10) and the model checkpoint callback. The model loss function is the binary_crossentropy function and the optimizer is RMSProp. Table 4 shows the parameter values ​​of each layer of the model structure.

[0209] Table 4 CNN-LSTM model layer parameter values

[0210]

[0211] The following examples are used to verify the beneficial effects of the present invention:

[0212] Embodiment 1:

[0213] Research Questions: This paper evaluates the effectiveness of the proposed method by answering the following five research questions (RQ).

[0214] RQ1: What is the detection effect of the MC odor detection method based on static program analysis proposed in this paper?

[0215] RQ2: Can traditional machine learning models identify the coexistence of object-oriented code smells and Android-specific code smells? If so, which model has the best detection effect?

[0216] RO3: Is the detection effect of traditional machine learning models better than that of deep learning models?

[0217] RQ4: Compared with manual detection of odor coexistence, is the detection effect of the method of the present invention better than that of the manual detection method?

[0218] RQ5: How much time does the method proposed in this invention take in the process of constructing a sample data set, training a traditional machine learning model, and using the trained model for prediction?

[0219] Experimental design

[0220] Regarding RQ1: Since there are existing Android-specific code smell detection tools, it is not possible to detect traditional object-oriented code smells, nor can it detect the coexistence of smells. In order to detect the coexistence of MIM and MC smells, the present invention proposes to use a program static analysis method to detect MC smells, and then on this basis proposes to use a static program analysis method to detect the coexistence of MIM and MC smells, and implements the tool ASSD. The accuracy of the MC smell detection method is the basis for the subsequent use of machine learning models for odor coexistence detection. Therefore, in order to verify the effectiveness of the MC smell detection method proposed by the present invention, the tool ASSD developed by the present invention is used to detect MC smells in 6 open source applications, and then the detection results are manually analyzed. Table 5 shows the detailed information of the 6 applications.

[0221] Table 5 Six Android applications used in the test

[0222]

[0223]

[0224] For RQ2: The present invention uses code text information as a feature set to input 6 traditional machine learning models. The model is tested by ten-fold cross validation, and the recognition effects of the 6 methods are compared using precision, recall and F1 values. The 6 traditional machine learning algorithms selected by the present invention are NB, DT, RT, RF, JRip, and BN. Among them, the first 5 algorithms refer to the research results of scholars such as Fontana, and the 5 learning algorithms with the best performance are selected, namely NB, DT, RT, RF, and JRip. Compared with naive Bayes, Bayesian networks can more flexibly represent the dependencies between variables, provide probabilistic classification predictions by modeling the dependencies between variables, and can adapt to different data conditions and feature independence assumptions. In some classification problems, excellent results are often achieved. Therefore, the present invention selects Bayesian networks as the sixth machine learning model. The training set of the model is the 70 Android applications shown in Section 2.2.1. The test set selects 8 open source applications as the test set. These 8 open source projects are screened using the standards of Section 3.2.1 and have no intersection with the 70 applications in the training set. The detailed information of the 8 applications used in the experiment is shown in Table 8. Table 8 includes a total of 11 applications, of which the first 8 applications are the 8 applications used to answer RQ2.

[0225] For RO3: With the widespread application of deep learning in detecting code smells, more and more software engineering researchers use deep learning to detect code smells and have achieved good results. However, the research of researchers such as Sharma shows that not all code smells are suitable for using deep learning as a detection method. In order to try to use deep learning to detect the feasibility of coexistence of Android code smells and compare it with the detection effect of traditional machine learning, the present invention selects the three most frequently used deep learning models in the existing code smell detection research as the basic model, improves it, and detects the coexistence of odors. The training set used in the experiment is the 70 applications shown in Section 2.2.1. 11 open source applications are used as the test set, and these 11 applications do not overlap with the 70 applications in the training set. Table 6 shows the detailed information of the 11 applications.

[0226] Table 6: 11 Android applications used in the test set

[0227]

[0228]

[0229] Regarding RQ4: Since the existing tools cannot detect the two smells studied in this invention, for comparative research, this invention adopts a comparison method with manual detection. Manual detection here means first using the tool DAAP to detect the source code to be analyzed and identify the MIM smell therein. Then, in the code segment containing the MIM smell, the MC smell is manually identified. After detecting the co-existing smells, the tool Tokenizer is used to convert the code into a digital vector, which is input into the subsequent machine learning model, and the evaluation index is obtained according to the model output. This invention selects the 8 open source applications shown in Table 6 as the test set. Table 6 includes a total of 11 applications, of which the first 8 applications are the 8 applications used to answer RQ4. The manual analysis process is as follows:

[0230] 1. Select evaluators: Two computer science graduate students with experience in software engineering were selected to perform manual testing (the first and third authors of this invention). The two students were very familiar with both smells and had used the detection tool to analyze the output of a single smell.

[0231] 2. Manual detection: For the 8 applications to be analyzed, each student independently analyzed the source code of 4 applications. The evaluator first used the tool DAAP to detect the source program to be analyzed, and then manually identified the MC smell in the code segment containing the MIM smell and recorded the detection results.

[0232] 3. Discussion and negotiation: After the test was completed, the two students exchanged the test results and performed a consistency check on the results to ensure that their assessments were reliable and consistent. When there was a disagreement, the two students discussed and negotiated on the issue of disagreement to understand the other party's point of view and reach a consensus. For samples that could not reach a consensus and had disagreements, statistical analysis was performed to determine the causes and effects of the disagreement. The second author of the present invention performed an independent assessment to help resolve the disagreement.

[0233] 4. Statistical analysis: After manual detection, the code is converted into a digital vector using the Tokenizer tool and input into the subsequent machine learning model. Evaluation indicators are obtained based on the model output, namely precision, recall, and F1 value.

[0234] Regarding RQ5: Studies have shown that model complexity is an important factor to consider when selecting a model, because complex models may require additional resource consumption (such as storage and computing time costs). Deep learning models are more complex than traditional machine learning models. Therefore, time complexity is an important metric when evaluating deep learning models. This research question quantitatively reveals the training and testing time of the proposed method. In order to evaluate the time efficiency of the method of the present invention. The present invention runs the entire process of the experiment on a computer (8GB RAM, Intel Core CPU i7-7700), and records the time consumption (in minutes) of generating data sets and training and testing processes of classifiers to analyze the time performance of the proposed method.

[0235] Model evaluation indicators

[0236] The present invention uses three classic evaluation indicators, namely, precision, recall and F1-score, to evaluate the performance of different classifiers. The relevant formula is as follows:

[0237]

[0238]

[0239]

[0240] Among them, TP (True Positive) means that the model has detected the actual co-existing code smells. FN (False Negative) means that the model predicts the actual fragments with co-existing code smells as fragments without co-existing code smells, resulting in missed detection. FP (False Positive) means that the model mistakenly identifies normal code fragments as fragments with code smells, resulting in false positives. Precision measures the proportion of samples that the model determines to have co-existing code smells, which actually co-exist. Recall measures the proportion of samples that are correctly identified by the model among all samples that actually have co-existing code smells. The F1 score is a balance between precision and recall. Using the calculation results of these evaluation indicators, the performance of different classifiers in identifying the co-existence of code smells can be fairly evaluated.

[0241] Experimental results and analysis

[0242] RQ1: What is the detection effect of the MC odor detection method based on static program analysis proposed in this paper?

[0243] Table 7 shows the precision, recall, and F1 values ​​for the six applications tested.

[0244] Table 7 Detection results of MC code smells of 6 apps using ASSD

[0245]

[0246] As shown in Table 7, the maximum precision is 1, the minimum is 0.729, and the average is 0.859; the maximum recall is 1, the minimum is 0.5, and the average is 0.892; the maximum F1 value is 0.945, the minimum is 0.666, and the average is 0.86. Therefore, the MC odor detection method based on static program analysis proposed in the present invention has a good effect and can detect MC odors in the program. It lays a good foundation for odor coexistence detection, thereby providing high-quality input samples for subsequent machine learning models.

[0247] RQ2: Can traditional machine learning models identify the coexistence of object-oriented code smells and Android-specific code smells? If so, which model has the best detection effect?

[0248] Table 8 shows the detection results of six traditional machine learning models. The precision, recall and F1 values ​​of the six models are all high, among which the random forest model (RF) has the best detection effect. It shows that it is feasible to detect the coexistence of MIM and MC odors using machine learning methods. The order of F1 values ​​of the detection results of the six classifiers is: RF>RT>JRip>DT>NB>BN. Therefore, when detecting the coexistence of MC and MIM odors, the RF model has the best detection effect. The detection results are analyzed as follows: First, RF is a classifier based on ensemble learning, which classifies by building multiple decision trees, while RT is a single decision tree. In addition, RF can effectively handle the correlation between features and has good generalization ability when processing large amounts of data. Secondly, RT is better than JRip. This is because although JRip usually classifies according to the rules in the data set, when processing large-scale data, JRip may be restricted by the rules and perform poorly. Finally, JRip's detection effect is better than DT. This is because compared with DT, JRip can better capture the patterns and regularities in the data and perform better in complex situations.

[0249] Table 8 Odor coexistence detection results of different classifiers

[0250]

[0251] The Fontana team used 16 traditional machine learning models to detect 4 object-oriented code smells, but they only considered a single type of object-oriented code smell, without considering the coexistence of smells, let alone Android code smells. The present invention selects the 5 models with the best detection effect in its research, namely DT, RF, JRip, NB and BN for comparison. The detection results of the Fontana team are: DT>RF>JRip>NB>BN. It can be seen that when using machine learning models to detect code smells, the detection effect of the model will vary depending on the type of smell and the way the smell exists. This result is consistent with the conclusion of researchers such as Sharma. That is, the detection effect of the machine learning model varies due to different types of smells. It can be seen that in actual program development and maintenance, the appropriate machine learning model should be selected according to the actual situation of the smell. No machine learning model can achieve good results in all code smell identification.

[0252] In summary, when dealing with the coexistence of MC and MIM odors, random forest is the best choice, followed by random tree.

[0253] RQ3: Is the detection effect of traditional machine learning models due to deep learning models?

[0254] Table 9 shows the detection results using three improved deep learning models. The model with the best detection effect is the improved LSTM, with an F1 value of 89.7%. Combining Tables 10 and 11, the top six machine learning classifiers with the best detection effect are: RF>LSTM>RT>CNN>RNN>JRip. Therefore, when detecting code smells where MC and MIM coexist in Android applications, RF has the best detection effect. Although the effects of the three improved deep neural network models are not inferior, the detection effect of the RF model is better.

[0255] Table 9 Odor coexistence detection results under deep learning

[0256]

[0257] The changes of the three performance indicators of the three improved deep neural network models on the training set and the validation set. From the curve change trend, it can be seen that with the increase in the number of iterations, the three indicators of the CNN, RNN and CNN-LSTM models are higher in the training set and the validation set and the difference is not large, indicating that the three models fit the rules of the data and can detect the coexistence of code smells well.

[0258] RQ4: Is the detection effect of the method of the present invention better than that of the manual detection method for the coexistence of odors? The first 8 Android applications shown in Table 8 were tested, and Table 10 lists the comparison results of 6 machine learning models and manual detection. The detection results of the 6 machine learning models used in the present invention are all better than the manual detection results, among which RF has the best detection effect. Compared with manual detection, the F1 value is improved by an average of 9.1% (90.1%-81%). In addition, the manual detection process takes about 120 hours.

[0259] Research shows that due to the scale and complexity of mobile software applications, manually analyzing smells from source code is a very difficult and challenging task, which is time-consuming and prone to errors. In addition, since manual detection of code smells relies heavily on the experience and judgment of developers, different people may have inconsistent smell detection results for the same code snippet.

[0260] RQ5: How much time does the proposed method take to construct a sample dataset, train a traditional machine learning model, and use the trained model for prediction?

[0261] Due to space limitations, Table 11 lists the time (in minutes) taken to generate training sets for 8 of the Android applications. Detailed information on the time consumed by all 70 applications is available at https: / / github.com / biboyu / MC-MIMDetection.

[0262] Table 10 Performance comparison of different classifiers

[0263]

[0264] As shown in Table 11, it takes a lot of time to collect sample data. It takes 1540 minutes (about 25.7 hours) to extract training set data from 70 Android applications, with an average of 22 minutes per project. Among them, the most time-consuming program is a2dpvolume-master (1007 classes, 1183 methods, 89051 lines of code, and the time consumption is 151 minutes), and the least time-consuming program is Dali-master (33 classes, 201 methods, 1548 lines of code, and the time consumption is 9 minutes).

[0265] Table 11 Time consumption for generating training set data

[0266]

[0267]

[0268] Table 12 Time consumption of the training and testing processes of the six machine learning models (in seconds)

[0269]

[0270] As shown in Table 12, compared to the training set generation process which took more than 25 hours, the average training time of the machine learning classifier on the test set was 0.1 seconds. The training time of the random forest model with the best detection effect was 1.77 seconds. During the test, all six classifiers had good time performance.

[0271] Due to the complexity of the model, deep learning usually requires more computing resources and time training. Table 13 shows the time consumption (in seconds) of the training and testing processes of the three improved deep learning models. Compared with the training set generation process that took more than 25 hours, the average training time of deep learning on the test set was 171 seconds. The training time of LSTM, the model with the best detection effect, was 3600 seconds, which was not superior to the other two deep learning models in terms of time performance. Combining the results of Tables 12 and 13, it can be seen that in terms of detection time, the traditional machine learning model has an advantage over the deep learning model.

[0272] Table 13 Detection time of different methods

[0273]

[0274] In summary, among the nine machine learning models, the RF model has advantages in detection accuracy and detection time. Therefore, the RF model is recommended when detecting the coexistence of MC and MIM odors in Android applications.

[0275] The present invention may also have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. An Android code smell coexistence detection method integrating static program analysis and machine learning, characterized by: The specific process of the method is: Step 1: Get a sample set as a label data set. The specific process is as follows: Step 1. Download the Android application developed in Java language from the open source code library GitHub as the code corpus; Step 1 and 2: Use JavaParser to parse the Android application source code to generate an abstract syntax tree of the code; Step 13: Use static program analysis methods to detect MC odors; Based on the MC odor detection results, a static program analysis method is used to detect the coexistence of MIM and MC odors; Step 14: Use two nested hash tables to store the odor detection results; Step 15: Automatically generate positive and negative samples based on step 14, and generate a sample set based on the positive and negative samples as a label data set; Step 2: Process the sample set to obtain a processed data set; Step 3: Build a deep neural network classifier and obtain a trained deep neural network classifier based on the processed data set; Step 4: Input the Android code smell to be tested into the trained deep neural network classifier, and output whether MIM and MC code smells coexist in the Android code smell to be tested.

2. The Android code smell coexistence detection method integrating static program analysis and machine learning according to claim 1 is characterized in that: In the step 1, an Android application developed in Java language is downloaded from the open source code library GitHub as a code corpus; The specific process is: Step 1: The application in the code corpus is at least 1000KB; Step 112: The number of stars of the application must be greater than 500; Step 113. The last update date of the Android application should be later than April 1, 2023.

3. The Android code smell coexistence detection method integrating static program analysis and machine learning according to claim 2 is characterized in that: In the step 1-3, a static program analysis method is used to detect the MC odor; based on the MC odor detection result, a static program analysis method is used to detect the coexistence of MIM and MC odors; the specific process is: 1) Use static program analysis method to detect MC odor; the specific process is: Use the visitor pattern to traverse all method calls in the abstract syntax tree; Recursively trace the call chain of each method call until the starting point of the chain; If the length of a call chain exceeds the set threshold, the call chain is considered to be an over-coupled message chain smell and is marked as an MC smell; 2) Based on the MC odor detection results, a static program analysis method is used to detect the coexistence of MIM and MC odors.

4. The Android code smell coexistence detection method integrating static program analysis and machine learning according to claim 3 is characterized in that: In step 1-4, two nested hash tables are used to store the odor detection results; the specific process is as follows: Record the name of each code smell and the number of times each code smell appears in the current class in a hash table called map2; Then, the classpath of each class is combined with map2 to form a more comprehensive hash table map1.

5. The Android code smell coexistence detection method integrating static program analysis and machine learning according to claim 4 is characterized in that: In the step 15, positive and negative samples are automatically generated based on the step 14, and a sample set is generated based on the positive and negative samples as a label data set; the specific process is: (1) Create the file directory smellyFile and the file directory nonsmellyFile respectively; The file directory smellyFile is used to store code segments containing the coexistence smell of MIM and MC; The file directory nonsmellySets is used to store code segments that do not contain MIM and MC odors; Create a file directory smellyFile to store positive samples and a file directory nonsmellyFile to store negative samples respectively; (2) Traverse smellySets and write the code segments where MIM and MC smells coexist into a java file to generate a positive sample set; (3) Traverse nonsmellySets and write the code segments that do not contain MIM and MC smells into a java file to generate a negative sample set; Generate a sample set based on positive samples and negative samples as a label dataset; MC is the smell of over-coupled message chains; MIM is the method smell of ignoring class member variables.

6. The Android code smell coexistence detection method integrating static program analysis and machine learning according to claim 5 is characterized in that: In step 2, the data set is preprocessed to obtain a preprocessed data set; the specific process is: Use the Tokenizer tool to convert positive and negative samples into integers and use digital vectors to represent program text information; Remove duplicate samples in program text information; Normalize the program text information represented by the digital vector; The program text information represented by the normalized digital vector is used as the input data of the deep neural network classifier.

7. The Android code smell coexistence detection method integrating static program analysis and machine learning according to claim 6 is characterized in that: The deep neural network classifier in step three is any one of naive Bayes, decision tree, random tree, random forest, rule-based induction algorithm, Bayesian network, improved convolutional neural network model, improved recurrent neural network model, and improved CNN-LSTM network model.

8. The Android code smell coexistence detection method integrating static program analysis and machine learning according to claim 7 is characterized in that: The improved convolutional neural network model includes an embedding layer, a first convolutional layer, a first maximum pooling layer, a second convolutional layer, a second maximum pooling layer, a third convolutional layer, a third maximum pooling layer, a Dropout layer, a Flatten layer, a first fully connected layer, a second fully connected layer, and an output layer in sequence; The mask_zero parameter is set to True in the embedding layer; The dropout rate of the Dropout layer is set to 0.5; The activation function of the first fully connected layer is a relu activation function; The activation function of the second fully connected layer is a sigmoid function; The loss function is binary_crossentropy function, and the optimizer is RMSProp; The data set preprocessed in step 2 is input into the improved convolutional neural network model until the loss function converges to obtain a trained improved convolutional neural network model.

9. The Android code smell coexistence detection method integrating static program analysis and machine learning according to claim 8 is characterized in that: The improved recurrent neural network model includes an embedding layer, a long short-term memory network LSTM, a Flatten layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, and an output layer in sequence; The number of neurons in the first fully connected layer is 64; The number of neurons in the second fully connected layer is 32; The unit digit of the neurons in the third fully connected layer is 1; The activation functions of the first fully connected layer and the second fully connected layer are relu functions; The activation function of the third fully connected layer is a sigmoid function; The output dimension of the long short-term memory network LSTM is set to 128; The output of the third fully connected layer is a one-dimensional vector; The loss function is binary_crossentropy function, and the optimizer is RMSProp; The data set preprocessed in step 2 is input into the improved recurrent neural network model until the loss function converges to obtain a trained improved recurrent neural network model.

10. The Android code smell coexistence detection method integrating static program analysis and machine learning according to claim 8 is characterized in that: The improved CNN-LSTM network model includes an embedding layer, a first convolutional layer, a first maximum pooling layer, a long short-term memory network LSTM, a Dropout layer, a Flatten layer, a first fully connected layer, and an output layer in sequence; The improved CNN-LSTM network model uses early stopping and model checkpoint callback; The number of filters in the first convolutional layer is 32, and the kernel size is 3; The first maximum pooling layer window size is 2; The output dimension of the LSTM layer is set to 128; The dropout rate of the Dropout layer is set to 0.5; The activation functions of the first convolutional layer and the first maximum pooling layer are relu activation functions; The activation function of the LSTM layer is a sigmoid activation function; The loss function of the improved CNN-LSTM network model is the binary_crossentropy function, and the optimizer is RMSProp; The data set preprocessed in step 2 is input into the improved CNN-LSTM network model until the loss function converges to obtain a trained improved CNN-LSTM network model.

Images