Code peculiar smell detection method and system based on AST code measurement and code semantics

Abstract

The invention provides a code peculiar smell detection method and system based on AST code measurement and code semantics. The method comprises the following steps of: firstly, constructing a FA-AST (Flow-Augmented Abstraction Syntax Tree) by analyzing a code snippet, and extracting three types of AST-based code metrics, namely a structure metric, a syntactic metric and a semantic metric from the FA-AST; then, a deep learning model is used for capturing contextual information and a dependency relationship of the code from the source code so as to extract code semantics; and finally, by fusing AST-based code measurement and code semantics, inputting a classification model to carry out training and testing so as to identify the peculiar smell in the code and evaluate the severity of the peculiar smell. Compared with software code measurement extracted by a traditional static code analysis tool, the AST-based code measurement provided by the invention is more comprehensive and easier to extract, and has obvious advantages in the aspect of improving the code odor detection precision and efficiency.

Description

Technical Field

[0001] The present invention belongs to the field of software engineering, and in particular relates to a code smell detection method and system based on AST code metrics and code semantics. Background Art

[0002] In the process of software development and maintenance, the existence of code smells is often the main reason for the decline of software quality and the increase of maintenance costs. Code smells refer to bad smells in the code that may cause defects or reduce the readability and maintainability of the code. Common code smells include God classes, long methods, data classes, and feature envy. These smells are usually caused by improper design or irregular implementation, but they do not necessarily lead to software errors directly.

[0003] Existing code smell detection technologies can be mainly divided into three categories: 1) Rule-based static analysis methods: These methods rely on predefined rule sets and detect code snippets that do not comply with these rules by parsing the code. For example, detection rules based on tools such as Checkstyle and PMD. The advantage of these methods is that they are simple and easy to implement, but the disadvantage is that they are prone to produce a large number of false positives and false negatives. 2) Code metric-based detection methods: This type of method calculates code metrics such as code complexity, annotation rate, coupling degree, and compares them with predefined thresholds to detect possible code smells. These methods are more dependent on the rationality of code metrics and have difficulty processing deep semantic information in the code. 3) Machine learning-based detection methods: This type of method builds a model that can automatically detect code smells by learning from a large amount of annotated data. Although these methods can reduce false positives to a certain extent, they are highly dependent on data and often do not work well when dealing with complex code structures and semantics.

[0004] Therefore, when facing complex code scenarios, the existing technologies still have some research gaps and limitations that need to be addressed. On the one hand, software code metrics are difficult to extract and vary greatly at different code levels: Existing code smell identification methods usually rely on software code metrics. However, the extraction process of code metrics is usually complex and difficult to unify, especially at different code granularity levels, the applicability and consistency of metrics are poor. This leads to a decrease in accuracy in identifying code smells, especially when dealing with large-scale or complex code bases. On the other hand, highly abstract code semantic information is difficult to convey the structural details of the code: When using code semantic information, existing methods often focus on highly abstract semantic features and ignore the structural details of the code. Although this method can identify code smells in some cases, it lacks a deep understanding of the code structure, which leads to poor performance in dealing with complex smell identification tasks.

[0005] Therefore, how to combine the structural features of the code with the semantic information to improve the accuracy of code smell detection is a technical problem that needs to be solved urgently. Summary of the invention

[0006] The purpose of the present invention is to solve the defect of low detection accuracy in the prior art when facing complex smell identification tasks, and to provide a code smell detection method and system based on AST code metrics and code semantics.

[0007] The specific technical solutions adopted by the present invention are as follows:

[0008] In a first aspect, the present invention provides a method for detecting code smell based on AST code metrics and code semantics, which comprises the following steps:

[0009] S1. Convert the code snippet to be detected into a vector representation through a deep learning-based code representation learning technology or a code embedding technology as the code semantics corresponding to the code snippet to be detected;

[0010] S2, extracting structural metrics, syntactic metrics and semantic metrics from the code snippet to be detected, and using them together as code metrics corresponding to the code snippet to be detected;

[0011] S3, using the code semantics and the code metrics as inputs of a first classification model, and obtaining a prediction result of whether the code snippet to be detected has code smell by the trained first classification model;

[0012] S4. Using the code semantics and the code metrics as inputs of a second classification model, and obtaining a prediction result of the code smell degree of the code snippet to be detected by the trained second classification model.

[0013] As a preferred embodiment of the first aspect, in S1, the code snippet to be detected is input into the Code2Vec code embedding model to parse the abstract syntax tree of the code snippet, thereby converting it into its corresponding vector representation.

[0014] As a preferred embodiment of the above-mentioned first aspect, in S2, the steps of extracting structural metrics, syntactic metrics and semantic metrics from the code snippet to be detected are as follows: first, obtain the abstract syntax tree AST of the code snippet to be detected, and add the edges corresponding to the control flow and the data flow in the abstract syntax tree AST to form a flow abstract syntax tree FA-AST; then, the structural metrics, syntactic metrics and semantic metrics are counted by traversing the nodes and edges in the flow abstract syntax tree FA-AST, wherein: the structural metrics are 10 structural parameters, namely: tree depth, tree width, number of nodes, number of leaf nodes, maximum parameter list length, number of statement blocks, number of code blocks, branching factor, node coverage The syntactic metric is the number of 20 types of nodes, namely: reference type, variable declarator, member reference, literal, method declaration, code block statement, return statement, collection, string, field declaration, formal parameter, expression statement, assignment, current object reference, method call, local variable declaration, basic type, conditional statement, binary operation and class creator; the semantic metric is the number of 9 types of edges, namely: parent-child node relationship, sibling node relationship, statement order relationship, identifier connection relationship, variable use relationship, branch when the If statement condition is true, branch when the If statement condition is false, loop execution relationship, loop control relationship.

[0015] As a preferred embodiment of the above-mentioned first aspect, the first classification model and the second classification model adopt a machine learning model or a deep learning model; preferably, the machine learning model is an XGB model or an RF model, and the code semantics and the normalized code metrics are directly spliced ​​as the input of the classification model; preferably, the deep learning model is a CNN model or a DNN model as the backbone network to construct two branch networks, and the code semantics and the normalized code metrics are each extracted features through different branch networks and then spliced ​​as the input of the Softmax classifier.

[0016] As a preferred embodiment of the first aspect, the first classification model is pre-trained in a code smell identification task using a labeled data set with binary classification labels, where the binary classification labels are used to indicate whether a code snippet has a code smell.

[0017] As a preferred embodiment of the first aspect, the second classification model is pre-trained in a code smell severity determination task using a labeled data set with multi-classification labels, wherein the multi-classification labels are used to indicate the severity of a code snippet when it is considered as a code smell.

[0018] In a second aspect, the present invention provides a code smell detection system based on AST code metrics and code semantics, which includes:

[0019] A code semantics acquisition module is used to convert the code snippet to be detected into a vector representation through a code representation learning technology or a code embedding technology based on deep learning, as the code semantics corresponding to the code snippet to be detected;

[0020] A code metric acquisition module is used to extract structural metrics, syntactic metrics and semantic metrics from the code snippet to be detected, and use them together as code metrics corresponding to the code snippet to be detected;

[0021] A first detection module, configured to use the code semantics and the code metrics as inputs of a first classification model, and obtain a prediction result of whether a code snippet to be detected has code smell by the trained first classification model;

[0022] The second detection module is used to use the code semantics and the code metrics as inputs of a second classification model, and obtain a prediction result of the code smell degree of the code fragment to be detected by the trained second classification model.

[0023] In a third aspect, the present invention provides a computer program product, comprising a computer program / instructions, characterized in that when the computer program / instructions are executed by a processor, the code smell detection method based on AST code metrics and code semantics as described in any one of the first aspects above can be implemented.

[0024] In a fourth aspect, the present invention provides a computer-readable storage medium, characterized in that a computer program is stored on the storage medium, and when the computer program is executed by a processor, the code smell detection method based on AST code metrics and code semantics as described in any one of the first aspects above is implemented.

[0025] In a fifth aspect, the present invention provides a computer electronic device, characterized in that it includes a memory and a processor;

[0026] The memory is used to store computer programs;

[0027] The processor is used to implement the code smell detection method based on AST code metrics and code semantics as described in any one of the first aspects above when executing the computer program.

[0028] The present invention combines code metrics based on abstract syntax tree (AST) and code semantics to propose a framework for identifying code smells. Specifically, the present invention proposes three AST-based code metrics, which are intended to replace traditional software code metrics. These code metrics are not only easy to extract, but also can be processed uniformly at different code granularities to accurately describe the structural details of the code. In addition, the fusion learning framework proposed in the present invention can not only identify code smells, but also evaluate their severity by combining AST-based code metrics and code semantics.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The present invention provides an alternative solution for structural measurement, syntactic measurement and semantic measurement based on AST, which effectively solves the limitations of traditional software code measurement in extraction difficulty and applicability.

[0031] 2. The detection framework of the present invention combines AST-based code metrics with code semantic features, significantly improving the accuracy and versatility of the code smell identification method.

[0032] 3. This paper verifies the effectiveness of AST-based code metrics and fusion learning framework in the task of identifying code smells and their importance through a variety of experiments, and outperforms the existing state-of-the-art methods on the same dataset. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the specific implementation of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the specific implementation or the prior art description. Some specific embodiments of the present invention will be described in detail in an exemplary but not restrictive manner with reference to the drawings. The same reference numerals in the drawings indicate the same or similar components or parts. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0034] Figure 1 This is a flowchart of the basic steps of the code smell detection method based on AST code metrics and code semantics proposed by the present invention.

[0035] Figure 2 This is a flowchart of a code smell detection method based on AST code metrics and code semantics provided in an embodiment of the present invention.

[0036] Figure 3 A schematic diagram of extracting three types of AST-based code metrics from code snippets provided by an embodiment of the present invention.

[0037] Figure 4 A schematic diagram of screening key node types as syntactic metrics through a voting mechanism provided by an embodiment of the present invention.

[0038] Figure 5 The fusion learning framework provided in the embodiment of the present invention is a schematic diagram of using XGB, RF, DNN and CNN as classifiers respectively.

[0039] Figure 6 A schematic diagram of the module composition of a code smell detection system based on AST code metrics and code semantics provided in an embodiment of the present invention.

[0040] Figure 7 The figure is a schematic diagram of the structure of a computer electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the above-mentioned purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation mode of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in each embodiment of the present invention can be combined accordingly without conflicting with each other.

[0042] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for the purpose of distinguishing descriptions, and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features.

[0043] First, the nouns involved in the embodiments of the present application are briefly introduced:

[0044] Code Semantics: Code semantics refers to the logical meaning and execution effect of the code, describing the behavior and function of the code at runtime. Different from the representation at the grammatical level, code semantics captures key information such as program logic, data flow, and control flow. In this invention, code semantics is converted into vector representation through deep learning-based code representation learning technology (such as Code2Vec) as one of the important features of code smell detection.

[0045] Abstract Syntax Tree (AST): A tree-structured code representation based on code syntax, where non-terminal nodes (internal nodes) represent operators (arithmetic or assignment) and terminal nodes (leaf nodes) represent operands (constants or identifiers).

[0046] Flow-Augmented Abstract Syntax Tree (FA-AST): A code representation based on a graph structure constructed based on an abstract syntax tree. It is constructed by adding control flow and data flow edges to the original abstract syntax tree.

[0047] Code Metrics: A set of metrics used to quantify the characteristics of code snippets, including traditional software metrics and AST-based metrics. In the present invention, code metrics are further subdivided into structural metrics, syntactic metrics, and semantic metrics to comprehensively cover the multi-dimensional characteristics of code.

[0048] Code Embedding: A method of converting code snippets into vector representations, usually implemented through deep learning techniques (such as Code2Vec, etc.). Code embedding captures the contextual semantic information of the code, making the code processable by machine learning models.

[0049] Extreme Gradient Boosting (XGB): An ensemble learning algorithm that improves the overall predictive power of a model by integrating multiple weak learners (usually decision trees). XGB is known for its efficiency, scalability, and ability to handle large-scale datasets, and is widely used in various classification and regression tasks.

[0050] Random Forest (RF): A decision tree-based ensemble learning method that improves the accuracy and stability of the model by building multiple decision trees and combining their prediction results. RF performs well in processing high-dimensional data and preventing overfitting, and is a commonly used classification and regression algorithm.

[0051] Convolutional Neural Network (CNN): A deep learning model widely used in image processing and computer vision. CNN extracts local features of data through convolutional layers and has strong feature learning capabilities. In this paper, CNN is used to process the structural metrics and code semantics of code snippets to achieve accurate identification of code smells.

[0052] Deep Neural Network (DNN): A multi-layer neural network structure that learns complex features of data through nonlinear transformations of multiple hidden layers. DNN has powerful expression capabilities and can handle complex pattern recognition tasks. In the present invention, DNN is used to fuse the multimodal information of the code to improve the effect of code smell detection.

[0053] The present invention provides a code smell detection method based on code metrics and code semantics of AST. First, the present invention extracts two types of information from each code snippet: code semantics and code metrics. These two types of information are then input into a classifier to identify code smells. Finally, each classifier performs two classification tasks: identifying the presence of code smells (binary classification task) and determining the severity of code smells (multi-classification task).

[0054] like Figure 1 As shown, the code smell detection method based on AST code metrics and code semantics provided by the present invention has the following basic steps:

[0055] S1. Convert the code snippet to be detected into a vector representation through a deep learning-based code representation learning technology or a code embedding technology as the code semantics corresponding to the code snippet to be detected;

[0056] S2, extracting structural metrics, syntactic metrics and semantic metrics from the code snippet to be detected, and using them together as code metrics corresponding to the code snippet to be detected;

[0057] S3, using the code semantics and the code metrics as inputs of a first classification model, and obtaining a prediction result of whether the code snippet to be detected has code smell by the trained first classification model;

[0058] S4. Using the code semantics and the code metrics as inputs of a second classification model, and obtaining a prediction result of the code smell degree of the code snippet to be detected by the trained second classification model.

[0059] The code semantics corresponding to the above-mentioned code snippet to be detected can be realized through existing models such as Code2Vec based on the abstract syntax tree AST. The code snippet to be detected is embedded in the Code2Vec code model by inputting it into the AST of the code snippet, thereby parsing the AST of the code snippet and converting it into its corresponding high-dimensional vector representation.

[0060] In addition, the steps of extracting structural metrics, syntactic metrics and semantic metrics from the code snippet to be detected are as follows: first, obtain the abstract syntax tree AST of the code snippet to be detected, and add edges corresponding to the control flow and data flow in the abstract syntax tree AST to form a flow abstract syntax tree FA-AST; then, the structural metrics, syntactic metrics and semantic metrics are counted by traversing the nodes and edges in the flow abstract syntax tree FA-AST.

[0061] It should be noted that the process of adding edges in the abstract syntax tree AST according to the control flow and data flow in the code snippet to form the stream abstract syntax tree FA-AST belongs to the prior art and will not be described in detail. In addition, the specific statistical methods of structural measurement, syntactic measurement and semantic measurement will be further explained through examples later.

[0062] It should be noted that the first classification model and the second classification model need to be trained in advance using annotated data sets. The first classification model and the second classification model can be trained using traditional machine learning algorithms (XGB and RF) or deep learning algorithms (CNN and DNN). The two classification models can use different models or the same model, and their training is independent.

[0063] The inputs of the first classification model and the second classification model are both code semantics and code metrics, but due to the different model structures, the specific processing methods for the two inputs need to be adjusted accordingly. For traditional machine learning algorithms, the present invention can directly use the code semantics and normalized code metrics as input after splicing; for deep learning algorithms, the present invention can use two branch networks to process the two types of inputs respectively. Both branch networks can be implemented using CNN or DNN. One branch network inputs code semantics, and the other branch network inputs normalized code metrics. The output features of the two branch networks are then spliced ​​and input into the Softmax classifier to implement the classification task.

[0064] In order to better demonstrate the specific implementation and technical effect of the above detection method of the present invention, it is demonstrated through this embodiment below.

[0065] like Figure 2 As shown, a flowchart of a code smell detection method based on AST code metrics and code semantics is shown in a preferred embodiment of the present invention, which can detect the code smell of a specified code fragment and its severity identification method. The method includes the following steps:

[0066] Step 101, the embodiment of the present invention uses a neural network model Code2Vec to map code snippets of different sizes into a vector space of fixed dimensions for representing code semantics, making it suitable for code similarity comparison, code search, and code recommendation tasks. Then, code smell identification is regarded as a special code recommendation task, that is, recommending smelly codes to developers to improve the efficiency of smelly code refactoring.

[0067] Step 102, the code metrics selected by the embodiment of the present invention include traditional software metrics (product metrics and process metrics) and AST-based code metrics (structural metrics, syntactic metrics, and semantic metrics). Among them, traditional software metrics are extracted through a software measurement tool JavaMetricsPP. It calculates two types of software code metrics: product metrics and process metrics. Product and process metrics are the main metrics used to evaluate and quantify software quality and development processes. They focus on the quality of software code and the efficiency of development processes. Product metrics evaluate the quality and characteristics of the software product itself, providing information about software code, design, and functionality. These metrics help evaluate the quality of software delivery and are critical to developers, testers, and project managers. Process metrics focus on the efficiency and quality of software development and maintenance processes. They provide insights into the development process and help identify areas for improvement. These metrics enable teams to understand efficiency and quality issues in development activities and take measures to enhance the development process.

[0068] It is worth noting that the traditional software metrics in this embodiment are only used to compare with the AST-based code metrics proposed in the present invention to evaluate the effectiveness of the AST-based code metrics proposed in the present invention in identifying code smells and their severity. However, in the code smell detection method of the present invention, only the code metrics composed of structural metrics, syntactic metrics and semantic metrics need to be used, without the need to use software metrics.

[0069] Combine the following Figure 3 The process shown introduces in detail the three types of AST-based code metrics provided by the present invention: structural metrics, syntactic metrics, and semantic metrics.

[0070] Structural metrics: The embodiment of the present invention first extracts AST from method or class code snippets written in Java based on the Javalang library. Then, the written Python script is used to traverse the nodes of the AST and automatically count the structural metrics based on the AST. As shown in Table 1, the embodiment of the present invention counts ten structural metrics based on the abstract syntax tree (AST) and describes their meanings in detail. These metrics comprehensively reflect various aspects of the code, such as size, complexity, and structure. Unlike linear text representation, AST can capture the structural information of the code. For example, the depth and width of AST can help analyze the nested relationship and hierarchy between code blocks, reflecting the overall structure and complexity of the code; while the number of lines of code based on code text can only provide the length information of the code, which is difficult to reflect the structural complexity. In addition, the embodiment of the present invention also counts the number of nodes, parameters and their types, further enriching the description of the code structure and complexity. This step also counts the number of code blocks and branching factors to more accurately reflect the logical structure of the code. Finally, the embodiment of the present invention calculates the coverage rate of the execution statement nodes and the repetition rate of the nodes, thereby reflecting the repetitiveness and distribution of the execution logic in the code. If the node repetition rate is high and the proportion of execution statement nodes is low, it may indicate that the code structure similarity is high but the logic is simple; conversely, if the proportion of execution statement nodes is high and the repetition rate is low, it may mean that the code logic is diverse and the structural complexity is low.

[0071] Table 1 10 structural metrics based on AST and their descriptions provided by the embodiment of the present invention

[0072]

[0073]

[0074] Syntactic metrics: In addition to structural information, the abstract syntax tree (AST) also contains rich syntactic details. The node types of AST represent syntactic structural units in the source code, such as statements, expressions, and function declarations. Each node type corresponds to a syntactic structure in the programming language. By traversing the AST, the syntactic information of the source code can be obtained and analyzed, which is of great value for program understanding and analysis. However, since AST contains multiple node types, some of the statistical information is irrelevant to code smell. Therefore, an embodiment of the present invention aims to select and count the top K node types that are most relevant to code smell as syntactic metrics of AST.

[0075] In order to effectively select node types, the embodiment of the present invention constructs a comprehensive feature selector, which adopts five feature selection methods: correlation coefficient analysis, variance analysis, principal component analysis, mutual information and LASSO (least absolute shrinkage and selection operator) regression. These methods select node types from different perspectives, avoid the limitations of a single method, and improve the robustness of feature selection. Figure 4 As shown, the embodiment of the present invention selects the top K node types most related to code smell through the following five steps:

[0076] 1) Get all AST node types contained in the dataset: S type ={t 1 ,t 2 ,...,t n};

[0077] 2) Create input data X and Y, where X = [[item 1 ={count(t 1 ),count(t 2 ),...,count(t n )}],item 2 ,...,item m ],Y=

[0078] [label 1 ,label 2 ,...,label m ];

[0079] 3) Input X and Y into each feature selection method and calculate the score of each node type;

[0080] 4) Each feature selection method recommends the top K node types with the highest scores;

[0081] 5) Use the majority voting mechanism to select the top K node types for constructing the syntactic metric.

[0082] The embodiment of the present invention is based on the code smell dataset MLCQ, and sets K=20, that is, extracting the top 20 node types most relevant to code smell, as shown in Table 2. Then, the embodiment of the present invention counts the number of each of these 20 node types in each abstract syntax tree (AST) and uses it as a syntactic metric. The MLCQ dataset contains four types of code smells (God class, long method, data class, and feature envy), and each sample is labeled with one of four severity levels (i.e., 0, 1, 2, or 3). The embodiment of the present invention uniformly considers all samples and their severity to create input data X and Y.

[0083] Semantic metric: The embodiment of the present invention adds a series of directed edges to supplement the control flow, data flow and structural details by traversing the nodes in the original abstract syntax tree (AST). To avoid redundant calculations due to directed edges that appear in pairs, the embodiment of the present invention treats these directed edges as undirected edges and counts them only once. As shown in Table 2, the embodiment of the present invention finally counts the number of 9 edge types in the flow enhanced abstract syntax tree (FA-AST) and uses it as a semantic metric. The construction methods of the 9 edges are:

[0084] ChildParent: In the abstract syntax tree (AST), connect each non-terminal node with its child node, and simultaneously connect each non-root node with its parent node according to the AST structure.

[0085] Sibling node relationship (NextSib): Connect each node in the AST to its next sibling node (from left to right).

[0086] Statement order relationship (NextStmt): Connect each statement subtree to its next sibling node to represent the relationship of sequential execution.

[0087] Identifier connection relationship (NextToken): Connect each terminal node (identifier token) with the next terminal node in the source code.

[0088] Variable usage relationship (NextUse): Connect each node representing a variable to the next occurrence of the variable in the code.

[0089] Loop execution relationship (While): Connect the condition node to the loop body node, and connect the loop body node back to the condition node to simulate the loop execution process.

[0090] Loop Control Relationship (For): Connects the control node to the loop body node, and connects the loop body node back to the control node.

[0091] Conditional execution relationship (If-true branch when the If statement condition is true and If-else branch when the If statement condition is false): Connect the conditional nodes to the corresponding ThenStatement and ElseStatement nodes respectively.

[0092] In addition, the embodiment of the present invention further adds a backtracking edge to each edge without a backtracking edge (such as CondTrue and NextStmt) to ensure a bidirectional connection.

[0093] Table 2 20 syntactic metrics and 9 semantic metrics based on AST provided by the embodiment of the present invention

[0094]

[0095] Step 103, the embodiment of the present invention respectively uses two traditional machine learning algorithms and two deep learning algorithms to implement the proposed code smell detection framework. In order to facilitate the subsequent work, the task of identifying whether there is a code smell is recorded as Task 1, and the task of determining the severity of the code smell is recorded as Task 2. Figure 5 As shown, the embodiment of the present invention discloses a framework structure formed by various algorithms and can be used to derive the design details of the model. The input data includes code semantics of dimension d and code metrics of length l. In particular, the embodiment of the present invention normalizes the code metrics while keeping the code semantics unchanged. Then, the normalized code metrics are spliced ​​with the code semantic features and used as input data for traditional machine learning algorithms. For deep learning algorithms, the embodiment of the present invention uses two neural network branches to extract two types of inputs (i.e., normalized code metrics and code semantic representations) respectively, and then splices the outputs of the two branches together for classification tasks. Specifically, Figure 5 (a) shows the classification model framework implemented using one of two traditional machine learning algorithms. The first classification model and the second classification model required for Task 1 and Task 2 can both adopt the XGB model or the RF model. In the two tasks, the d-dimensional code semantics (without normalization) and the l-dimensional code metrics (normalized first) are directly concatenated as the input of the corresponding classification model, and the classification model outputs the corresponding classification results. Figure 5 (b) and (c) respectively show the framework implemented using the traditional feedforward neural network model, specifically the deep neural network (DNN) and the convolutional neural network (CNN). Unlike the two traditional machine learning algorithms, these two deep learning models need to use CNN (DNN can also be used) as the backbone network to build two branch networks. The d-dimensional code semantics (no normalization is required) and the l-dimensional code metrics (normalized first) are each extracted through different branch networks and then spliced ​​as the input of the Softmax classifier. The output of the Softmax classifier is the classification result. The embodiment of the present invention implements all the above models based on the PyTorch framework.

[0096] Step 104, train different classification models on the data set annotated with binary classification labels to implement the code smell identification task (i.e., Task 1), where the binary classification label is used to indicate whether a code snippet is a code smell. Since the severity of code smell in the MLCQ data set used in this embodiment is divided into four levels: critical, major, minor, and none, the embodiment of the present invention converts the code smell identification task into a binary classification task. In the binary classification task, the embodiment of the present invention regards all samples corresponding to the first three severity levels as positive samples and the others as negative samples to train the model's ability to identify code smells.

[0097] Step 105, different classification models are trained on the data set annotated with multi-classification labels to implement the code smell severity determination task (i.e., Task 2), wherein the multi-classification labels are used to indicate the severity of a code snippet when it is considered as a code smell. The embodiment of the present invention converts the code smell severity determination task into a four-classification task, and in the four-classification task, each sample is mapped to one of the four severity levels of critical, major, minor, and none by a label mapping method, so as to further train the model's ability to identify the severity of code smell.

[0098] It should be noted that the training in the above steps 104 and 105 is performed independently on the initialized models, rather than jointly.

[0099] All experiments in the embodiments of the present invention are run on a computer equipped with an 8-core 3.6GHz CPU and an RTX3090 24G GPU. Table 3 shows the hyperparameter settings of all algorithms involved in the method provided in the embodiments of the present invention. The embodiments of the present invention adopt the Code2Vec algorithm based on TensorFlow implementation, and use the MLCQ dataset publicly available on GitHub to train and generate a code semantic representation model with a dimension of 128. Since Code2Vec is designed to generate code embeddings for method-level code snippets, the embodiments of the present invention generate the corresponding code semantic representation by averaging all method embeddings in each class-level code snippet.

[0100] Table 3 Hyperparameter settings

[0101]

[0102] The experimental results of step 104 and step 105 show that the AST-based code metrics proposed in the embodiment of the present invention can effectively identify four types of code smells (God classes, long methods, data classes, and feature envy), and outperform traditional software code metrics in both tasks, indicating that the proposed AST-based code metrics are effective and have the potential to replace traditional software code metrics.

[0103] In addition, the embodiment of the present invention is also compared with the SOTA (State-Of-The-Art) method based on the pre-training model. As shown in Table 4, et al. (2022) and et al. (2024) on the same dataset MLCQ, and made a fair comparison with the method provided by the present invention. It can be observed that although the method provided by the present invention adopts fewer AST-based code metrics and code embedding dimensions, at least two technical combination schemes perform best in each type of code smell identification task, indicating that the method proposed by the present invention is more effective in identifying code smells and has higher coding efficiency. In addition, since the SOTA method only relies on software code metrics or code embedding, the best technology pairings for identifying different code smells are different. In contrast, the method proposed by the present invention combines code semantics and AST-based code metrics, and has better versatility in identifying code smells.

[0104] Table 4 Comparison of experimental results between the method provided by the present invention and the SOTA method

[0105]

[0106] After the above steps, the two pre-processed code data are input into the model provided by the present invention for training. After several rounds, the trained model can be used to identify code smells and their severity.

[0107] In the reasoning phase under the application scenario, for any code snippet to be detected, after extracting the code semantics and the code metrics therefrom, the code semantics and the code metrics are used as the input of the classification model trained in step 104, and a prediction result of whether the code snippet to be detected has code smell can be obtained; similarly, the code semantics and the code metrics are used as the input of the classification model trained in step 105, and a prediction result of the degree of code smell of the code snippet to be detected can be obtained.

[0108] It should be noted that each step of the above-mentioned code smell detection method based on AST code metrics and code semantics can essentially be implemented in the form of a computer program.

[0109] Therefore, based on the same inventive concept, Figure 6As shown, the present invention also provides a code smell detection system based on AST code metrics and code semantics corresponding to the code smell detection method based on AST code metrics and code semantics provided in the above embodiment, which includes:

[0110] A code semantics acquisition module is used to convert the code snippet to be detected into a vector representation through a code representation learning technology or a code embedding technology based on deep learning, as the code semantics corresponding to the code snippet to be detected;

[0111] A code metric acquisition module is used to extract structural metrics, syntactic metrics and semantic metrics from the code snippet to be detected, and use them together as code metrics corresponding to the code snippet to be detected;

[0112] A first detection module, configured to use the code semantics and the code metrics as inputs of a first classification model, and obtain a prediction result of whether a code snippet to be detected has code smell by the trained first classification model;

[0113] The second detection module is used to use the code semantics and the code metrics as inputs of a second classification model, and obtain a prediction result of the code smell degree of the code fragment to be detected by the trained second classification model.

[0114] In addition, based on the same inventive concept, Figure 7 As shown, the present invention also provides a computer electronic device corresponding to the code smell detection method based on AST code metrics and code semantics provided in the above embodiment, which includes a memory and a processor;

[0115] The memory is used to store computer programs;

[0116] The processor is used to implement the code smell detection method based on AST code metrics and code semantics as described above when executing the computer program;

[0117] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0118] Therefore, based on the same inventive concept, the present invention provides a computer-readable storage medium corresponding to a code smell detection method based on AST code metrics and code semantics, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the code smell detection method based on AST code metrics and code semantics as described above can be implemented.

[0119] Therefore, based on the same inventive concept, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, can implement the code smell detection method based on AST code metrics and code semantics as described above.

[0120] Specifically, in the computer-readable storage medium of the above three embodiments, the stored computer program is executed by the processor to perform the above steps S1 to S4.

[0121] It is understandable that the above storage medium may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. The storage medium may also be a U disk, a mobile hard disk, a magnetic disk or an optical disk, etc., which can store program codes.

[0122] It can be understood that the above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0123] It should also be noted that those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the various embodiments provided in this application, the division of steps or modules in the system and method is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or steps can be combined or integrated together, and a module or step can also be split.

[0124] The above-described embodiments are only some preferred implementations of the present invention, but are not intended to limit the present invention. A person skilled in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A code smell detection method based on AST code metrics and code semantics, characterized in that: The following steps are involved: S1. Convert the code snippet to be detected into a vector representation through a deep learning-based code representation learning technology or a code embedding technology as the code semantics corresponding to the code snippet to be detected; S2, extracting structural metrics, syntactic metrics and semantic metrics from the code snippet to be detected, and using them together as code metrics corresponding to the code snippet to be detected; S3, using the code semantics and the code metrics as inputs of a first classification model, and obtaining a prediction result of whether the code snippet to be detected has code smell by the trained first classification model; S4. Using the code semantics and the code metrics as inputs of a second classification model, and obtaining a prediction result of the code smell degree of the code snippet to be detected by the trained second classification model.

2. The code smell detection method based on AST code metrics and code semantics as claimed in claim 1, characterized in that: In S1, the code snippet to be detected is input into the Code2Vec code embedding model, and the abstract syntax tree of the code snippet is parsed to be converted into its corresponding vector representation.

3. The code smell detection method based on AST code metrics and code semantics as claimed in claim 1, characterized in that: In S2, the steps of extracting structural metrics, syntactic metrics and semantic metrics from the code snippet to be detected are as follows: first, obtaining the abstract syntax tree AST of the code snippet to be detected, and adding edges corresponding to the control flow and the data flow in the abstract syntax tree AST to form a flow abstract syntax tree FA-AST; Then, the structural metrics, syntactic metrics and semantic metrics are counted by traversing the nodes and edges in the flow abstract syntax tree FA-AST, wherein: the structural metrics are 10 structural parameters, namely: tree depth, tree width, number of nodes, number of leaf nodes, maximum parameter list length, number of statement blocks, number of code blocks, branching factor, node coverage and node repetition rate; the syntactic metrics are the number of 20 types of nodes, namely: reference type, variable declarator, member reference, literal, method declaration, code block statement, return statement, collection, string, field declaration, formal parameter, expression statement, assignment, current object reference, method call, local variable declaration, basic type, conditional statement, binary operation and class creator; the semantic metrics are the number of 9 types of edges, namely: parent-child node relationship, sibling node relationship, statement order relationship, identifier connection relationship, variable use relationship, branch when the If statement condition is true, branch when the If statement condition is false, loop execution relationship, loop control relationship.

4. The code smell detection method based on AST code metrics and code semantics as claimed in claim 1, characterized in that: The first classification model and the second classification model adopt a machine learning model or a deep learning model; preferably, the machine learning model is an XGB model or an RF model, and the code semantics and the normalized code metrics are directly spliced ​​as the input of the classification model; preferably, the deep learning model is a CNN model or a DNN model as the backbone network to construct two branch networks, and the code semantics and the normalized code metrics are each extracted features through different branch networks and then spliced ​​as the input of the Softmax classifier.

5. The code smell detection method based on AST code metrics and code semantics as claimed in claim 1, characterized in that: The first classification model is pre-trained in a code smell identification task using a labeled data set with binary classification labels, where the binary classification labels are used to indicate whether a code snippet has a code smell.

6. The code smell detection method based on AST code metrics and code semantics as claimed in claim 1, characterized in that: The second classification model is pre-trained in a code smell severity determination task using a labeled data set with multi-classification labels, where the multi-classification labels are used to indicate the severity of a code snippet when it is considered as a code smell.

7. A code smell detection system based on AST code metrics and code semantics, characterized in that: include: A code semantics acquisition module is used to convert the code snippet to be detected into a vector representation through a code representation learning technology or a code embedding technology based on deep learning, as the code semantics corresponding to the code snippet to be detected; A code metric acquisition module is used to extract structural metrics, syntactic metrics and semantic metrics from the code snippet to be detected, and use them together as code metrics corresponding to the code snippet to be detected; A first detection module, configured to use the code semantics and the code metrics as inputs of a first classification model, and obtain a prediction result of whether a code snippet to be detected has code smell by the trained first classification model; The second detection module is used to use the code semantics and the code metrics as inputs of a second classification model, and obtain a prediction result of the code smell degree of the code fragment to be detected by the trained second classification model.

8. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the code smell detection method based on AST code metrics and code semantics as described in any one of claims 1 to 7 can be implemented.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the code smell detection method based on AST code metrics and code semantics according to any one of claims 1 to 7 is implemented.

10. A computer electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement the code smell detection method based on AST code metrics and code semantics as described in any one of claims 1 to 7 when executing the computer program.

Images