A code smell recognition method based on code semantics and metrics

Abstract

The code smell recognition method based on code semantics and metrics comprises the following steps: constructing a code smell dataset; performing data preprocessing on a code snippet to obtain code semantic representation information and code metric information; inputting the code semantic representation information into a code semantic representation learning model to obtain code semantic embedding; inputting the code metric information into a code metric relationship learning model to obtain code metric relationship embedding; fusing the code semantic embedding and the code metric relationship embedding to obtain a fusion learning model; constructing a loss function of the fusion learning model, training the fusion learning model, and obtaining a code smell recognition model; and inputting a code snippet to be detected into the code smell recognition model to obtain a code smell recognition result. The method designed in the application can automatically recognize various code smells and is suitable for different languages and code granularities.

Description

Technical Field

[0001] This invention provides a code smell identification method based on code semantics and metrics, belonging to the field of code smell identification in software engineering. Background Technology

[0002] The presence of code smells indicates that the code violates basic design principles and negatively impacts design quality. The accumulation of code smells leads to technical debt, a metaphor for suboptimal solutions developers use to achieve rapid delivery. Technical debt includes various code smells, anti-patterns, hard coding, unnecessary code duplication, and failure to adhere to existing design conventions. Code smells are a significant source of technical debt, indicating quality issues in the software system. A large number of code smells in a software system result in poor maintainability, hindering software evolution. Therefore, identifying code smells and regularly refactoring them is essential for maintaining low technical debt.

[0003] Code smells are any features in a code snippet that may indicate a deeper problem. Identifying whether a code snippet has a smell is subjective and varies depending on the programming language, the developer, and the development methodology. Therefore, it is essential to design an AI model that can automatically identify code smells, applicable to different languages ​​and code granularities.

[0004] Existing tools for detecting code smells rely on preset rules and fixed metric thresholds. Developers use these tools as technical advisors to track code smells. However, there is no unified standard for judging whether a code snippet has a smell; in other words, it is subjective. Research shows that these tools cannot accurately capture the most important parts that need refactoring from code metrics. Therefore, some inventors use machine learning or deep learning methods to automatically detect code smells, hoping that the model will learn from human experience in identifying code smells rather than relying on fixed rules and thresholds. However, the code information they consider is relatively limited, and the methods they use are relatively simple, resulting in low accuracy in identification.

[0005] As mentioned above, current code smell detection tools rely on preset rules and metric thresholds, ignoring the implicit semantics within the code and the interrelationships between metrics. On one hand, preset rules typically identify smells by matching specific code patterns. While some methods use neural networks to learn code context information hidden in abstract syntax trees, they don't adequately learn code semantics and ignore code metrics. On the other hand, metric-based methods are very effective at identifying measurable smells. However, it's difficult for software engineers to manually adjust often contradictory and interrelated thresholds. Summary of the Invention

[0006] To address the two problems mentioned above in existing technologies, this invention provides a code smell identification method based on code semantics and metrics, comprising the following steps:

[0007] S1 constructs a code smell dataset, in which the samples are code snippets that were actually refactored before and after the project's historical version iterations due to code smell issues, collected through automated methods.

[0008] S2 performs data preprocessing on the code sniffs in the code smell dataset to obtain code semantic representation information and code measurement information;

[0009] S3 constructs a code semantic representation learning model, inputs the code semantic representation information into the code semantic representation learning model, and obtains the code semantic embedding;

[0010] S4 constructs a code metric relationship learning model, inputs the code metric information into the code metric relationship learning model, and obtains the code metric relationship embedding;

[0011] S5 fuses the code semantic embedding and the code metric relationship embedding to obtain a fusion learning model, which is used to detect code smells;

[0012] S6 constructs the loss function of the fusion learning model, trains the fusion learning model, and obtains the code odor recognition model;

[0013] S7 processes the code segment to be detected through the data preprocessing in step S2 to obtain the code semantic representation information and code measurement information of the code segment to be detected. The code semantic representation information and code measurement information of the code segment to be detected are then input into the code odor identification model to obtain the code odor identification result.

[0014] Preferably, in step S1, the code snippets originate from a large open-source code repository; the automation method includes: using code smell detection tools and preset rules to automatically collect code snippets from the code repository that have been refactored before and after being affected by code smells during the project's historical version iterations.

[0015] Preferably, in step S2, the method for preprocessing to obtain code semantic representation information is as follows: using a code syntax parsing tool to parse the class-level and method-level abstract syntax trees of the code, and then constructing a flow-enhanced abstract syntax tree to obtain the code graph representation by adding control flow and data flow edges to the abstract syntax tree. The flow-enhanced abstract syntax tree provides rich code structure and semantics;

[0016] The method for preprocessing to obtain code metrics information is as follows: use a code smell detection tool to calculate object-oriented code metrics; save the code semantic representation information and code metrics information in a JSON file in dictionary format for easy model reading.

[0017] Preferably, in step S3, the code semantic representation learning model includes a word embedding layer, several graph neural network layers, and a convolutional neural network layer; the semantic information of the flow-enhanced abstract syntax tree is learned by stacking several graph neural network layers, and a non-linear activation function is connected after each graph neural network layer.

[0018] Step S3 specifically includes the following steps:

[0019] First, the stream-enhanced abstract syntax tree is fed into a word embedding layer so that all text information in the stream-enhanced abstract syntax tree is represented by word vectors, resulting in a vectorized stream-enhanced abstract syntax tree. Then, the vectorized stream-enhanced abstract syntax tree is input into the semantic representation learning model. Next, a global pooling layer based on an attention mechanism is used to read out the output features of each graph neural network layer. Then, the output features are concatenated and input into an output layer composed of a convolutional neural network layer and a non-linear activation function for fusion, resulting in code semantic embedding.

[0020] Preferably, in step S4, the code metric relationship learning model includes a metric embedding layer, a position embedding layer, several Transformer layers, and a convolutional neural network layer; the constraint relationship between code metrics is learned by stacking several Transformer layers.

[0021] Step S4 specifically includes the following steps: First, the code metric information is sent to the metric embedding layer to vectorize and encode the code metric to obtain the metric embedding. Then, the metric embedding is input to the position embedding layer to encode the metric arrangement order to obtain the position-encoded metric embedding. The position-encoded metric embedding is input to a network consisting of several stacked Transformer layers. The features output by the last Transformer layer are concatenated and input to an output layer consisting of a convolutional neural network layer and a non-linear activation function for fusion to obtain the code metric relationship embedding.

[0022] Preferably, step S5 specifically includes the following steps: fusing the code semantic embedding obtained in S3 and the code metric relationship embedding obtained in S4 by concatenation operation, and then inputting the fused features into a classifier composed of a fully connected network to obtain a fusion learning model for identifying code smells.

[0023] Preferably, in step S6, the loss function is the loss function of the classifier in the fusion learning model, which is constructed based on the cross-entropy loss function, and the loss function is used to train the fusion learning model.

[0024] Preferably, the object-oriented code metrics include: number of lines of code, cyclomatic complexity, number of parameters, number of lines of code, number of fields, number of public fields, number of methods, number of public methods, weighted methods for each class, number of subclasses, inheritance tree depth, insufficient aggregation in methods, fan-in, and fan-out.

[0025] This invention constructs a semantically rich flow-enhanced abstract syntax tree (AST) by adding control flow and data flow edges to the AST, and employs a graph neural network to learn code semantic representations. Furthermore, it uses a Transformer layer based on a self-attention mechanism to learn the relationships between metrics. Finally, it identifies code smells by fusing data from both learning modalities of the code. This invention has the following beneficial effects:

[0026] 1. This invention provides a method for automatically collecting code smell datasets, which can conveniently and quickly obtain a large number of real data samples for model training;

[0027] 2. This invention is the first to simultaneously consider using code semantic information and code measurement information to jointly identify code smells, making it suitable for identifying more types of code smells;

[0028] 3. This invention provides a carefully designed fusion learning model that can fully learn the constraints between code semantic information and code metrics, enabling the model to simultaneously analyze two important types of code information related to code smells, resulting in higher accuracy in code smell identification. Attached Figure Description

[0029] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Some specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings in an exemplary and non-limiting manner. The same reference numerals in the drawings indicate the same or similar parts or components. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0030] Figure 1 The flowchart illustrates the code smell identification method based on code semantics and metrics provided in this embodiment of the invention.

[0031] Figure 2 The dataset construction method (automatic sampling strategy for positive and negative samples) provided in the embodiments of the present invention.

[0032] Figure 3This is a schematic diagram illustrating the relationship between learning code metrics in a Transformer layer based on a self-attention mechanism, as provided in an embodiment of the present invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0034] To better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that all other embodiments obtained by those skilled in the art without creative effort, as described herein, fall within the scope of protection of the present invention.

[0035] First, a brief introduction to the terms used in the embodiments of this application:

[0036] Abstract Syntax Tree (AST): A tree-structured code representation based on code syntax, where non-terminal nodes (internal nodes) represent operators (arithmetic or assignment), and terminal nodes (leaf nodes) represent operands (constants or identifiers).

[0037] Flow-Augmented Abstract Syntax Tree (FA-AST): A code representation based on a graph structure constructed from an abstract syntax tree. It is constructed by adding edges such as control flow and data flow to the original abstract syntax tree.

[0038] Graph Neural Networks (GNNs) are a framework that utilizes deep learning to directly learn from graph-structured data. Their superior performance has attracted significant attention and in-depth exploration by researchers. By formulating specific strategies for nodes and edges in a graph, GNNs transform graph-structured data into standardized representations, which are then fed into various neural networks for training. They have achieved excellent results in tasks such as node classification, edge information propagation, and graph clustering.

[0039] Transformer: A neural network module based on self-attention mechanism and feedforward neural network construction, which has achieved very good performance in fields such as natural language processing and computer vision.

[0040] Convolutional Neural Network (CNN): A very common algorithm (model) in deep learning, it is widely used in image processing.

[0041] Token embedding: An algorithm for automatically encoding text. It generates a vocabulary containing the word embeddings of all words in the vocabulary by taking the number of words in the desired vocabulary and the desired word embedding dimension as input.

[0042] Metric embedding: Similar to lexical embedding, it automatically generates a metric table containing metric embeddings of all metrics in the metric library by inputting the number of metrics in the required metric library and the dimension of the required metric embedding.

[0043] Position embedding: An algorithm that encodes the positions of a sequence of input vectors to mark the order of each vector. In this invention, it is used to mark the order of a series of output metrics in order to encode the different meanings of metrics at different positions.

[0044] This invention provides a code smell identification method based on code semantics and metrics. First, a code smell dataset is constructed by automatically collecting a large number of samples of code smells before and after actual refactoring during historical version iterations of a project. Then, the code snippets in the constructed code smell dataset are preprocessed to obtain code semantic representation information and code metric information. Next, the code semantic representation information is input into a code semantic representation learning model to obtain code semantic embeddings; the code metric information is input into a code metric relation learning model to obtain code metric relation embeddings. Then, the code semantic embeddings and code metric relation embeddings are fused to obtain a fusion learning model, which is used to detect code smells. Finally, a loss function is constructed for the fusion learning model, which is used to train the fusion learning model to obtain a code smell identification model, used to identify code snippets with meaningful code smells.

[0045] The process of the solution provided by this invention is as follows: 1) Automatically collect a large number of code smell samples before and after actual reconstruction during the historical version iteration of the project, constructing a code smell dataset; 2) Perform data preprocessing on the code fragments in the constructed code smell dataset to obtain code semantic representation information and code measurement information; 3) Input the code semantic representation information into a code semantic representation learning model to obtain code semantic embedding; 4) Input the code measurement information into a code measurement relation learning model to obtain code measurement relation embedding; 5) Fuse the code semantic embedding and code measurement relation embedding to obtain a fusion learning model, which is used to detect code smells; 6) Construct a loss function for the fusion learning model to train the fusion learning model, obtaining a code smell identification model, which is used to identify code fragments with meaningful code smells; 7) Input the code fragment to be detected into the code smell identification model after data preprocessing to obtain the code smell identification result.

[0046] This invention provides a method for identifying code smells based on code semantics and metrics. Figure 1 This is a flowchart illustrating the code odor identification method provided in an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0047] Step 101: Automatically collect a large number of code smell samples before and after actual refactoring during the project's historical version iterations, constructing a code smell dataset. This invention collects the dataset based on two assumptions: 1) the code smell detection tool has a certain degree of reliability and can partially detect code smells requiring refactoring; 2) with project version iterations, some smells may be introduced, while others may be eliminated. Based on these assumptions, this invention automatically collects a large number of code smell samples before and after actual refactoring during the project's historical version iterations from large online open-source code repositories using code detection tools and preset rules. Specifically, such as... Figure 2 As shown, this invention collects code snippets from all versions of a project to form a code snippet set C, and uses C... i，j Represents the i-th code snippet of the j-th version (if it exists), using S. i，j ∈(-1, 0, 1) represents C i，j The state of S. i，j =-1 means C i，j It has not yet been created or removed, S i，j =0 indicates C i，j It exists and is identified by the tool as odorless, S i，j =1 indicates C i，j It exists and is identified by the tool as an aberration code. This invention uses the following rules to collect the dataset:

[0048] Rule 1 ( Figure 2 Case 1): When S i，j =1,S i，j+1 When = 0, C i，j Treating C as a positive sample i，j+1 Consider it a negative sample;

[0049] Rule 2 ( Figure 2 Case 2): When S i，j =1,S i，j+1 When C = -1, i，j Consider it a positive sample;

[0050] Rule 3 ( Figure 2 Cases 3 & 4): Code snippets saved from the initial version to the latest version are always detected as non-smelly by the tool. Figure 2 Case 3) or odor ( Figure 2Case 4), whose total versions account for more than 90% of the total project versions, is considered a negative sample, and will... Figure 2 Case 3 and Figure 2 In Case 4, the number of negative samples was set to be equal.

[0051] Step 102 involves preprocessing the code snippets from the constructed code smell dataset to obtain code semantic representation information and code metric information. This invention uses a code syntax parsing tool to extract class-level and method-level abstract syntax trees from the project code. Then, it constructs a flow-enhanced abstract syntax tree by adding control flow and data flow edges to the original abstract syntax tree, providing rich code structure and semantics. Regarding the relational information of code metrics, this invention utilizes a code smell detection tool to calculate 13 common object-oriented code metrics, as shown in Table 1. Furthermore, this invention stores both types of code information in a dictionary format in a JSON file for easy model reading.

[0052] Table 1. Object-Oriented Code Metrics Details

[0053]

[0054] Step 103: Input the code semantic representation information into the code semantic representation learning model to obtain the code semantic embedding. The code semantic representation learning model mainly includes a word embedding layer, several graph neural network layers, and a convolutional neural network layer. This invention mainly learns the semantic information of the code graph representation flow-enhanced abstract syntax tree by stacking several graph neural network layers, with each graph neural network layer followed by an activation function. Specifically, this invention first feeds the flow-enhanced abstract syntax tree into the word embedding layer so that all text information in the flow-enhanced abstract syntax tree is represented by word vectors. Then, the vectorized flow-enhanced abstract syntax tree data is input into a network composed of several graph neural network layers. Then, a global pooling layer based on an attention mechanism is used to read out the input features of each graph neural network layer. Then, these read features are concatenated. Finally, they are input into the output layer composed of a convolutional neural network layer and a non-linear activation function to obtain the code semantic embedding. The node information update formula of the graph neural network layer used in this invention is as follows:

[0055]

[0056] in This represents the weight of the edge from source node j to target node i. To enable the model to autonomously learn the weights of different edge types, this invention randomly initializes a trainable parameter. Make The formula for the output layer is defined as follows:

[0057]

[0058] Where eLU is an activation function, f i (i = 1, 2, ..., n) represents the output features, || represents the concatenation operation, d represents the feature dimension, and d′ represents the output feature dimension.

[0059] Step 104: Input the code metric information into the code metric relation learning model to obtain the code metric relation embedding. The code metric relation learning model mainly includes a metric embedding layer, a position embedding layer, several Transformer layers, and a convolutional neural network layer. This invention mainly uses stacked Transformer layers to learn the constraints between code metrics. The Transformer model consists of an encoder and a decoder, where the encoder part is composed of several stacked encoder layers, each consisting of a self-attention layer and a feedforward layer. This invention uses the encoder part of the Transformer as the feature extraction network of the code metric relation learning model to extract code metric relation information. Its principle is as follows: Figure 3 As shown, on the one hand, the Transformer layer automatically calculates the correlation between each metric and other metrics, eliminating the need to manually select the most correlated metric; on the other hand, it outputs global metric information from the perspective of each metric, containing the mutual constraints between all metrics. In the code metric relation learning model, this invention takes the code metric sequence as input and obtains the metric embedding from a randomly initialized metric matrix. Secondly, the positional embedding assigns positional information to each metric in the metric sequence, making different metrics with the same value have different meanings. Then, these positionally encoded data are input into an encoder consisting of several Transformer layers and output features of the same size as the input. Finally, these feature vectors are concatenated and input into an output layer consisting of a convolutional neural network layer and a non-linear activation function to obtain the code metric relation embedding. The calculation formula for the output layer is the same as the formula for calculating the output layer in step 103. In step 105, the code semantic embedding and the code metric relation embedding are fused to obtain a fused learning model for identifying code smells. This invention concatenates and fuses the code semantic embedding obtained in step 103 and the code metric relationship embedding obtained in step 104, and then inputs the concatenated and fused features into a classifier composed of a fully connected network to obtain a fusion learning model for identifying code smells.

[0060] Step 106: Construct the loss function of the fusion learning model to train the fusion learning model of code semantic representation and metric relation learning, thus obtaining the code smell identification model. To optimize the sample imbalance problem, this invention constructs the loss function of the binary classifier of the fusion learning model based on the cross-entropy loss function:

[0061]

[0062] Where N represents the batch size, α balances positive and negative samples, γ adjusts the easily identifiable and difficult-to-identify samples, and BCE represents the binary cross-entropy loss function. In this invention, α is set to 1 and γ is set to 2.

[0063] Step 107: Input the code segment to be detected into the code odor recognition model after data preprocessing in step 102 to obtain the code odor recognition result.

[0064] After the above steps, the two types of preprocessed code data are input into the model provided by this invention for training. After several rounds, the trained model can be used to identify code smells in real-world projects.

[0065] The above description is only a part of the specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying code smells based on code semantics and metrics, characterized in that, Includes the following steps: S1 constructs a code smell dataset, in which the samples are code snippets that were actually refactored before and after the project's historical version iterations due to code smell issues, collected through automated methods. S2 performs data preprocessing on the code sniffs in the code smell dataset to obtain code semantic representation information and code measurement information; S3 constructs a code semantic representation learning model, inputs the code semantic representation information into the code semantic representation learning model, and obtains the code semantic embedding; S4 constructs a code metric relationship learning model, inputs the code metric information into the code metric relationship learning model, and obtains the code metric relationship embedding; S5 fuses the code semantic embedding and the code metric relationship embedding to obtain a fusion learning model, which is used to detect code smells; S6 constructs the loss function of the fusion learning model, trains the fusion learning model, and obtains the code odor recognition model; S7 After the code segment to be detected is processed by the data preprocessing in step S2, the code semantic representation information and code measurement information of the code segment to be detected are obtained. The code semantic representation information and code measurement information of the code segment to be detected are input into the code odor identification model to obtain the code odor identification result. In step S2, the method for preprocessing to obtain code semantic representation information is as follows: use a code syntax parsing tool to parse the abstract syntax tree at the class level and method level of the code, and then construct a flow-enhanced abstract syntax tree to obtain the code graph representation by adding control flow and data flow edges to the abstract syntax tree; The method for obtaining code metrics information through preprocessing is to use code smell detection tools to calculate object-oriented code metrics; The code semantic representation information and code measurement information are stored in a JSON file in dictionary format; In step S3, the code semantic representation learning model includes a word embedding layer, several graph neural network layers, and a convolutional neural network layer; the semantic information of the stream-enhanced abstract syntax tree is learned by stacking several graph neural network layers, and a non-linear activation function is connected after each graph neural network layer. Step S3 specifically includes the following steps: First, the stream-enhanced abstract syntax tree is fed into a word embedding layer so that all text information in the stream-enhanced abstract syntax tree is represented by word vectors, resulting in a vectorized stream-enhanced abstract syntax tree. Then, the vectorized stream-enhanced abstract syntax tree is input into the semantic representation learning model. Next, a global pooling layer based on an attention mechanism is used to read out the output features of each graph neural network layer. Then, the output features are concatenated and input into an output layer composed of a convolutional neural network layer and a non-linear activation function for fusion, resulting in code semantic embedding.

2. The code smell identification method based on code semantics and metrics according to claim 1, characterized in that, In step S4, the code metric relationship learning model includes a metric embedding layer, a position embedding layer, several Transformer layers, and a convolutional neural network layer; the constraint relationship between code metrics is learned by stacking several Transformer layers. Step S4 specifically includes the following steps: First, the code metric information is sent to the metric embedding layer to vectorize and encode the code metric to obtain the metric embedding. Then, the metric embedding is input to the position embedding layer to encode the metric arrangement order to obtain the position-encoded metric embedding. The position-encoded metric embedding is input to a network consisting of several stacked Transformer layers. The features output by the last Transformer layer are concatenated and input to an output layer consisting of a convolutional neural network layer and a non-linear activation function for fusion to obtain the code metric relationship embedding.

3. The code smell identification method based on code semantics and metrics according to claim 1, characterized in that, In step S1, the code snippet originates from a large open-source code repository; The automation method includes: using code smell detection tools and preset rules to automatically collect code snippets from the code repository that have been actually refactored before and after the project's historical version iterations due to code smell issues.

4. The code smell identification method based on code semantics and metrics according to claim 1, characterized in that, Step S5 specifically includes the following steps: fusing the code semantic embedding obtained in S3 and the code metric relationship embedding obtained in S4 by concatenation operation, and then inputting the fused features into a classifier composed of a fully connected network to obtain a fusion learning model for identifying code smells.

5. The code smell identification method based on code semantics and metrics according to claim 1, characterized in that, In step S6, the loss function is the loss function of the classifier in the fusion learning model, which is optimized based on the cross-entropy loss function. The loss function is used to train the fusion learning model.

6. The code smell identification method based on code semantics and metrics according to claim 3, characterized in that, The object-oriented code metrics include: lines of code, cyclomatic complexity, number of parameters, number of lines of code, number of fields, number of public fields, number of methods, number of public methods, weighted methods for each class, number of subclasses, inheritance tree depth, insufficient aggregation in methods, fan-in, and fan-out.

Images