Code odor detection method based on deep semantics and complex structure
By parsing code using the Code BERT and Joern frameworks, and combining Transformer and graph neural networks, multimodal information fusion for code smell detection was achieved, solving the problem of inaccurate code smell detection in existing technologies and improving detection efficiency and accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
- Filing Date
- 2025-11-24
- Publication Date
- 2026-05-05
AI Technical Summary
Existing technologies lack models that integrate deep semantic information and complex structural information of code, resulting in inaccurate and incomplete code smell detection.
The code is parsed using the Code BERT language model and the Joern framework to obtain deep semantic and complex structural information. A classifier is trained using Transformer and graph neural network, and a weighted fusion model is used to predict probabilities, thus achieving decision-level fusion of multimodal information.
It improves the accuracy and robustness of code smell detection, reduces the workload of manual review, increases detection efficiency, and promotes software quality assurance.
Smart Images
Figure CN121979757A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of software engineering, specifically to the field of code smell detection and static software analysis, and relates to a code smell detection method based on deep semantics and complex structure. Background Technology
[0002] Code smells refer to code segments within a program's source code that reveal deeper system problems or potentially cause program errors. Programmers frequently generate code smells during programming due to poor system design, heavy workloads, or other reasons, indicating potential errors in specific parts of the code. These smells can further lead to larger problems during subsequent development, evolution, and maintenance. Only some code smells can be explicitly identified by the programmer through code comments. However, most code smells lack obvious indicators.
[0003] Code smells are parts of the source code that may require refactoring. Developers can help refactor programs by detecting code smells. However, manually identifying code smells is both difficult and inaccurate, so automated code smell detection methods are needed to assist programmers. Code smell detection can help programmers discover potential code defects and improve code quality. To this end, many methods have been proposed for automatic or semi-automatic code smell identification. Most of these methods rely on manually designed heuristics to map manually selected source code metrics to predictions.
[0004] Therefore, there is currently a lack of a model that can integrate deep semantic information and complex structural information of code, as well as a method to detect code smells from different perspectives and integrate the results. Summary of the Invention
[0005] The technical problem to be solved by this invention is to address the shortcomings of the existing technology by providing a code smell detection method based on deep semantics and complex structure. This method uses methods such as large-scale code model encoding and the Joern framework to parse code structure to obtain datasets; it employs static code analysis, data preprocessing, and fine-tuning training to construct a systematic and efficient code smell detection model; and it uses a weight-threshold mechanism to fuse different models, enabling code smell detection from both semantic and structural perspectives. This helps developers improve code quality and conduct in-depth research in the field of code smell detection.
[0006] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows:
[0007] A code smell detection method based on deep semantics and complex structure includes:
[0008] Step 1: Parse the code using the encoder of the Code BERT code language model to obtain deep semantic information;
[0009] Step 2: Use the Joern framework to parse the code and obtain complex structural information;
[0010] Step 3: Train different models for deep semantic information and complex structural information respectively;
[0011] Step 4: Combine the predicted probabilities from different models to obtain the final result.
[0012] To optimize the above technical solution, the specific measures also include:
[0013] Step 1 above uses the encoder of the Code BERT code-big-language model to parse the code and obtain deep semantic information, constructing a preliminary deep semantic feature representation, including:
[0014] First, the system collected and organized a large-scale code corpus containing various types of code smells as the object of deep semantic encoding. By deeply analyzing the contextual semantics and abstract syntactic structure of representative code fragments in this corpus, a preliminary concept of feature vectors was proposed. Then, a high-quality, expert-annotated benchmark dataset of code smells was collected for model pre-training and fine-tuning. Through systematic analysis of this dataset, it was found that most code smells are closely related to specific semantic patterns in their context. Therefore, starting from the code's token sequence and abstract syntax tree (AST), by focusing on the vectors generated by the Code BERT encoder, the deep semantic indicators of the smells were inversely derived by analyzing their distribution and correlation in high-dimensional space. This process required significant computational resources for model training and parameter tuning. Through repeated iterative optimization, continuously optimizing and adjusting the model's encoding layer and attention weights, a preliminary feature extraction model capable of representing the deep semantics of code was established.
[0015] Step 2 above uses the Joern framework to parse the code and obtain complex structural information, constructing a Code Property Graph (CPG) feature representation, including:
[0016] First, the system utilizes the Joern parsing tool to batch construct graphs from the same code corpus used in step 1, serving as the object of structural analysis. Through in-depth analysis of the Abstract Syntax Tree (AST), Control Flow Graph (CFG), and Program Dependency Graph (PDG) corresponding to these codes, a preliminary graph feature engineering concept is proposed. Then, Joern's queryable language (JoernQL) is further used for data flow graph (DFG) extraction and analysis. By systematically analyzing the differences in graph structure between smelly and non-smelly code, it is found that most code smells manifest as specific structural patterns or abnormal data flow paths on the graph. Therefore, starting from the node types, edge relationships, and topological features of the Code Attribute Graph (CPG), by focusing on the subgraph structures representing code dependencies, calls, and control relationships, the system analyzes their correspondence with smell patterns to inversely deduce the structural indicators of smells. This process requires significant graph computation and traversal overhead. Through repeated iterative experiments, the extraction methods and dimensions of graph features are continuously optimized and adjusted, initially establishing a feature vector space capable of representing complex structural relationships in code.
[0017] Step 3 above trains different models for deep semantic information and complex structural information respectively, constructing heterogeneous odor classifiers, including:
[0018] First, based on the deep semantic feature vectors obtained in step 1, the system selects sequence-sensitive models such as Transformer or Multilayer Perceptron (MLP) as classifiers for semantic branches. By deeply analyzing the distribution of semantic features in the vector space, a loss function is designed to capture subtle semantic differences. Then, based on the Code Attribute Graph (CPG) or its derived graph features obtained in step 2, graph-sensitive models such as Graph Neural Network (GNN) or Graph Convolutional Network (GCN) are selected as classifiers for structural branches. Through systematic analysis of graph topology and odor labels, it is found that different types of GNN layers (such as GAT, GIN) have different abilities to capture different odors. Therefore, this invention chooses to use the GAT_JK_Pool architecture, which integrates Jumping Knowledge (JK) connections and a double pooling (Mean / Max Pooling) mechanism after the graph attention layer (GAT) to extract and fuse features from graph convolutions of different depths. Compared to the basic GCN model, which relies solely on the last layer's features and uses a single pooling method (such as global average pooling), GAT_JK_Pool has its unique features: it utilizes JK connections to collect the outputs of all three layers of GAT and performs both average and max pooling operations on the outputs of each layer. This incremental, multi-scale feature fusion method greatly enhances the model's ability to understand and represent complex graph structure information, thereby effectively improving the model's performance in handling graph-level classification tasks such as code smells.
[0019] Step 4 above fuses the predicted probabilities of different models to obtain the final result, achieving decision-level fusion of multimodal information, including:
[0020] First, the system obtains the predicted probability vectors output by the semantic model and the structural model for the same code sample in step 3 as input for decision fusion. By deeply analyzing the differences in preferences and confidence levels of the two models across different odor categories, a preliminary concept of probability-weighted fusion is proposed. Then, more complex fusion strategies are explored on the validation set. This invention employs a probability-weighted fusion strategy to fuse the prediction results of multimodal models (Code BERT model based on deep semantic information of the code and GCN model based on complex structural information of the code). This method aims to leverage the powerful semantic understanding capabilities of text sequence models and the insight into code structural dependencies of graph models to obtain a final judgment that is more robust and accurate than any single model. The core of the fusion process lies in weighted averaging: the predicted probabilities of the two models for the smell class (i.e., the positive class) are assigned different weights and 1-weights, and a weighted average probability is obtained. To determine the optimal fusion effect, this invention performs a grid search on the validation set, systematically trying different combinations of weights and the final classification threshold, and finally selecting the parameter pair that maximizes the F1-score. Through this optimized fusion mechanism, the present invention can more accurately achieve a balance between semantic and structural information, thereby improving the accuracy and recall of code smell identification.
[0021] The automated code odor identification method of the present invention can detect the types of code odors by using the Code BERT model and the GCN (Graph Convolutional Network) model as benchmarks and utilizing their semantic information and structural information respectively. It makes full use of the deep feature information of the code from multiple aspects to improve the accuracy of code odor detection. Finally, it integrates multiple trained binary classification models into a whole model to improve the flexibility of the model in predicting different code odors.
[0022] The present invention has the following beneficial effects:
[0023] This invention systematically collects and parses code using the encoder of the Code BERT code language model and the Joern code analysis framework, obtaining deep semantic information (step 1) and complex structural information (step 2) to establish a preliminary multimodal feature system. Subsequently, through iterative optimization and feature engineering, a three-dimensional code smell representation system encompassing both code semantics and topological structure is formed. This system not only improves the depth and breadth of smell representation but also provides strong feature support for subsequent accurate detection.
[0024] This invention employs a heterogeneous model independent training method (step 3), utilizing sequence models such as Transformers to process deep semantic information and graph neural networks (GNNs) to process complex structural information. This provides two complementary feature perspectives for classifier training, reducing the bias and false positives / false negatives caused by single-modality analysis. This method not only enhances the classifier's generalization ability to different types of odor patterns but also improves detection accuracy and robustness.
[0025] This invention utilizes deep learning's decision-level fusion technology (step 4), combining the predicted probabilities of the semantic model and the structural model, and fully leveraging the advantages of weighted fusion or hybridization. The idea is to improve detection accuracy by fusing models, thereby achieving automated code smell detection. This significantly reduces the workload of manual code review and analysis, and improves detection efficiency. Through this automation technology, a large number of potential code smells can be quickly and accurately identified and classified, promoting the efficiency and effectiveness of software quality assurance work.
[0026] The detection method and technical solution provided by this invention offer new tools and paradigms for research on automated code quality assurance based on deep learning. This helps developers and software maintainers gain a deeper understanding of the underlying semantics and structural essence of code smells, promoting scientific research and technological progress in the fields of software engineering and intelligent software development. Attached Figure Description
[0027] Figure 1 This invention provides a flowchart of a method for constructing and training a multimodal model to address code smells;
[0028] Figure 2 A flowchart of a method for constructing a multimodal dataset to address code smells, provided by this invention;
[0029] Figure 3 This invention provides a framework diagram for constructing complex structural models; Detailed Implementation
[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0031] Although the steps in this invention are arranged by numbers, this is not intended to limit the order of the steps. Unless the order of the steps is explicitly stated or the execution of a step requires other steps as a basis, the relative order of the steps can be adjusted. It is understood that the term "and / or" as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. The overall process steps used in this invention are as follows: Figure 1 As shown.
[0032] Step 1: Obtain deep semantic information by parsing the code through the encoder of the Code BERT code big language model.
[0033] The core of this step is to transform the original text string of the code into a fixed-dimensional mathematical vector that can represent its program logic and intent.
[0034] First, collect the code snippets to be analyzed and tokenize the code text using the Tokenizer that accompanies the Code BERT pre-trained model. This process breaks down the code string into a series of "subwords." For example... Figure 2 As shown, this invention processes semantic information into semantic vectors using a word segmenter. Special symbols are added at the beginning and end of the token sequence. <s> (sequence start) and< / s> (End of sequence) marker. Additionally, padding or truncation is performed as needed to ensure all input sequences have a uniform length.
[0035] Model Input Construction: The tokenization process generates two key tensors: input_ids: a sequence of indices corresponding to the tokens in the vocabulary. attention_mask: a binary sequence indicating which tokens are real codes (value 1) and which are padding tokens (value 0), so that the model ignores them when calculating attention.
[0036] Forward Propagation and Semantic Extraction: The input_ids and attention_mask are fed into the encoder of the Code BERT model, performing one forward pass. The model outputs the last hidden state of all tags. To obtain a single semantic vector representing the entire code snippet, the first tag is typically extracted (…). <s>The hidden state corresponds to a tag (similar to the [CLS] tag in BERT). This vector is considered the "deep semantic information" of the code snippet. It compresses the code's context, variable relationships, and operational intent, and can be used for subsequent classification tasks.
[0037] Step 2: Use the Joern framework to parse the code and obtain complex structural information;
[0038] The core of this step is to use static analysis tools to extract the explicit structure of the code, especially the dependencies and control relationships between code elements, and represent it as a graph structure.
[0039] Code Property Graph (CPG) Generation: Joern's parser analyzes the entire project or file containing the target code. Joern parses the source code and generates a comprehensive "Code Property Graph" (CPG). For example... Figure 2 As shown, this invention uses Joern to parse structural information into a Code Attribute Graph (CPG). CPG is a powerful graph representation that integrates the following within a single graph: Abstract Syntax Tree (AST): representing the syntactic structure of the code; Control Flow Graph (CFG): representing the execution order and branches of the code; Program Dependency Graph (PDG): representing data and control dependencies between statements; and Data Flow Graph (DFG): tracing the flow path of data (variables) in the program.
[0040] Graph Feature Extraction or Subgraph Construction: For each code segment to be analyzed (e.g., the same function in step one), locate and extract its corresponding subgraph from the complete CPG. This subgraph (or its features) is the "complex structural information" of the code. Convert the subgraph into a format that can be processed by a Graph Neural Network (GNN). This includes: Node Feature Matrix (X): Create a feature vector for each node in the graph (e.g., CALL, METHOD, IDENTIFIER). Adjacency Matrix (A): Indicate whether there are edges between nodes (e.g., AST, CFG, DFG, etc. relationships).
[0041] Step 3: Train different models for deep semantic information and complex structural information respectively;
[0042] The core of this step is to build two independent, specialized classifiers: a fine-tuned model based on Code BERT for understanding deep semantic information, and a GCN model for understanding complex structural information.
[0043] Deep semantic models (Model A: Code BERT model):
[0044] Model Architecture: This paper uses the Code BERT model as the core. The invention employs a pre-trained Code BERT model with a classification head added on top. This classification head is a fully connected layer (i.e., a multilayer perceptron, MLP) that receives the output of Code BERT. <s>The model uses a 768-dimensional semantic vector ([CLS]) to represent the code snippet and maps it to the final classification label. The input to the model consists of the input_ids and attention_mask tensors obtained in step one. Fine-tuning is the core training method, where the tokenized input of the code snippet and its corresponding label (e.g., 0 or 1) are fed into the model. During training, the loss is backpropagated, updating not only the weights of the classification head but also subtly adjusting the weights of the underlying Code BERT encoder. This allows the Code BERT model to better adapt its semantic representation to the specific downstream task of "code smell detection."
[0045] Complex structural model (Model B: GCN model):
[0046] The general structure of the GCN model used in this invention is as follows: Figure 3 As shown, the structural model is based on a graph neural network (GCN) containing three layers of graph convolutions (GCNConv), with all GCN layers having 128 hidden channels. Each graph convolutional block (conv1, conv2, conv3) is stacked strictly in the order of GCNConv->BatchNorm->LeakyReLU->Dropout. BatchNorm (batch normalization) is used to accelerate convergence and stabilize training. After the three convolutional layers are processed, global_mean_pool (global average pooling) is used to aggregate the final feature vectors of all nodes in the graph into a single, fixed-dimensional graph-level representation vector. Finally, a linear layer is used to map this 128-dimensional graph representation vector to a single output channel. This output value will be used for code smell detection. During training, a binary cross-entropy loss with Logits is used. This loss function integrates Sigmoid activation and is specifically designed for binary classification tasks with one output channel. Furthermore, its pos_weight parameter is used to pass in pre-calculated class weights to effectively handle class imbalance in the training data. The Adam optimizer is used for parameter updates, and L2 regularization is configured to reduce model complexity. A scheduler is configured for the training process. This scheduler monitors the F1 score on the validation set, and when the F1 score does not improve within patience = 20 epochs, the learning rate is automatically multiplied by factor = 0.1 (i.e., reduced by a factor of 10) to help the model escape local optima in the later stages of training.
[0047] Step 4: Combine the predicted probabilities from different models to obtain the final result.
[0048] The core of this step is to design a Weight-Threshold mechanism to optimally combine the output results of the two models and make a final prediction.
[0049] For a new code snippet to be detected, it is input into two pre-trained models simultaneously: through Model A (the Code BERT fine-tuned model), the predicted probability P of the semantic model is obtained. semantic Through Model B (the GCN model), the predicted probability P of the structural model is obtained. structure A fusion weight α (whose value ranges from 0 to 1) is introduced to balance the importance of the two models. A final fusion score P is calculated through weighted averaging: final :
[0050] P final = α * P semantic + (1 - α) * P structure
[0051] A decision threshold T (Threshold) (whose value also ranges from 0 to 1, but is no longer limited to 0.5) is introduced. The final fusion score is compared with this threshold: if P final > T, then this code snippet is classified as "smelly". If P final < T, then this code snippet is classified as "non-smelly".
[0052] d and T are not randomly set, but hyperparameters that need to be optimized. The goal of optimization is to maximize the key performance metric: F1 score on the validation set. Specifically, when implementing, grid search is adopted: traverse all possible values of α (from 0.1 to 0.9, with a step of 0.1); traverse all possible values of T (for example, from 0.1 to 0.9, with a step of 0.05). Calculate the F1 score of each pair of <α, T> combinations on the validation set. Finally, select the <α, T> combination that can make the F1 score reach the highest as the final parameters of this fusion mechanism and use it for subsequent test set evaluation and actual deployment.< / s> < / s>
Claims
1. A code smell detection method based on deep semantics and complex structure, characterized in that, include: Step 1: Parse the code using the encoder of the Code BERT code language model to obtain deep semantic information; Step 2: Use the Joern framework to parse the code and obtain complex structural information; Step 3: Train different models for deep semantic information and complex structural information respectively; Step 4: Combine the predicted probabilities from different models to obtain the final result.
2. The code smell detection method based on deep semantics and complex structure according to claim 1, characterized in that... Step 1 involves using the encoder of the Code BERT code-big-language model to parse code and obtain deep semantic information, constructing a preliminary deep semantic feature representation, including: First, the system collected and organized a large-scale code corpus containing various types of code smells as the object of deep semantic encoding. By deeply analyzing the contextual semantics and abstract syntactic structure of representative code fragments in these corpora, a preliminary concept of feature vectors was proposed. Then, a high-quality, expert-annotated benchmark dataset of code smells was collected for model pre-training and fine-tuning. Through systematic analysis of this dataset, it was found that most code smells are closely related to specific semantic patterns in their context. Therefore, starting from the token sequence and abstract syntax tree (AST) of the code, the system focused on the vectors generated by the Code BERT encoder and analyzed their distribution and correlation in high-dimensional space to inversely deduce the deep semantic indicators of the smells. This process required a large amount of computing resources for model training and parameter tuning. Through repeated iterative optimization, the encoding layer and attention weights of the model were continuously optimized and adjusted, and a feature extraction model capable of representing the deep semantics of code was initially established.
3. The code smell detection method based on deep semantics and complex structure according to claim 1, characterized in that, Step 2 involves parsing the code using the Joern framework to obtain complex structural information and constructing a Code Attribute Graph (CPG) feature representation, including: First, the system uses the Joern parsing tool to batch construct graphs from the same code corpus in step 1 as the object of structural analysis. By deeply analyzing the Abstract Syntax Tree (AST), Control Flow Graph (CFG), and Program Dependency Graph (PDG) corresponding to these codes, a preliminary concept of graph feature engineering is proposed. Then, Joern's queryable language is used to extract and analyze the Data Flow Graph (DFG). By systematically analyzing the differences in graph structure between smelly and non-smelly code, it is found that most code smells are manifested as specific structural patterns or abnormal data flow paths on the graph. Therefore, starting from the node types, edge relationships, and topological features of the code attribute graph (CPG), the system focuses on the subgraph structures that represent code dependencies, calls, and control relationships, and analyzes their correspondence with smell patterns to inversely deduce the structural indicators of smells. This process requires a large amount of graph computation and traversal overhead. Through repeated iterative experiments, the extraction methods and dimensions of graph features are continuously optimized and adjusted, and a feature vector space that can represent complex structural relationships of code is initially established.
4. The code smell detection method based on deep semantics and complex structure according to claim 1, characterized in that, Step 3 trains different models for deep semantic information and complex structural information respectively, constructing heterogeneous odor classifiers, including: First, based on the deep semantic feature vectors obtained in step 1, the system selects sequence-sensitive models such as Transformer or Multilayer Perceptron (MLP) as classifiers for semantic branches. By deeply analyzing the distribution of semantic features in the vector space, a loss function is designed to capture subtle semantic differences. Then, based on the code attribute graph (CPG) or its derived graph features obtained in step 2, graph-sensitive models such as Graph Neural Networks (GNN) or Graph Convolutional Networks (GCN) are selected as classifiers for structural branches. Through systematic analysis of graph topology and odor labels, it is found that different types of GNN layers, such as GAT and GIN, have different abilities to capture different odors. Therefore, we choose to use the GAT_JK_Pool architecture. This model integrates Jumping Knowledge connections and double pooling Mean / Max after the graph attention layer (GAT). The pooling mechanism extracts and fuses features from graph convolutions of different depths. Compared to the basic GCN model, which only relies on the features of the last layer and uses a single pooling method such as global average pooling, GAT_JK_Pool has its unique features: it uses JK connections to collect the outputs of all three layers of GAT and performs average and max pooling operations on the output of each layer. This incremental, multi-scale feature fusion method greatly improves the model's ability to understand and represent complex graph structure information, thereby effectively enhancing the model's performance in handling graph-level classification tasks such as code smells.
5. The code smell detection method based on deep semantics and complex structure according to claim 1, characterized in that, Step 4, which fuses the predicted probabilities of different models to obtain the final result, achieves decision-level fusion of multimodal information, including: First, the system obtains the predicted probability vectors output by the semantic model and the structural model for the same code sample in step 3 as input for decision fusion. By deeply analyzing the differences in preferences and confidence levels of the two models across different odor categories, a preliminary probability-weighted fusion concept is proposed. Then, more complex fusion strategies are explored on the validation set. We adopt a probability-weighted fusion strategy to combine the prediction results of the multimodal model CodeBERT, which is based on deep semantic information of the code, and the GCN, which is based on complex structural information of the code. This method aims to leverage the powerful semantic understanding capabilities of text sequence models and the insight into code structural dependencies of graph models to obtain higher prediction results than either single model. The final judgment on the robustness and accuracy of a model lies in the weighted averaging process: the predicted probabilities of the two models for the smell class (i.e., the positive class) are assigned different weights (weight and 1-weight), and the weighted average probability is obtained. In order to determine the best fusion effect, we performed a grid search on the validation set and systematically tried different combinations of weights and the final classification threshold. Finally, we selected the parameter pair that maximizes the F1-score. Through this optimized fusion mechanism, we can more accurately achieve a balance between semantic and structural information, thereby improving the accuracy and recall of code smell identification. Our automated code smell identification method uses the Code BERT model and the GCN graph convolutional network model as benchmarks, respectively utilizing their semantic and structural information to detect different types of code smells. It makes full use of the deep feature information of the code from multiple aspects to improve the accuracy of code smell detection. Finally, it merges multiple trained binary classification models into a whole model to improve the flexibility of the model in predicting different code smells.