Transformer Code Representations for Label-Free Code Smell Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional code smell detection methods rely heavily on manual annotation of large datasets, which is costly, time-consuming, and prone to errors, and supervised learning models struggle with generalization due to the lack of large-scale labeled data.
Innovation Solution
A self-supervised learning approach using transformer-based code representations (CoRT) that learns intermediate representations from unlabeled data by predicting masked reserved words, allowing for effective detection of code smells without manual labeling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual annotation of large datasets is used for code smell detection, then detection accuracy can be improved, but the cost and time consumption increase significantly
Solution Approach 1:
The system uses self-supervised learning where the model learns to predict masked reserved words in code, automatically generating training signals without human intervention. The transformer model processes code sequences, masks certain reserved words, and learns to predict them, thereby self-training on unlabeled code data while maintaining detection accuracy.
Solution Approach 2:
The approach performs preliminary learning on large amounts of unlabeled code data before actual code smell detection is needed. By pre-training the transformer model on code structure and syntax patterns through the masked reserved word prediction task, the system prepares robust representations that generalize well to detection tasks without requiring manual annotation at deployment time.
2Reliability
If supervised learning models are used for code smell detection, then detection performance can be improved, but the need for large-scale labeled data increases cost and complexity
Solution Approach 1:
The system eliminates the need for external labeled data by creating its own training signals through masking reserved words in the code. The model learns from the structure and syntax patterns inherent in the code itself, generating supervision signals automatically without human annotators or complex data labeling pipelines.
Solution Approach 2:
The masked reserved word prediction task serves as an intermediary learning objective that bridges the gap between raw code data and code smell detection. By learning to predict reserved words in context, the model indirectly learns code structure and patterns that are transferable to detection tasks, without requiring direct labeled detection data.
3Ease of manufacture
If conventional threshold-based methods are used for code smell detection, then implementation simplicity is maintained, but detection accuracy and adaptability decrease
Solution Approach 1:
The system replaces mechanical threshold-based rule systems with a transformer-based neural network that learns code patterns automatically. Instead of manually defining thresholds for code metrics, the model learns hierarchical representations of code structure and syntax, enabling more accurate and adaptable detection without sacrificing implementation feasibility through standardized transformer architectures.
Data Source
AI summary
A device, method, and non-transitory computer readable medium that for analyzing computer source code to detect code smells is disclosed. The method includes inputting, via processing circuitry, the source code and creating, via the processing circuitry, pseudo labels by a proxy task based on a vector of tokens for the source code. In addition, the method includes training, via the processing circuitry, a transformer model on the pseudo labels, as a pre-trained model that outputs a prediction of a value of tokens in the vector of tokens, and applying, via the processing circuitry, the pre-trained model to a plurality of fine-tuning models for respective downstream tasks, where each fine-tuning model is created by training the pre-trained model. The method also includes outputting, via the processing circuitry, from each fine-tuning model, an indication of whether a code smell has been detected in the source code.