Source Code Anomaly Detection via Token Probability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Manual inspection of source code for errors, bugs, and deviations from high-quality coding standards is labor-intensive and inefficient, especially in complex software projects, and existing static analysis tools rely on manually defined rules and are insufficient.
Innovation Solution
A machine learning-based anomaly detection system that uses a trained generative language model to compute the probability of tokens in source code, detecting low probability regions and suggesting alternative tokens to improve code quality by identifying deviations from high-quality coding standards.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual inspection of source code is performed, then detection precision of errors and bugs is improved, but productivity deteriorates due to labor-intensive process
Solution Approach 1:
The system enables automated self-inspection of source code through machine learning models that automatically detect anomalies, errors, and code smells without requiring manual reviewer intervention, thus maintaining high detection precision while dramatically improving productivity
Solution Approach 2:
The patent replaces the mechanical manual inspection process with an automated electronic system using trained language models and probability-based anomaly detection, substituting human labor with computational processes that achieve both high precision and scalability
2Productivity
If existing static analysis tools are used, then productivity is improved through automation, but detection precision deteriorates due to reliance on manually defined rules
Solution Approach 1:
The system transitions from static rule-based parameters to dynamic probability parameters by using trained language models that compute likelihood scores for code tokens, enabling the detection system to adapt to diverse coding patterns and contexts while maintaining high productivity through automation
Solution Approach 2:
The patent introduces dynamic anomaly detection by computing probability scores that adapt to different code contexts and patterns, replacing rigid manual rules with flexible, context-aware probabilistic assessments that improve detection precision while maintaining automated efficiency
3Reliability
If comprehensive code inspection is performed, then reliability is improved, but loss of time increases due to labor-intensive nature
Solution Approach 1:
The system enables continuous automated code inspection that can operate without interruption, maintaining high reliability through comprehensive analysis while eliminating the time loss associated with manual inspection processes by running probabilistic anomaly detection algorithms automatically
Solution Approach 2:
The automated system performs comprehensive reliability checks on source code without requiring human time investment, using trained machine learning models to continuously assess code quality and detect anomalies, thus maintaining high reliability while eliminating manual time costs
Data Source
AI summary
A method, computer system, and a computer program product for software anomaly detection is provided. The present invention may include, receiving a target source code including a sequence of tokens. The present invention may also include, determining a probability of a candidate token in the sequence of tokens based on a context of other tokens in the sequence of tokens. The present invention may further include, in response to the determined probability of the candidate token satisfying a low probability threshold, detecting a low probability region in the received target source code, wherein the detected low probability region is associated with the candidate token satisfying the low probability threshold.