AI Code Detection Using Masked Line Completion Probabilities
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of distinguishing between human-written and AI-generated computer code has become increasingly difficult due to the advancements in large language models, leading to concerns about code quality, security, plagiarism, and ethical misuse, with existing detection methods being inadequate, especially for codes generated by models like GPT-4.
Innovation Solution
A method involving masking lines of code to create perturbed codes, generating missing code using an AI code generator model, and employing a surrogate model to predict probabilities, comparing these against a detection threshold to determine AI-generated code.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing detection methods are used, then detection simplicity is maintained, but detection precision deteriorates due to inability to distinguish AI-generated code from human-written code
Solution Approach 1:
The detection method segments the code analysis process into multiple distinct stages: perturbation generation, code completion, probability prediction, and threshold comparison. Each stage processes specific aspects of the code independently, allowing for precise measurement of AI-generated characteristics without requiring a monolithic complex detection system.
Solution Approach 2:
The patent introduces intermediary elements including perturbed codes as intermediate representations, completion probabilities as intermediate metrics, and detection thresholds as intermediate decision boundaries. These intermediaries bridge the gap between raw code input and final detection output, enabling precise differentiation between AI-generated and human-written code through measurable probability distributions.
2Measurement precision
If AI code generator models are employed to generate perturbed codes, then detection precision improves through probability comparison, but computational cost increases
Solution Approach 1:
The method applies partial action by generating only the necessary perturbed codes required for detection rather than comprehensive code transformations. The surrogate model predicts probabilities for specific perturbed versions rather than analyzing all possible code variations, reducing computational energy while maintaining detection precision through targeted probability comparisons.
Data Source
AI summary
Systems and methods for detecting artificial intelligence (AI) generated computer code. Lines of code can be masked from a candidate code to obtain perturbed codes. Missing code can be generated from the perturbed codes by employing an AI code generator model to obtain machine-filled codes. Probabilities of the candidate code probability and the machine-filled codes as AI-generated can be predicted by employing a surrogate model. The candidate code can be distinguished as AI-generated by comparing the probabilities against a detection threshold to obtain detection results.