AI Code Detection Using Masked-Line Perturbation Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge lies in effectively distinguishing between human-written and AI-generated computer code, as large language models have made machine-generated code indistinguishable from human-written code, leading to concerns about security, plagiarism, and code quality.
Innovation Solution
A method involving masking lines of code to create perturbed codes, generating missing code using an AI code generator model, and employing a surrogate model to predict probabilities, comparing these against a detection threshold to determine AI-generated code.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If large language models are used to generate computer code, then code fluency and diversity are improved, but the ability to distinguish machine-generated code from human-written code deteriorates
Solution Approach 1:
The system performs preliminary actions by masking code lines before detection, creating perturbed versions of the code. This preprocessing step enables the detection model to analyze patterns that reveal whether the original code was AI-generated, addressing the detection difficulty without affecting the fluency of the generated code itself
Solution Approach 2:
The patent introduces an intermediary detection system that acts as a mediator between AI code generation and human review. This intermediary analysis layer, which uses perturbation-based detection, allows fluent AI-generated code to be identified without requiring changes to the code generation process itself
2Productivity
If AI code generator models are employed to generate code, then productivity is improved, but code quality control deteriorates due to inability to identify AI-generated code
Solution Approach 1:
The system implements feedback by using detection results to inform future code generation and review processes. When code is identified as AI-generated through the perturbation-based detection method, this feedback enables quality control measures to be applied selectively, maintaining high productivity while improving code quality assurance
3Measurement precision
If masking lines of code is performed to create perturbed codes, then detection accuracy is improved, but processing complexity increases
Solution Approach 1:
The detection process is segmented into distinct steps: masking code lines to create perturbations, analyzing the perturbed code, and comparing results against the original. This segmentation allows the system to achieve high detection accuracy through systematic analysis while managing processing complexity through modular implementation
Data Source
AI summary
Systems and methods for detecting artificial intelligence (AI) generated computer code. Lines of code can be masked from a candidate code to obtain perturbed codes. Missing code can be generated from the perturbed codes by employing an AI code generator model to obtain machine-filled codes. Probabilities of the candidate code probability and the machine-filled codes as AI-generated can be predicted by employing a surrogate model. The candidate code can be distinguished as AI-generated by comparing the probabilities against a detection threshold to obtain detection results.