AI Code Detection Using Masked Perturbation and Surrogate Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of distinguishing between human-written and AI-generated computer code has become increasingly difficult due to the advancements in large language models, leading to concerns about code quality, security, plagiarism, and ethical misuse, with existing detection methods failing to effectively identify AI-generated code.
Innovation Solution
A method involving masking lines of code to create perturbed codes, generating missing code using an AI code generator model, and employing a surrogate model to predict probabilities, comparing these against a detection threshold to determine AI-generated code.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If large language models are used to generate computer code, then code fluency and diversity are improved, but the ability to distinguish AI-generated code from human-written code deteriorates
Solution Approach 1:
The patent applies preliminary action by masking certain lines of code before generation to create perturbed codes. This preprocessing step modifies the input code structure in advance, allowing the detection system to capture subtle patterns that distinguish AI-generated code from human-written code, thereby resolving the detection difficulty despite improved code fluency
Solution Approach 2:
The patent introduces an intermediary detection system that uses surrogate models to evaluate the probability of code being AI-generated. This intermediary layer acts as a mediator between the code generation process and final code evaluation, enabling differentiation of AI-generated code even when fluency is high
2Ease of operation
If existing detection methods are used, then detection simplicity is maintained, but detection accuracy deteriorates
Solution Approach 1:
The patent segments the detection process into multiple components: masking perturbation, AI code generation, surrogate model evaluation, and probability comparison. This segmentation allows each component to be optimized independently while maintaining overall system simplicity, achieving both ease of operation and high detection accuracy
Solution Approach 2:
The patent changes key parameters in the detection process by using probability thresholds and perturbation levels as adjustable parameters. This allows the detection system to adapt to different code types and complexity levels, improving measurement precision while maintaining operational simplicity through parameter adjustment rather than structural complexity
Data Source
AI summary
Systems and methods for detecting artificial intelligence (AI) generated computer code. Lines of code can be masked from a candidate code to obtain perturbed codes. Missing code can be generated from the perturbed codes by employing an AI code generator model to obtain machine-filled codes. Probabilities of the candidate code probability and the machine-filled codes as AI-generated can be predicted by employing a surrogate model. The candidate code can be distinguished as AI-generated by comparing the probabilities against a detection threshold to obtain detection results.