Large language model generation code detection method and system

Abstract

The invention provides a large language model generation code detection method and system, which is applied to the technical field of artificial intelligence, and comprises the following steps: obtaining a to-be-detected code; a to-be-detected code is input to a trained shared encoder, a code feature vector is obtained, the shared encoder is obtained through multi-target joint training, and the multi-target joint training is used for optimizing classification loss, domain confrontation loss, comparison loss and difficult sample loss at the same time; l2 normalization is carried out on the code feature vector, and the code feature vector is mapped to a hyperspherical space to obtain spherical embedding; the sphere is embedded and input into a sphere category classifier based on sphere logistic regression for classification processing, a detection result of the to-be-detected code output by the sphere category classifier is obtained, the detection result comprises AI generation and human writing, and a decision boundary of the sphere category classifier is an intersection line of a hyperplane and a hypersphere for classification. According to the invention, the AI generation code and the human compiled code can be accurately distinguished.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for detecting code generated by a large language model. Background Technology

[0002] Large language model-generated code detection technology aims to distinguish between human-written code and code generated by large language models through technical means. With the widespread application of large language models in the software development process, this technology is becoming increasingly important in ethical and security scenarios such as preventing academic misconduct and cheating in technical interviews.

[0003] For example, code watermarking technology for large language models assists in recognition by embedding specific patterns during the generation process. However, this method inevitably reduces the quality of the generated code and cannot be applied to large, non-open-source models. Zero-shot detection technology, on the other hand, does not require additional fine-tuning. Instead, it utilizes the inherent features of the content generated by the large language model (such as syntactic features and word collocation) and uses a proxy model to score and make judgments. However, zero-shot detection generally suffers from insufficient accuracy, and its performance is highly dependent on the proxy model, resulting in poor stability.

[0004] It is evident that the code detection method generated by large language models in related technologies suffers from technical problems such as low accuracy and poor generalization. Summary of the Invention

[0005] This invention provides a method and system for detecting code generated by a large language model, which addresses the shortcomings of existing methods for detecting code generated by large language models, which have low accuracy, and enables accurate differentiation between AI-generated code and human-written code.

[0006] This invention provides a method for detecting code generated by a large language model, comprising the following steps.

[0007] Obtain the code to be detected; input the code to be detected into a trained shared encoder to obtain a code feature vector, wherein the shared encoder is obtained through multi-objective joint training, which is used to simultaneously optimize classification loss, domain adversarial loss, contrastive loss, and hard sample loss; perform L2 normalization on the code feature vector and map it to a hyperspherical space to obtain a spherical embedding; input the spherical embedding into a spherical classifier based on spherical logistic regression for classification processing to obtain the detection result of the code to be detected output by the spherical classifier, wherein the detection result includes: AI-generated and human-written, and the decision boundary of the spherical classifier is the intersection of the hyperplane used for classification and the hypersphere.

[0008] According to the method for detecting code generated by a large language model provided by the present invention, the step of inputting the spherical embedding into a spherical category classifier based on spherical logistic regression for classification processing, and obtaining the detection result of the code to be detected output by the spherical category classifier, includes: The classification probability of the code to be detected is determined according to the following formula: ; in, Represents the classification probability. This indicates the category label of the code to be detected. Indicates the category label index. This indicates that the i-th code to be detected is obtained. w represents the weight vector corresponding to category k. j This represents the weight vector corresponding to category j. Represents the transpose matrix. This represents the bias term corresponding to category k. This represents the bias term corresponding to category j.

[0009] According to a method for detecting code generated by a large language model provided by the present invention, before inputting the code to be detected into a trained shared encoder to obtain a code feature vector, the method further includes: acquiring a training code sample set, wherein the training code sample set includes multiple code segments, each code segment having a category label and a domain label; inputting the training code sample set into a large language model for rewriting to obtain a hard sample set of the code sample set, wherein the hard sample set includes semantically equivalent variants of each code segment; encoding based on the training code sample set and the hard sample set using a shared encoder to obtain a code feature vector set; and processing the code feature vectors. The eigenvector set is normalized and mapped to a hyperspherical space to obtain a spherical embedding sample set. Class classification training is performed based on this spherical embedding sample set to determine the classification loss. Domain adversarial training is performed based on this spherical embedding sample set to determine the domain adversarial loss. Inter-class and intra-class contrastive training is performed based on this spherical embedding sample set to determine the contrastive loss. Hard sample contrastive training is performed based on the spherical embedding sample set and the spherical embeddings corresponding to the hard sample set to determine the hard sample loss. Based on the classification loss, domain adversarial loss, contrastive loss, and hard sample loss, the total loss for multi-objective joint training is determined. Based on the total loss, the parameters of the shared encoder are optimized to obtain a trained shared encoder.

[0010] According to a method for detecting code generated by a large language model provided by the present invention, the step of performing hard sample comparison training based on the spherical embedding sample set and the hard sample set to determine the hard sample loss includes: obtaining code segments corresponding to the original sample spherical embeddings in the spherical embedding sample set; determining the hard sample spherical embeddings of the hard samples corresponding to the code segments; and determining the hard sample comparison training loss based on the spherical similarity between the original sample spherical embeddings and the hard sample spherical embeddings, wherein: for code segments whose category label is AI-generated, the spherical distance between the original sample spherical embeddings and the hard sample spherical embeddings is reduced; for code segments whose category label is human-written, the spherical distance between the original sample spherical embeddings and the hard sample spherical embeddings is increased.

[0011] According to a method for detecting code generated by a large language model provided by the present invention, the method for performing domain adversarial training based on the spherical embedding sample set and determining the domain adversarial loss includes: constructing an adversarial training framework consisting of a shared encoder, a spherical domain classifier, and a gradient inversion layer; wherein the spherical domain classifier is constructed based on spherical logistic regression and is used to identify the domain labels of the spherical embeddings of the original samples; the gradient inversion layer does not process the spherical embeddings of the original samples during forward propagation, and multiplies the gradient output by the spherical domain classifier by negative one before passing it to the shared encoder during backward propagation; the spherical embedding set is input into the adversarial training framework to obtain the probability that each spherical embedding of the original samples belongs to each domain, as output by the adversarial training framework; and the domain adversarial loss is determined based on the probability and the domain label of the code fragment, wherein the domain adversarial loss adopts the form of a cross-entropy loss function.

[0012] According to the present invention, a method for detecting code generated by a large language model includes the following steps: performing inter-class and intra-class comparison training based on the spherical embedding sample set to determine the comparison loss. The method comprises: defining code segments in the training code sample set that have the same code category label as positive sample pairs, and constructing a positive sample pair set; defining code segments in the training code sample set that have different code category labels as negative sample pairs, and constructing a negative sample pair set; determining the similarity between the code segment and the positive and negative sample pairs based on the spherical inner product; and for each code segment in the training code sample set, determining the comparison loss based on the similarity between the code segment and samples in the positive sample pair set, and the similarity between the code segment and samples in the negative sample pair set.

[0013] This invention also provides a detection system for code generated by a large language model, comprising the following modules: an acquisition module for acquiring code to be detected; an encoding module for inputting the code to be detected into a trained shared encoder to obtain a code feature vector, wherein the shared encoder is obtained through multi-objective joint training, which simultaneously optimizes classification loss, domain adversarial loss, contrastive loss, and hard sample loss; a mapping module for performing L2 normalization on the code feature vector and mapping it to a hyperspherical space to obtain a spherical embedding; and a detection module for inputting the spherical embedding into a spherical classifier based on spherical logistic regression for classification processing to obtain the detection result of the code to be detected output by the spherical classifier, wherein the detection result includes: AI-generated and human-written, and the decision boundary of the spherical classifier is the intersection of the hyperplane used for classification and the hypersphere.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the detection method for generating code from a large language model as described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the detection method for generating large language models as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the detection method for generating code from a large language model as described above.

[0017] The method and system for detecting code generated by a large language model provided by this invention first acquire the code to be detected, providing an input source for subsequent processing. Second, the code to be detected is input into a shared encoder obtained through multi-objective joint training. This encoder, by simultaneously optimizing classification loss, domain adversarial loss, contrastive loss, and hard sample loss, can extract code feature vectors with domain invariance, discriminativeness, and robustness, effectively overcoming the distribution difference problem of traditional methods in cross-domain scenarios. Next, the code feature vectors are L2 normalized and mapped to a hyperspherical space to obtain a spherical embedding. This step utilizes the geometric properties of a sphere to constrain the feature distribution on a unit sphere, reducing inter-domain offset and enhancing the generalization ability of the features. Finally, the spherical embedding is input into a spherical classifier based on spherical logistic regression. The decision boundary of this classifier is the intersection of the hyperplane and the hypersphere, which is more suitable for the spherical feature space, thereby accurately outputting detection results generated by AI or written by humans, improving the accuracy and stability of cross-domain detection. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the detection method for code generated by a large language model provided by the present invention.

[0020] Figure 2 This is a schematic diagram of the learning framework provided by the present invention.

[0021] Figure 3 This is a schematic diagram of the code generation detection generalization framework provided by the present invention.

[0022] Figure 4 This is a diagram showing the code generation and detection effect under the attack scenario provided by the present invention.

[0023] Figure 5 This is a schematic diagram of the module of the detection system for generating code from a large language model provided by the present invention.

[0024] Figure 6 This is a schematic diagram of the physical structure of the electronic device provided by the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0026] This invention belongs to the field of large language model generated code detection, specifically involving a method for detecting large language model generated code. It aims to solve the problem of accurately identifying large language model generated code across different programming languages, code generation models, and application fields, and is applicable to scenarios such as education and industry where it is necessary to distinguish between code written by humans and artificial intelligence (AI).

[0027] Supervised fine-tuning detection technology, which trains feature extractors and classifiers on large-scale labeled datasets, performs well when the training and test data are in the same distribution. However, recent research shows that its detection performance drops significantly when faced with out-of-distribution data—including training on unseen programming languages, unseen generative models, or unseen application domains. To maintain the effectiveness of this technology, it is usually necessary to re-label and retrain on out-of-distribution data, which is not only costly but also hinders its practical deployment in industry.

[0028] Among related technologies, a pre-trained code model is used as a feature encoder, and the feature space distribution is optimized through a carefully designed contrastive learning mechanism: during training, the embedding distance between human-written code and AI-generated code is deliberately increased, while the embedding difference between similar samples (human-to-human and AI-to-AI generated code) is reduced, enabling the model to learn a strong discriminative feature representation, thereby achieving a high accuracy in code generation detection tasks. Another related technology focuses on improving the cross-domain generalization ability of AI-generated content detection. Its core is to combine domain adversarial learning with supervised fine-tuning on large-scale multi-domain code generation datasets: by introducing a domain classifier and a gradient inversion layer to build an adversarial training mechanism, the feature encoder is forced to strip away domain-specific information and retain domain-general information, thereby achieving the model's generalization ability in detecting Artificial Intelligence Generated Content (AIGC) in unknown domain scenarios.

[0029] Existing technologies suffer from two significant drawbacks: First, they exhibit weak cross-domain generalization ability. Detection methods centered on supervised fine-tuning are highly susceptible to overfitting to specific patterns in the training domain when faced with unfamiliar programming languages ​​(e.g., migrating from Python to Go), new code generation models (e.g., switching from GPT-4 to Claude), or unfamiliar application scenarios (e.g., transitioning from industrial project code to algorithmic competition problems). This is due to significant differences in code style, syntax features, and semantic habits between different domains, leading to a sharp decline in detection accuracy. Performance losses typically reach 10%-20%, making it difficult to adapt to complex and ever-changing real-world detection needs. Second, there is an imbalance between the discriminative power and domain invariance of features. While adversarial learning-based solutions can align feature distributions across different domains and mitigate domain-specific interference through adversarial training, this global domain alignment may overlook the compactness of features within categories. For example, excessive elimination of domain differences may result in samples of the same category (e.g., human-written code in different domains) being scattered across the feature space, blurring the essential difference between human-generated and AI-generated code, ultimately affecting the core discriminative ability of the generation detection task.

[0030] This invention aims to address the cross-domain generalization bottleneck of existing large language model-generated code detection technologies. Specifically, it includes: improving detection accuracy in scenarios with unknown programming languages, unknown generation models, and unknown application domains (such as competitive programming, basic programming, etc.), and reducing performance loss caused by domain shift; enhancing intra-class compactness and inter-class discriminability while ensuring embedding domain invariance, and balancing feature generalization and discriminativity; furthermore, strengthening the model's detection stability in adversarial rewriting attack environments; and providing comprehensive verification of the cross-domain detection performance of this technology by constructing a multi-dimensional (language, model, scenario) evaluation dataset, ultimately achieving reliable large language model-generated code detection in complex real-world scenarios.

[0031] Although existing code generation detection technologies have achieved certain results in specific scenarios, their lack of cross-domain generalization ability makes it difficult to cope with complex and ever-changing real-world detection needs.

[0032] This invention aims to learn an intrinsic code embedding representation to achieve generalization performance in code generation detection. This representation must satisfy three core characteristics: domain independence, where the code representation should remove domain-specific information while retaining inter-domain common information; strong discriminativeness, where code embedding representations of the same category are tightly clustered, while embedding representations of different categories should be clearly separated; and robustness, where the code embedding representation can ensure stable large-scale model code generation detection even under adversarial perturbation attacks.

[0033] Contrastive learning generates positive and negative sample pairs through data augmentation, and optimizes the model by bringing similar samples closer together and dissimilar samples further apart using an encoder and contrastive loss. Domain adversarial learning uses a feature extractor and a domain classifier, and constructs an adversarial mechanism through a gradient inversion layer, allowing the extractor to learn common features (confused domain classifier) ​​and alleviate domain bias. Similarity learning uses a feature extractor and a similarity measurement module, and optimizes similar samples by using triplet loss and other methods to make similar samples closer and dissimilar samples farther apart.

[0034] Based on this, the present invention gradually achieves the acquisition of this universal code representation by synergistically integrating adversarial training, contrastive learning, and hard sample augmentation. Specifically, the present invention includes three key additional components: an adversarial domain classification module that uses domain labels to reduce specific domain bias; an inter-class and intra-class contrastive learning objective to enhance the discriminative ability of domain-invariant embeddings; and a hard sample learning strategy that further enhances the robustness of the extracted code representation by comparing semantically similar samples.

[0035] Optionally, the method for detecting the large language model generated code in this embodiment can be executed by a server, by a terminal device, or by both a server and a terminal device. Taking the execution of the method for detecting the large language model generated code in this embodiment by a server as an example.

[0036] Figure 1 This is a flowchart illustrating the detection method for code generated by a large language model provided by the present invention, as shown below. Figure 1 As shown, the method includes the following steps.

[0037] Step 101: Obtain the code to be detected.

[0038] This step involves preparing the input data. The code to be tested can be any form of code snippet, such as source code obtained from a file system, database, or real-time input stream. During implementation, the code to be tested should be provided in text format, ensuring its structural integrity (e.g., retaining indentation, comments, and other syntax elements) for subsequent processing.

[0039] After obtaining the code to be detected, it undergoes preliminary preprocessing, such as removing irrelevant characters or standardizing the encoding format, but the core semantic content remains unchanged.

[0040] Step 102: Input the code to be detected into the trained shared encoder to obtain the code feature vector.

[0041] The shared encoder is obtained through multi-objective joint training, which is used to simultaneously optimize classification loss, domain adversarial loss, contrastive loss, and hard sample loss.

[0042] A shared encoder is a pre-trained neural network model whose parameters are optimized through multi-objective joint training. It can map code snippets into high-dimensional vector representations. Upon receiving the code to be detected, the shared encoder first performs lexical analysis and syntax parsing to generate a token sequence; then, it encodes contextual information through a multi-layer self-attention mechanism, outputting a code feature vector of dimension d.

[0043] Multi-objective joint training simultaneously optimizes four types of loss functions: classification loss, domain adversarial loss, contrastive loss, and hard-sample loss. During training, the shared encoder iteratively updates its parameters on the training code sample set. The classification loss is calculated using a spherical classifier to ensure basic discriminative ability; the domain adversarial loss is calculated using a gradient reversal layer and a spherical domain classifier to enhance feature domain invariance; the contrastive loss is calculated based on inter-class and intra-class contrastive learning to improve feature discriminativity; and the hard-sample loss is calculated using semantically equivalent hard samples to improve robustness. After training, the shared encoder can extract general features with domain invariance, discriminativity, and robustness, effectively overcoming cross-domain distribution differences.

[0044] The code feature vectors output by the shared encoder capture deep semantic information of the code, providing input for subsequent spherical space mapping. Its advantage lies in balancing generalization and accuracy through multi-objective training, avoiding the performance degradation of traditional methods in unknown domains.

[0045] Step 103: Perform L2 normalization on the code feature vector and map it to the hypersphere space to obtain the spherical embedding.

[0046] In this embodiment of the invention, the code feature vector is L2 normalized, the Euclidean norm of the code feature vector is calculated, and then the vector is scaled to a unit norm vector. Then, by multiplying by the hypersphere radius r (r is a preset constant, such as r=10), the vector is mapped to the hypersphere space to obtain a spherical embedding.

[0047] Through L2 normalization, the feature distribution of the code feature vector is constrained to a hypersphere with radius r. This eliminates the domain-specific bias caused by the vector magnitude (such as style differences between different programming languages) and instead focuses on the feature direction (angular information). This mapping can take advantage of the inherent stability of spherical geometry, reduce inter-domain distribution shifts, and improve the generalization ability of features in unknown scenarios.

[0048] Step 104: Input the spherical embedding into a spherical category classifier based on spherical logistic regression for classification processing, and obtain the detection result of the code to be detected output by the spherical category classifier.

[0049] The detection results include: AI-generated and human-written results; the decision boundary of the spherical category classifier is the intersection of the hyperplane and the hypersphere used for classification.

[0050] The spherical category classifier is built upon spherical logistic regression, and its decision boundary is defined as the intersection of the hyperplane and the hypersphere. Specifically, the classifier receives spherical embeddings as input and calculates the probability that each spherical embedding belongs to a specific category.

[0051] The decision-making process of a spherical classifier relies on the interaction between the hyperplane and the hypersphere, defined as follows: ( A unit weight vector, To constrain The weight vector defines a hyperplane, which intersects with the hypersphere to form a circular decision boundary; the bias term adjusts the boundary position, enabling the spherical classifier to flexibly distinguish different categories. Finally, the spherical classifier selects the category with the highest probability as the detection result (e.g., AI-generated or human-written).

[0052] Traditional linear classifiers are susceptible to domain shifts in Euclidean space, while spherical classifiers, through spherical constraints, make the decision boundary more stable, significantly improving cross-domain detection accuracy. The output can be directly used in downstream applications (such as academic integrity checks or code quality assessments) without additional calibration.

[0053] In this embodiment of the invention, firstly, the code to be detected is obtained to provide an input source for subsequent processing. Secondly, the code to be detected is input into a shared encoder obtained through multi-objective joint training. This encoder, by simultaneously optimizing classification loss, domain adversarial loss, contrastive loss, and hard sample loss, can extract code feature vectors with domain invariance, discriminativeness, and robustness, effectively overcoming the distribution difference problem of traditional methods in cross-domain scenarios. Next, the code feature vectors are L2 normalized and mapped to a hyperspherical space to obtain a spherical embedding. This step utilizes the geometric properties of the sphere to constrain the feature distribution on a unit sphere, reducing inter-domain offset and enhancing the generalization ability of the features. Finally, the spherical embedding is input into a spherical classifier based on spherical logistic regression. The decision boundary of this classifier is the intersection of the hyperplane and the hypersphere, which is more suitable for the spherical feature space, thereby accurately outputting detection results generated by AI or written by humans, improving the accuracy and stability of cross-domain detection.

[0054] According to the method for detecting code generated by a large language model provided by the present invention, a spherical embedding is input into a spherical category classifier based on spherical logistic regression for classification processing, and the detection result of the code to be detected output by the spherical category classifier is obtained, including: The classification probability of the code to be detected is determined using the following formula: ; in, Represents the classification probability. The category label representing the code to be detected. Indicates the category label index. This indicates that the i-th code to be detected is obtained. w represents the weight vector corresponding to category k. j This represents the weight vector corresponding to category j. Represents the transpose matrix. This represents the bias term corresponding to category k. This represents the bias term corresponding to category j.

[0055] refer to Figure 2 , Figure 2 This is a schematic diagram of the learning framework provided by the present invention, wherein, reference Figure 2 (a) The spherical classification backbone includes the code to be detected C0, the shared encoder, L2 normalization, the spherical category classifier, and .

[0056] This invention uses UniXCoder as the base encoder (i.e., a shared encoder) to process the input code snippets. Encode as a code feature vector, i.e., a high-dimensional vector. Subsequently through Normalization maps it to hyperspherical space, resulting in a spherical embedding. (in The radius of the hypersphere, (Using the Euclidean norm), making the feature distribution more adaptable to cross-domain generalization scenarios. Simultaneously, a spherical classifier was constructed based on spherical logistic regression, whose decision boundary is the intersection of the hyperplane and the hypersphere, defined as... ( For the weight vector, To constrain The bias term is used to calculate the classification probability using the above formula.

[0057] The classification performance is optimized using cross-entropy loss, where the cross-entropy loss formula is as follows: ; in, Represents classification loss, Indicates the number of samples in the batch. Indicates the index of the sample. Represents probability. Represents the category label of the i-th sample (e.g., (Generate labels for AI) This represents the spherical embedding corresponding to the i-th sample.

[0058] Through the embodiments of the present invention, the classification probability of the code to be detected is determined by a specific formula. By using a spherical classifier based on spherical logistic regression, the detection results of the code to be detected can be calculated based on the spherical embedding of the code to be detected, the weight vectors corresponding to each category, and the bias terms. This can output the detection results of the code to be detected more accurately.

[0059] According to the method for detecting code generated by a large language model provided by the present invention, before inputting the code to be detected into a trained shared encoder to obtain the code feature vector, the method further includes: Obtain a training code sample set, which includes multiple code snippets, each with a category label and a domain label; The training code sample set is input into the large language model for rewriting, resulting in the hard sample set output by the large language model. The hard sample set includes semantically equivalent variants of each code segment. Based on the training code sample set and the hard sample set, a code feature vector set is obtained by encoding through a shared encoder; The code feature vector set is normalized and mapped to a hyperspherical space to obtain a spherical embedding sample set; Class classification training is performed based on a spherical embedding sample set to determine the classification loss; Domain adversarial training is performed based on a spherical embedding sample set to determine the domain adversarial loss. Inter-class and intra-class contrastive training is performed based on a spherical embedding sample set to determine the contrastive loss; Robust training for hard samples is performed based on the spherical embedding sample set and the spherical embedding corresponding to the hard sample set, and the hard sample loss is determined. The total loss for multi-objective joint training is determined based on classification loss, domain adversarial loss, contrastive loss, and hard sample loss. Based on the total loss, the parameters of the shared encoder are optimized to obtain a trained shared encoder.

[0060] refer to Figure 3 , Figure 3 This is a schematic diagram of the code generation detection generalization framework provided by this invention. It is specifically divided into two parts: (a) a training phase and (b) a testing phase. In the training phase, the code to be detected, difficult samples generated by a large language model, and other codes within the batch are input into a shared encoder to generate a code representation. This representation is then mapped to a hyperspherical space through L2 normalization, forming a spherical code representation. Based on this, difficult sample learning and intra-class / inter-class contrastive learning are performed in parallel. Multi-objective joint training is then conducted via a spherical classifier and a spherical neighborhood classifier connected through a GRL (Gradient Reversal Layer). Finally, the total loss is calculated to optimize the encoder parameters. In the testing phase, the code to be detected, C0, is input into the trained encoder. Its output code representation, after L2 normalization, is input into the spherical classifier to determine whether the code was written by a human or generated by a machine.

[0061] This invention consists of four parts: a spherical classification backbone, spherical domain adversarial training, inter-class and intra-class comparative learning, and hard sample learning. For example... Figure 3 As shown, the training phase process is as follows.

[0062] The initial code samples to be detected are first rewritten as semantically equivalent "hard example" variants. Then, the original samples, their hard example samples, and other samples in the batch are encoded by a shared encoder and normalized to generate spherical embeddings in the hyperspherical space. On this basis, the features extracted by the encoder are optimized through multi-objective learning, including classification learning (identifying AI-generated code and human-written code), adversarial learning (generating domain-invariant embeddings), inter-class and intra-class comparison learning (improving discriminability), and hard example sample learning (ensuring semantic robustness).

[0063] During the testing phase, the initial code sample to be detected is directly input into the trained encoder. After obtaining the code representation, it is normalized and processed by a spherical classifier to obtain the final predicted label.

[0064] Through the embodiments of the present invention, a training code sample set with category labels and domain labels is obtained, a hard sample set is generated using a large language model, and after encoding by a shared encoder and normalization mapping to obtain a spherical embedding sample set, different losses are determined from multiple dimensions such as category classification, domain adversarial, inter-class and intra-class comparison, and hard sample robustness and are jointly optimized, finally obtaining a trained shared encoder, which effectively improves the encoder's ability to extract and process code features, and enhances the accuracy and robustness of the detection method.

[0065] According to the present invention, a method for detecting code generated by a large language model, based on a spherical embedding sample set, performs domain adversarial training and determines the domain adversarial loss, including: Construct an adversarial training framework consisting of a shared encoder, a spherical domain classifier, and a gradient inversion layer; Among them, the spherical domain classifier is built based on spherical logistic regression and is used to identify the domain labels of the spherical embeddings of the original samples; the gradient inversion layer does not process the spherical embeddings of the original samples during the forward propagation, and multiplies the gradient output by the spherical domain classifier by negative one before passing it to the shared encoder during the backward propagation. By inputting the spherical embedding set into the adversarial training framework, the probability that each original sample spherical embedding belongs to each domain is obtained from the output of the adversarial training framework. Based on the domain labels of probabilities and code snippets, the domain adversarial loss is determined, where the domain adversarial loss adopts the form of a cross-entropy loss function.

[0066] refer to Figure 2 (b) Spherical domain adversarial training includes: the code to be detected C0, a shared encoder, L2 normalization, GRL, a spherical domain classifier, and domain adversarial loss. .

[0067] Spherical domain confrontation training ( Figure 2 As shown in (b), based on the spherical classification backbone, this invention constructs an adversarial framework consisting of a shared encoder, a spherical domain classifier, and a gradient inversion layer (GRL). The spherical domain classifier based on spherical logistic regression is used to identify sample domain labels. In forward propagation, GRL is used for spherical embedding. No processing is performed. Instead, during backpropagation, the gradient output by the spherical discriminator is multiplied by negative one and fed into the encoder, forming an adversarial relationship between the encoder and the spherical classifier.

[0068] Spherical domain classifiers use domain adversarial loss The optimization aims to minimize the domain classification error, while the encoder maximizes the domain adversarial loss. This makes it difficult to identify the domain to which the extracted features belong.

[0069] ; in, Indicating resistance to loss, Indicates the number of samples in the batch. Indicates the index of the sample. Indicates the number of fields. Represents a domain index. This represents the domain label of the i-th sample. Represents probability. This represents the spherical embedding corresponding to the i-th sample.

[0070] Through the embodiments of the present invention, an adversarial training framework is constructed that includes a shared encoder, a spherical domain classifier based on spherical logistic regression, and a gradient inversion layer. By utilizing the special gradient processing of the gradient inversion layer during backpropagation, the probability of the sample spherical embedding belonging to each domain is obtained by inputting the spherical embedding set into the framework. Then, based on the probability and domain label, the domain adversarial loss is determined in the form of a cross-entropy loss function, which helps to improve the model's ability to detect codes of different domains.

[0071] According to the present invention, a method for detecting code generated by a large language model, based on a spherical embedding sample set, performs inter-class and intra-class contrastive training to determine the contrastive loss, including: In the training code sample set, code fragments with the same category label as the code fragment are defined as positive sample pairs, and a set of positive sample pairs is constructed. In the training code sample set, code fragments that have different category labels from the code fragments are defined as negative sample pairs, and a set of negative sample pairs is constructed. The similarity between the code fragment and positive and negative sample pairs is determined based on the inner product of the sphere. For each code snippet in the training code sample set, the contrastive loss is determined based on the similarity between the code snippet and samples in the positive sample pair set, and the similarity between the code snippet and samples in the negative sample pair set.

[0072] refer to Figure 2 (c) Inter-class-intra-class contrastive learning includes: the code to be detected and other code in the batch, shared encoder, L2 normalization, inter-class-intra-class contrastive learning, and contrastive loss. .

[0073] Inter-class and intra-class comparative learning Figure 2 As shown in (c): To address the problem of classification performance degradation caused by the loss of fine-grained category information due to adversarial training in spherical domains, this invention introduces inter-class and intra-class contrastive learning on the basis of contrastive learning to ensure that domain-invariant features have strong discriminative ability.

[0074] Unlike conventional contrastive learning, which constructs positive and negative samples through data augmentation, this invention constructs positive and negative samples based on in-batch data: for batch samples ( This represents the input code snippet. For category labels, (representation of field labels) and their spherical embedding Samples of the same category are defined as positive sample pairs. Outlier samples are defined as negative sample pairs. Similarity is calculated using the inner product of spheres. Furthermore, by optimizing the contrastive loss, the embeddings of samples of the same class are forced to cluster and the embeddings of samples of different classes are separated, thereby enhancing the discriminativeness of the learned embedding representations.

[0075] The contrast loss is expressed by the following formula: ; in, Indicates comparative loss, For temperature parameters, Indicates the number of samples in the batch. Indicates the index of the code sample. Indicates a positive sample pair. Indicates a positive sample. Represents the similarity function. This represents the i-th spherical embedding. Indicates negative sample pairs, This represents all samples.

[0076] By defining code segments with specific conditions (same category label and different domain label, different category label and different domain label) in the training code sample set as positive and negative sample pairs and constructing a set, the similarity between sample pairs is determined by the inner product of the sphere, and the contrast loss is determined based on the similarity between the code segment and the set of positive and negative sample pairs. This can effectively enhance the model's ability to distinguish between different categories and domain code features, and improve the accuracy and robustness of detection.

[0077] According to the present invention, a method for detecting code generated by a large language model, based on spherical embeddings corresponding to a spherical embedding sample set and a hard sample set, performs comparative training on hard samples to determine the hard sample loss, including: Obtain the code snippet corresponding to the original spherical embedding of the sample in the spherical embedding sample set; Determine the hard sample spherical embedding of the hard sample corresponding to the code snippet; The hard sample loss is determined based on the spherical similarity between the original sample spherical embedding and the hard sample spherical embedding, where: For code snippets with category labels generated by AI, the spherical distance between the original sample spherical embedding and the hard sample spherical embedding is reduced; For code snippets with category labels written by humans, push the spherical distance between the original sample spherical embedding and the hard sample spherical embedding.

[0078] refer to Figure 2 (d) in the equation includes the code to be detected, hard sample generation, hard samples, shared encoder, L2 normalization, hard sample learning, and hard sample loss. .

[0079] Hard sample learning ( Figure 2 As shown in (d): In order to better cope with the detection of code generation in complex and diverse real-world scenarios, this invention utilizes hard sample learning to enhance the robustness of code representation.

[0080] Specifically, firstly, an auxiliary large language model is used to analyze the code snippets. Perform semantically equivalent rewriting to generate "hard samples". And encode it as a hard sample spherical embedding. Then design the hard sample loss. For AI-generated code use To narrow the embedding distance between candidate code and rewritten code (i.e., hard samples), where Represents a similarity function; code written by humans use , By maximizing the embedding distance between candidate code and rewritten code, this loss function reduces the model's dependence on surface semantics.

[0081] In this embodiment of the invention, the formula for the total training loss is: ,in The weight parameters set for the experience ensure that the framework learns an intrinsic code embedding representation that is domain invariant, discriminative, and robust.

[0082] Through the embodiments of the present invention, by obtaining the code fragments corresponding to the spherical embeddings of the original samples and the spherical embeddings of the difficult samples in the spherical embedding sample set, the loss of the difficult samples is determined based on the spherical similarity between the two, and the strategy of bringing the spherical distance closer or further away is adopted for code fragments with different category labels, which can effectively enhance the robustness of the model to difficult samples (samples that are difficult to distinguish) and improve the model's ability to distinguish between AI-generated code and human-written code.

[0083] The following describes the detection method for the large language model generated code provided by this invention, with examples in practical application scenarios.

[0084] To simulate cross-domain code generation detection in real-world scenarios, this invention constructs a multi-dimensional code generation detection dataset. Specifically, this invention first collects human-written C++, Python, and Java code from GitHub and Codeforce platforms. It then uses the GPT-4o and CodeLLama models to regenerate this human-written code to obtain AI-generated code data. After data deduplication and cleaning, 25,843 human-written code entries and 28,915 AI-generated code entries are obtained. This data is divided into a training dataset, an intra-domain validation set, and an intra-domain test set in an 8:1:1 ratio. Next, for cross-domain code generation detection and evaluation, this invention also collects human-written Go, JavaScript, C#, and PHP code from GitHub, Codeforce, LeetCode, and GeeksforGeeks platforms. The collected human-written code is then regenerated using the Claude, DeepSeekCoder, and StarCoder models to obtain AI-generated code data. This data is then deduplicated and cleaned, resulting in 2,902 code samples. This data is used as the cross-domain generated code detection dataset.

[0085] This invention exhibits excellent discriminative ability and superior detection performance within the domain. Experimental results show that on the domain test set, the accuracy of this invention reaches 93.21%, with an F1-Score of 0.9350 and an AUC of 0.9816, demonstrating outstanding performance among all comparison methods and accurately distinguishing between AI-generated code and human-written code (Table 1).

[0086] This invention possesses strong cross-domain generalization ability and performs exceptionally well in unknown domain detection. On the out-of-domain test set, the accuracy of this invention is 83.43%, the F1-Score is 0.8492, and the AUC is 0.9260. Compared with other existing methods, the average unknown domain detection accuracy is improved by at least 6.83% (Table 1).

[0087] Referring to Table 1, which is a table showing the detection effect of code generation inside and outside the domain provided by the present invention.

[0088] Table 1

[0089] refer to Figure 4 , Figure 4 This is a diagram showing the code generation and detection effect under the attack scenario provided by the present invention.

[0090] This invention exhibits excellent robustness and remains stable even in adversarial scenarios. In attack experiments, this invention achieved an accuracy of 89.6% in detecting adversarially modified code generation, far exceeding other comparative methods. Furthermore, its performance degradation from standard to adversarial scenarios is only 3.1%, significantly lower than the attenuation of existing methods, effectively resisting semantic perturbations. Figure 4 ).

[0091] The detection system for large language model generated code provided by this invention will be described below. The detection system for large language model generated code described below can be referred to in correspondence with the detection method for large language model generated code described above.

[0092] refer to Figure 5 , Figure 5 This is a schematic diagram of the module of the detection system for generating code from a large language model provided by the present invention.

[0093] Module 501 is used to acquire the code to be detected; The encoding module 502 is used to input the code to be detected into the trained shared encoder to obtain the code feature vector. The shared encoder is obtained through multi-objective joint training, which is used to simultaneously optimize the classification loss, domain adversarial loss, contrastive loss and hard sample loss. The mapping module 503 is used to perform L2 normalization on the code feature vector and map it to the hyperspherical space to obtain a spherical embedding. The detection module 504 is used to input the spherical embedding into a spherical category classifier based on spherical logistic regression for classification processing, and to obtain the detection result of the code to be detected output by the spherical category classifier. The detection result includes: AI-generated and human-written. The decision boundary of the spherical category classifier is the intersection of the hyperplane and the hypersphere used for classification.

[0094] Specifically, the detection system for large language model generated code provided by the present invention can implement all the method steps implemented in the above-described method embodiment for detecting large language model generated code, and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0095] Figure 6 This is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as... Figure 6As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640. The processor 610, communications interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a detection method for code generated by a large language model. This method includes: acquiring the code to be detected; inputting the code to be detected into a trained shared encoder to obtain a code feature vector, wherein the shared encoder is obtained through multi-objective joint training, which simultaneously optimizes classification loss, domain adversarial loss, contrastive loss, and hard sample loss; performing L2 normalization on the code feature vector and mapping it to a hyperspherical space to obtain a spherical embedding; inputting the spherical embedding into a spherical classifier based on spherical logistic regression for classification processing to obtain the detection result of the code to be detected output by the spherical classifier, wherein the detection result includes: AI-generated and human-written code, and the decision boundary of the spherical classifier is the intersection of the hyperplane used for classification and the hypersphere.

[0096] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0097] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the detection method for code generated by the large language model provided by the above methods. The method includes: acquiring the code to be detected; inputting the code to be detected into a trained shared encoder to obtain a code feature vector, wherein the shared encoder is obtained through multi-objective joint training, which is used to simultaneously optimize classification loss, domain adversarial loss, contrastive loss, and hard sample loss; performing L2 normalization on the code feature vector and mapping it to a hyperspherical space to obtain a spherical embedding; inputting the spherical embedding into a spherical classifier based on spherical logistic regression for classification processing to obtain the detection result of the code to be detected output by the spherical classifier, wherein the detection result includes: AI-generated and human-written, and the decision boundary of the spherical classifier is the intersection of the hyperplane used for classification and the hypersphere.

[0098] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a method for detecting code generated by a large language model as provided by the above methods. The method includes: acquiring code to be detected; inputting the code to be detected into a trained shared encoder to obtain a code feature vector, wherein the shared encoder is obtained through multi-objective joint training, which is used to simultaneously optimize classification loss, domain adversarial loss, contrastive loss, and hard sample loss; performing L2 normalization on the code feature vector and mapping it to a hyperspherical space to obtain a spherical embedding; inputting the spherical embedding into a spherical classifier based on spherical logistic regression for classification processing to obtain the detection result of the code to be detected output by the spherical classifier, wherein the detection result includes: AI-generated and human-written, and the decision boundary of the spherical classifier is the intersection of the hyperplane used for classification and the hypersphere.

[0099] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0100] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting code generated by a large language model, characterized in that, include: Obtain the code to be detected; The code to be detected is input into the trained shared encoder to obtain the code feature vector. The shared encoder is obtained through multi-objective joint training, which is used to simultaneously optimize classification loss, domain adversarial loss, contrastive loss and hard sample loss. The code feature vectors are L2 normalized and mapped to a hypersphere space to obtain a spherical embedding; The spherical embedding is input into a spherical category classifier based on spherical logistic regression for classification processing, and the detection result of the code to be detected is obtained from the output of the spherical category classifier. The detection result includes AI generation and human writing. The decision boundary of the spherical category classifier is the intersection of the hyperplane and the hypersphere used for classification.

2. The method for detecting code generated by a large language model according to claim 1, characterized in that, The step of inputting the spherical embedding into a spherical category classifier based on spherical logistic regression for classification processing, and obtaining the detection result of the code to be detected output by the spherical category classifier, includes: The classification probability of the code to be detected is determined according to the following formula: ； in, Represents the classification probability. This indicates the category label of the code to be detected. Indicates the category label index. This represents the i-th spherical embedding. w represents the weight vector corresponding to category k. j This represents the weight vector corresponding to category j. Represents the transpose matrix. This represents the bias term corresponding to category k. This represents the bias term corresponding to category j.

3. The method for detecting code generated by a large language model according to claim 1, characterized in that, Before inputting the code to be detected into the trained shared encoder to obtain the code feature vector, the method further includes: Obtain a training code sample set, wherein the training code sample set includes multiple code snippets, each of which carries a category label and a domain label; The training code sample set is input into a large language model for rewriting to obtain a hard sample set output by the large language model, wherein the hard sample set includes semantic / functional equivalent variants of each code segment; Based on the training code sample set and the hard sample set, a code feature vector set is obtained by encoding through a shared encoder; The code feature vector set is normalized and mapped to a hyperspherical space to obtain a spherical embedding sample set; Based on the spherical embedded sample set, perform category classification training and determine the classification loss; Domain adversarial training is performed based on the spherical embedding sample set to determine the domain adversarial loss. Based on the spherical embedding sample set, perform inter-class and intra-class contrastive training to determine the contrastive loss; Based on the spherical embedding sample set and the spherical embeddings corresponding to the difficult sample set, perform comparative training on difficult samples to determine the difficult sample loss; Based on the classification loss, domain adversarial loss, contrastive loss, and hard sample loss, the total loss for multi-objective joint training is determined. Based on the total loss, the parameters of the shared encoder are optimized to obtain a trained shared encoder.

4. The method for detecting code generated by a large language model according to claim 3, characterized in that, The step of performing hard sample comparison training based on the spherical embedding sample set and the hard sample set, and determining the hard sample loss, includes: Obtain the code snippet corresponding to the original sample spherical embedding in the spherical embedding sample set; Determine the hard sample spherical embedding of the hard sample corresponding to the code fragment; The hard sample loss is determined based on the spherical similarity between the original sample spherical embedding and the hard sample spherical embedding, wherein: For code snippets whose category labels are generated by AI, the spherical distance between the original sample spherical embedding and the difficult sample spherical embedding is reduced; For the category label being a code snippet written by humans, the spherical distance between the original sample spherical embedding and the hard sample spherical embedding is increased.

5. The method for detecting code generated by a large language model according to claim 3, characterized in that, The process of performing domain adversarial training based on the spherical embedding sample set and determining the domain adversarial loss includes: Construct an adversarial training framework consisting of a shared encoder, a spherical domain classifier, and a gradient inversion layer; The spherical domain classifier is constructed based on spherical logistic regression and is used to identify the domain labels of the spherical embeddings of the original samples. The gradient inversion layer does not process the spherical embeddings of the original samples during forward propagation, but multiplies the gradient output by the spherical domain classifier by negative one and passes it to the shared encoder during backward propagation. The spherical embedding set is input into the adversarial training framework to obtain the probability that each original sample spherical embedding belongs to each domain, as output by the adversarial training framework. Based on the probability and the domain label of the code snippet, the domain adversarial loss is determined, wherein the domain adversarial loss adopts the form of a cross-entropy loss function.

6. The method for detecting code generated by a large language model according to claim 3, characterized in that, The step of performing inter-class and intra-class contrastive training based on the spherical embedding sample set to determine the contrastive loss includes: In the training code sample set, code fragments with the same category label as the code fragment are defined as positive sample pairs, and a set of positive sample pairs is constructed. In the training code sample set, code fragments that have different category labels from the code fragments are defined as negative sample pairs, and a set of negative sample pairs is constructed. The similarity between the code fragment and positive and negative sample pairs is determined based on the inner product of the sphere. For each code segment in the training code sample set, a contrastive loss is determined based on the similarity between the code segment and samples in the positive sample pair set, and the similarity between the code segment and samples in the negative sample pair set.

7. A detection system for code generated by a large language model, characterized in that, include: The acquisition module is used to acquire the code to be detected. The encoding module is used to input the code to be detected into the trained shared encoder to obtain the code feature vector. The shared encoder is obtained through multi-objective joint training, which is used to simultaneously optimize classification loss, domain adversarial loss, contrastive loss and hard sample loss. The mapping module is used to perform L2 normalization on the code feature vector and map it to the hyperspherical space to obtain a spherical embedding; The detection module is used to input the spherical embedding into a spherical category classifier based on spherical logistic regression for classification processing, and obtain the detection result of the code to be detected output by the spherical category classifier. The detection result includes AI generation and human writing. The decision boundary of the spherical category classifier is the intersection of the hyperplane and the hypersphere used for classification.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the detection method for large language model generated code as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the detection method for large language model generated code as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the detection method for large language model generated code as described in any one of claims 1 to 6.

Images