Detection of AI-generated computer code

Abstract

A system and method for detecting artificial intelligence (AI) generated computer code. Perturbation codes can be obtained by masking code lines from candidate codes (110). Missing codes can be generated from perturbation codes using an AI code generation model (120) to obtain machine-filled codes. The probability that candidate codes and machine-filled codes are AI-generated can be predicted using a surrogate model (130). By comparing this probability with a detection threshold, candidate codes can be identified as AI-generated (140), and a detection result can be obtained.

Description

[Technical Field]

[0001] Related application information This application claims priority to U.S. Provisional Application No. 63 / 521,191, filed on 15 June 2023, and U.S. Patent Application No. 18 / 731,845, filed on 3 June 2024, both of which are incorporated herein by reference.

[0002] This invention relates to computer code analysis, and more specifically, to the detection of artificial intelligence-generated computer code. [Background technology]

[0003] Description of related technologies Remarkable advances in large-scale pre-trained language models (LLMs) have brought machine-generated text closer to human-written text in terms of both fluency and diversity. LLMs are now being used not only to generate text, but also computer code. This has created a problem: it is difficult to distinguish between machine-generated and human-written computer code. [Overview of the Initiative]

[0004] According to one aspect of the present invention, a computer implementation method for detecting artificial intelligence (AI)-generated computer code is provided. This method includes: obtaining perturbation codes by masking code lines from candidate codes; obtaining machine-filled codes by generating missing codes from the perturbation codes using an AI code generation model; predicting the probability that the candidate codes and the machine-filled codes are AI-generated using a surrogate model; and identifying the candidate codes as AI-generated by comparing the probability with a detection threshold to obtain a detection result.

[0005] According to another aspect of the present invention, a system for detecting artificial intelligence (AI)-generated computer code is provided. The system comprises a memory and one or more processor devices that communicate with the memory, configured to obtain perturbation codes by masking code lines from candidate codes, obtain machine-filled codes by generating missing codes from the perturbation codes using an AI code generation model, predict the probability that the candidate codes and the machine-filled codes are AI-generated using a surrogate model, and identify the candidate codes as AI-generated by comparing the probability with a detection threshold to obtain a detection result.

[0006] According to yet another aspect of the present invention, a non-temporary computer program product is provided which includes a computer-readable storage medium containing program code for detecting artificial intelligence (AI) generated computer code, the program code, when executed on a computer, causes the computer to perform the following actions: obtain perturbation code by masking lines of code from candidate code; obtain machine-filled code by generating missing code from the perturbation code using an AI code generation model; predict the probability that the candidate code and the machine-filled code are AI-generated using a surrogate model; and identify the candidate code as AI-generated by comparing the probability with a detection threshold to obtain a detection result.

[0007] These features and advantages, as well as other features and advantages, will become apparent from the following detailed description of exemplary embodiments of the present invention, which should be read in conjunction with the accompanying drawings. [Brief explanation of the drawing]

[0008] This disclosure provides further details in the following description of preferred embodiments with reference to the following drawings.

[0009] [Figure 1] This flowchart provides a high-level overview of a method for detecting AI-generated computer code according to one embodiment of the present invention.

[0010] [Figure 2] This is a block diagram showing a computing system for detecting AI-generated computer code according to one embodiment of the present invention.

[0011] [Figure 3] This is a block diagram showing a software program for detecting AI-generated computer code according to one embodiment of the present invention.

[0012] [Figure 4] This block diagram shows a system integrating a practical application for detecting AI-generated computer code, according to one embodiment of the present invention.

[0013] [Figure 5] This is a block diagram illustrating an overview of a deep learning neural network, based on one embodiment of the present invention. [Modes for carrying out the invention]

[0014] According to embodiments of the present invention, a system and method for detecting artificial intelligence (AI)-generated computer code are provided.

[0015] In one embodiment, a candidate code can be identified as AI-generated by comparing its predicted probability with a detection threshold. A surrogate model can be used to predict the probability that both the candidate code and the machine-filled code are AI-generated. The machine-filled code can be obtained by generating missing codes from perturbation codes using an AI code generation model. The perturbation code can be obtained by masking code lines from the candidate code.

[0016] In one embodiment, the identified candidate code can be flagged as AI-generated to ensure transparency in the computer code generation process for the decision-making entity to execute an action. In one embodiment, this action is to protect a health management system that handles patients' vital data by patching the candidate code flagged for potential security risks.

[0017] Thanks to the remarkable progress of pre-trained large language models (LLMs), machine-generated computer code has come closer to code written by humans. The popularity of large language models has revolutionized natural language processing (NLP) tasks while also raising concerns about potential abuse for generating malicious or unethical code. Therefore, identifying the origin of code has become an urgent issue. On the other hand, research on detecting AI-generated code is still lagging compared to the development of LLMs. There is also little research in the literature on this pressing issue. As AI-generated code gradually approaches the human level, effectively detecting AI-generated code involves fundamental difficulties. For this reason, recently, there has been a discussion about whether AI-generated code can be detected. However, especially in the era of ChatGPT TM there is still a lack of practical tests regarding the detection of AI-generated code.

[0018] In the prior art, the detection of code generated from large language models such as GPT TM -4 has not been considered. Furthermore, in the prior art directed at training-based text detectors, code detection has not been successful due to specific statistical characteristics inherent in the code structure. To address these drawbacks, in this embodiment, a training-free detection of AI-generated code is proposed to mitigate the risks associated with its indiscriminate use.

[0019] Detecting AI-generated computer code is useful in various scenarios to ensure code quality, security, and maintainability. Some application examples are shown below.

[0020] Code Review: AI-generated code may not adhere to best practices, coding conventions, or quality standards. Detecting such code allows developers to ensure that human-written code meets the necessary standards before it is integrated into the main codebase.

[0021] Security Analysis: AI-generated code may contain vulnerabilities, whether intentional or not. Detecting these vulnerabilities helps identify potential security risks, such as injection attacks, buffer overflows, and other potentially exploitable vulnerabilities.

[0022] Plagiarism Detection: In academic or professional settings, it is crucial to identify instances of code that has been plagiarized or copied from existing sources. By distinguishing between human-written code and AI-generated code, educators and organizations can detect instances of plagiarism and ensure respect for intellectual property rights.

[0023] Debugging and Maintenance: AI-generated code can be difficult to debug and maintain because it lacks the logical consistency and intent found in human-written code. Identifying AI-generated sections allows developers to focus on those parts and address them appropriately during debugging and maintenance.

[0024] Automated Testing: AI-generated code can introduce errors and unpredictable behavior that might go undetected by traditional testing. Identifying such code enables a targeted testing strategy to ensure the accuracy and reliability of software under development.

[0025] Code Refactoring and Optimization: AI-generated code may contain redundancy, unnecessary complexity, and inefficient structure. By detecting these sections, developers can refactor and optimize the code to improve performance and maintainability.

[0026] Intellectual property protection: In scenarios where proprietary or confidential code needs to be protected, detecting AI-generated code can identify potential leaks or misuse of such code.

[0027] Next, referring in detail to drawings where the same number represents identical or similar elements, we first refer to Figure 1, a flowchart showing a high-level overview of a method for detecting AI-generated computer code according to one embodiment of the present invention.

[0028] In one embodiment, the candidate code 501 (shown in Figure 2) can be identified as AI-generated and the detection result 518 (shown in Figure 3) obtained by comparing the predicted candidate code probability 514 (shown in Figure 3) and the machine-filled code probability 515 (shown in Figure 3) with the detection threshold 517 (shown in Figure 3). The probability that candidate code 514 and machine-filled code 515 are AI-generated can be predicted using a surrogate model 511 (shown in Figure 3). The machine-filled code 505 (shown in Figure 3) can be obtained by generating a missing code 520 (shown in Figure 3) from a perturbation code 503 (shown in Figure 3) using an AI code generator model 504 (shown in Figure 4). The perturbation code 503 can be obtained by masking the code line from candidate code 501.

[0029] In block 110, the code line of candidate code 501 can be masked to obtain perturbation code 503.

[0030] In one embodiment, candidate code 501 can be a text stream written in a programming language for solving a given problem. TM Java TM , C, Swiff TM It can be written in various programming languages ​​such as Python. For example, candidate code 501 is Python TMIt can be the computer code described below, and can solve the following problem: "Chanek is given an array a with indices from 1 to n. Define f(a) as the number of indices where a_i = i. Elements can be selected and deleted from the current array, and the remaining elements can be concatenated. For example, if we delete the third element from the array [4, 2, 3, 1]." The candidate code 501 can be as follows.

Number

[0031] In one embodiment, the candidate code 501 can be input by an entity via a peripheral device 595 (shown in FIG. 2). In another embodiment, the candidate code 501 can be stored in and accessed from a memory 591 (shown in FIG. 2). In another embodiment, the candidate code 501 can be obtained via a network.

[0032] In one embodiment, a code line can be masked by replacing it with mask text 521. For example, the mask text 521 (shown in FIG. 3) can be "<MASK:1 (for example, the number of masks)>". In one embodiment, the number of code lines to be masked can be an element of a predefined set of numerical values m. For example, m ∈ {1, 2, 4, 8, 16, 32, 64, 80, 100}. In another embodiment, the number of code lines to be masked can be determined by a mask rate. For example, the mask rate r can be 15% of the total number of code lines of the candidate code 501.

[0033] In one embodiment, the perturbation code 503 can be a revised version of the candidate code 501 using the mask text 521. For example, using the above example, the perturbation code 503 can be as follows. [[ID=二十一]]

Number

[0034] In block 120, the missing codes from the perturbation code 503 can be generated by using the AI ​​code generator model 504 to obtain the machine filling code 505.

[0035] In one embodiment, the missing code 520 can be generated from the perturbation code 503 by deleting the mask text 521 within the perturbation code 503, performing fill-in-intermediate (FIM) using the AI ​​code generator model 504, and obtaining the machine-filled code 505. In another embodiment, the missing code 520 can be generated from the perturbation code 503 by directly inputting the perturbation code 503 into the AI ​​code generator model 504 to perform FIM and obtain the machine-filled code 505. FIM can be a code generation task that can be learned by the AI ​​code generator model 504 to fill in the missing code 520 within the perturbation code 503.

[0036] In one embodiment, the AI ​​code generator model 504 can be an autoregressive model such as Incoder-6B. The autoregressive model can be used because it can be pre-trained on the FIM task and can perform code completion without reducing its left-to-right generation ability. In another embodiment, the AI ​​code generator model 504 is a generatively pre-trained transformer (GPT). TM -4, GPT TM This can be -35-turbo), large-scale language model related (LLaMa-13B), or text-davinci-edit-001. Other AI code generator models are also available.

[0037] In one embodiment, the AI ​​code generator model 504 can be pre-trained using a human-evaluated generative model dataset (HumanEval), (HumanEval-X), or a handwritten code evaluation dataset such as the CodeContests dataset.

[0038] In block 130, the candidate code probability 514 and the machine-filled code probability 515 can be predicted using the surrogate model 511.

[0039] In one embodiment, a surrogate model 511 can be used to predict whether candidate code probabilities 514 and machine-filled code probabilities 515 are AI-generated.

[0040] In one embodiment, the surrogate model 511 can be an autoregressive model trained on FIM tasks such as the Python code generator model (PyCodeGPT-110M), "PolyCoder-160M", "CodeParrot-1.5B", and "LLaMa-13B".

[0041] In one embodiment, the surrogate model 511 can be pre-trained on a human-evaluated generative model dataset (HumanEval), (HumanEval-X), or a handwritten code evaluation dataset such as the CodeContests dataset.

[0042] In one embodiment, the candidate code probability 514 can be predicted by a surrogate model by predicting the probability of generating a remainder code 508 given a prefix code 507 obtained from the candidate code 501. In one embodiment, the prefix code 507 and the remainder code 508 can be obtained by splitting the candidate code 501 by a split ratio 522 (shown in Figure 3). In one embodiment, the split ratio 522 can be 90%. In another embodiment, the split ratio 522 can be 50%.

[0043] In one embodiment, the surrogate model 511 can predict the candidate code probability 514 by predicting the probability of generating a remainder code 508 given a prefix code obtained from the candidate code 501 by calculating the rightmost token logit.

[0044] In one embodiment, given a prefix filling code 509 obtained from a machine filling code 505, the surrogate model 511 can predict the machine filling code probability 515 by predicting the probability of generating a surplus filling code 510. In one embodiment, the prefix filling code 509 and the surplus filling code 510 can be obtained by dividing the machine filling code with a division ratio 522. In one embodiment, the division ratio 522 can be 90%. In another embodiment, the division ratio can be 50%.

[0045] In one embodiment, the surrogate model 511 can predict the machine-filled code probability 515 by predicting the probability of generating a surplus-filled code 510 given a prefix-filled code 509 obtained from the machine-filled code 505 by calculating the token logit at the right end.

[0046] In another embodiment, an n-gram 519 of candidate code 501 and machine-filled code can be obtained. An n-gram is an sequence of n adjacent symbols or words in a specific order.

[0047] In block 140, by comparing the detection threshold 517 with the probability, it can be determined that candidate code 501 was generated by AI, and the detection result 518 can be obtained.

[0048] In one embodiment, the detection threshold 517 can be a predetermined ratio that can range from 0 to 1. In one embodiment, the detection threshold 517 can be 0.9. In another embodiment, the detection threshold 517 can be 0.95. The detection threshold 517 can be obtained by maintaining the true positive rate (TPR) while minimizing the false positive rate (FPR).

[0049] In one embodiment, the probability between the machine-filled code 515 and the candidate code 514 is compared with a detection threshold 517 to obtain the following detection result 518.

[0050]

Number

[0051] Here, p(Y0|X) can be the candidate code probability 514, which is the probability of generating the remainder code 508 (Y0) based on the prefix code 507 (X). p(Y k |X) is the machine filling code probability 515, which is the probability of obtaining the remainder filling code 510 (Y n ) based on the corresponding prefix filling code 509 (X n ). N can be the number of machine filling codes, n can be an element of N, and T can be the detection threshold 517.

[0052] In another embodiment, the n-gram divergence 519 between the candidate code 501 and the machine filling code 505 can be compared with the detection threshold 517 to identify that the candidate code 501 is AI-generated, and the detection result 518 can be obtained. To calculate the n-gram divergence 519 between the candidate code 501 and the machine filling code 505, it can be obtained as follows.

[0053]

Number

[0054] Here, grams(Y,n) can represent the set of all array n-grams 504 within the array Y, where Y can include the array of the remainder code Y from the candidate code 501 of the n-gram of the candidate code 501 and the array Y o of the remainder filling code 510 of the n-gram of the machine filling code 505. k can be an element of the sample size K, N can be the number of arrays, f(n) can be a weight function empirically selected for various lengths n, and |Y k | can be the normalized length of the array Yk used to normalize grams(Y0,n), and T can be the detection threshold 517. In one embodiment, f(n) can be nlog(n), and n​​o = 4, and N = 25.

[0055] In another embodiment, the model output probability 513 between candidate code 501 and machine filling code 505 is obtained from the surrogate model 511 and compared with the detection threshold 517 to obtain the detection result 518.

[0056]

number

[0057] Here, p(Y o |X) can be the model output probability of the remainder code array Yo and prefix code X, and p(Y k |X) is the remainder-filled code array Y k The model output probability can be set to the prefix-filling code X, k can be a number within the sample size K, N can be the number of elements in the sequence, and T can be the detection threshold 517.

[0058] Figure 2 is a block diagram showing a computing system for detecting AI-generated computer code 500 according to one embodiment of the present invention.

[0059] The arithmetic unit 500 includes, exemplary, a processor device 594, an input / output (I / O) subsystem 590, memory 591, a data storage device 592, and a communication subsystem 593, and / or other components and devices commonly found in a server or similar arithmetic unit. In other embodiments, the arithmetic unit 500 may include other or additional components, such as those commonly found in a server computer (e.g., various input / output devices). Furthermore, in some embodiments, one or more of the exemplary components may be incorporated into or form part of other components. For example, in some embodiments, memory 591 or a part thereof may be incorporated into the processor device 594.

[0060] The processor device 594 can be implemented as any type of processor capable of performing the functions described herein. The processor device 594 can be implemented as a single processor, multiple processors, a central processing unit (CPU), a graphics processing unit (GPU), a single-core processor or a multi-core processor, a digital signal processor, a microcontroller, or any other processor or processing / control circuit.

[0061] Memory 591 can be implemented as any type of volatile or non-volatile memory, or data storage device, capable of performing the functions described herein. When in operation, memory 591 can store various data and software (such as operating systems, applications, programs, libraries, and drivers) used during the operation of the arithmetic unit 500. Memory 591 is communicatively connected to the processor device 594 via the I / O subsystem 590, which can be implemented as circuits and / or components that facilitate input / output operations with the processor device 594, memory 591, and other components of the arithmetic unit 500. For example, the I / O subsystem 590 can be implemented as, or otherwise include, a memory control hub, an input / output control hub, a platform control hub, an integrated control circuit, a firmware device, communication links (such as point-to-point links, bus links, wires, cables, optical guides, and traces on printed circuit boards), and / or other components and subsystems that facilitate input / output operations. In some embodiments, the I / O subsystem 590 can form part of a system-on-a-chip (SOC) and be integrated onto a single integrated circuit chip together with other components of the processor device 594, memory 591, and arithmetic unit 500.

[0062] The data storage device 592 can be implemented as any type of device(s) configured for short-term or long-term data storage, such as a memory device or circuit, a memory card, a hard disk drive, a solid-state drive, or other data storage device. The data storage device 592 can store program code for detecting the AI-generated computer code 100. Some or all of these program code blocks may be included in a given arithmetic system.

[0063] The communication subsystem 593 of the computing unit 500 can be implemented as any network interface controller, or other communication circuits, devices, or sets thereof, that enable communication between the computing unit 500 and other remote devices over a network. The communication subsystem 593 can be configured to employ any one or more communication technologies (e.g., wired or wireless communication) and associated protocols (e.g., Ethernet, InfiniBand®, Bluetooth®, Wi-Fi®, WiMAX, etc.) to enable such communication.

[0064] As shown in the figure, the arithmetic unit 500 may also include one or more peripheral devices 592. Peripheral devices 592 may include any number of additional input / output devices, interface devices, and / or other peripheral devices. For example, in some embodiments, peripheral devices 592 may include a display, a touchscreen, graphics circuitry, a keyboard, a mouse, a speaker system, a microphone, a network interface, and / or other input / output devices, interface devices, a GPS, a camera, and / or other peripheral devices.

[0065] Of course, the arithmetic unit 500 may include other elements (not shown) that are readily conceivable to those skilled in the art, and certain elements may be omitted. For example, the arithmetic unit 500 may include various other sensors, input devices, and / or output devices depending on its specific implementation, as will be readily understood to those skilled in the art. For example, various types of wireless and / or wired input and / or output devices may be used. Furthermore, additional processors, controllers, memories, etc., in various configurations may also be used. These and other variations of the processing system 500 are readily conceivable to those skilled in the art based on the teachings of the present invention as described herein.

[0066] Next, referring to Figure 3, this is a block diagram showing a software program for detecting AI-generated computer code 300 according to one embodiment of the present invention.

[0067] In one embodiment, the candidate code 501 can be masked using the masking module 502 to obtain the perturbation code 503. The perturbation code 503 with missing codes can be generated using the AI ​​code generator model 504 to obtain the machine-filled code 505. The candidate code 501 can be split into a prefix code 507 and a remainder code 508 using the splitting module 506. The machine-filled code 505 can be split into a prefix-filled code 509 and a remainder code 510. The candidate code probability 514 can be predicted using the prefix code 507 and the remainder code 508 and the prediction module 512. The machine-filled code probability 515 can be predicted using the prefix-filled code 509 and the remainder code 510 and the prediction module 512. The prediction module can utilize the surrogate model 511. The model output probability 513 can be obtained using the surrogate model 511. The detection result 518 can be obtained by comparing the candidate code probability 514 and the machine-filled code probability 515 with the detection threshold 517.

[0068] In another embodiment, the detection result 518 can be obtained by comparing the N-gram divergence 519 between the remainder code 508 and the remainder filling code 510 with the detection threshold 517. In yet another embodiment, the detection result 518 can be obtained by comparing the model output probability 513 of the prefix code 507 and the remainder code 508 with the model output probability 513 of the corresponding prefix filling code 509 and the remainder filling code 510 with the detection threshold 517.

[0069] Next, referring to Figure 4, this is a block diagram showing a system integrating a practical application for detecting AI-generated computer code 600, according to one embodiment of the present invention.

[0070] In one embodiment, entity 601 can obtain candidate code 501 from the health management system 602, which may contain patient vital data 603. AI-generated malicious code 611 could be introduced into the health management system 602, potentially causing security risks such as data breaches and exposing patient vital data 603 to malicious actors. Candidate code 501 can be input into a computer system 605 equipped with the capability to detect AI-generated computer code 100, and a flagged candidate code 608 can be obtained. The flagged candidate code 608 can be presented to a decision-making entity 609 to perform action 610. Action 610 can patch the health management system 602 to ensure data breaches and remove the flagged candidate code 608 as malicious code. In another embodiment, the decision-making entity 609 can update the configuration of the health management system 602 and autonomously perform action 610.

[0071] In another embodiment, AI-generated malicious code 611 can be introduced into the health management system 602 to generate false patient vital data. Action 610 may involve patching the health management system 602 to remove the false patient vital data and remove candidate code 608 which has been flagged as malicious code. In another embodiment, the decision-making entity 609 may update the configuration of the health management system 602 to autonomously perform action 610.

[0072] In another embodiment, AI-generated malicious code 611 can be introduced into the health management system 602 to generate a false or inaccurate medical diagnosis based on the patient's vital data 603. Action 610 can patch the health management system 602 to remove the false or inaccurate medical diagnosis and remove candidate code 608 that has been flagged as malicious code. In another embodiment, the decision-making entity 609 can update the configuration of the health management system 602 and autonomously perform action 610.

[0073] In another embodiment, AI-generated malicious code 611 can be introduced into the health management system 602, which can generate inappropriate information such as hate speech, obscene language, defamatory language, threats, and blackmail based on the patient's vital data 603. Action 610 can patch the health management system 602, remove the inappropriate information such as hate speech, obscene language, defamatory language, threats, and blackmail, and remove candidate code 608 that has been flagged as malicious code. In another embodiment, the decision-making entity 609 can update the configuration of the health management system 602 and autonomously execute action 610.

[0074] In another embodiment, the health management system 602 may be a different computer system not limited to health, such as a corporate system, a public system, or an educational institution system. Other computer systems are also possible.

[0075] In the following embodiments, system 602 can be a corporate system, a public data system, an educational institution system, and so on.

[0076] In another embodiment, action 610 may be performed by a decision entity 609 that determines that the flagged candidate code 608 conforms to best practices, coding conventions, or quality standards. The system 602 can then integrate the approved candidate code into the main codebase.

[0077] In another embodiment, action 610 may be a decision entity 609 that labels the flagged candidate code 608 as plagiarism. System 602 can provide the decision entity and the creator of the candidate code 501 with evidence of plagiarism.

[0078] In another embodiment, action 610 may be a decision entity 609 that checks candidate code 608 flagged for debugging and maintenance. The system 602 can then create a code flag that can contain candidate code 608 flagged for debugging.

[0079] In another embodiment, action 610 can create a code hook containing flagged candidate code 608 that flags the behavior of the code and targets a test strategy to ensure the correctness and reliability of the software under development.

[0080] In another embodiment, action 610 may label flagged candidate code 608 as containing redundancy, unnecessary complexity, or inefficient structure when refactoring and optimizing the code to improve performance and maintainability.

[0081] In another embodiment, action 610 may alert the original author of the flagged candidate code 608 that such code has been misused as their own code.

[0082] Next, referring to Figure 5, this is a block diagram illustrating an overview of a deep learning neural network according to one embodiment of the present invention.

[0083] A neural network is a general-purpose system that improves its function and accuracy by being exposed to additional empirical data. Neural networks are trained by exposure to empirical data. During training, the neural network remembers and adjusts multiple weights applied to the input empirical data. By applying these adjusted weights to the data, it can identify whether the data belongs to a specific predefined class from a group of classes, or output the probability that the input data belongs to each class.

[0084] The empirical data obtained from a series of examples (also known as training data) can be formatted as a string of values ​​and fed into a neural network. Each example can be associated with a known result or output. Each example can be represented as a pair (x,y), where x is the input data and y is the known output. The input data can contain a variety of different data types and can contain multiple different values. The network can have one input node for each value that makes up the example's input data, and individual weights can be applied to each input value. The input data can be formatted as a vector, array, or string, for example, depending on the architecture of the neural network being built and trained.

[0085] A neural network "learns" by comparing the neural network output generated from input data with known values ​​from examples and adjusting stored weights to minimize the difference between the output and the known values. This adjustment of stored weights can be performed using backpropagation, where the effect of the weights on the output is determined by calculating a mathematical gradient and adjusting the weights to bring the output closer to the minimum difference. This optimization is called gradient descent and is a non-restrictive example of how training can be performed. The accuracy of the neural network can be tested and validated using a subset of examples with known values ​​that were not used in training.

[0086] During operation, the trained neural network can be used on new data that has not been previously used for training or validation by generalization. The weights of the tuned neural network can be applied to the new data, where the weights estimate a function constructed from the training examples. The parameters of the estimated function captured by the weights are based on statistical inference.

[0087] A deep neural network 1000, such as a multilayer perceptron, may have an input layer 911 with source nodes 912, one or more computation layers 926 with one or more computation nodes 932, and an output layer 940 with one output node 942 for each category into which the input example may be classified. The input layer 911 may have a number of source nodes 912 equal to the number of data values ​​912 in the input data 911. The computation nodes 932 in the computation layer 926 are located between the source nodes 912 and the output nodes 942 and are not directly observed, and are therefore also called hidden layers. Each node 932, 942 in the computation layer generates a linear combination of weighted values ​​from the output values ​​of the nodes in the previous layer and applies a nonlinear activation function that is differentiable over the range of the linear combination. The weights applied to the values ​​from each node in the previous layer are, for example, w1, w2, ...w n-1 ,w nIt can be represented as follows: The output layer provides the network's overall response to the input data. Deep neural networks can be fully connected, where each node in the computational layer is connected to all other nodes in the previous layer, or the connection configuration between layers can be other configurations. If there are missing links between nodes, the network is called partially connected.

[0088] In one embodiment, the computation layer 926 of the AI ​​code generator model 504 can generate a sequence of codes based on a candidate code 501 and the context and syntax of the candidate code 501. The output layer 940 of the AI ​​code generator model 504 can then provide the network's overall response to the candidate text 501 as the generated missing code 520. In another embodiment, the computation layer 926 of the surrogate model 511 can generate probability weights for the sequence of generated missing codes 520 based on the candidate code 501. The output layer 940 of the surrogate model 511 can then provide the network's overall response to the candidate code 501 as the generated missing code 502 and the model output probability 513.

[0089] Training a deep neural network may involve two phases: a forward phase in which the weights of each node are fixed and the input is propagated through the network, and a backward phase in which error values ​​are backpropagated through the network and the weight values ​​are updated.

[0090] One or more computational (hidden) layers 926 computation nodes 932 perform a nonlinear transformation on the input data 912 that generates the feature space. Classes or categories can be separated more easily in the feature space than in the original data space.

[0091] The embodiments described herein may be entirely hardware, entirely software, or may include both hardware and software elements. In preferred embodiments, the present invention is implemented in software including, but not limited to, firmware, resident software, and microcode.

[0092] Embodiments may include computer program products accessible from computer-usable or computer-readable media that provide program code for use by or in connection with a computer or any instruction execution system. Computer-usable or computer-readable media may include any device that stores, communicates, propagates, or transfers programs for use by or in connection with an instruction execution system, apparatus, or device. The medium may be magnetic, optical, electronic, electromagnetic, infrared, or semiconductor systems (or apparatus or devices), or propagation media. The medium may include computer-readable storage media such as semiconductor or solid-state memory, magnetic tape, removable computer diskettes, random-access memory (RAM), read-only memory (ROM), rigid magnetic disks, and optical disks.

[0093] Each computer program can be substantially stored in a machine-readable storage medium or device (e.g., program memory or magnetic disk) that is readable by a general-purpose or special-purpose programmable computer, in order to configure and control the operation of the computer when the storage medium or device is read by the computer in order to perform the procedures described herein. The system of the present invention can also be considered to be implemented on a computer-readable storage medium configured with a computer program, in which case the configured storage medium causes the computer to operate in a specific predetermined manner to perform the functions described herein.

[0094] A data processing system suitable for storing and / or executing program code may include at least one processor directly or indirectly coupled to a memory element via a system bus. The memory element may include local memory, bulk storage, and cache memory that provides at least some temporary storage for the program code to reduce the number of times the code is retrieved from bulk storage during execution. Input / output or I / O devices (including, but not limited to, keyboards, displays, pointing devices, etc.) may be coupled to the system directly or via an intermediary I / O controller.

[0095] Network adapters can also be integrated into a system to enable a data processing system to connect to other data processing systems or remote printers or storage devices via an intervening private or public network. Modems, cable modems, and Ethernet cards are just a few of the types of network adapters currently available.

[0096] As used herein, the terms “hardware processor subsystem” or “hardware processor” can refer to a processor, memory, software, or combination thereof that works together to perform one or more specific tasks. In useful embodiments, a hardware processor subsystem may include one or more data processing elements (e.g., logic circuits, processing circuits, instruction execution devices, etc.). The one or more data processing elements may be contained in a central processing unit, a graphics processing unit, and / or a separate processor- or arithmetic element-based controller (e.g., logic gates, etc.). A hardware processor subsystem may include one or more onboard memories (e.g., caches, dedicated memory arrays, read-only memory, etc.). In some embodiments, a hardware processor subsystem may include one or more memories (e.g., ROM, RAM, Basic Input / Output System (BIOS), etc.) that may be onboard or offboard, or that may be dedicated for use by the hardware processor subsystem.

[0097] In some embodiments, a hardware processor subsystem may include and execute one or more software elements. These software elements may include an operating system and / or one or more applications and / or specific code to achieve a specified result.

[0098] In other embodiments, the hardware processor subsystem may include dedicated circuits that perform one or more electronic processing functions to achieve a specified result. Such circuits may include one or more application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and / or programmable logic arrays (PLAs).

[0099] These and other variations of the hardware processor subsystem are also intended in accordance with embodiments of the present invention.

[0100] In this specification, any reference to “one embodiment” or “one embodiment” of the present invention, and to other modifications, means that certain features, structures, properties, etc., described in relation to the embodiments are included in at least one embodiment of the present invention. Therefore, expressions such as “in one embodiment” or “in one embodiment” appearing elsewhere in this specification, and any other modifications, do not necessarily all refer to the same embodiment. However, it should be understood that, considering the teachings of the present invention provided herein, features of one or more embodiments can be combined.

[0101] For example, in the case of "A / B," the use of any of the following " / ," "and / or," or "at least one," such as "A and / or B" or "at least one of A and B," will be understood as intended to include the selection of only the first listed option (A), only the second listed option (B), or both options (A and B). As further examples, in the case of "A, B, and / or C" and "at least one of A, B, and C," such expressions are intended to include the selection of only the first listed option (A), only the second listed option (B), only the third listed option (C), only the first and second listed options (A and B), only the first and third listed options (A and C), only the second and third listed options (B and C), or all three options (A, B, and C). This can be extended as many times as there are listed items.

[0102] The foregoing is to be understood in all respects to be illustrative and not restrictive, and the scope of the invention disclosed herein is to be determined not from the detailed description but from the claims as interpreted in accordance with the full width permitted by patent law. The embodiments shown and described herein are merely illustrative of the invention, and those skilled in the art should understand that various modifications can be implemented without departing from the scope and spirit of the invention. Those skilled in the art can implement various other combinations of features without departing from the scope and spirit of the invention. Thus, while aspects of the invention have been described with the detail and specificity required by patent law, what is claimed and intended to be protected by the patent is as stated in the appended claims.

Claims

1. A computer implementation method for detecting artificial intelligence (AI) generated computer code, The process involves masking the code lines from the candidate code (110) to obtain the perturbation code, By using an AI code generation model, missing codes are generated from the perturbation codes (120), and machine filling codes are obtained. By using a surrogate model, the probability that the candidate code and the machine-filled code are AI-generated is predicted (130), A computer implementation method that includes identifying the candidate code as AI-generated by comparing the probability with a detection threshold (140) and obtaining a detection result.

2. The computer implementation method according to claim 1, further comprising flagging the candidate code as AI-generated and detecting malicious code for a decision-making entity to perform an action.

3. The computer implementation method according to claim 2, wherein the action is to protect a health management system that processes critical patient data by patching candidate code that has been flagged for potential security risks.

4. The computer implementation method according to claim 1, further comprising predicting the probability of a candidate code by predicting the probability of generating a remainder code given a prefix code.

5. The computer implementation method according to claim 4, further comprising identifying the candidate code by comparing the n-gram divergence between the prefix code and the remainder code with the n-gram divergence between the prefix-filled code and the remainder-filled code.

6. The computer implementation method according to claim 1, further comprising predicting the probability of a machine-filled code by predicting the probability of generating a surplus-filled code given a prefix-filled code.

7. The computer implementation method according to claim 1, further comprising identifying the candidate code by calculating the difference between the probability of the candidate code and the probability of the machine-filled code.

8. A system for detecting artificial intelligence (AI) generated computer code, Memory (592) and, The system comprises one or more processor devices (594) that communicate with the memory (592), and the processor devices are Mask the code lines from the candidate code (110) to obtain the perturbation code, By using an AI code generation model, missing codes are generated from the perturbation codes (120), and machine filling codes are obtained. By using a surrogate model, the probability that the candidate code and the machine filling code were AI-generated is predicted (130). A system configured to obtain a detection result by identifying the candidate code as AI-generated by comparing the probability with a detection threshold (140).

9. The system according to claim 8, further comprising the processor device flagging the candidate code as AI-generated and detecting malicious code for a decision-making entity to perform an action.

10. The system according to claim 9, wherein the processor device performs the action to protect a healthcare management system that processes critical patient data by patching candidate codes that have been flagged for potential security risks.

11. The system according to claim 8, wherein the prediction of probabilities by the processor device further includes predicting the probability of a candidate code by predicting the probability of generating a remainder code given a prefix code.

12. The system according to claim 11, wherein identifying the candidate code by the processor device further includes comparing the n-gram divergence between the prefix code and the remainder code with the n-gram divergence between the prefix-filled code and the remainder-filled code.

13. The system according to claim 8, wherein predicting the probability by the processor device further includes predicting the probability of a machine-filled code by predicting the probability of generating a surplus-filled code given a prefix-filled code.

14. The system according to claim 8, wherein identifying the candidate code by the processor device further comprises calculating the difference between the probability of the candidate code and the probability of the machine-filled code.

15. A non-temporary computer program product including a computer-readable storage medium containing program code for detecting artificial intelligence (AI) generated computer code, wherein the program code, when executed on a computer, The process involves masking the code lines from the candidate code (110) to obtain the perturbation code, By using an AI code generation model, missing codes are generated from the perturbation codes (120), and machine filling codes are obtained. By using a surrogate model, the probability that the candidate code and the machine-filled code are AI-generated is predicted (130), The detection result is obtained by comparing the aforementioned probability with a detection threshold to identify the candidate code as one generated by AI (140), A non-temporary computer program product to be executed by the aforementioned computer.

16. The non-temporary computer program product according to claim 15, further comprising flagging the candidate code as AI-generated and detecting malicious code for a decision-making entity to perform an action.

17. The non-temporary computer program product according to claim 16, wherein the action is to protect a healthcare management system that processes critical patient data by patching candidate code that has been flagged for potential security risks.

18. The non-temporary computer program product according to claim 15, further comprising predicting the probability of a candidate code by predicting the probability of generating a remainder code given a prefix code.

19. The non-temporary computer program product according to claim 15, further comprising predicting the probability of a machine-filled code by predicting the probability of generating a surplus-filled code given a prefix-filled code.

20. The non-temporary computer program product according to claim 18, wherein identifying the candidate code further comprises calculating the difference between the probability of the candidate code and the probability of the machine-filled code.

Images