Processing device, processing method, and processing program

Abstract

This processing device (10) comprises: a code generation instruction unit (11) that causes a large language model (LLM) to generate code; an evaluation unit (12) that evaluates the security vulnerabilities of the code generated by the LLM; and a prompt generation unit (13) that, on the basis of the evaluation result from the evaluation unit (12), adds advice for improving security to a prompt instructing the generation of the code.

Description

Processing device, processing method, and processing program

[0001] The present invention relates to a processing device, a processing method, and a processing program.

[0002] In recent years, large language models (LLMs) have made remarkable progress and are now being used in a variety of fields. In particular, code generation using LLMs has become a powerful use case, as evidenced by the popularity of GitHub Copilot (Non-Patent Document 1), and has significantly improved the productivity of engineers.

[0003] GitHub Copilot, [Retrieved April 1, 2024], Internet <URL: https: / / github.com / features / copilot> Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, Ramesh Karri, "Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions," IEEE Symposium on Security and Privacy 2022.

[0004] However, although the code generated by LLM has high functionality (it is a program that operates according to the user's instructions), it is known that much of the code is vulnerable.

[0005] The present invention has been made in view of the above, and has as its object to provide a processing device, a processing method, and a processing program that can improve the security of code generated by an LLM.

[0006] In order to solve the above-mentioned problems and achieve the object, the processing device of the present invention is characterized by having a generation instruction unit that causes a large-scale language model to generate code, an evaluation unit that evaluates security vulnerabilities of the code generated by the large-scale language model, and a generation unit that, based on the evaluation results by the evaluation unit, adds advice for improving security to a prompt that instructs the generation of code.

[0007] Furthermore, the processing method of the present invention is a processing method executed by a processing device, and is characterized by including the steps of: causing a large-scale language model to generate code; evaluating the security vulnerabilities of the code generated by the large-scale language model; and adding advice for improving security to a prompt that instructs code generation based on the evaluation results of the evaluating step.

[0008] In addition, the processing program of the present invention causes a computer to execute the steps of causing a large-scale language model to generate code, evaluating the security vulnerabilities of the code generated by the large-scale language model, and adding advice for improving security to a prompt that instructs code generation based on the evaluation results in the evaluating step.

[0009] According to the present invention, it is possible to improve the security of the code generated by the LLM.

[0010] FIG. 1 is a diagram illustrating processing content proposed in an embodiment. FIG. 2 is a block diagram illustrating an example of the configuration of a processing device according to an embodiment. FIG. 3 is a diagram illustrating the flow of processing according to an embodiment. FIG. 4 is a diagram illustrating an example of a method for processing information extracted from an external database (DB). FIG. 5 is a flowchart illustrating the processing procedure of a processing method according to an embodiment. FIG. 6 is a diagram illustrating experimental conditions for an evaluation experiment. FIG. 7 is a diagram illustrating experimental results of the evaluation experiment. FIG. 8 is a diagram illustrating experimental results of the evaluation experiment. FIG. 9 is a diagram illustrating experimental results of the evaluation experiment. FIG. 10 is a diagram illustrating experimental results of the evaluation experiment. FIG. 11 is a diagram illustrating an example of a computer in which a processing device is realized by executing a program.

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are designated by the same reference numerals.

[0012] [Embodiment] Fig. 1 is a diagram illustrating the processing content proposed in the embodiment. As shown in Fig. 1, in the embodiment, appropriate secure coding advice (see box W1) is automatically provided in response to a code generation instruction input to the LLM. In the embodiment, a prompt including the instruction and advice shown in Fig. 1 is set in the LLM to instruct code generation. This allows the LLM to generate code taking this advice into consideration, thereby improving the security of the code generated by the LLM.

[0013] [Processing Device] First, a processing device according to an embodiment will be described. Fig. 2 is a block diagram showing an example of the configuration of the processing device according to an embodiment.

[0014] As shown in FIG. 2, the processing device 10 according to the embodiment communicates with a generating server 20 and an external DB 30 .

[0015] The generation server 20 has an LLM 21. The LLM 21 is a large-scale language model that generates and outputs data in accordance with prompts set by the processing device 10. In this embodiment, a prompt including coding instructions is set in the LLM 21, and the LLM 21 generates and outputs code in accordance with the prompt. The generation server 20 outputs the code generated by the LLM 21 to the processing device 10.

[0016] The external DB 30 is a DB that stores vulnerability information. The external DB 30 holds, for example, information related to CWE (Common Weakness Enumerations). CWE is one of the standard specifications for classifying and identifying security vulnerabilities in software, hardware, and the like that make up IT (Information Technology) systems, and is often adopted as a common standard for vulnerability classification. The external DB 30 stores, for example, CWE descriptions (vulnerability information) acquired from the NVD (National Vulnerability Database). The NVD is a DB that collects information related to security vulnerabilities in software, IT products, and the like, and is managed and published by the National Institute of Standards and Technology (NIST).

[0017] The processing device 10 sets a prompt in the LLM 21 and instructs the LLM 21 to generate code. At this time, the processing device 10 has a function of automatically providing appropriate advice on secure coding in response to the code generation instruction input to the LLM 21.

[0018] [Processing Device] Next, the processing device 10 will be described. The processing device 10 is realized, for example, by loading a predetermined program into a computer or the like including a ROM (Read Only Memory), a RAM (Random Access Memory), a CPU (Central Processing Unit), etc., and having the CPU execute the predetermined program. The processing device 10 is also capable of communicating with other devices connected via a network or the like via a communication interface that transmits and receives various types of information to and from other devices.

[0019] The processing device 10 includes a code generation instruction unit 11 (instruction unit), an evaluation unit 12, and a prompt generation unit 13 (generation unit).

[0020] The code generation instruction unit 11 sets a prompt instructing the LLM 21 to generate code, and causes the LLM 21 to generate code.

[0021] The evaluation unit 12 receives the code generated by the LLM 21 and evaluates the security vulnerability of the code generated by the LLM 21. The evaluation unit 12 evaluates the security vulnerability of the code generated by the LLM 21 using a static analysis tool (for example, SAST (Static Application Security Testing)).

[0022] The prompt generation unit 13 adds advice for improving security to a prompt that instructs code generation based on the evaluation result by the evaluation unit 12. The prompt generation unit 13 constructs advice based on information output by the static analysis tool. The prompt generation unit 13 searches the external DB 30 based on the information output by the static analysis tool, and constructs advice based on the obtained vulnerability information.

[0023] [Processing Flow] Next, the processing flow according to the embodiment will be described with reference to Fig. 3.

[0024] As shown in FIG. 3, for example, the code generation instruction unit 11 uses the instruction given by the user of the processing device 10 as it is to instruct the LLM 21 to generate code ((1) in FIG. 3).

[0025] The evaluation unit 12 uses the SAST 12A to analyze the code ((2) in FIG. 3) generated by the LLM 21. If the SAST 12A identifies a problem with the code ((3) in FIG. 3), the prompt generation unit 13 searches the external DB 30 for advice on improving security in response to the problem.

[0026] For example, a case will be described in which the SAST 12A outputs a CWE ID as a problem. In this case, the processing device 10 performs processing such as acquiring vulnerability information from the NVD to prepare an external DB 30 that stores information related to each CWE ID. The processing device 10 extracts, from the external DB 30, countermeasures and points of caution corresponding to the CWE ID output from the SAST 12A as information for constructing advice ((4) in FIG. 3).

[0027] The prompt generator 13 appropriately processes the extracted information and adds it to the code generation instruction given by the user as advice to generate a prompt. The code generation instruction unit 11 sets this prompt in the LLM 21 and instructs the LLM 21 to generate code again ((5) in Figure 3), causing the LLM 21 to generate code with high security ((6) in Figure 3).

[0028] In this way, the processing device 10 first causes the LLM 21 to generate code based on the instruction, and then evaluates the code in the SAST 12 A. The processing device 10 then generates coding practices from the CWE detected by the SAST 12 A, adds the coding practices to the code generation instruction as advice, and causes the processing device 10 to generate code again.

[0029] 4 is a diagram showing an example of a method for processing information extracted from the external DB 30. As shown in Fig. 4, for example, the prompt generation unit 13 processes the vulnerability description corresponding to the CWE ID into secure coding advice (box W22) by having the LLM process the vulnerability description.

[0030] Specifically, the prompt generation unit 13 inputs the SAST results ((1), box W21 in Figure 4) into the LLM along with a prompt that instructs advice conversion. Then, the prompt generation unit 13 generates a prompt that instructs code generation, as exemplified in Figure 1, by adding the output from the LLM (box W22) as advice ((2) in Figure 4). In this way, the prompt generation unit 13 converts the description of the vulnerability corresponding to the CWE ID into coding advice (box W22) and adds the converted advice to the prompt.

[0031] [Processing Procedure] Next, the processing procedure of the processing method according to the embodiment will be described with reference to a flowchart of FIG.

[0032] In the processing device 10, the code generation instruction unit 11 sets a prompt instructing the LLM 21 to generate code, and instructs the LLM 21 to generate code (step S11).

[0033] Next, the evaluation unit 12 receives the code generated by the LLM 21 (step S12) and evaluates the security vulnerability of this code (step S13).

[0034] If the evaluation unit 12 points out a security vulnerability in the code (step S14: Yes), the prompt generation unit 13 searches for and obtains an explanation of the vulnerability from the external DB 30 in order to improve security in accordance with the vulnerability (step S15).

[0035] The prompt generator 13 constructs advice based on the information obtained from the external DB 30 and attaches it to a prompt that instructs the generation of code (step S16).

[0036] The code generation instruction unit 11 then sets the prompt with the advice in the LLM 21, causing the LLM 21 to generate code (step S17). The processing device 10 then receives the code generated by the LLM 21 (step S18). If the evaluation unit 12 does not identify any security vulnerabilities in the code (step S14: No), the code generated by the LLM 21 is deemed safe and is provided to the user.

[0037] [Evaluation Experiment] An evaluation experiment was carried out on the processing method of this embodiment.

[0038] Figure 6 shows the experimental conditions for the evaluation experiment. As shown in Figure 6, in the evaluation experiment, 316 vulnerable functions of Python programs were prepared, and natural language instruction prompts for generating these functions were prepared. In addition, for each function, the functions after the vulnerabilities were fixed and the results of SAST using semgrep were also prepared.

[0039] Using the above dataset (316 Python samples), GPT3.5 and GPT4 were used to generate code under two conditions: a first condition and a second condition. Under the first condition, the instruction prompt was set as is, and GPT3.5 and GPT4 were used to generate code. Under the second condition, the processing method according to the present embodiment was applied, and advice for improving security was added to the prompt, after which code was generated using GPT3.5 and GPT4.

[0040] The generated code is then subjected to SAST, and if no vulnerabilities are detected, it is deemed to have passed. The functionality of the generated code is measured (reference value) by its similarity to the correct function. In this manner, the rate at which the generated code passes SAST (the rate at which no vulnerabilities are identified) is measured under each of the first and second conditions. In this embodiment, the vulnerabilities detected by SAST are CWE IDs, and the external DB 30 stores CWE descriptions for each CWE ID obtained from the NVD. Furthermore, the advice for improving security provided in the prompt is converted into advice using GPT3.5 based on the CWE description (i.e., the method shown in FIG. 4).

[0041] Figures 7 to 10 show the experimental results of the evaluation experiment. Figure 7 shows the evaluation results when GPT3.5 was used to generate code under the first condition. Figure 8 shows the evaluation results when GPT3.5 was used to generate code under the second condition. Figure 9 shows the evaluation results when GPT4 was used to generate code under the first condition. Figure 10 shows the evaluation results when GPT4 was used to generate code under the second condition.

[0042] As shown in Figures 7 and 8, for GPT3.5, the rate of passing the SAST tool was 55% under the first condition, but improved to 78% under the second condition. Also, as shown in Figures 9 and 10, for GPT4, the rate of passing the SAST tool was 55% under the first condition, but improved to 85% under the second condition.

[0043] As described above, in both cases where GPT3.5 and GPT4 were used, applying the second condition (the processing method according to the present embodiment) significantly improved the rate of passing the SAST tool. Furthermore, the results showed that, between GPT3.5 and GPT4, GPT4, which has higher accuracy, is more secure.

[0044] [Effects of the embodiment] In the embodiment, a flow is constructed in which appropriate advice on secure coding is automatically given based on the evaluation results for the code generation instructions input to the LLM 21. As can be seen from the results of the evaluation experiment, this embodiment can improve the security of the code generated by the LLM 21.

[0045] In the embodiment, the processing device 10 automatically provides appropriate advice on secure coding to the prompt, so that the user can have the LLM 21 execute coding that takes security into consideration simply by inputting coding instructions to the LLM 21 as before.

[0046] In the embodiment, the processing device 10 uses a static analysis tool to evaluate security vulnerabilities in the code generated by the large-scale language model, but the present invention is not limited to this. In the processing device 10, the evaluation unit 12 may use a dynamic analysis tool to evaluate security vulnerabilities in the code generated by the LLM 21, and the prompt generation unit 13 may construct advice based on information output by the dynamic analysis tool.

[0047] [System Configuration of the Embodiment] The processing device 10 is a functional concept and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of the functions of the processing device 10 is not limited to that shown in the figure, and all or part of it can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc.

[0048] Furthermore, all or any part of the processes performed by the processing device 10 may be realized by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a program analyzed and executed by the CPU and the GPU (Graphics Processing Unit). Furthermore, each process performed by the processing device 10 may be realized as hardware using wired logic.

[0049] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually. Alternatively, all or part of the processes described as being performed manually can be performed automatically using a known method. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters described above and illustrated can be changed as appropriate unless otherwise specified.

[0050] 11 is a diagram showing an example of a computer in which a program is executed to realize the processing device 10. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0051] The memory 1010 includes a ROM 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0052] The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. That is, the program that defines each process of the processing device 10 is implemented as a program module 1093 in which code that can be executed by the computer 1000 is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing processes similar to those of the functional configuration of the processing device 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced with an SSD (Solid State Drive).

[0053] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in memory 1010 or hard disk drive 1090. Then, CPU 1020 reads out program module 1093 or program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as necessary and executes them.

[0054] The program module 1093 and program data 1094 may not necessarily be stored in the hard disk drive 1090, but may also be stored in a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0055] Although the present invention has been described above as an embodiment, the present invention is not limited to the descriptions and drawings that form part of the disclosure of the present invention. In other words, other embodiments, examples, and operational techniques that can be made by those skilled in the art based on the present invention are all included in the scope of the present invention.

[0056] REFERENCE SIGNS LIST 10 Processing device 11 Code generation instruction unit 12 Evaluation unit 12A SAST 13 Prompt generation unit 20 Generation server 21 LLM 30 External DB 1000 Computer

Claims

1. A processing device comprising: a generation instruction unit that causes a large-scale language model to generate code; an evaluation unit that evaluates security vulnerabilities in the code generated by the large-scale language model; and an attachment unit that adds advice for improving security to a prompt that instructs the generation of code based on the evaluation results by the evaluation unit.

2. The processing device described in claim 1, characterized in that the evaluation unit uses a static analysis tool to evaluate security vulnerabilities in the code generated by the large-scale language model, and the assignment unit constructs the advice based on information output by the static analysis tool.

3. The processing device according to claim 2, characterized in that the assignment unit searches a database storing vulnerability information based on the information output by the static analysis tool, and constructs the advice based on the obtained vulnerability information.

4. The processing device described in claim 1, characterized in that the evaluation unit uses a dynamic analysis tool to evaluate security vulnerabilities in the code generated by the large-scale language model, and the assignment unit constructs the advice based on information output by the dynamic analysis tool.

5. A processing method executed by a processing device, comprising: a step of causing a large-scale language model to generate code; a step of evaluating the security vulnerabilities of the code generated by the large-scale language model; and a step of adding advice for improving security to a prompt that instructs the generation of code based on the evaluation results of the evaluation step.

6. A processing program for causing a computer to execute the steps of: causing a large-scale language model to generate code; evaluating the security vulnerabilities of the code generated by the large-scale language model; and adding advice for improving security to a prompt that instructs code generation based on the evaluation results in the evaluating step.

Images