Processing device, processing method, and processing program

Abstract

This processing device (10) comprises: an advice generation unit (11) that generates advice for improving security on the basis of vulnerability information of software code when a large language model (LLM) (21) creates the code in response to a code generation instruction, and stores the advice in an advice database (DB) group (30); a code generation instruction unit (12) that receives the code generation instruction for the LLM (21); and a prompt generation unit (13) that searches the advice DB group (30) for advice for improving security, and adds the found advice to a prompt instructing that the code be generated.

Description

Processing device, processing method, and processing program

[0001] The present invention relates to a processing device, a processing method, and a processing program.

[0002] In recent years, large language models (LLMs) have made remarkable progress and are now being used in a variety of fields. In particular, code generation using LLMs has become a powerful use case, as evidenced by the popularity of GitHub Copilot (Non-Patent Document 1), and has significantly improved the productivity of engineers.

[0003] GitHub Copilot, [Retrieved April 19, 2024], Internet <URL: https: / / github.com / features / copilot> Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, Ramesh Karri, "Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions," IEEE Symposium on Security and Privacy 2022.

[0004] However, although the code generated by LLM has high functionality (it is a program that operates according to the user's instructions), it is known that much of the code is vulnerable.

[0005] The present invention has been made in view of the above, and has an object to provide a processing device, a processing method, and a processing program that can improve the security of code generated by an LLM.

[0006] In order to solve the above-mentioned problems and achieve the object, the processing device of the present invention is characterized by having a generation unit that generates advice for improving security when a large-scale language model creates code in response to a code generation instruction based on vulnerability information of software code and stores the advice in a database, an instruction unit that accepts code generation instructions for the large-scale language model, and an assignment unit that searches the database for advice for improving security and assigns the searched advice to a prompt that instructs code generation.

[0007] In addition, the processing method of the present invention is a processing method executed by a processing device, and is characterized by including the steps of generating advice for improving security when a large-scale language model creates code in response to a code generation instruction based on vulnerability information of software code and storing the advice in a database, accepting a code generation instruction for the large-scale language model, and searching the database for advice for improving security and adding the searched advice to a prompt that instructs code generation.

[0008] In addition, the processing program of the present invention causes a computer to execute the following steps: generating advice for improving security when a large-scale language model creates code in response to a code generation instruction based on vulnerability information of software code, and storing the advice in a database; accepting a code generation instruction for the large-scale language model; and searching the database for advice for improving security and adding the searched advice to a prompt that instructs code generation.

[0009] According to the present invention, it is possible to improve the security of the code generated by the LLM.

[0010] FIG. 1 is a diagram illustrating processing content proposed in an embodiment. FIG. 2 is a block diagram illustrating an example of the configuration of a processing device according to an embodiment. FIG. 3 is a diagram illustrating a processing flow according to an embodiment. FIG. 4 is a diagram illustrating an example of advice generation processing. FIG. 5 is a diagram illustrating an example of advice generation processing. FIG. 6-1 is a diagram illustrating an example of advice generation processing. FIG. 6-2 is a diagram illustrating an example of advice generation processing. FIG. 6-3 is a diagram illustrating an example of advice generation processing. FIG. 7 is a diagram illustrating an example of advice generation processing. FIG. 8 is a diagram illustrating an example of advice generation processing. FIG. 9 is a diagram illustrating an example of advice generation processing. FIG. 10 is a flowchart illustrating a processing procedure of a processing method according to an embodiment. FIG. 11 is a flowchart illustrating a processing procedure of a processing method according to an embodiment. FIG. 12 is a diagram illustrating experimental conditions for an evaluation experiment. FIG. 13 is a diagram illustrating experimental conditions for an evaluation experiment. FIG. 14-1 is a diagram illustrating experimental results of the evaluation experiment. FIG. 14-2 is a diagram illustrating experimental results of the evaluation experiment. FIG. 14-3 is a diagram illustrating experimental results of the evaluation experiment. FIG. 14-4 is a diagram illustrating experimental results of the evaluation experiment. FIG. 15-1 is a diagram showing the results of the evaluation experiment. FIG. 15-2 is a diagram showing the results of the evaluation experiment. FIG. 15-3 is a diagram showing the results of the evaluation experiment. FIG. 15-4 is a diagram showing the results of the evaluation experiment. FIG. 16-1 is a diagram showing the results of a qualitative experiment when a third advice database (DB) based on configuration information was used. FIG. 16-2 is a diagram showing the results of a qualitative experiment when a third advice DB based on configuration information was used. FIG. 17-1 is a diagram showing the results of a qualitative experiment when a third advice DB based on configuration information was used. FIG. 17-2 is a diagram showing the results of a qualitative experiment when a third advice DB based on configuration information was used. FIG. 18 is a diagram showing an example of a computer in which a processing device is implemented by executing a program.

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are designated by the same reference numerals.

[0012] [Embodiment] Fig. 1 is a diagram illustrating the processing content proposed in the embodiment. As shown in Fig. 1, in the embodiment, appropriate secure coding advice (see box W1) is automatically provided in response to a code generation (coding) instruction input to the LLM. In the embodiment, a prompt including the instruction and advice shown in Fig. 1 is set in the LLM to instruct code generation. This allows the LLM to generate code taking this advice into consideration, thereby improving the security of the code generated by the LLM.

[0013] [Processing Device] First, a processing device according to an embodiment will be described. Fig. 2 is a block diagram showing an example of the configuration of the processing device according to an embodiment.

[0014] 2, the processing device 10 according to the embodiment communicates with a generating server 20. The processing device 10 also communicates with an external server.

[0015] The generation server 20 has an LLM 21. The LLM 21 is a large-scale language model that generates and outputs data in accordance with a prompt set by the processing device 10. In this embodiment, a prompt including an instruction for code generation is set in the LLM 21, and the LLM 21 generates and outputs code in accordance with the prompt. The generation server 20 outputs the code generated by the LLM 21 to the processing device 10.

[0016] The processing device 10 sets a prompt in the LLM 21 and instructs the LLM 21 to generate code. At this time, the processing device 10 has a function of automatically providing appropriate advice on secure coding in response to the code generation instruction input to the LLM 21.

[0017] The processing device 10 uses various methods to construct a database for Retrieval-Augmented Generation (RAG) that stores advice for improving security. The processing device 10 then searches the database for advice corresponding to code generation instructions and adds the advice to the prompt, thereby directly generating a prompt for SecureCoding. Note that the processing device 10 does not use an analysis tool such as a SAST tool when generating the prompt.

[0018] [Processing Device] Next, the processing device 10 will be described. The processing device 10 is realized, for example, by loading a predetermined program into a computer or the like including a ROM (Read Only Memory), a RAM (Random Access Memory), a CPU (Central Processing Unit), etc., and having the CPU execute the predetermined program. The processing device 10 is also capable of communicating with other devices connected via a network or the like via a communication interface that transmits and receives various types of information to and from other devices.

[0019] The processing device 10 includes an advice DB group 30 (database), an advice generator 11 (generator), a code generation instruction unit 12 (instructor), and a prompt generator 13 (prompter). Fig. 3 is a diagram illustrating the flow of processing according to the embodiment.

[0020] The advice DB group 30 is a database for the RAG, and stores advice for improving security when the LLM 21 creates code in response to a code generation instruction. The advice for improving security is generated by the advice generator 11 based on vulnerability information of the software code. The advice DB group 30 is provided with a first advice DB 30-1, a second advice DB 30-2, and a third advice DB 30-3 for each vulnerability information source, depending on the source of the vulnerability information.

[0021] The first advice DB 30-1 stores advice whose information source is vulnerability information (e.g., descriptions of Common Weakness Enumerations (CWE)) collected from an external DB (e.g., the National Vulnerability Database (NVD)) that stores vulnerability information on software code. The NVD is a DB that collects information on security vulnerabilities in software, IT products, etc., and is managed and published by the National Institute of Standards and Technology (NIST).

[0022] The second advice DB30-2 stores advice based on information sources including code previously generated by the LLM and the results of an evaluation of the security vulnerabilities of the code previously generated by the LLM using an analysis tool (e.g., SAST (Static Application Security Testing)).

[0023] The third advice DB 30-3 stores advice whose information source is vulnerability information based on software configuration information (for example, SBOM (Software Bill of Materials)).

[0024] The advice generation unit 11 collects vulnerability information on various software code from various information sources ((1) in FIG. 3). Based on the vulnerability information on the software code, the advice generation unit 11 generates advice for improving security when the LLM 21 creates code in response to a code generation instruction ((2) in FIG. 3). The advice generation unit 11 stores the generated advice in the advice DB group 30 ((3) in FIG. 3).

[0025] The advice generation unit 11 stores the code generation instructions and advice in a first advice DB 30-1, a second advice DB 30-2, or a third advice DB 30-3, which is provided for each vulnerability information source. For example, the advice generation unit 11 vectorizes the code generation instructions and advice and stores them in the first advice DB 30-1, the second advice DB 30-2, or the third advice DB 30-3.

[0026] Next, the processing device 10 uses the advice stored in the advice DB group 30 to generate a prompt to be set in the LLM 21 .

[0027] First, the code generation instruction unit 12 receives an instruction to generate code ((4) in FIG. 3).

[0028] In response to a code generation instruction from a user, the prompt generation unit 13 generates a prompt that instructs code generation. At this time, the prompt generation unit 13 searches the advice DB group 30 for advice for improving security (e.g., vector search) ((5) in FIG. 3). The prompt generation unit 13 then assigns the searched advice to the prompt that instructs code generation. The prompt generation unit 13 assigns the searched advice to the code generation instruction to create a prompt ((6) in FIG. 3).

[0029] The prompt generator 13 sets the prompt after providing the advice to the LLM 21 and instructs the LLM 21 to generate a code. This allows the LLM 21 to generate a code with high security ((7) in FIG. 3). The processing device 10 receives the code generated by the LLM 21 and transmits it to the user.

[0030] [Advice Generation Unit] Next, we will explain the advice generation unit 11. The advice generation unit 11 handles three types of information sources for generating advice for improving security: CWE information, past coding information by LLM, and configuration information such as SBOM.

[0031] First, a method for constructing the first advice DB 30-1 based on the information of the CWE will be described. Fig. 4 is a diagram for explaining an example of advice generation processing.

[0032] The CWE information 51 that can be acquired from an NVD or the like includes items such as description and extended_description, which are used to describe the threat classification.

[0033] The advice generator 11 generates advice for improving security based on the explanation of the CWE information using the LLM 41 (FIG. 4), and stores the advice in the first advice DB 30-1. The LLM 41 is GPT3.5 (registered trademark) or the like.

[0034] The advice generator 11 collects CWE information 51 (FIG. 4) from an NVD or the like, inputs the description of the collected CWE information to an LLM 41 in which a prompt 52 (FIG. 4) for advice conversion is set, and converts the information into advice. The LLM 41 may be the LLM 21.

[0035] Next, a method for constructing the second advice DB 30-2 from past coding results will be described. Figures 5 and 6-1 to 6-3 are diagrams for explaining an example of advice generation processing.

[0036] 5, the advice generator 11 accumulates past code generation instructions for the LLM 61, code 62 (see, for example, FIG. 6-1) that the LLM 61 previously generated based on these instructions, and evaluation results (issues) 63 (see, for example, FIG. 6-2) of security vulnerabilities of the code 62 that the LLM 61 previously generated using an analysis tool (e.g., SAST). The issues in FIG. 5 are security concerns identified in the code generated by the LLM 61 using the SAST tool.

[0037] The advice generation unit 11 converts the accumulated information into advice for improving security using an LLM64 (FIG. 5) such as GPT3.5, and accumulates the advice in the second advice DB 30-2. The advice generation unit 11 inputs the accumulated past instruction content, vulnerable code 62, and issues 63, which are the evaluation results, into the LLM 64 and converts them into advice. A prompt 65 (see, for example, FIG. 6-3) for advice conversion is set in the LLM 64.

[0038] The analysis tool is not limited to an SAST tool, and may be, for example, the output of a DAST (Dynamic Application Security Testing) tool or an IAST (Interactive Application Security Testing) tool, or the results of a code review. The past code does not necessarily have to be generated by the LLM 61. The LLMs 61 and 64 may be the same LLM or may be the LLM 21.

[0039] Next, a method for constructing the third advice DB 30-3 from configuration information (for example, SBOM) will be described. Figures 7 to 9 are diagrams for explaining an example of advice generation processing.

[0040] Here, it is assumed that a project is under development and that there is an SBOM, which is a list of OSS (Open Source Software) that is being used or is planned to be used in the project.

[0041] 7, the advice generation unit 11 searches a CVE database 71 based on this SBOM to obtain a list of known vulnerabilities, their descriptions 72, and PoC code 73 that triggers the vulnerabilities. Based on this obtained information, the advice generation unit 11 generates advice for improving security using an LLM 74 such as GPT3.5, and stores the advice in the third advice DB 30-3. In this way, the third advice DB 30-3 stores advice for coding that does not exploit vulnerabilities, even for OSSs with vulnerabilities (OSSs for which CVEs have been issued).

[0042] The advice generation unit 11 searches the CVE database 71 for data 77 ( FIG. 8 ) stored in one entry, for example. Then, the advice generation unit 11 acquires a description 72 from an API (Application Programming Interface) ((1) in FIG. 8 ), for example. The advice generation unit 11 then follows the link of the Exploit tag acquired from the API ((2) in FIG. 8 ). Since the link destination contains data corresponding to the PoC code 73, the advice generation unit 11 acquires the PoC code 73 by scraping it ((3) in FIG. 8 ).

[0043] The advice generator 11 inputs a list of known vulnerabilities, their descriptions 72, and PoC codes 73 that trigger the vulnerabilities into an LLM 74 that has a prompt 75 (FIG. 9) for advice conversion set therein, and converts them into advice 76 (FIG. 9). The LLM 74 may be the LLM 21.

[0044] [Prompt Generator] Next, the processing of the prompt generator 13 will be described.

[0045] The prompt generation unit 13 searches the first to third advice DBs 30-1 to 30-3, for example, using a vector search, and performs processing to add necessary information to a program generation instruction from the user. The prompt generation unit 13 searches the first to third advice DBs 30-1 to 30-3 using RAG for advice. The prompt generation unit 13 may use a search method other than a vector search.

[0046] The prompt generator 13 may also search for advice based on code generation instructions using a search method appropriate for the database being searched. Here, a vector search method that differs for each database will be described. This search method was also used in the evaluation experiment (described later).

[0047] First, the prompt generation unit 13 performs a search based on the similarity between instructions and advice in the first advice DB 30-1, which is based on CWE information. This similarity is measured by vectorizing all instructions and various pieces of advice using, for example, the Embedding API of OpenAI (registered trademark), and using the cosine similarity between them. Note that the advice to be stored in the first advice DB 30-1 is vectorized and saved in advance by the advice generation unit 11.

[0048] Similarly, if there is an instruction to generate an LLM corresponding to a past code in the second advice DB 30-2 based on past coding results, the prompt generation unit 13 searches the second advice DB 30-2 using the similarity between the instruction and the current instruction.If no instruction with a similarity within a predetermined range is found, the prompt generation unit 13 searches based on the similarity between the current instruction and the advice itself.

[0049] The prompt generation unit 13 performed a search in the third advice DB 30-3 based on the configuration information based on the similarity between the instruction and the explanation / advice of each OSS. Note that the advice acquired from each of the first to third advice DBs 30-1 to 30-3 is a hyperparameter, and in the experiment described below, the prompt generation unit 13 acquired the top two results from each of the first to third advice DBs 30-1 to 30-3 as advice to be assigned to the instruction.

[0050] [Advice Storage Processing] Next, a processing procedure of the processing method according to the embodiment will be described. First, a processing of storing advice in the advice DB group 30 will be described. Fig. 10 is a flowchart showing the processing procedure of the processing method according to the embodiment.

[0051] The advice generator 11 collects various vulnerability information about software code (step S11). As described above, the vulnerability information includes CWE information, past coding information based on LLM, and / or configuration information such as SBOM.

[0052] The advice generator 11 generates advice for improving security when the LLM 21 creates code in response to the code generation instruction based on the vulnerability information of the software code collected in step S11 (step S12).The advice generator 11 stores the advice in a DB corresponding to the information source among the advice DB group 30 (step S13).

[0053] [Code Generation Processing] Next, the code generation processing will be described with reference to a flowchart of FIG.

[0054] The code generation instruction unit 12 receives a code generation instruction from the LLM 21 (step S21). The prompt generation unit 13 generates a prompt instructing code generation in response to the user's code generation instruction. The prompt generation unit 13 searches the advice DBs 30 for advice for improving security (step S22).

[0055] The prompt generator 13 assigns the advice found in step S22 to a prompt instructing code generation (step S23). The prompt generator 13 sets the advice-added prompt in the LLM 21 and instructs the LLM 21 to generate code (step S24). The processing device 10 receives the code generated by the LLM 21 (step S25), and transmits the received code to the user.

[0056] [Evaluation Experiment] An evaluation experiment was carried out on the processing method of this embodiment.

[0057] Figures 12 and 13 show the experimental conditions for the evaluation experiment. As shown in Figure 12, a total of 316 vulnerable functions in Python programs were prepared for the evaluation experiment, and natural language prompts for generating these functions were provided. In addition, for each function, the functions after the vulnerabilities were fixed and the results of SAST using semgrep were also prepared.

[0058] As shown in FIG. 13, in the evaluation experiment, the IDs and descriptions of all CWEs were extracted from the NVD and converted to create a first advice DB 30-1 based on the CWEs. In the evaluation experiment, the first 108 entries of the above dataset were used to create a second advice DB 30-2 based on past coding results. The latter 108 entries were used to generate code using GPT3.5 and GPT4 under the first and second conditions. The percentage of code generated under the first and second conditions that passed the SAST tool (the percentage of code that did not identify vulnerabilities) was then measured.

[0059] Under the first condition, the instruction prompt is set as is, and code is generated using GPT3.5 and GPT4. Under the second condition, the processing method according to the present embodiment is applied, and advice for improving security is added to the prompt, and then code is generated using GPT3.5 and GPT4. Note that, since large-scale experiments could not be conducted using the third advice DB 30-3 based on configuration information, only a few qualitative evaluations were conducted manually.

[0060] Figures 14-1 to 15-4 show the results of the evaluation experiment. First, Figures 14-1 to 14-4 show the evaluation results when GPT3.5 was used to generate code. Figures 15-1 to 15-4 show the evaluation results when GPT4 was used to generate code.

[0061] Figure 14-1 shows the evaluation results when GPT3.5 was used to generate code under the first condition, with only 57% correct. Figure 14-2 shows the evaluation results when GPT3.5 was used to generate code under the second condition, with advice from the first advice DB 30-1 based on CWE, with 65% correct. Figure 14-3 shows the evaluation results when GPT3.5 was used to generate code under the second condition, with advice from the second advice DB 30-2 based on past coding results, with 80% correct. Figure 14-4 shows the evaluation results when GPT3.5 was used to generate code under the second condition, with advice from the first advice DB 30-1 and the second advice DB 30-2, with 81% correct.

[0062] Figure 15-1 shows the evaluation results when GPT4 was allowed to generate code under the first condition, with only 57% being normal. Figure 15-2 shows the evaluation results when GPT4 was allowed to generate code under the second condition with advice from the first advice DB 30-1, with 75% being normal. Figure 15-3 shows the evaluation results when GPT4 was allowed to generate code under the second condition with advice from the second advice DB 30-2, with 82% being normal. Figure 15-4 shows the evaluation results when GPT4 was allowed to generate code under the second condition with advice from the first advice DB 30-1 and the second advice DB 30-2, with 87% being normal.

[0063] As such, it can be seen that in both cases where GPT3.5 and GPT4 are used, the rate at which the SAST tool passes is significantly improved by using the processing method according to this embodiment.

[0064] 16-1 to 17-2 are diagrams showing the results of a qualitative experiment using the third advice DB 30-3 based on configuration information. Fig. 16-1 is an example of a prompt instructing the generation of advice from configuration information. Fig. 16-2 is an example of advice.

[0065] The advice generator 11 sets a prompt 80 (FIG. 16-1) in the LLM 74 and inputs a list of known vulnerabilities, their descriptions 72, and PoC code 73 that triggers the vulnerabilities. As a result, the advice generator 11 generates advice 81 (FIG. 16-2) based on the configuration information.

[0066] Figure 17-1 corresponds to the case where the instruction prompt is set as is and coding is performed, while Figure 17-2 corresponds to the case where advice 81 based on configuration information (Figure 16-2) is added to the prompt. Python tarfiles have a vulnerability that poses the risk of directory traversal if the extract method, etc., is used without any countermeasures. When handling tarfiles in Python, this OSS method is basically used, so it is difficult to avoid this vulnerability other than by implementing it properly. In contrast, in the embodiment, as shown in Figure 17-2, by adding advice 81 based on configuration information, exception handling is performed appropriately, and it can be seen that the code is almost zero likely to trigger a directory traversal vulnerability.

[0067] [Effects of the embodiment] The processing device 10 according to the embodiment uses a plurality of techniques to construct an advice DB group 30 that stores advice for improving security. The processing device 10 then constructs a flow in which, in response to a code generation instruction input to the LLM 21, the processing device 10 searches the advice DB group 30 and automatically assigns the retrieved appropriate secure coding advice to a prompt. As can be seen from the results of the evaluation experiment, this embodiment can improve the security of code generated by the LLM 21.

[0068] In the embodiment, the processing device 10 automatically provides appropriate advice on secure coding to the prompt, so that the user can have the LLM 21 execute coding that takes security into consideration simply by inputting code generation instructions to the LLM 21 as before.

[0069] Furthermore, in the embodiment, when generating prompts, analysis tools such as SAST tools are not used, and code instructions to the LLM 21 only need to be given once, so code generation can be carried out quickly.

[0070] The processing device 10 can also assign to the prompt a combination of advice searched from multiple DBs among the advice DB group 30. In this case, the processing device 10 can instruct the LLM 21 to generate code based on multiple pieces of vulnerability information rather than relying on only one type of vulnerability information, thereby stably improving the security of the code generated by the LLM 21.

[0071] [System Configuration of the Embodiment] The processing device 10 is a functional concept and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of the functions of the processing device 10 is not limited to that shown in the figure, and all or part of it can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc.

[0072] Furthermore, all or any part of the processes performed by the processing device 10 may be realized by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a program analyzed and executed by the CPU and GPU. Furthermore, each process performed by the processing device 10 may be realized as hardware using wired logic.

[0073] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually. Alternatively, all or part of the processes described as being performed manually can be performed automatically using a known method. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters described above and illustrated can be changed as appropriate unless otherwise specified.

[0074] 18 is a diagram showing an example of a computer in which a program is executed to realize the processing device 10. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0075] The memory 1010 includes a ROM 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0076] The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. That is, the program that defines each process of the processing device 10 is implemented as a program module 1093 in which code that can be executed by the computer 1000 is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing processes similar to those of the functional configuration of the processing device 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced with an SSD (Solid State Drive).

[0077] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in memory 1010 or hard disk drive 1090. Then, CPU 1020 reads out program module 1093 or program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as necessary and executes them.

[0078] The program module 1093 and program data 1094 may not necessarily be stored in the hard disk drive 1090, but may also be stored in a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0079] Although the present invention has been described above as an embodiment, the present invention is not limited to the descriptions and drawings that form part of the disclosure of the present invention. In other words, other embodiments, examples, and operational techniques that can be made by those skilled in the art based on the present invention are all included in the scope of the present invention.

[0080] REFERENCE SIGNS LIST 10 Processing device 11 Advice generation unit 12 Code generation instruction unit 13 Prompt generation unit 20 Generation server 21 LLM 30 Advice DB group 30-1 First advice DB 30-2 Second advice DB 30-3 Third advice DB

Claims

1. A processing device comprising: a generation unit that generates advice for improving security when a large-scale language model creates code in response to a code generation instruction based on vulnerability information of software code, and stores the advice in a database; an instruction unit that accepts code generation instructions for the large-scale language model; and an assignment unit that searches the database for advice for improving security and assigns the searched advice to a prompt that instructs code generation.

2. The processing device described in claim 1, characterized in that the generation unit collects vulnerability information from a database storing vulnerability information for software code, and converts the collected vulnerability information into advice for improving security using a large-scale language model.

3. The processing device described in claim 1, characterized in that the generation unit accumulates code previously generated by a large-scale language model and evaluation results obtained by using an analysis tool to evaluate the security vulnerabilities of the code previously generated by the large-scale language model, and converts the accumulated information into advice for improving security using the large-scale language model.

4. The processing device described in claim 1, characterized in that the generation unit acquires vulnerability information based on software configuration information and converts the acquired vulnerability information into advice for improving security using a large-scale language model.

5. The processing device described in claim 1, characterized in that the generation unit stores the code generation instructions and the advice in a database provided for each source of vulnerability information according to the source of the vulnerability information, and the assignment unit searches for the advice based on the code generation instructions using a search method according to the database to be searched.

6. A processing method executed by a processing device, comprising the steps of: generating advice for improving security when a large-scale language model creates code in response to a code generation instruction based on vulnerability information of software code, and storing the advice in a database; accepting a code generation instruction for the large-scale language model; and searching the database for advice for improving security and adding the found advice to a prompt that instructs code generation.

7. A processing program for causing a computer to execute the following steps: generating advice for improving security when a large-scale language model creates code in response to a code generation instruction based on vulnerability information of software code, and storing the advice in a database; accepting a code generation instruction for the large-scale language model; and searching the database for advice for improving security and adding the found advice to a prompt that instructs code generation.

Images