Evaluation device, evaluation method, and evaluation program

Abstract

This evaluation device (10) comprises: an information collection unit (11) that collects software source code; and a code evaluation unit (13) that gives an instruction regarding risk assessment of the source code to a large language model (LLM) and sets the answer from the LLM as an evaluation result.

Description

Evaluation device, evaluation method, and evaluation program

[0001] The present invention relates to an evaluation device, an evaluation method, and an evaluation program.

[0002] Software is not only used in IT (Information Technology) systems, but also in controlling machines that affect human life, such as automobiles and aircraft, and critical infrastructure such as electricity and water that are essential to daily life. To accommodate a wide range of applications and requirements, the functionality required of software is increasing, and its complexity is also growing.

[0003] When using software created by other parties (suppliers), including OSS (Open Source Software), users are exposed to security risks. Therefore, users should evaluate the reliability of the software from a security perspective before using it.

[0004] Conventionally, there are two methods for evaluating the behavior of software.

[0005] One technique for evaluating software behavior is to use a mechanism similar to regular expressions to detect parts of a program that perform an operation specified by a user (static analysis of source code) (Non-Patent Documents 1 and 2).

[0006] Furthermore, as a technology for evaluating the behavior of software, there is a method (malignant code detection method) that performs dynamic analysis in addition to static analysis to detect malicious code (Non-Patent Document 3).Malignant code detection methods check typical behavior of malicious code, such as code that generates a process, whether file contents are being sent and received, whether received data is being used to generate a process without being verified, whether communication is being performed with an unintended destination, whether sensitive information is being accessed, etc., and detect any of the above if any of the following is true.

[0007] Semgrep, [online], [Retrieved April 16, 2024], Internet <URL: https: / / semgrep.dev / > CodeQL, [online], [Retrieved April 16, 2024], Internet <URL: https: / / codeql.github.com / > Ruian Duan, Omar Alrawi, Ranjita Pai Kasturi, Ryan Elder, Brendan Saltaformaggio, and Wenke Lee, “Towards Measuring Supply Chain Attacks on Package Managers for Interpreted Languages”, NDSS, 2021.

[0008] In static analysis of source code, rules exist for detecting each event, and these rules must be created for each programming language. Furthermore, in static analysis of source code, the rules must be updated in response to changes in the programming language specifications or API (Application Programming Interface).

[0009] Furthermore, malicious code detection methods have rules for detecting typical malicious code behavior, and just like static analysis of source code, they require the creation of rules for each programming language and the updating of rules as specifications change. For example, to detect network communications in Python, the APIs that communicate must be registered. For example, the Python module contains functions such as urllib.urlopen(), urllib2.urlopen(), and http.client.HTTPConnection(), but when a new API (such as requests.get()) appears, the rules must be updated.

[0010] As described above, conventional evaluation techniques require frequent rule updates, which means that if the rules are not updated, risk information for software behavior may not be properly evaluated.

[0011] The present invention has been made in consideration of the above, and aims to provide an evaluation device, an evaluation method, and an evaluation program that can appropriately evaluate risk information for software operation without frequently updating rules.

[0012] In order to solve the above-mentioned problems and achieve the objectives, the evaluation device of the present invention is characterized by having a collection unit that collects software source code, and an evaluation unit that gives instructions regarding risk evaluation of the source code to a large-scale language model and uses the response from the large-scale language model as the evaluation result.

[0013] In addition, the evaluation method of the present invention is an evaluation method executed by an evaluation device, and is characterized by including the steps of collecting source code of software, providing instructions regarding risk evaluation of the source code to a large-scale language model, and using the response from the large-scale language model as the evaluation result.

[0014] In addition, the evaluation program of the present invention causes a computer to execute the steps of collecting software source code, providing instructions regarding risk evaluation of the source code to a large-scale language model, and using the answer from the large-scale language model as the evaluation result.

[0015] According to the present invention, risk information for software behavior can be appropriately evaluated without frequently updating rules.

[0016] FIG. 1 is a diagram illustrating an overview of evaluation processing according to the first embodiment. FIG. 2 is a diagram illustrating an example of the configuration of a system including an evaluation device according to the first embodiment. FIG. 3 is a diagram schematically illustrating an example of the configuration of an evaluation device according to the first embodiment. FIG. 4 is a flowchart illustrating the processing steps of an evaluation method according to the first embodiment. FIG. 5 is a flowchart illustrating evaluation result reference processing according to the first embodiment. FIG. 6 is a diagram illustrating an example of the data configuration of data stored in the software database (DB) shown in FIG. 3. FIG. 7 is a diagram illustrating an evaluation approach in the code evaluation unit shown in FIG. 3. FIG. 8 is a diagram illustrating evaluation viewpoints. FIG. 9-1 is a diagram illustrating an example of a prompt in the first approach and an example of an LLM response. FIG. 9-2 is a diagram illustrating an example of a prompt in the second approach and an example of an LLM response. FIG. 10 is a diagram illustrating an example of the data configuration of evaluation results stored in the evaluation result DB shown in FIG. 3. FIG. 11 is a diagram schematically illustrating an example of the configuration of an evaluation device according to the second embodiment. FIG. 12 is a diagram illustrating the processing of the evaluation target determination unit shown in FIG. 11. FIG. 13 is a flowchart illustrating the processing steps of an evaluation method according to the embodiment. FIG. 14 is a diagram illustrating an example of a computer that implements the evaluation device by executing a program.

[0017] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are designated by the same reference numerals.

[0018] First Embodiment A first embodiment relates to a software evaluation technique, and in particular to a technique for evaluating the operation of software from its source code and evaluating whether the software performs an operation that poses a security risk.

[0019] In the first embodiment, by utilizing large language models (LLMs), malicious, inappropriate, and unnecessary software behavior can be detected without frequent rule updates, thereby contributing to ensuring software transparency.

[0020] [Outline of Evaluation Process] Fig. 1 is a diagram showing an outline of the evaluation process according to the first embodiment. As shown in Fig. 1, before a user uses software created by a developer, an evaluation device 10 according to the embodiment collects information about the software ((1) in Fig. 1) and gives instructions regarding a risk evaluation of the source code to the LLM 21 ((2) in Fig. 1). Based on the response from the LLM 21, the evaluation device 10 obtains an evaluation result of the software's source code ((3) in Fig. 1). The evaluation result is an evaluation of whether or not the software performs operations that pose a security risk.

[0021] The evaluation device 10 then provides the user with evaluation information on the software name and / or source code ((4) in Figure 1), allowing the user to determine whether or not the software can be used ((5) in Figure 1).

[0022] [System Configuration] Fig. 2 is a diagram showing an example of the configuration of a system including the evaluation device according to embodiment 1. As shown in Fig. 2, the evaluation device 10 communicates between a client 30 (user terminal) that requests the evaluation results of software and a server 20 that has an LLM 21.

[0023] [Evaluation Device] Fig. 3 is a diagram schematically illustrating an example of the configuration of an evaluation device according to the first embodiment. The evaluation device 10 according to the embodiment is realized by, for example, loading a predetermined program into a computer or the like including a ROM (Read Only Memory), a RAM (Random Access Memory), a CPU (Central Processing Unit), etc., and having the CPU execute the predetermined program. The evaluation device 10 also has a communication interface for transmitting and receiving various information to and from other devices connected via a network, etc. The evaluation device 10 is realized by a general-purpose computer such as a workstation or a personal computer.

[0024] The evaluation device 10 has an information collection unit 11 (collection unit), a software DB 12, a code evaluation unit 13 (evaluation unit), an evaluation result DB 14, and an evaluation result reference unit 15 (output unit). Fig. 4 is a flowchart showing the processing procedure of the evaluation method according to the first embodiment. Fig. 5 is a flowchart showing the evaluation result reference process according to the first embodiment.

[0025] The information collection unit 11 collects the source code of the software (step S1 in FIG. 4) and registers it in the software DB 12.

[0026] The software DB 12 stores the source code of the software collected by the information collection unit 11 .

[0027] The code evaluation unit 13 divides the source code to be evaluated into function units, and issues instructions regarding risk evaluation to the LLM 21, examining the functional parts and parts not included in the functions. The code evaluation unit 13 stores the response from the LLM 21 in the evaluation result DB 14 as the evaluation result of the software source code. The code evaluation unit 13 calculates an evaluation score for the software source code based on the response from the LLM 21 (step S2 in FIG. 4 ). The code evaluation unit 13 stores the evaluation score as the evaluation result in the evaluation result DB 14 (step S3 in FIG. 4 ).

[0028] The evaluation result DB 14 stores the evaluation results of the software source code evaluated by the code evaluation unit 13 .

[0029] Based on the software name and / or source code content of the search target, the evaluation result reference unit 15 searches for the corresponding evaluation results and evaluation scores from the evaluation result DB 14. The evaluation result reference unit 15 outputs information from the evaluation result DB 14 in response to the search query to the search request source (e.g., client 30) (step S11 in FIG. 5).

[0030] [Information Collecting Unit] The information collecting unit 11 will now be described. The information collecting unit 11 collects source code of software to be evaluated. Furthermore, when a user has source code that they wish to evaluate and the source code of this software is provided by the user, the information collecting unit 11 registers the provided source code in the software DB 12.

[0031] 6 is a diagram showing an example of the data structure of data stored in the software DB 12 shown in FIG. 3. As shown in FIG. 6, the software DB 12 stores the source code of each piece of software according to the following items: source code ID, software name, version, function, and code. The information collection unit 11 registers source code in units of functions. The information collection unit 11 also treats code in the global scope as a single function (e.g., __global__).

[0032] [Code Evaluation Unit 13] Next, a description will be given of the code evaluation unit 13. Fig. 7 is a diagram for explaining an evaluation approach in the code evaluation unit 13 shown in Fig. 3 .

[0033] The code evaluation unit 13 inspects each function in the software DB 12, gives instructions regarding risk evaluation to the LLM 21, and saves the answer as the evaluation result. The evaluation approaches taken by the code evaluation unit 13 include the following first approach (first evaluation) and second approach (second evaluation).

[0034] The first approach indicates evaluation viewpoints and has the LLM 21 analyze the source code in accordance with the viewpoints. The code evaluation unit 13 evaluates the code by indicating evaluation viewpoints and having the LLM 21 analyze the code in accordance with the viewpoints (first approach in FIG. 7). FIG. 8 is a diagram illustrating evaluation viewpoints. For example, the viewpoints are as shown in the table in FIG. 8. Viewpoints can be added, and specific examples for each viewpoint can also be adjusted.

[0035] The second approach involves having the LLM 21 explain the source code and then having the LLM 21 determine whether the content of the explanation is positive or negative, thereby evaluating the code (the second approach in FIG. 7).

[0036] Next, we will explain the prompts to the LLM 21. Figure 9-1 is a diagram showing an example of a prompt in the first approach and an example of a response from the LLM 21.

[0037] As shown in Figure 9-1, prompt P1 in the first approach instructs the LLM 21 to be a software evaluation service and to evaluate whether there are problems with the source code being evaluated from viewpoints 1 to 4. Prompt P1 also instructs the LLM 21 to output a score corresponding to whether or not there is a problem with the behavior for each viewpoint.

[0038] 9-1 is set by the code evaluation unit 13, the LLM 21 outputs an answer R1. The LLM 21 outputs an answer and an evaluation score for the source code to be evaluated for each of the viewpoints 1 to 4. For example, the evaluation score is set as "-1" for problematic behavior, "+1" for non-problematic behavior, and "0" if it cannot be determined.

[0039] Figure 9-2 shows an example of a prompt in the second approach and an example of an answer from LLM21.

[0040] As shown in Figure 9-1, prompt P2 in the second approach instructs the LLM 21 to further analyze the sentiment of the software commentary service and the commentary on the source code being evaluated to determine whether the commentary is positive or negative. For example, in the second approach, the evaluation score is set to "-1" if the commentary is negative, "+1" if the commentary is positive, and "0" if it cannot be determined.

[0041] 9-2 is set by the code evaluation unit 13, the LLM 21 outputs an answer R2. For example, answer R2 indicates that a problem with the source code being evaluated is pointed out, and therefore contains negative emotions.

[0042] The code evaluation unit 13 then calculates the weighted sum of the evaluation results of the first approach and the second approach for the source code to be evaluated as the evaluation score of the source code to be evaluated.

[0043] Fig. 10 is a diagram showing an example of the data configuration of the evaluation results stored in the evaluation result DB 14 shown in Fig. 3. The overall evaluation score is determined by adding a weight W to the evaluation score of each first approach and the evaluation score of the second approach.

[0044] For example, if the weights W for the evaluation of the first approach and the evaluation of the second approach are [1,1,1,1,1], the evaluation score for code ID "1" will be "1x1+1x1+1x1+1x1+1x1=5" (see evaluation result 14-1 in FIG. 10). The code evaluation unit 13 calculates the weighted sum of the evaluation score for the first approach and the evaluation score for the second approach for each source code, and registers this in the evaluation result DB 14 together with the code ID and the evaluation scores for the first and second approaches.

[0045] [Evaluation Result Reference Unit] When a search request is sent from the client 30 using software name, version, function, etc. as search keywords, the evaluation result reference unit 15 identifies the software ID from the software DB 12, and if the software ID exists in the evaluation result DB 14, it can refer to the evaluation score in the evaluation result DB 14. For example, when the main function of version 2.0 of software B is searched for (source code ID "3" in FIG. 6), an evaluation score of "-3" (line L3 in FIG. 10) is returned to the client 30.

[0046] [Effects of Embodiment 1] The evaluation device 10 according to embodiment 1 divides the source code of the software into function units, inspects the function parts and the parts not included in the functions, gives instructions regarding risk evaluation to the LLM 21, and regards the response from the LLM 21 as the evaluation result.

[0047] In this way, by utilizing the LLM 21, the evaluation device 10 detects malicious, inappropriate, and unnecessary software behavior without frequently updating rules. Therefore, the evaluation device 10 can appropriately evaluate risk information regarding software behavior without frequently updating rules, contributing to ensuring software transparency. Users can then determine whether to use software based on the evaluation information of the software's behavior provided by the evaluation device 10.

[0048] As described above, the evaluation device 10 visualizes whether or not there is a security risk in the operation of software, and software users can determine whether or not to use the software based on the risk information before using the software.

[0049] The code evaluation unit 13 may perform only one of the first approach and the second approach. In this case, the code evaluation unit 13 may calculate an evaluation score based on the result of either the first approach or the second approach that has been executed.

[0050] Second Embodiment FIG. 11 is a diagram schematically illustrating an example of the configuration of an evaluation device according to a second embodiment.

[0051] Since there are a large number of parts to be evaluated in the soft code of software, a large number of requests are generated to the LLM 21, resulting in high monetary costs such as usage fees. As shown in the evaluation device 210 in Fig. 11, compared to the evaluation device 10, an evaluation target determination unit 216 and an evaluation target DB 217 are added, and candidates for the soft code of software to be evaluated are narrowed down.

[0052] The evaluation target determination unit 216 is provided before the code evaluation unit 13. The evaluation target determination unit 216 determines whether each code in the software DB 12 is to be evaluated. The evaluation target determination unit 216 narrows down the source code to be evaluated based on the length of the source code and / or whether the source code calls other functions or APIs (Application Programming Interfaces). The evaluation target determination unit 216 stores the source code determined to be to be evaluated in the evaluation target DB 217.

[0053] FIG. 12 is a diagram illustrating the processing of the evaluation target determination unit 216 shown in FIG.

[0054] The evaluation target determination unit 216 determines whether the length of the source code is equal to or greater than a certain length (e.g., 100 characters). If the code length is shorter than the certain length, it is considered that the code will not be able to cause the behavior to be detected. Therefore, the evaluation target determination unit 216 excludes source code that is shorter than the certain length (e.g., 100 characters) from the evaluation targets of the LLM 21.

[0055] The evaluation target determination unit 216 then determines whether the source code to be evaluated calls other functions or APIs. If the source code to be evaluated does not call other functions or APIs, it is highly likely that no operation will be performed, and therefore the evaluation target determination unit 216 excludes it from the evaluation targets of the LLM 21.

[0056] In this way, the evaluation target determination unit 216 narrows down the source code to be evaluated from the source code in the software DB 12 ((1) in FIG. 12). The code evaluation unit 13 gives instructions regarding risk evaluation to the LLM 21 for the source code narrowed down by the evaluation target determination unit 216 ((2) in FIG. 12), and obtains the evaluation results ((3) in FIG. 12).

[0057] [Evaluation Process] Fig. 13 is a flowchart showing the processing procedure of the evaluation method according to the embodiment. Step S21 in Fig. 13 is the same process as step S1 in Fig. 4.

[0058] The evaluation target determination unit 216 determines whether each code in the software DB 12 is an evaluation target (step S22).

[0059] The code evaluation unit 13 issues instructions regarding risk evaluation to the LLM 21 for the source code whose number has been narrowed down by the evaluation target determination unit 216, and obtains a response from the LLM 21 as the evaluation result (step S23). Step S24 in Fig. 13 is the same process as step S3 in Fig. 4.

[0060] [Effects of the Second Embodiment] In this way, in the second embodiment, by appropriately narrowing down the candidates for the soft code of the software to be evaluated, it is possible to appropriately evaluate risk information for the operation of the software without generating a large number of requests to the LLM 21.

[0061] [System Configuration of the Embodiment] The evaluation devices 10 and 210 are conceptual functional units and do not necessarily have to be physically configured as shown in the drawings. In other words, the specific form of distribution and integration of the functions of the evaluation devices 10 and 210 is not limited to that shown in the drawings, and all or part of the evaluation devices 10 and 210 can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc.

[0062] Furthermore, all or any part of the processes performed in the evaluation devices 10 and 210 may be realized by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a program analyzed and executed by the CPU and GPU. Furthermore, each process performed in the evaluation devices 10 and 210 may be realized as hardware using wired logic.

[0063] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually. Alternatively, all or part of the processes described as being performed manually can be performed automatically using a known method. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters described above and illustrated can be changed as appropriate unless otherwise specified.

[0064] 14 is a diagram showing an example of a computer in which the evaluation device 10, 210 is realized by executing a program. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0065] The memory 1010 includes a ROM 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0066] The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the evaluation device 10, 210 is implemented as a program module 1093 in which code executable by the computer 1000 is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, the program module 1093 for executing processes similar to those of the functional configuration of the evaluation device 10, 210 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced with an SSD (Solid State Drive).

[0067] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in memory 1010 or hard disk drive 1090. Then, CPU 1020 reads out program module 1093 or program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as necessary and executes them.

[0068] The program module 1093 and program data 1094 may not necessarily be stored in the hard disk drive 1090, but may also be stored in a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0069] Although the present invention has been described above as an embodiment, the present invention is not limited to the description and drawings that form part of the disclosure of the present invention. In other words, other embodiments, examples, and operational techniques that can be made by those skilled in the art based on the present invention are all included in the scope of the present invention.

[0070] 10, 210 Evaluation device 11 Information collection unit 12 Software DB 13 Code evaluation unit 14 Evaluation result DB 15 Evaluation result reference unit 20 Server 21 LLM 30 Client 216 Evaluation target determination unit 217 Evaluation target DB

Claims

1. An evaluation device comprising: a collection unit that collects software source code; and an evaluation unit that gives instructions regarding risk evaluation of the source code to a large-scale language model and uses the response from the large-scale language model as the evaluation result.

2. The evaluation device described in claim 1, characterized in that the evaluation unit divides the source code into function units, inspects the function parts and parts not included in the functions, gives instructions regarding risk evaluation of the source code to a large-scale language model, and uses the response from the large-scale language model as the evaluation result.

3. The evaluation device described in claim 1, characterized in that the evaluation unit: indicates an evaluation viewpoint; performs a first evaluation in which the large-scale language model analyzes the source code in accordance with the viewpoint; and calculates an evaluation score based on the results of the first evaluation.

4. The evaluation device described in claim 1, characterized in that the evaluation unit performs a second evaluation in which the large-scale language model explains the source code and further determines whether the content of the explanation is positive or negative, and calculates an evaluation score based on the result of the second evaluation.

5. The evaluation device described in claim 1, characterized in that the evaluation unit performs: a first evaluation in which an evaluation viewpoint is indicated and the large-scale language model is caused to analyze the source code in accordance with the viewpoint; and a second evaluation in which the large-scale language model is caused to explain the source code and further cause the large-scale language model to determine whether the content of the explanation is positive or negative; and calculates a weighted sum of the results of the first evaluation and the second evaluation as an evaluation score.

6. The evaluation device according to claim 1, further comprising a determination unit that narrows down the source code to be evaluated based on the length of the source code and / or whether the source code calls other functions or APIs (Application Programming Interfaces).

7. An evaluation method executed by an evaluation device, comprising the steps of: collecting software source code; providing instructions regarding risk evaluation of the source code to a large-scale language model; and determining the response from the large-scale language model as the evaluation result.

8. An evaluation program for causing a computer to execute the steps of: collecting software source code; providing instructions regarding risk evaluation of the source code to a large-scale language model; and using the answer from the large-scale language model as the evaluation result.

Images