Systems, methods, and programs for detecting vulnerabilities in web applications

Abstract

This supports static analysis of web applications that appropriately utilize large-scale language models. [Solution] A vulnerability testing server 10 according to one embodiment of the present invention provides a vulnerability testing service to a user operating a user terminal 30 that is connected to the communication network 20 via a communication network 20, performing static analysis of vulnerabilities in a web application. The server 10 obtains vulnerability information indicating vulnerabilities in the web application detected based on the source code of the web application from a large-scale language model, and determines whether or not there is a false positive for the vulnerability based on the obtained vulnerability information, thereby enabling it to respond to false positives caused by hallucination occurring in the large-scale language model.

Description

Technical Field

[0001] The present invention relates to a technique for detecting vulnerabilities in web applications.

Background Art

[0002] Conventionally, detecting vulnerabilities in web applications has been performed by static analysis of source code (see, for example, Patent Document 1 below). Also, in such static analysis, the use of large language models (LLMs) has been considered.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, when using a large language model for the above-described static analysis, it is necessary to fully consider the characteristics of the large language model. For example, there is a possibility that a phenomenon called hallucination, in which the large language model generates information not based on facts, may occur, and it is desirable to use it considering such characteristics.

[0005] One object of an embodiment of the present invention is to assist static analysis of web applications in which a large language model is appropriately used. Other objects of the embodiments of the present invention will become apparent by referring to the entire specification.

Means for Solving the Problems

[0006] A system according to one embodiment of the present invention is a system for detecting vulnerabilities in a web application, comprising one or more computer processors, wherein the one or more computer processors input a first prompt to a first large-scale language model for obtaining vulnerability information indicating vulnerabilities in the web application detected based on the source code of the web application, and perform the steps of obtaining the vulnerability information from the first large-scale language model, and determining whether or not there is a false positive for the vulnerability indicated by the vulnerability information based on the vulnerability information.

[0007] A method according to one embodiment of the present invention is a method for detecting vulnerabilities in a web application, which is executed by one or more computers, and comprises the steps of: inputting a first prompt to a first large-scale language model for obtaining vulnerability information indicating vulnerabilities in the web application that are detected based on the source code of the web application, and obtaining the vulnerability information from the first large-scale language model; and determining, based on the vulnerability information, whether or not the vulnerability indicated by the vulnerability information is a false positive.

[0008] A program according to one embodiment of the present invention is a program for detecting vulnerabilities in a web application, which causes one or more computers to input a first prompt to a first large-scale language model for obtaining vulnerability information indicating vulnerabilities in the web application that are detected based on the source code of the web application, and to obtain the vulnerability information from the first large-scale language model, and to determine whether or not there is a false positive for the vulnerability indicated by the vulnerability information based on the vulnerability information. [Effects of the Invention]

[0009] Various embodiments of the present invention support static analysis of web applications that appropriately utilize large-scale language models. [Brief explanation of the drawing]

[0010] [Figure 1] A schematic diagram showing the network configuration including a vulnerability testing server 10 according to one embodiment of the present invention. [Figure 2] A diagram illustrating the information managed by inspection management table 1511. [Figure 3] A diagram illustrating the information managed by the inspection details management table 1512. [Figure 4] A diagram illustrating inspection screen 50. [Figure 5] A flowchart illustrating the processes that server 10 executes when performing vulnerability testing. [Figure 6] A diagram illustrating a prompt for obtaining endpoint information. [Figure 7] A diagram illustrating a prompt for obtaining specification information. [Figure 8] A diagram illustrating a prompt for obtaining vulnerability information. [Figure 9] A diagram illustrating the response from a large-scale language model to a prompt for obtaining vulnerability information. [Figure 10] This diagram illustrates a prompt for obtaining vulnerability information when a false positive corresponding to Case 1 is detected. [Figure 11] Figure 10 illustrates the response from a large-scale language model to a prompt for obtaining vulnerability information. [Figure 12] A diagram illustrating examples of strings used to indicate false positives based on vulnerability types and development environment combinations. [Figure 13] This diagram illustrates a prompt for obtaining vulnerability information when a false positive corresponding to Case 2 is detected. [Figure 14] Figure 13 illustrates the response from a large-scale language model to a prompt for obtaining vulnerability information. [Figure 15] This diagram illustrates a prompt for obtaining vulnerability information when a false positive corresponding to Case 3 is detected. [Figure 16] Figure 15 illustrates the response from a large-scale language model to a prompt for obtaining vulnerability information. [Figure 17] A diagram illustrating a prompt for obtaining vulnerability information when a false detection corresponding to Case 4 is determined. [Figure 18] A diagram illustrating a response from a large language model to which a prompt for obtaining vulnerability information illustrated in FIG. 17 is input. [Figure 19] A diagram illustrating a prompt for obtaining false detection information. [Figure 20] A diagram illustrating a response from a large language model to which a prompt for obtaining false detection information is input. [Figure 21] A diagram illustrating a prompt for obtaining false detection information when the false detection is determined to be "present". [Figure 22] A diagram illustrating a response from a large language model to which a prompt for obtaining false detection information illustrated in FIG. 21 is input.

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In each drawing, the same or similar components may be assigned the same reference numerals.

[0012] FIG. 1 is a configuration diagram schematically showing the configuration of a network including a vulnerability inspection server 10 according to an embodiment of the present invention. As shown in the figure, the server 10 is communicably connected to a user terminal 30 via a communication network 20 such as the Internet. In FIG. 1, only one user terminal 30 is shown, but the server 10 is communicably connected to a plurality of user terminals 30. The vulnerability inspection server 10 provides a vulnerability inspection service for performing static analysis of the vulnerability of a web application to a user who operates the user terminal 30. The vulnerability inspection server 10 is an example of a device that implements all or part of the system of the present invention.

[0013] First, the hardware configuration of the vulnerability testing server 10 will be described. The vulnerability testing server 10 is configured as a general-purpose computer and, as shown in Figure 1, includes a computer processor 11, main memory 12, input / output interface 13, communication interface 14, and storage (memory device) 15. These components are electrically connected via buses and the like (not shown).

[0014] The computer processor 11 is configured as a CPU or GPU, and loads various programs stored in storage 15, etc., into the main memory 12 and executes various instructions contained in those programs. The main memory 12 is configured as, for example, DRAM.

[0015] The input / output interface 13 includes various input / output devices for exchanging information with the operator, etc. The input / output interface 13 includes, for example, information input devices such as keyboards and pointing devices (e.g., mice, touch panels, etc.), audio input devices such as microphones, and image input devices such as cameras. Furthermore, the input / output interface 13 includes image output devices such as displays and audio output devices such as speakers.

[0016] The communication interface 14 is implemented as hardware such as a network adapter, various communication software, or a combination thereof, and is configured to enable wired or wireless communication via the communication network 20, etc.

[0017] The storage 15 is composed of, for example, a magnetic disk or flash memory. The storage 15 stores various programs, including the operating system, and various data. For example, as shown in Figure 1, the storage 15 has various tables 151 that manage various information for providing vulnerability testing services. Also, for example, the storage 15 stores a server-side program 40 according to one embodiment of the present invention. This program 40 is a program that causes the server 10 to function as all or part of a system for providing vulnerability testing services. At least a part of the server-side program 40 may be configured to be executed on the user terminal 30 side via a terminal-side program 42, which will be described later.

[0018] In this embodiment, the vulnerability testing server 10 may be configured using multiple computers, each having the hardware configuration described above. For example, the server 10 may be composed of multiple server devices.

[0019] The vulnerability testing server 10 configured in this way can be configured to function as both a web server and an application server. It performs various processes in response to requests from the user terminal 30 and sends screen data (e.g., HTML data) and control data, etc., to the user terminal 30 according to the results of those processes. The user terminal 30 then displays a web page or other screen based on the received data.

[0020] Next, the hardware configuration of the user terminal 30 will be described. The user terminal 30 is configured as a general-purpose computer and, as shown in Figure 1, includes a computer processor 31, main memory 32, input / output interface 33, communication interface 34, and storage (memory device) 35. These components are electrically connected via buses and the like (not shown).

[0021] The computer processor 31 is configured as a CPU or GPU, and loads various programs stored in storage 35, etc., into the main memory 32 and executes various instructions contained in those programs. The main memory 32 is configured as, for example, DRAM.

[0022] The input / output interface 33 includes various input / output devices for exchanging information with the operator, etc. The input / output interface 33 includes, for example, information input devices such as keyboards and pointing devices (e.g., mice, touch panels, etc.), audio input devices such as microphones, and image input devices such as cameras. Furthermore, the input / output interface 33 includes image output devices such as displays and audio output devices such as speakers.

[0023] The communication interface 34 is implemented as hardware such as a network adapter, various communication software, and a combination thereof, and is configured to enable wired or wireless communication via the communication network 20, etc.

[0024] The storage 35 is composed of, for example, a magnetic disk or flash memory. The storage 35 stores various programs, including the operating system, and various data. The programs stored in the storage 35 can be downloaded and installed from an application market or the like. The storage 35 also stores the terminal-side program 42 described above. The program 42 is configured as a web browser or other application (for example, a terminal-side application for the vulnerability testing service in this embodiment), and as described above, it can be configured to execute at least a part of the server-side program 40.

[0025] In this embodiment, the user terminal 30 may be configured as a smartphone, tablet device, or personal computer, etc.

[0026] A user operating the user terminal 30 configured in this way can utilize the vulnerability testing service provided by the server 10 by communicating with the server 10 via the terminal-side program 42 installed on the storage 35, etc.

[0027] Next, the functions of the vulnerability testing server 10 configured in this way will be described. As shown in Figure 1, the computer processor 11 of the server 10 is configured to function as a management function control unit 112 and a testing control unit 114 by executing instructions contained in a program loaded into the main memory 12 (for example, at least a part of the server-side program 40).

[0028] The management function control unit 112 is configured to perform various processes related to controlling the management functions of the vulnerability testing service. For example, the management function control unit 112 transmits screen data and control data for various screens related to the management functions to the user terminal 30, performs various processes in response to user input via the screens output on the user terminal 30, and transmits screen data and control data to the user terminal 30 according to the results of those processes. The management functions controlled by the management function control unit 112 include, for example, login processing (user authentication), billing control, and user account management.

[0029] The inspection control unit 114 is configured to perform various processes related to the control of vulnerability testing. For example, the inspection control unit 114 transmits screen data and control data for various screens used to control the testing to the user terminal 30, performs various processes in response to user input via the screens output on the user terminal 30, and transmits screen data and control data corresponding to the results of those processes to the user terminal 30.

[0030] In this embodiment, the inspection control unit 114 is configured to input a first prompt to a first large-scale language model (LLM) to obtain vulnerability information indicating vulnerabilities in the web application detected based on the source code of the web application, and to obtain the vulnerability information from the first large-scale language model. The large-scale language model is a machine learning model in natural language processing that is trained using a large amount of data, and may include, for example, GPT provided by OpenAI, Inc., PaLM or LaMDA provided by Google, Inc., or LLaMa provided by Meta Inc., Inc. The prompt is information for giving instructions or requests to the large-scale language model, and is input, for example, via an API corresponding to such a large-scale language model. The first prompt can also be said to be a prompt for instructing the detection of vulnerabilities based on the source code.

[0031] Furthermore, the inspection control unit 114 is configured to determine whether or not a vulnerability indicated by the vulnerability information obtained from the first large-scale language model has been falsely detected. False detections include, for example, overdetection due to hallucination.

[0032] Thus, the vulnerability testing server 10 in this embodiment obtains vulnerability information indicating vulnerabilities in the web application detected based on the source code of the web application from a large-scale language model, and determines whether or not there are false positives of the vulnerability based on the obtained vulnerability information. Therefore, it is possible to address false positives caused by hallucination occurring in the large-scale language model. In other words, the server 10 supports static analysis of web applications in which the large-scale language model is appropriately utilized.

[0033] In this embodiment, the determination of whether or not a false positive has occurred based on vulnerability information can be achieved by various methods. For example, the determination of whether or not a false positive has occurred may be made based on predetermined rules (rule-based).

[0034] For example, vulnerability information obtained from the first large-scale language model may include the type of vulnerability and the output destination (Sink) related to the vulnerability, and the inspection control unit 114 may be configured to determine whether or not there is a false positive based on the type of vulnerability and the output destination. For example, the first prompt is configured as information that instructs or requests the provision of vulnerability information including the type of vulnerability and the output destination related to the vulnerability. For example, the inspection control unit 114 is configured to determine that it is a false positive if the output destination contains a string indicating a false positive (for example, this string is set for each web application development environment (language, framework, or library, etc.)) corresponding to the type of vulnerability. The output destination related to the vulnerability can also be said to be the location where the vulnerability is triggered or the location where the input value is used. The string indicating a false positive is, for example, a string corresponding to the syntax in which HTML special character escaping is performed automatically when the type of vulnerability is "cross-site scripting" (for example, "<%=*%>" (* is any string) when the development environment is "EJS"). Furthermore, for example, the string indicating a false positive is a string indicating a function used on the client side (for example, "ajax" if the development environment is "jQuery") when the type of vulnerability is "server-side request forgery" (a vulnerability that allows malicious requests to be sent from the server side). This configuration makes it possible to determine whether or not a false positive has occurred based on the type of vulnerability and the output destination related to that vulnerability.

[0035] Furthermore, for example, vulnerability information obtained from the first large-scale language model may include the type of input source related to the vulnerability, and the inspection control unit 114 may be configured to determine whether or not a false positive has occurred based on the type of input source. For example, the first prompt may be configured as information that instructs or requests the provision of vulnerability information including the type of input source related to the vulnerability. An input source related to a vulnerability can also be called the source (origin) of a vulnerable input value. For example, the inspection control unit 114 may be configured to determine that a false positive has occurred if the type of input source related to the vulnerability is a predetermined type that indicates a false positive. For example, if the type of input source is a hardcoded constant or a static variable (because these types of input sources are used as fixed values ​​within the program and the input values ​​cannot be manipulated from the outside), it may be determined that a false positive has occurred if the data is from a configuration file (because the contents of configuration files generally cannot be manipulated from the outside), and if the data is from an environment variable (because the contents of environment variables generally cannot be manipulated from the outside). Furthermore, for example, vulnerability information may include the type of vulnerability, and the inspection control unit 114 is configured to determine whether or not there is a false positive based on the type of vulnerability and the type of input source related to the vulnerability (for example, it determines that there is a false positive if the combination of the type of vulnerability and the input source related to the vulnerability is a predetermined combination that indicates a false positive). For example, the combination of vulnerability type "open redirect" or "cross-site scripting" and input source type "data from the HTTP Host header" is determined to be a false positive (because the Host header cannot be manipulated with JavaScript due to browser security restrictions). This configuration makes it possible to determine whether or not there is a false positive of a vulnerability based on the type of input source related to the vulnerability.

[0036] In this embodiment, the first prompt for obtaining vulnerability information may be automatically generated. For example, the inspection control unit 114 may be configured to generate the first prompt for obtaining vulnerability information of a web application based on the source code of the web application. Such a configuration makes it possible to easily obtain vulnerability information utilizing a large-scale language model.

[0037] In this embodiment, a large-scale language model may be used to determine whether or not a false positive has occurred, in addition to detecting vulnerabilities. Specifically, the inspection control unit 114 may be configured to generate a second prompt to obtain false positive information regarding the false positive of a vulnerability indicated by the vulnerability information, based on the vulnerability information obtained from the first large-scale language model, input the second prompt to the second large-scale language model, obtain the false positive information from the second large-scale language model, and determine whether or not a false positive has occurred based on the false positive information. This configuration makes it possible to determine whether or not a vulnerability has been falsely detected by utilizing a large-scale language model.

[0038] In this case, the second large-scale language model for obtaining false positive information may be the same large-scale language model as the first large-scale language model for obtaining vulnerability information. This configuration makes it easy to obtain vulnerability information and false positive information using the same large-scale language model.

[0039] Furthermore, the second large-scale language model used to acquire false positive information may be a different large-scale language model from the first large-scale language model used to acquire vulnerability information. This configuration allows for the determination of false positives using a large-scale language model with a different hallucination tendency than the large-scale language model used to acquire vulnerability information, thereby supporting the appropriate determination of whether or not a false positive has occurred.

[0040] Next, a specific example of one embodiment of the vulnerability testing server 10 of this embodiment, which has such functions, will be described. First, in this example, the information managed by each table included in the various tables 151 will be described.

[0041] Figure 2 illustrates the information managed by the inspection management table 1511 in this example. In this example, the inspection management table 1511 manages information related to vulnerability inspections and, as shown in the figure, associates an "inspection ID" that identifies an individual inspection with information about the user performing the inspection, "basic information" including the date and time of the inspection, the "source code file storage path" which is the path where the source code file containing the source code of the web application to be inspected is stored, and the "source code file name" which is the file name of the said source code file.

[0042] Figure 3 illustrates the information managed by the inspection details management table 1512 in this example. In this example, the inspection details management table 1512 manages endpoint-level information in an inspection, and as shown in the figure, it manages information such as the "HTTP method" corresponding to the process executed at the endpoint, the "path" that identifies the endpoint, the "code" corresponding to the process executed at the endpoint, the "specification information" which is information describing the specifications of the endpoint in natural language, the "vulnerability information" which is information about vulnerabilities at the endpoint, and the "false positive information" which is information about false positives of the vulnerability.

[0043] In this example, the endpoint, sometimes called an API endpoint, serves as the entry point for a client to access a specific function or data, and can also be described as the point or unit from which external attacks can enter. The client can obtain a response from the endpoint by sending an HTTP request.

[0044] The above explains the information managed by each table in this example. Next, we will explain the processes executed by the vulnerability testing server 10 and the screens displayed on the user terminal 30 in this example.

[0045] Figure 4 illustrates an inspection screen 50 displayed on the user terminal 30. This screen 50 is used by users of the vulnerability inspection service to perform vulnerability inspections by static analysis of source code. As shown in the figure, it has a first button 52 with the text "Specify source code file" and a second button 54 with the text "Start inspection".

[0046] The first button 52 is an object for specifying a file containing the source code of the web application to be tested for vulnerabilities. When this button 52 is selected, a screen (not shown) for specifying a specific file is displayed, and the user can specify the file containing the source code through this screen. Once the file is specified, the source code file storage path and the source code file name are registered in the test management table 1511. Note that it may be possible to specify multiple files as files containing source code.

[0047] The second button 54 is an object for initiating vulnerability testing. Figure 5 is a flowchart illustrating the processes that the server 10 executes in response to the selection of the button 54 (i.e., when performing vulnerability testing). First, the server 10 generates a prompt for obtaining endpoint information, as shown in the figure (step S100). Specifically, a prompt instructing the server to obtain endpoint information is generated based on the source code of the web application.

[0048] Figure 6 illustrates a prompt for obtaining endpoint information. As shown in the figure, this prompt instructs the system to extract the endpoint path and the function / call destination from the source code of the web application. Specifically, it instructs the system to extract the HTTP method, path, and code as endpoint-specific information. The contents of the source code are obtained from the source code file identified by the source code file storage path and source code file name managed in the inspection management table 1511.

[0049] Returning to the flowchart in Figure 5, Server 10 then acquires and registers endpoint information (step S110). Specifically, it inputs a prompt for acquiring endpoint information to the large language model and registers the endpoint information (HTTP method, path, and code) of each of the multiple endpoints output from the large language model into the inspection details management table 1512.

[0050] Next, server 10 generates a prompt for obtaining specification information for the target endpoint (step S120). Specifically, an unprocessed endpoint is identified as the target endpoint from among multiple endpoints, and a prompt for obtaining specification information for that target endpoint is generated. This prompt instructs the server to obtain specification information based on the endpoint information.

[0051] Figure 7 illustrates a prompt for obtaining specification information. As shown in the figure, this prompt instructs the user to describe the endpoint specifications based on the endpoint information (HTTP method, path, and code). Specifically, it instructs the user to describe the HTTP method executed at the endpoint, the path that identifies the endpoint, and a description of the endpoint's specifications. The endpoint information (HTTP method, path, and code) is obtained from the inspection details management table 1512.

[0052] Returning to the flowchart in Figure 5, the server 10 then acquires and registers the specification information (step S130). Specifically, it inputs a prompt for acquiring specification information to the large-scale language model and registers the specification information output from the large-scale language model in the inspection detail management table 1512. In this example, the specification information is acquired using the same large-scale language model as the one used to acquire the endpoint information. In other examples of this embodiment, the specification information may be acquired using a large-scale language model different from the one used to acquire the endpoint information.

[0053] Next, server 10 generates a prompt (first prompt) for obtaining vulnerability information for the target endpoint (step S140). Specifically, a prompt is generated that instructs the server to obtain vulnerability information for the endpoint based on the corresponding specification information.

[0054] Figure 8 illustrates a prompt for obtaining vulnerability information. As shown in the figure, the prompt instructs the system to identify vulnerabilities in the endpoint based on the endpoint's specification information and code (obtaining the type of vulnerability, vulnerability name, location where the vulnerability occurs (output destination), source of the vulnerable input value, type of source, vulnerability description, and corresponding code). The endpoint's specification information and code are obtained from the inspection details management table 1512. As illustrated in Figure 8, the prompt instructs the system to select the type of source of the vulnerable input value from the following 13 predetermined values. (1) config: Data from the configuration file (2) env: Data from environment variables (3) static_variable: hardcoded constant or static variable (4) deserialized_data: Data deserialized in Java or other languages (5) session_variable: Data from session variables (6) cookie_variable: Data from HTTP cookie (7) http_host_header_variable: Data from the HTTP Host header (8) http_referer_header_variable: Data from the Referer header (9) http_path_variable: Data from HTTP path parameters (10) http_query_variable: Data from HTTP query parameters (11) http_body_variable: Data from the HTTP body (12) jwt_data: Data from JSON Web Token (JWT) (13)other_source: Other data not included in the above categories

[0055] Returning to the flowchart in Figure 5, the server 10 then acquires and registers vulnerability information (step S150). Specifically, it inputs a prompt for acquiring vulnerability information to the large-scale language model and registers the vulnerability information output from the large-scale language model in the inspection details management table 1512. In this example, vulnerability information is acquired using the same large-scale language model as the one used to acquire endpoint information and specification information. In other examples of this embodiment, vulnerability information may be acquired using a large-scale language model different from the one used to acquire endpoint information and / or specification information.

[0056] Figure 9 illustrates the response from a large-scale language model to which a prompt for obtaining vulnerability information has been entered. As shown in the figure, the vulnerability information for the endpoint is provided based on the endpoint's specification information and code, including the type of vulnerability ("CWE-89"), the vulnerability name ("SQL Injection"), the location where the vulnerability is triggered (output destination), the source of the vulnerable input value (input source), the type of the source, a description of the vulnerability, and the relevant code.

[0057] Returning to the flowchart in Figure 5, Server 10 then determines and registers whether or not there are false positives in the acquired vulnerability information (step S160). In this example, the determination of whether or not there are false positives is performed on a rule basis, and specifically, a false positive (over-detection) is determined in the following cases. Case 1: The relevant code included in the vulnerability information is not present in the source code. Case 2: When a string indicating a false positive corresponding to the type of vulnerability included in the vulnerability information is included in the location (output destination) where the vulnerability is triggered. Case 3: When the type of source (input source) of the vulnerable input value included in the vulnerability information is a value that indicates a false positive. Case 4: When the combination of the type of vulnerability included in the vulnerability information and the type of source (input source) of the vulnerable input value indicates a false positive. The result of the determination of whether or not a false positive occurred (false positive information) is managed in the inspection details management table 1512, as described above.

[0058] The determination for Case 1 is made by determining whether the relevant code included in the vulnerability information is included in the code in the corresponding vulnerability information acquisition prompt (the code in the corresponding endpoint, which is managed in the inspection details management table 1512).

[0059] Figures 10 and 11 illustrate the prompt for retrieving vulnerability information and the response from the large-scale language model when a false positive corresponding to Case 1 is determined. In this example, the relevant code included in the vulnerability information (the value of "code_snippet" in Figure 11) is not included in the code in the corresponding vulnerability information retrieval prompt (the code listed in the "# source code" column in Figure 10), so it is determined to be a false positive corresponding to Case 1. For example, a false positive corresponding to Case 1 is likely to occur when the large-scale language model reports a general example code corresponding to the type of vulnerability (e.g., open redirect) as a vulnerability.

[0060] Furthermore, the determination of Case 2 is made by determining whether the string indicating a false positive corresponding to the type of vulnerability included in the vulnerability information is included in the location (output destination) where the vulnerability is triggered, also included in the vulnerability information. The string indicating a false positive corresponding to the type of vulnerability is pre-configured for each web application development environment. For example, as illustrated in Figure 12, for the combination of vulnerability type "CWE-79 (Cross-Site Scripting)" and development environment, the string corresponding to the syntax in which HTML special characters are automatically escaped (which can be said to be a secure syntax with respect to cross-site scripting) is pre-configured as the string indicating a false positive. This correspondence between vulnerability types, development environments, and strings indicating false positives may be managed in tables included in various tables 151.

[0061] Figures 13 and 14 illustrate the prompt for obtaining vulnerability information and the response from the large-scale language model when a false positive corresponding to Case 2 is determined. In this example, the string indicating a false positive (<%= variable %>) corresponding to the combination of the type of vulnerability ("CWE_ID" value in Figure 14, which is "CWE-79") and the development environment (EJS) included in the vulnerability information is also included in the location where the vulnerability is triggered (output destination) ("<%= params.last_name %>" value in Figure 14, which is "sink"), and therefore it is determined to be a false positive corresponding to Case 2. Note that the development environment of the web application is determined based on the extension in the source code file name.

[0062] Furthermore, the determination of Case 3 is made by determining whether the type of source (input source) of the vulnerable input value included in the vulnerability information is of a predetermined type that indicates a false positive. In this example, the predetermined type is, for example, "hardcoded constants or static variables" (hardcoded constants (written directly into the source code) and static variables cannot be sources of vulnerable input values ​​because they cannot specify the input necessary for an attack).

[0063] Figures 15 and 16 illustrate the prompt for obtaining vulnerability information and the response from the large-scale language model when a false positive corresponding to Case 3 is determined. In this example, the type of source (input source) of the vulnerable input value included in the vulnerability information is "hardcoded constant or static variable" (the value of "source_type" in Figure 16 is "static_variable"), so it is determined to be a false positive corresponding to Case 3.

[0064] Furthermore, the determination of Case 4 is made by determining whether the combination of the type of vulnerability included in the vulnerability information and the type of source (input source) of the vulnerable input value, also included in the vulnerability information, is a combination that indicates a false positive. Combinations that indicate a false positive may be managed in tables included in various tables 151.

[0065] Figures 17 and 18 illustrate the prompt for obtaining vulnerability information and the response from the large-scale language model when a false positive corresponding to Case 4 is determined. In this example, the combination of the type of vulnerability included in the vulnerability information (the value of "CWE_ID" in Figure 18, which is "CWE-601") and the type of source (input source) of the vulnerable input value (the value of "source_type" in Figure 18, which is "http_host_header_variable") is a predetermined combination that indicates a false positive, and is therefore determined to be a false positive corresponding to Case 4 (regarding open redirects (CWE-601), the HTTP Host header cannot be manipulated by JavaScript due to browser security constraints, and therefore cannot be a source of vulnerable input values).

[0066] Returning to the flowchart in Figure 5, if there are any unprocessed endpoints (YES in step S170), the server 10 returns to step S120 and targets the next endpoint, generating a prompt for obtaining specification information, obtaining and registering the specification information, generating a prompt for obtaining vulnerability information, obtaining and registering the vulnerability information, and determining and registering whether or not there are any false positives (steps S120-S160). This series of processes is repeated until there are no more unprocessed endpoints (NO in step S170).

[0067] In this example, since the presence or absence of false positives corresponding to cases 1-4 is determined based on vulnerability information, it becomes possible to address false positives caused by hallucination, etc.

[0068] In the example described above, the determination of whether or not a false positive is present is performed using a rule-based method, but instead of this, or in addition to this, it may be done by utilizing a large-scale language model. This large-scale language model may be the same large-scale language model used to obtain endpoint information, specification information, and / or vulnerability information, or it may be a different large-scale language model.

[0069] Figure 19 illustrates a prompt for obtaining false positive information regarding false positives generated by server 10. As shown in the figure, the prompt instructs the system to determine whether a vulnerability has been falsely detected (to obtain the reason for the determination and whether a false positive occurred) based on the endpoint's specification information, code, and vulnerability information. The endpoint's specification information, code, and vulnerability information are obtained from the inspection details management table 1512.

[0070] Figure 20 illustrates the response from a large-scale language model when a prompt for obtaining false positive information is entered. As shown in the figure, the reason for the determination and whether or not a false positive occurred are provided from the endpoint specification information, code, and vulnerability information. Figures 19 and 20 correspond to the case where "no" false positives are determined (in Figure 20, the value of "false_positive" is "false"). On the other hand, Figures 21 and 22 illustrate the prompt and response for obtaining false positive information when "yes" false positives are determined (in Figure 22, the value of "false_positive" is "true"). Server 10 can determine whether or not a false positive occurred based on the false positive information provided by the large-scale language model.

[0071] In the example described above, endpoint information and endpoint specification information are obtained using a large-scale language model, but this information may also be provided individually by the user. Alternatively, endpoint specification information may be obtained from a file containing web application specification information provided by the user (for example, a button for specifying such a file is placed on inspection screen 50). Furthermore, in this case, the extraction of endpoint-specific specification information from the specification information described in the file may be performed automatically using a large-scale language model.

[0072] In the example described above, the prompt for obtaining vulnerability information was automatically generated by server 10, but it could also be generated manually. In this case, the vulnerability information prompt specified by the user would be input to the large-scale language model.

[0073] The vulnerability testing server 10 according to this embodiment, as described above, obtains vulnerability information indicating vulnerabilities in the web application detected based on the source code of the web application from a large-scale language model, and determines whether or not there are false positives of the vulnerabilities based on the obtained vulnerability information. Therefore, it is possible to address false positives caused by hallucination that occurs in the large-scale language model. In other words, the server 10 supports static analysis of web applications in which the large-scale language model is appropriately utilized.

[0074] In another embodiment of the present invention, some or all of the functions of the vulnerability testing server 10 in the above-described embodiment may be realized by the cooperation of the vulnerability testing server 10 and the user terminal 30, or by the user terminal 30 alone.

[0075] The processes and procedures described herein can be implemented by software, hardware, or any combination thereof, in addition to those explicitly described herein. For example, the processes and procedures described herein can be implemented by implementing the logic corresponding to those processes and procedures on a medium such as an integrated circuit, volatile memory, non-volatile memory, or magnetic disk. Furthermore, the processes and procedures described herein can be implemented as computer programs corresponding to those processes and procedures and executed by various types of computers.

[0076] Even if a process or procedure described herein is described as being performed by a single device, software, component, or module, such process or procedure may be performed by multiple devices, multiple software programs, multiple components, and / or multiple modules. Furthermore, the software and hardware elements described herein can also be realized by integrating them into fewer components or by decomposing them into more components. [Explanation of Symbols]

[0077] 10 Vulnerability testing server 11 Computer Processors 112 Management Function Control Unit 114 Inspection Control Unit 15 Storage 151 Various Tables 1511 Inspection Management Table 1512 Inspection Details Management Table 30 User terminals 40 Server-side programs 42 Terminal-side programs 50 Examination screen

Claims

1. A system for detecting vulnerabilities in web applications, comprising one or more computer processors, The steps include inputting a first prompt to a first large-scale language model to obtain vulnerability information indicating vulnerabilities in the web application detected based on the source code of the web application, and obtaining the vulnerability information from the first large-scale language model, The steps include determining whether or not the vulnerability indicated by the vulnerability information is a false positive based on the vulnerability information, and performing the steps described above. system.

2. The vulnerability information includes the type of vulnerability and the output destination related to the vulnerability. The step of determining whether or not a false positive is performed based on the type of vulnerability and the output destination. The system according to claim 1.

3. The step of determining whether or not there is a false positive is to determine that it is a false positive if a predetermined string indicating the false positive corresponding to the type of vulnerability is included in the output destination. The system according to claim 2.

4. The predetermined string is set for each development environment of the web application. The system according to claim 3.

5. The vulnerability information includes the type of input source related to the vulnerability, The step of determining whether or not a false detection has been performed, based on the type of input source, The system according to claim 1.

6. The step of determining whether or not a false detection has occurred is to determine that a false detection has occurred if the type of the input source is a predetermined type that indicates a false detection. The system according to claim 5.

7. The vulnerability information further includes the type of vulnerability, The step of determining whether or not a false positive occurs involves determining whether or not a false positive occurs based on the type of vulnerability and the type of input source. The system according to claim 5.

8. The step of determining whether or not a false positive occurs is to determine that a false positive occurs if the combination of the type of vulnerability and the type of input source is a predetermined combination indicating a false positive. The system according to claim 7.

9. The one or more computer processors further perform the step of generating the first prompt based on the source code of the web application. The system according to claim 1.

10. The one or more computer processors described above further include: Based on the vulnerability information, a second prompt is generated to obtain false positive information regarding the false positive; The steps of inputting the second prompt to the second large-scale language model and obtaining the false positive information from the second large-scale language model are performed. The step of determining whether or not a false detection has occurred involves determining whether or not a false detection has occurred based on the false detection information. The system according to claim 1.

11. The second large-scale language model is the same large-scale language model as the first large-scale language model. The system according to claim 10.

12. The second large-scale language model is a large-scale language model that differs from the first large-scale language model. The system according to claim 10.

13. A method for detecting vulnerabilities in a web application, which is executed by one or more computers, The steps include inputting a first prompt to a first large-scale language model to obtain vulnerability information indicating vulnerabilities in the web application detected based on the source code of the web application, and obtaining the vulnerability information from the first large-scale language model, The method includes the step of determining whether or not the vulnerability indicated by the vulnerability information is a false positive, based on the vulnerability information. method.

14. A program for detecting vulnerabilities in web applications, installed on one or more computers, The steps include inputting a first prompt to a first large-scale language model to obtain vulnerability information indicating vulnerabilities in the web application detected based on the source code of the web application, and obtaining the vulnerability information from the first large-scale language model, The procedure involves performing the steps of determining whether or not the vulnerability indicated by the vulnerability information is a false positive, based on the vulnerability information. program.

Images