Security Testing System

Abstract

Grasp the appropriate operation procedures for complex websites including sequential operations and perform automatic tours efficiently and effectively. 【Solution means】Set the data of the manual of the target website 3 in the prompt, input it into the LLM 4 to create test cases, set each test case and the HTML information of the corresponding web page in the prompt, input it into the LLM 4, create operation procedure information in units of web page transitions, simulate the operations related to the web page according to the operation procedure information, obtain the URLs related to the links in the web page, register them in the tour target page list 14, and perform vulnerability inspections on each web page registered in the page list 16 created by touring each web page registered in the tour target page list 14 using the DAST method.

Description

Technical Field

[0001] The present invention relates to application testing technology, and particularly to a technology effective when applied to a security testing system for inspecting the presence or absence of vulnerabilities in a web application.

Background Art

[0002] A web application is premised on network usage, and it is very important to inspect and test for the presence or absence of vulnerabilities from a security perspective. Various tools and services for inspecting the presence or absence of vulnerabilities in web applications are available, and research and development are being carried out every day.

[0003] The security testing methods for web applications are roughly divided into SAST (Static Application Security Testing) and DAST (Dynamic Application Security Testing). In contrast to SAST that statically analyzes source code, etc., in DAST, for a running application, a pseudo attack (inspection) request is sent from the perspective of an attacker, and it is determined whether there are vulnerabilities based on changes in the behavior of the application. Therefore, in the security testing mechanism by DAST, it is necessary to specify the page to be attacked (inspected) in the web application. For this purpose, the target web application (website) is automatically or manually traversed to analyze the configuration such as links and collect page information.

[0004] As a technology related to the automatic traversal of such websites, for example, in Patent No. 7320211 (Patent Document 1), in the vulnerability inspection of a website, when automatically traversing the website, AI (Artificial Intelligence) is used to determine the inspection-required functions that require vulnerability inspection, and to determine the relevance between a plurality of executable operations that can be executed on a web page and the inspection-required functions, and to preferentially execute the operations identified as having a high relevance.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] According to the above prior art, when automatically traversing a website, among the multiple operations that can be executed on each web page, it is possible to start from the operations with high priority and perform the traversal. Therefore, for example, even in cases where there is an upper limit set by factors such as the number and hierarchy of web pages to be automatically traversed, the elapsed time, etc., it is said that it is possible to suppress the omission of inspection of important functions.

[0007] However, there are cases where a website includes operations with an order, such that the next operation cannot be performed unless a specific operation is carried out first. For example, in an EC (Electronic Commerce) website, there are cases where the order screen cannot be transitioned to unless the orderer's address is entered, or cases where the order screen cannot be transitioned to unless an item is added to the cart. In such cases, communication does not occur unless the operations are performed in the appropriate procedure. Therefore, in order to comprehensively grasp the operations that can be executed on each web page without omission, it is necessary to correctly understand the functions of the website and simulate the operations in the appropriate procedure.

[0008] Regarding this point, for example, it is difficult to comprehensively cover all patterns of when and in what order operations need to be performed and pre-define them as rules in advance. On the other hand, if all possible operations in terms of form are tried one by one, it is theoretically possible to grasp the appropriate operation procedure. However, depending on the scale and complexity of the web page, the number of combinations of operation procedures that need to be tried can become extremely large, making it difficult to simulate the operations within a realistic time. It is also possible to grasp the appropriate operation procedure by analyzing the HTML (HyperText Markup Language) of the web page. However, in websites that perform unique validation checks or websites with many operation elements, it may be difficult to grasp the appropriate operation procedure only by analyzing the HTML, and the number of trial runs for reproducing the operations may also become extremely large.

[0009] Therefore, an object of the present invention is to provide a security test system that can grasp the appropriate operation procedure for a complex web site including sequential operations and perform an efficient and effective automatic tour.

[0010] The above and other objects and novel features of the present invention will become apparent from the description of this specification and the accompanying drawings.

Means for Solving the Problems

[0011] Among the inventions disclosed in the present application, the outline of representative ones will be briefly described as follows.

[0012] A security test system, which is a representative embodiment of the present invention, is a security test system for inspecting the presence or absence of security vulnerabilities in a web application. It sets the manual data of the web application to be inspected as a prompt and inputs it into an LLM (Large Language Model) to create test cases related to the web application. Then, it sets each test case and the HTML information of the corresponding web page as a prompt and inputs it into the LLM to create operation procedure information related to the web page in units of web page transitions.

[0013] Then, it simulates the operations related to the web page according to the operation procedure information, obtains the URLs related to the links in the web page, registers them in the list of pages to be traversed, and performs vulnerability inspections on each web page registered in the list of pages created by traversing each web page registered in the list of pages to be traversed using the DAST method.

Advantages of the Invention

[0014] Among the inventions disclosed in this application, the effects obtained by representative ones are briefly described as follows.

[0015] That is, according to a representative embodiment of the present invention, in the mechanism for inspecting the vulnerabilities of a website, it is possible to grasp appropriate operation procedures and perform efficient and effective automatic traversal even for complex websites including sequential operations.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

MODE FOR CARRYING OUT THE INVENTION

[0017] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In all the drawings for explaining the embodiments, the same parts are generally denoted by the same reference numerals, and repeated explanations thereof are omitted. On the other hand, for the parts described with reference numerals in a certain drawing, they will not be shown again in the explanations of other drawings, but may be referred to with the same reference numerals. (Embodiment 1) <Overview> In the security test of a website by DAST, it is necessary to find all the publicly available URLs (Uniform Resource Locators) (that is, the URLs to be inspected) by traversing the target website. (More precisely, it is not all the generated communications, but for URLs with common input / output part processing, since the results will not change even if each URL is inspected separately, it is sufficient to inspect (traverse) at least one URL, which means all the URLs with different input / output part processing.)

[0018] To find a URL, there are static methods that analyze the HTML of a website and dynamic methods that simulate actual operations on the website and collect the generated communications. When using the latter method, as described above, there are cases where a website includes sequential operations, where the next operation cannot be performed unless a specific operation has been carried out, and communications do not occur unless the operations are performed in the appropriate order. Therefore, to simulate actual operations, it is necessary to correctly understand the functions of the website and simulate the operations in the appropriate order.

[0019] Regarding this point, for example, it is difficult to comprehensively cover all patterns of when and in what order operations need to be performed and pre-define them as rules in advance. On the other hand, if all possible operations are tried exhaustively, it is theoretically possible to grasp the appropriate operation procedures. However, depending on the scale and complexity of the web page, the number of combinations of operation procedures that need to be tried can become extremely large, making it difficult to simulate the operations within a realistic time.

[0020] Here, websites that include sequential operations as described above may include two types of web pages: those that require multiple operations to be performed in the appropriate order within a web page and those that require operations to be performed in the appropriate order across multiple web pages.

[0021] As an example of the type that requires multiple operations to be performed in the appropriate order within a web page, on a screen for registering the delivery address of a product on an e-commerce site, even if the "Register" button is pressed without entering the name and address in the name and address fields, an error message indicating that the name and address need to be entered will occur, and it will not be possible to transition to the registration completion screen (to transition to the registration completion screen, the operation of entering the name and address must be performed before pressing the "Register" button).

[0022] Also, as a type that requires operations to be performed in an appropriate order across multiple web pages, for example, when moving to the cart screen before adding an item to the cart on an e-commerce site, since the cart is empty, the "proceed to checkout" button is not displayed and it is not possible to proceed to the order screen. However, if you add an item to the cart and then move to the cart screen, since there is an item in the cart, you can press the "proceed to checkout" button and proceed to the order screen. There are cases like this.

[0023] In the security test system according to Embodiment 1 of the present invention, as a response to a type of web page that requires multiple operations to be performed in an appropriate order within the former web page, in addition to, or instead of, a method (conventional method) of obtaining an appropriate operation procedure based on rules that predefine in what cases and in what order to perform operations, a method of using a generative AI such as ChatGPT (registered trademark) or a large language model (hereinafter sometimes collectively referred to as "LLM") to obtain an operation procedure close to the content when a human operates is used to create an appropriate operation procedure (operation procedure manual).

[0024] Also, as a response to a type of web page that requires operations to be performed in an appropriate order across multiple web pages of the latter, the LLM determines whether it is inferred that there will be a change in behavior between when the target web page was visited in the past and when it is visited again currently. If it is inferred that a change has occurred, even for the same URL, the operation procedure is considered different and it is individually targeted for a visit (inspection). As will be described later, in this embodiment, by grouping and handling web pages that have different URLs but are substantially the same, the number of web pages to be visited is substantially reduced, and the efficiency of the visit and inspection processes is improved. However, for a web page that is determined to be individually targeted for re-visiting because there is a change in behavior compared to when it was visited in the past even for the same URL, it is removed from the grouping target and made the target of re-visiting.

[0025] <System Configuration> FIG. 1 is a diagram showing an outline of a configuration example of a security test system according to Embodiment 1 of the present invention. The security test system 1 is composed of, for example, a server device or a virtual server constructed on a cloud computing service, etc., to which a user terminal 2 such as a PC (Personal Computer) used by the user is connected via a network such as the Internet, VPN (Virtual Private Network), or LAN (Local Area Network) not shown, and accesses it using a Web browser or a dedicated application not shown.

[0026] The security test system 1 realizes various functions related to the implementation of security tests by executing, for example, an OS (Operating System), a DBMS (DataBase Management System), middleware such as a Web server program, and software operating thereon, which are expanded from a recording device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) onto a memory by a CPU (Central Processing Unit) not shown. This security test system 1 has, for example, each part such as an inspection management unit 11, a tour processing unit 12, and an inspection execution unit 13 implemented by software. It also has each data store such as a tour target page list 14, an operation procedure manual creation rule 15, a page list 16, and an inspection result 17 implemented by a database or a file table, etc.

[0027] The inspection management unit 11 has, for example, functions such as a management function related to vulnerability inspection and a user interface function, which include receiving input and setting of information related to a target Web site 3 to be inspected from the user via the user terminal 2, receiving input of various settings and instructions related to the implementation of the inspection, and creating a report based on the inspection result 17, which is the result of the inspection, and presenting it to the user via the user terminal 2.

[0028] The loop processing unit 12 automatically loops through the target web site 3 specified by the user, for example, registers the detected web pages in the loop target page list 14, and has a function of creating a page list 16 including the web pages to be finally inspected. At this time, as described above, for the web pages including the sequential operations included in the target web site 3, the appropriate operation procedures are grasped using the operation procedure creation rule 15 and the LLM4, and the operation procedure manual for the target web page is created. The details of the processing content of the loop processing unit 12 will be described later.

[0029] The inspection execution unit 13 has a function of performing vulnerability inspection on each web page registered in the page list 16 by the DAST method, for example. That is, for each target web page, when sending a request to the target web site 3, it is sent as a pseudo attack request by entering an illegal value in the form. Then, it analyzes whether the response from the target web site 3 is a different response from normal, and determines the presence or absence of vulnerability from, for example, the ratio of response difference, error message, etc., and records the determination result in the inspection result 17. Note that the normal response is a response to an access by a normal operation without changing the request content (pseudo attack). For example, before performing a pseudo attack, a normal response (including an error response) may be obtained by sending a request equivalent to that during automatic looping, or the response obtained during automatic looping may be used.

[0030] <Flow of processing> Figure 2 is a flowchart showing an overview of the flow of automatic circuit processing by the circuit processing unit 12 in Embodiment 1 of the present invention. When starting the automatic circuit processing, first, the circuit processing unit 12 acquires the start page of the target Web site 3 specified by the user and registers it in the circuit target page list 14 (S01). Then, it is determined whether or not it meets the end condition of the automatic circuit (S02). For example, if any of the following conditions are met: the circuit target page list 14 is empty, the number of pages visited has reached a preset upper limit value, or the elapsed time since the start of the circuit has reached a preset timeout time, it is determined that the end condition is met (Yes in step S02), and the automatic circuit processing is terminated.

[0031] If it does not meet the end condition (No in step S02), based on the circuit target page list 14, a Web page (URL) to be circuited is determined from those not yet circuited (S03), the page of the target Web site 3 is accessed, and the page is analyzed (S04).

[0032] Figure 3 is a diagram showing an overview of a specific example of the circuit target page list 14 in Embodiment 1 of the present invention. Among the URLs in the list, those with a circuit status of "circuited" are already circuited and are not targets for circuiting. Also, those with a circuit status of "similar URL to ○○" are not targets for circuiting because they are similar to the "circuited" URLs (as described above, in this embodiment, by grouping and handling Web pages that have different URLs but are substantially the same, the reduction of Web pages to be circuited is achieved). In the example of Figure 3, the next URL to be circuited is, for example, the "No. 6" URL as the one found first among those with a circuit status of "not yet circuited".

[0033] Returning to FIG. 2, after statically analyzing the target web page (S04), a rule-based operation procedure manual is created for appropriate operation procedures (S05). In the present embodiment, the operation procedure manual refers to data (for convenience, referred to as "operation procedure 'book'") that includes information on one or more operation contents in the target web page (URL). There may be a plurality of operation procedure manuals for one web page. When the execution of the operation content of one operation procedure manual is completed, the target web page is reloaded before moving on to the execution of the operation of the next operation procedure manual so that the operation contents do not interfere between the operation procedure manuals during simulation.

[0034] The method for creating a rule-based operation procedure manual is as follows, for example. First, by analyzing the HTML of the target web page in step S04 of FIG. 2, based on event listeners and the like registered in the elements, the executable operations are grasped. At this time, for example, operations on elements to be operated on at one time, such as elements surrounded by the HTML Form tag, are grouped.

[0035] Here, the "elements to be operated on at one time" are not limited to elements within the Form tag. For example, elements that display a drop-down menu when clicked, or elements that require operations over multiple times, such as drag & drop, are targeted. Therefore, not only multiple operations on one element, but also operations on different multiple elements may be grouped into one. These rules are registered in operation procedure manual creation rule 15. After that, specific operation contents (selection values, input contents, etc.) are determined for each element, and an operation procedure manual is created.

[0036] FIG. 4 is a diagram showing an overview of an example of a web page in Embodiment 1 of the present invention. Here, on the "Address Registration" screen below the "Top" screen, it shows that each element of "Postal Code", "Prefecture", "City", "Return", and "Register" surrounded by the Form tag is arranged. The operations executable on this screen can be grasped as follows: for example, since "Top" is an a tag, it can be clicked; since "Postal Code" is an input tag, input is possible; since "Prefecture" is a select tag, the selected value can be changed; since "City" is an input tag, input is possible; and since the "Return" and "Register" buttons each have a click event set, they can be clicked.

[0037] Among these elements, the operations related to "Top" and "Return" are not subject to grouping, while the operations related to "Postal Code", "Prefecture", "City", and "Register", which are elements within the Form tag, are grouped together. Note that although "Return" is an operation within the Form tag, since it is an operation that causes a screen transition, it is to be separated as an operation procedure manual.

[0038] After that, specific operation contents are determined for each element and an operation procedure manual is created. In the example of FIG. 4, for example, "Operation Procedure Manual 1" is created with the content of clicking the "Top" link. Also, "Operation Procedure Manual 2" is created with the content of entering "123-4567" in the "Postal Code", selecting "Tokyo" as the "Prefecture", entering "Chuo-ku" in the "City", and clicking the "Register" button. Also, "Operation Procedure Manual 3" is created with the content of clicking the "Return" button. Note that specific set values, etc. can be determined by appropriate methods such as registering patterns in Operation Procedure Manual Creation Rule 15.

[0039] Returning to FIG. 2, in addition to creating an operation procedure manual based on rules (S05), in this embodiment, LLM4 is used to create an operation procedure manual close to the content when a human operates (S06).

[0040] FIG. 5 is a diagram showing an overview of another example of a web page in Embodiment 1 of the present invention. Here, an example of a screen for registering a delivery address on an EC site is shown, and the specification is such that the delivery address cannot be registered unless the required fields (* mark) of name, address, and telephone number are input in advance. Note that the Form tag is not used here.

[0041] Here, in the creation of the rule-based operation procedure manual in step S05 of FIG. 2, for a plurality of operations corresponding to the rules set in operation procedure creation rule 15 as "elements to be operated at one time" such as elements within the Form tag, they can be grouped into one operation procedure manual as described above.

[0042] On the other hand, depending on the website, there are cases where, as in the example of FIG. 5, the HTML standard Form tag is not used and it is implemented with a unique tag, and it is not possible to correctly recognize the "elements to be operated at one time", and the operation procedure manual may be separated. For example, according to the creation of the rule-based operation procedure manual in step S05 of FIG. 2 for the example of the web page in FIG. 5, for example, "Operation Procedure Manual 1" is created with the content of entering "○○" in the "surname" of "name", "Operation Procedure Manual 2" is created with the content of entering "△△" in the "given name" of "name", "Operation Procedure Manual 3" is created with the content of entering "●●" in the "sei" of "name (kana)",... (omitted)... and "Operation Procedure Manual X" is created with the content of clicking the "register" button, and the operation procedure manuals are separated in this way.

[0043] As a result of the separation of the operation procedure manuals, for example, even if the "surname" of "name" is input in "Operation Procedure Manual 1", since the target web page is reloaded before "Operation Procedure Manual 2" is executed, the operation of entering the "given name" starts again in the state where the "surname" is not input, and there is a problem that the content to be operated at one time (input to the required fields in the example of FIG. 5) cannot be correctly performed.

[0044] On the other hand, in creating the operation procedure manual using the LLM4 in step S06, when implementing a function equivalent to the Form tag with unique tags, or when a complex operation that requires an appropriate operation procedure for the "elements to be operated at once" cannot be created in creating the operation procedure manual based on rules in step S05, etc., an operation procedure manual close to the content when operated by a human is created. Thereby, for example, it is possible to create "Operation Procedure Manual 1" with the content of entering "○○" in the "Surname" of the "Name", which is a required field, entering "△△" in the "Given Name", entering "●●" in the "Sei" of the "Name in Kana",... (omitted), and clicking the "Register" button.

[0045] FIG. 6 is a diagram showing an overview of an example of creating an operation procedure manual using the LLM4 in Embodiment 1 of the present invention (step S06 in FIG. 2). Here, by setting the HTML content of the screen as in the example of FIG. 5 in the template of the prompt, a prompt for causing the LLM4 to create an operation procedure manual as shown in the upper figure of FIG. 6 is obtained. In the example of FIG. 6, although the HTML information of the target page is input, information obtained by a screen capture can also be used. Also, it is not always necessary to input the entire text of the HTML, and it may be input after performing trimming processing to remove unnecessary information.

[0046] By inputting the operation procedure manual prompt as shown in the upper figure into the LLM4, an operation procedure manual (operation procedure information in JSON (JavaScript Object Notation) format, etc.) as shown in the lower figure can be obtained. Then, by simulating the operation with the obtained operation procedure manual, for example, it is highly likely that requests that could not be found in the simulation of the operation using the operation procedure manual obtained based on rules (for example, a new address registration request in the screen example of FIG. 5) can be found.

[0047] In addition, in this embodiment, by performing both the creation of an operation procedure manual based on conventional rules (step S05 in FIG. 2) and the creation of an operation procedure manual using LLM4 (step S06 in FIG. 2), the comprehensiveness of the operations in the operation procedure manual is efficiently improved. However, it may be sufficient to create only the operation procedure manual using LLM4. Also, in the example of FIG. 2, it is described that the creation of the operation procedure manual based on rules → the creation of the operation procedure manual using LLM4 is executed in this order, but the reverse order or parallel execution may also be possible.

[0048] Returning to FIG. 2, then, the screen operation of the target web page is simulated according to the operation procedure manuals created in steps S05 and S06 (S07). And it is determined whether or not the screen display has changed due to the operation (S08). When the screen has changed (Yes in step S08), using the information of the changed screen (for example, DOM (Document Object Model), screen capture, etc.), return to step S05 to repeat the creation of the operation procedure manual (by rules and LLM4) and the simulation of the screen operation according to the created operation procedure manual.

[0049] On the other hand, when there is no change in the screen display due to the operation (No in step S08), a list of pages to be traversed 14 is created from the list of URLs found within the target web page (S09). At this time, if the target web page has already been traversed, it is highly likely that the URLs found will be the same even if the web page is traversed again. Therefore, in this embodiment, in order to reduce unnecessary traversals, it is determined based on rules whether the found URL overlaps (is substantially the same) with an existing web page that has been traversed in the past, and the overlapping ones are grouped and excluded from the traversal target (S10).

[0050] However, even if the URL is substantially the same as an existing web page that has been visited in the past, if the preconditions (operations performed) for reaching that URL are different, the behavior on the target web page may change, and in that case, it is necessary to visit it again. Therefore, in this embodiment, LLM4 is used to determine whether the behavior of the target web page is likely to change compared to the previous visit, and if it is inferred that a change will occur, it is excluded from the grouping target and made the target of re-visiting (S11).

[0051] FIG. 7 is a diagram showing an overview of another specific example of the page list 14 to be visited in Embodiment 1 of the present invention. In the example of FIG. 7, the "No." column shows the order in which each URL was found, and the "parent" column shows the No. of the parent screen where the link to the target URL was found. That is, it shows that the cart screen of "No. 2" and the product list screen of "No. 3" were found by analyzing the top screen of "No. 1" in step S09 of FIG. 2. And by analyzing the screen of "No. 3", the screen for adding the product of "No. 4" to the cart was found, and by analyzing the screen of "No. 4", it shows that the cart screen of "No. 5" was found.

[0052] At this time, in the process of step S10 in FIG. 2, since the cart screen of "No. 5" is similar (substantially the same) to the cart screen of "No. 2" and has already been visited, it is grouped with the screen of "No. 2" and determined not to require a visit. However, since the cart screen of "No. 2" shows the initial cart and there are no products added to the cart and it is empty, it is not possible to proceed to the checkout screen. On the other hand, in the cart screen of "No. 5", since it has transitioned after a product was added to the cart on the screen of "No. 4", it is expected that a new operation (URL) to proceed to the checkout can be found by re-visiting (that is, these web pages include sequential operations across multiple web pages).

[0053] Therefore, in the present embodiment, in the process of step S11 in FIG. 2, among the list of URLs grouped based on rules in step S10, based on the preconditions (operations) for reaching the target URL, it is inferred whether there is a change in the behavior of the web page between the past visit (the screen of "No. 2") and the current visit (the screen of "No. 5"), that is, whether a re-visit is necessary, using LLM4 to make a judgment.

[0054] FIG. 8 and FIG. 9 are diagrams showing an overview of an example of determining a web page that requires re-visit using LLM4 in Embodiment 1 of the present invention (step S11 in FIG. 2). Here, in the content of the page list 14 to be visited as in the example of FIG. 7, by setting the already found (visited) URLs and the URLs found in this visit in the template of the prompt, a prompt for causing LLM4 to determine URLs that require re-visit as shown in FIG. 8 is obtained.

[0055] Here, in the data of the visited URLs, in addition to the properties of the screen ID ("id") and the parent ID ("parentId"), the summary ("summary") property contains a sentence indicating the summary of the operation. It may include not only the summary of the operation performed in this visit but also all the operations performed since the start of the visit. This is because, for example, in the example of FIG. 7 above, after adding a product to the cart on the screen of "No. 4", directly moving to the cart screen of "No. 5", but if it is a website with a structure where the cart screen cannot be moved to without returning to the top screen of "No. 1" once after adding the product to the cart, it is impossible to appropriately judge whether a re-visit of the cart screen of "No. 5" is necessary without tracing the process of moving to the top screen.

[0056] By inputting a prompt as shown in the example of FIG. 8 into LLM4, an output as shown in FIG. 9 can be obtained. This output includes information on URLs that require recirculation. In this embodiment, it is determined whether recirculation is necessary using LLM4 as described above. However, in cases where the number of URLs to be determined is large, for efficiency, for some or all of them, instead of determining using LLM4, it may be determined based on rules.

[0057] Return to FIG. 2, and register the URLs determined to require recirculation in step S11 in the page list 14 to be circulated (or, if the target URL is already registered in the page list 14 to be circulated, change its circulation status) (S12). Then, return to step S02 and repeat the subsequent processing until the end condition of automatic circulation is met (Yes in step S02).

[0058] <Data Configuration> FIG. 10 is a diagram showing an overview of an example of the data configuration of the page list 14 to be circulated in Embodiment 1 of the present invention. The page list 14 to be circulated is a table that holds a list of pages (URLs) to be the target of (or that have been) automatic circulation. For example, it has items such as a page ID to be circulated, a parent page ID, a circulation request ID, a URL, a URL summary, a method, a request, a response, a capture, a page circulation status, an operation content, and an operation summary.

[0059] The item of the page ID to be circulated holds information on an ID that uniquely identifies the page to be circulated. The item of the parent page ID holds information on an ID that identifies the parent page where the page to be circulated was found. Also, the item of the circulation request ID holds information on an ID that uniquely identifies a circulation request for the target website 3 that includes the target page. By making it possible to identify individually for each circulation request, for example, information on the status as a whole (circulation pending, in circulation, circulation completed, etc.) for each circulation can be managed separately by a management table (not shown) or the like.

[0060] The URL item holds information on the URL of the target page. Also, the URL summary item holds a sentence (e.g., "Top screen" or "Add product to cart") indicating an overview of what kind of page the target page is, obtained as a result of the analysis of the target page. Each of the method, request, response, and capture items holds the method (GET, POST, PUT, etc.) of the target page, information on the request to the target page and the response thereto, and information on the screen capture of the target page. Also, the page traversal status item holds information on the traversal status (awaiting traversal, traversing, traversal completed, etc.) of the target page.

[0061] The operation details item holds the specific operation details (e.g., [{"name":"open","url":"https: / / example.com / top / "},{"name":"clickAt","target":" / / a[.='リンク']"}]) required to reach the URL of the target page. Also, the operation summary item holds a sentence (e.g., "Open the top screen and click [link]") indicating an overview of the above operation details.

[0062] As described above, according to the security test system 1 which is Embodiment 1 of the present invention, in addition to the conventional method of obtaining an appropriate operation procedure based on rules that pre-define in what cases and in what order to perform operations as a response to a type of web page that requires performing multiple operations in an appropriate order within the web page, by using the LLM4 and the method of obtaining an operation procedure close to the content when a human operates, an appropriate operation procedure manual can be created to simulate the screen operation.

[0063] In addition, as a response to web pages of a type that requires operations to be performed in an appropriate order across multiple web pages, LLM4 determines whether it is presumed that there will be a change in behavior between when the target web page was visited in the past and when it is visited again currently. If it is presumed that a change has occurred, even if the URL is the same, the operation procedure is considered different, and it is removed from the grouping target and made the target of re - visitation. By doing these, it is possible to grasp the appropriate operation procedure for the target website 3 including operations with orderliness and perform automatic visitation efficiently and effectively. (Embodiment 2) <Overview> According to the above - described Embodiment 1, for a website including operations with orderliness, by analyzing the HTML of the web page and using LLM4, an appropriate operation procedure document can be created to simulate screen operations and perform automatic visitation effectively and efficiently.

[0064] However, for websites that perform unique validation checks or websites with a large number of operation elements, it may be difficult to create an appropriate operation procedure document only by analyzing HTML, and the number of trial times to reproduce the operations may also become extremely large. For example, in an accounting processing application, when a user applies for expense settlement, they cannot transition to the application page without specifying the data of receipts and invoices. However, to specify the data of receipts and invoices, they need to transition to another page and select from a list. Furthermore, the data of receipts and invoices displayed in the list needs to be input and newly registered in another page in advance. Such a case can be an example of a website that performs unique validation checks and has a large number of operation elements.

[0065] Even if it is possible to comprehensively navigate a website according to the operation manual, for example, in a website with a large number of URLs to be navigated, such as a website that includes web pages with a large number of links related to various functions and menus, it is more efficient to group these URLs in a business - meaningful unit, prioritize each unit, and perform vulnerability checks rather than equally and sequentially checking all of these URLs one by one. However, it is difficult to do this from the list of URLs to be navigated.

[0066] In the security test system according to Embodiment 2 of the present invention, for a website that performs its own validation check or a website with many operation elements, from the manual of the website (a document that explains the functions, usage methods, operation methods, etc. of the website for users), the LLM4 creates test cases (operation instructions in this embodiment) that are close to the content when operated by a human. Then, when creating an operation manual from the HTML of the website by the LLM4 using the same method as in Embodiment 1 above, the above - mentioned test cases are also input to the LLM4 to create an appropriate operation manual for a complex website.

[0067] Also, for a website with a large number of URLs to be navigated, by grouping these into business - meaningful units, attaching function names that can be understood by users, and outputting them as scenarios, it becomes easier for users to preferentially check specific operations and functions.

[0068] <System Configuration> FIG. 11 is a diagram showing an overview of a configuration example of the security test system according to Embodiment 2 of the present invention. The configuration of the security test system 1 in this embodiment is basically the same as the configuration example of the security test system 1 shown in FIG. 1 in Embodiment 1 described above. However, it is assumed that the target website 3 to be subjected to vulnerability testing is a complex website such as a website that performs its own validation check or a website with many operation elements.

[0069] 1 in that the security test system 1 of this embodiment further includes a test case creation unit 18 implemented by software. The test case creation unit 18 has a function of receiving an input of the manual 5 for the target website 3 from a user and creating a test case (operation instruction) similar to the content of a human operation by the LLM 4. When the crawling processor 12 creates an operation manual from the HTML of the target website 3 by the LLM 4, the crawling processor 12 can create an appropriate operation manual even for a complex target website 3 by inputting the test case created by the test case creation unit 18 to the LLM 4. The crawling processor 12 also has a function of outputting the above-mentioned scenario to the crawl target page list 14.

[0070] <Processing flow> Fig. 12 is a flowchart outlining an example of the flow of an operating procedure manual creation process by the patrol processing unit 12 in the second embodiment of the present invention. The process here corresponds to the operating procedure manual creation process by the LLM 4 in step S06 in the automatic patrol process shown in Fig. 2 in the above-mentioned first embodiment. In the operating procedure manual creation process, first, the test case creation unit 18 accepts input of the manual 5 from the user (S21), and inputs the manual 5 into the LLM 4 to create a test case (operating instruction) (S22).

[0071] Figures 13 to 15 are diagrams outlining an example of creating test cases from manual 5 by LLM4 in Embodiment 2 of the present invention. In the examples of Figures 13 to 14, examples of prompts for causing LLM4 to create test cases from the manual are shown, and examples of output of test cases in JSON format created for the manual input by the user are shown. Such a prompt itself may be prepared in advance, or it may be created based on a template of a prompt prepared in advance according to the project. Note that the format of the manual to be input can be various ones such as, for example, a PDF file, an HTML file, a screen capture, etc., and is not particularly limited. Also, it is not always necessary to input the entire text of the manual, and it may be input after performing trimming processing to remove unnecessary information.

[0072] By inputting a prompt as shown in the examples of Figures 13 to 14 into LLM4, a test case as shown in Figure 15 can be obtained as output. In this test case, test cases in JSON format are output for each item described in the manual (in the example in the figure, "Schedule", "Task Management", "Contact", "File Sharing"). In the example in the figure, for example, operations such as "Click on 'Schedule' on the schedule app screen", "Click on 'Add Schedule'", and "Input necessary information and click 'Register'" each become test cases. The test case may include a plurality of screen operations. Note that in this embodiment, for the unit of processing / function in each item in the manual (the unit surrounded by the dashed line in the figure), the test cases are grouped into business - meaningful units (scenarios) by the processing described later.

[0073] Returning to FIG. 12, when the test cases are created, the loop processing unit 12 starts a loop process that repeats for all the test cases obtained in step S22 (S23). In the loop process, first, access is made to the first web page such as the top page to obtain the HTML of the page (S24). Then, based on the obtained HTML and the test case to be processed, an operation procedure manual is created by the LLM4 until the next web page is transitioned (S25).

[0074] After that, the content of the created operation procedure manual is executed to transition to the next page, and the HTML of the web page of the transition destination is obtained (S26). Then, based on the content of the obtained HTML, etc., it is determined whether the operation here is the final operation (S27). If it is the final operation (Yes in S27), the processing for the target test case is terminated, and the processing for the next test case is moved to (S27, S23). If it is not the final operation (No in S27), return to step S25, and repeat the creation of the operation procedure manual based on the obtained HTML and the test case to be processed. The created operation procedure manual may be handled separately from the already created operation procedure manual, or may be in the form of appending to the already created operation procedure manual.

[0075] FIGS. 16 and 17 are diagrams showing an outline of an example of creating an operation procedure manual from HTML and a test case in Embodiment 2 of the present invention. On the left side of FIG. 16, an example of the obtained HTML is shown, and on the upper right side, the screen operations (the portions surrounded by broken lines) in the test case to be processed are shown. And an example of the operation procedure manual for executing the test case created by the LLM4 with these contents as input is shown in the lower right figure.

[0076] Perform screen operations based on the operation procedure document created in the example of FIG. 16 to transition the screen, and obtain the HTML of the destination Web page. An example of the obtained HTML is shown on the left side of FIG. 17, and the screen operations (the parts surrounded by broken lines) in the test case to be processed are shown in the upper part on the right side. And an example of the operation procedure document for executing the test case created by LLM4 with these contents as input is shown in the figure in the lower part on the right side. In this way, the operation procedure document is sequentially created in units of one screen transition.

[0077] FIGS. 18 to 20 are diagrams showing an overview of an example of creating an operation procedure document by LLM4 in Embodiment 2 of the present invention. FIGS. 18 to 19 show examples of prompts for causing LLM4 to create an operation procedure document, and also describe the control of the entire loop process of steps S23 to S28 in FIG. 12. Such a prompt can be created, for example, by setting HTML or a test case in a template in which the method of creating an operation procedure document, the JSON format of the output, constraint conditions, etc. are described. Note that the information of the Web page to be input can be various types such as, for example, the response HTML and the screen capture, and is not particularly limited. Also, it is not always necessary to input the entire text of the HTML, and it may be input after performing trimming processing to remove unnecessary information.

[0078] By inputting a prompt as shown in the example of FIGS. 18 to 19 into LLM4, an operation procedure document as shown in FIG. 20 (only a part is shown in the example in the figure) can be obtained as an output.

[0079] Returning to Fig. 12, when the creation of the operation procedure manuals for all test cases is completed (S28), they are output as scenarios grouped into business - meaningful units for the test cases (S29), and the operation procedure manual creation process ends. For example, as shown in the example of Fig. 15, for test cases represented in JSON format, scenarios can be created by, for example, targeting JSON data at a predetermined hierarchy (the unit surrounded by the dashed line in the figure), summarizing the operation procedure manuals created for the test cases included in each JSON data, and attaching the key value (label) of the JSON data as the name to this summary.

[0080] Note that in this embodiment, as described above, test cases are created from the manual of the target website 3 input by the user, and the operation procedure manuals are obtained by getting the HTML for the obtained test cases. However, for example, when information equivalent to test cases can be directly obtained according to conditions such as the description format of the manual, it is also possible to directly create the operation procedure manuals by taking the manual as input and getting the HTML corresponding to the operation items in the manual, and it is also possible to create scenarios.

[0081] As described above, according to the security test system 1 which is the second embodiment of the present invention, for websites that perform unique validation checks or complex websites with many operation elements, test cases close to the content when operated by humans can be created from the manual of the website by the LLM4. Also, for websites with a large number of URLs to be traversed, by grouping them into business - meaningful units and outputting them as scenarios, it becomes easier for the user to preferentially inspect specific operations or functions.

[0082] As described above, the invention made by the present inventor has been specifically described based on the embodiments. However, the present invention is not limited to the above-described embodiments, and it goes without saying that various modifications can be made without departing from the gist thereof. Further, the above embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Also, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. Further, it is possible to add, delete, or replace other configurations for a part of the configuration of each embodiment.

[0083] In addition, each of the above configurations, functions, processing units, processing means, etc. may be realized in hardware by designing a part or all of them, for example, by using an integrated circuit. Also, each of the above configurations, functions, etc. may be realized in software by a processor interpreting and executing a program for realizing each function. Information such as programs, tables, files, etc. for realizing each function can be stored in a recording device such as a memory, a hard disk, an SSD, or a recording medium such as an IC card, an SD card, or a DVD.

[0084] Also, in each of the above figures, control lines and information lines are shown as those considered necessary for explanation, and do not necessarily show all the control lines and information lines in actual implementation. In practice, it may be considered that almost all configurations are interconnected.

Industrial Applicability

[0085] The present invention can be used in a security test system for inspecting the presence or absence of vulnerabilities in web applications.

Explanation of Signs

[0086] 1... Security test system, 2... User terminal, 3... Target web site, 4... LLM, 5... Manual 11…Inspection Management Department, 12…Patrol Processing Department, 13…Inspection Execution Department, 14…Patrol Target Page List, 15…Operation Procedure Document Creation Rule, 16…Page List, 17…Inspection Result, 18…Test Case Creation Department

Claims

1. A security test system for testing whether or not there is a security vulnerability in a web application, comprising: A manual data of the Web application to be inspected is set as a prompt and input to an LLM (Large Scale Language Model) to generate a test case consisting of a textual description of the operation contents related to the Web application; inputting information of the test cases and the corresponding Web pages in HTML (HyperText Markup Language) or screen captures into the LLM as prompts, and creating operation procedure information related to the Web pages, the operation procedure information including commands for realizing screen transitions related to the operation contents in the test cases, in units of Web page transitions; simulating an operation related to the Web page in accordance with the operation procedure information, acquiring a URL related to a link in the Web page, and registering the URL in a list of pages to be visited; A security testing system that performs vulnerability testing using a Dynamic Application Security Testing (DAST) method for each Web page registered in a page list created by crawling each Web page registered in the crawl target page list.

2. 2. The security test system according to claim 1, A security test system that compiles the operation procedure information related to the test cases of a predetermined unit, sets label information attached to the test cases of the predetermined unit, and outputs the result as a scenario.

3. A security test system for testing whether or not there is a security vulnerability in a web application, comprising: manual data of the Web application to be inspected and HTML (HyperText Markup Language) or screen capture information of a Web page corresponding to an operation item in the manual are set as prompts and input to a large-scale language model (LLM), and operation procedure information related to the Web page is created in units of transitions of the Web page; simulating an operation related to the Web page in accordance with the operation procedure information, acquiring a URL related to a link in the Web page, and registering the URL in a list of pages to be visited; A security testing system that performs vulnerability testing using a Dynamic Application Security Testing (DAST) method for each Web page registered in a page list created by crawling each Web page registered in the crawl target page list.

Images