Web application fuzz testing method based on multimodal large model assisted web crawler

Abstract

The present invention provides a web application fuzz testing method based on a multimodal large model assisted web crawler, comprising the following steps: using LLM to collect web application page information, extracting association information between web pages, and constructing a state machine based on the association information between web pages to guide the order of crawling web pages; using MLLM to take screenshots of web pages and obtain the HTML content of the web pages, generating a natural language description of the task to be completed by the user on the page and the operations required to complete the task; using LLM to convert the natural language description into test code in sequence, and judging whether the test code successfully completes the task based on the state change of the web page before and after execution. If successful, the corresponding form content and operation sequence are generated to generate a valid HTTP request. The present invention has the advantages of simple implementation, low cost, high testing efficiency and coverage.

Description

Technical Field

[0001] The present invention relates to the technical field of fuzz testing and automated testing, and in particular to a Web application fuzz testing method based on a multimodal large model assisted web crawler. Background Art

[0002] With the increasing number of web applications and the diversification of supported frameworks, automated and efficient web vulnerability detection technology is crucial. Fuzz testing is an automated testing technique that automatically or semi-automatically generates random data based on certain rules. This data is then fed into the entry point of a dynamically running program under test, while simultaneously monitoring the program for any anomalies. Fuzz testing is currently widely used to detect web application vulnerabilities. Fuzz testing of web applications is an automated testing method that searches for web vulnerabilities through a typical web interface and numerous HTTP requests. Fuzz testing can be performed by using crawlers to interact with running web application instances to acquire prior knowledge.

[0003] Vulnerability detection for web applications requires in-depth inspection of each web page. Web crawlers can automatically access the pages of web applications and discover the URLs, HTML form fields, and other input points therein. This information is crucial for further vulnerability detection. For example, for URLs that the crawler has not crawled, the test cases generated by subsequent fuzz testing will not be able to cover the content of these URLs. However, traditional web crawlers usually adopt a simple depth-first or breadth-first strategy, which can only obtain the content of the page and cannot discover the status and association information between pages (such as dependencies). Therefore, in the fuzz testing of traditional web applications, the web page information crawled by the web crawler is directly used. The actual generated form content is not related to the page, which may result in the generation of a large number of invalid operation sequences, and then a large number of invalid HTTP requests, resulting in low test efficiency and coverage.

[0004] Some practitioners have proposed using gray-box coverage-guided mutation fuzz testing to detect SQL and command-line injection vulnerabilities in web applications. This involves running a black-box crawler to automatically discover HTTP requests and parameters, and using the Gremlins testing tool to randomly trigger user input events (such as clicks, form filling, scrolling, and typing). However, this testing method randomly triggers user input, meaning that the form content is randomly filled in. As a result, the generated form content is often illogical and unrelated to the page content. The randomly generated operation sequence also results in a large number of repeated and useless requests or untriggerable requests, resulting in low efficiency and coverage.

[0005] Another approach proposed by some practitioners is to combine multiple machine learning methods to obtain "interaction intent" consistent with human cognition. This information can then be used to simulate the "understand-operate-check" testing and verification process that testers perform on client products. By selecting three modalities of page information: image information, render tree information, and text information, the goal of understanding UI interaction intent is achieved, ultimately enabling correct interaction across numerous form pages. However, this approach is only applicable to mobile applications, not web applications, and relies on multimodal self-attention deep learning. Integrating multimodal information requires complex model design and training, significantly increasing the difficulty of development and debugging. It also requires a large amount of labeled data to support the training of multimodal deep learning models. Summary of the Invention

[0006] The technical problem to be solved by the present invention is: In response to the above-mentioned problems in the prior art, a method and system for fuzzy testing of Web applications based on a multimodal large model assisted web crawler is provided with high efficiency and simple operation.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] A web application fuzz testing method based on a multimodal large model assisted web crawler includes the following steps:

[0009] Using an LLM (Large Language Model) to collect web application page information and extract association information between web pages, a state machine is constructed based on the association information between web pages to guide the order of crawling web pages. The page information includes web page functions and web page URL (Uniform Resource Locator) information. The association relationships include any one or more of state relationships, interaction logic, dependency relationships, and navigation relationships. The state relationship refers to the relationship between pages that may be affected by state or content. The interaction logic refers to the dynamic change or page redirection caused by user interaction between pages. The dependency relationship refers to the dependency relationship between page content or data. The navigation relationship refers to the relationship of the user's browsing path formed by connecting different pages.

[0010] Use MLLM (Multimodal Large Language Model) to take a screenshot of a web page and obtain the HTML (hypertext markup language) content of the web page. Generate a natural language description of the task based on the obtained HTML content, which is used to describe the task that the user needs to complete on the web page and the operations required to complete the task;

[0011] The natural language description generated by MLLM is split according to tasks to form natural language descriptions of multiple tasks. LLM is used to convert the natural language description of each task into test code in turn. The generated test code is judged whether it successfully completes the corresponding task based on the state changes of the web pages before and after executing the test code. If it is successfully completed, the form content and operation sequence corresponding to the task are generated according to the association information between the web pages to guide the generation of valid HTTP requests.

[0012] Furthermore, the use of LLM to collect web application page information and extract association information between web pages includes:

[0013] Define a first prompt word template to prompt the LLM to collect web page functions, web page URL information, and extract association information between web pages;

[0014] Multiple rounds of dialogue are conducted using the first prompt word template. In each round of dialogue, the LLM is first prompted to output the web page function and web page URL information in a specified format, and then the LLM is prompted to output the association information between the web pages in a specified format. During each round of dialogue, the answer to the previous round of dialogue is provided to the LLM, and the LLM conducts the next round of dialogue based on the answer to the previous round of dialogue. After completing multiple rounds of dialogue, the final web page function, web page URL information, and association information between the web pages are obtained.

[0015] Furthermore, the method of taking a screenshot of a web page using MLLM and obtaining the HTML content of the web page includes:

[0016] A second prompt word template is defined to prompt the MLLM to generate a natural language description of the task based on the HTML content of the web page;

[0017] In the second prompt word template, MLLM is first prompted to generate code to use puppeteer to simulate operations to complete the screenshot of the web page and obtain the HTML content, and then prompted to output natural language to describe the tasks that the user needs to complete on the current web page and the operations required to complete the tasks.

[0018] Furthermore, the second prompt word template also includes adding an attribute element-id to each element of the HTML content as a unique identifier of each element.

[0019] Furthermore, after using puppeteer to simulate the operation to complete the screenshot of the web page and obtain the HTML content, it also includes removing invisible elements from the obtained HTML content, removing attributes in the elements that are useless for understanding the elements, and extracting interactive elements, retaining attributes that are effective for inferring the functions of the elements, to obtain the final HTML content.

[0020] Furthermore, by defining a third prompt word template, each task uses the third prompt word template to conduct multiple rounds of dialogue. In each round of dialogue, the LLM is first prompted to obtain a natural language description of a task and convert it into test code. Then, the LLM is prompted to judge whether the generated test code successfully completes the corresponding task based on the state changes of the Web pages before and after executing the test code. If successful, the LLM is prompted to output the form content and operation sequence corresponding to the task based on the association information between the Web pages, and continue to obtain the natural language description of the next task to generate the corresponding form content and operation sequence until all tasks are completed.

[0021] Furthermore, judging whether the generated test code successfully completes the corresponding task based on the state change of the web page before and after executing the test code includes:

[0022] Submit the page content before and after executing the generated test code to LLM, and use LLM to infer page state changes;

[0023] It is determined whether the currently generated test code has successfully completed the task according to the page status change, wherein if the page changes, it is determined that the task has been successfully completed; otherwise, it is determined that the task has not been successfully completed.

[0024] Furthermore, if it is determined that the currently generated test code successfully completes the task, the natural language description of the next task is continued to be obtained to generate the corresponding form content and operation sequence. If it is determined that the currently generated test code does not successfully complete the task, the LLM is asked again to generate new executable test code for the current task. When the number of questions reaches the preset number of questions, the current task is abandoned and the natural language description of the next task is continued to be selected to generate the corresponding form content and operation sequence.

[0025] A web application fuzz testing system based on a multimodal large model assisted web crawler includes a processor and a memory connected to each other, the memory is used to store computer programs, and the processor is programmed or configured to execute the above-mentioned web application fuzz testing method based on a multimodal large model assisted web crawler.

[0026] A computer-readable storage medium stores a computer program / instruction, which is programmed or configured to execute the above-mentioned Web application fuzz testing method based on a multimodal large model assisted web crawler through a processor.

[0027] Compared with the prior art, the advantages of the present invention are: the present invention uses LLM to collect page functions, URLs and other page information of Web applications and extracts the association information between each Web page. During the web crawler process, a state machine is constructed according to the association information between Web pages to guide the correct order of crawling Web pages. LLM can be fully utilized to assist in guiding the web crawler process, so that more parts of the Web page are covered. At the same time, MLLM is used to take screenshots of Web pages and obtain the HTML content of the Web page, and then a natural language description of the tasks that users need to complete on the Web page and the operations required to complete the tasks is generated according to the HTML content. LLM is then used to convert the natural language description of each task into test code in turn, and whether the test code can successfully complete the task is judged according to the state changes of the Web page before and after execution. Reasonable form content and effective operation sequences can be generated to avoid generating invalid HTTP requests, which can greatly improve the efficiency and coverage of Web application fuzz testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a schematic diagram of the implementation process of the Web application fuzz testing method based on a multimodal large model assisted web crawler in this embodiment.

[0029] Figure 2 This is a schematic diagram of the effect of the first prompt word template defined in a specific application embodiment of the present invention.

[0030] Figure 3 This is a schematic diagram of the effect of the third prompt word template defined in a specific application embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0032] Vulnerability detection for web applications requires in-depth inspection of each web page. During fuzz testing to explore web application vulnerabilities, the key goal of the crawler is to collect as many URLs as possible and obtain valid parameter information. However, there are certain relationships between pages, and web applications are complex and diverse, potentially including different functions. Many pages may also require specific inputs. The designs of different web application pages, and even different pages within the same web application, vary significantly. Therefore, it is impossible to accurately generate specific inputs based solely on the structure and style of static pages. Furthermore, web pages are often dynamic. Traditional web crawlers based on simple depth-first or breadth-first strategies are unable to discover the state and relationships between pages. As a result, the generated form content is unrelated to the page, resulting in a large number of invalid operation sequences, resulting in low efficiency and coverage of web application fuzz testing.

[0033] For example, if you directly access a page that requires logging in or performing specific operations without considering the prerequisites, traditional crawlers may repeatedly access the same page or enter an infinite loop because they lack memory and state management of visited pages. This makes it impossible to effectively continue exploring other pages and paths, affecting the efficiency and coverage of web application fuzz testing.

[0034] The Large Language Model (LLM), through pre-training on large amounts of text data, can learn rich language patterns and structures, enabling it to understand and process complex linguistic phenomena. LLMs leverage natural language processing capabilities to understand user needs and automatically generate crawler code or strategies, reducing manual intervention and maintenance costs. LLMs can also accurately infer relationships between web pages. If these relationships are considered during the crawling process to determine the crawling order, crawler efficiency can be effectively improved, allowing for wider coverage of web applications. The Multimodal Large Language Model (MLLM) combines natural language understanding with the ability to process other media (such as images and videos), enabling the model to acquire information from multiple sensory inputs and conduct comprehensive analysis and understanding. MLLMs combine page screenshots and page content to accurately generate natural language descriptions of tasks and operations on web pages. LLMs, combined with page relationships, can guide web crawlers, generating reasonable form inputs and valid operation sequences. The generated form content and pages are correlated, effectively improving test efficiency and coverage.

[0035] The present invention is aimed at fuzz testing of Web applications. By using LLM to collect page information such as page functions and URLs of Web applications and extract the association information between each Web page, a state machine is constructed according to the association information between Web pages during the web crawler process to guide the correct order of crawling Web pages. LLM can be fully utilized to assist in guiding the web crawler process, so that more parts of the Web page are covered. At the same time, MLLM is used to screenshot the Web page and obtain the HTML content of the Web page, and then a natural language description of the task that the user needs to complete on the Web page and the operations required to complete the task is generated according to the HTML content. Each task is then split into a single task. The LLM is used to convert the natural language description of each task into test code in turn, and judge whether the test code can successfully complete the task according to the state changes of the web page before and after execution. If the test code can successfully complete the task, the form content and operation sequence are generated. The LLM-based web crawler process can be combined to generate reasonable form content and effective operation sequences, thereby generating valid HTTP requests, improving the effectiveness of the requests captured in the crawler stage, and avoiding invalid HTTP requests caused by invalid operation sequences. Therefore, the efficiency of web application fuzz testing can be greatly improved, and the coverage of web application fuzz testing can also be effectively improved to discover more web vulnerabilities.

[0036] like Figure 1 As shown, the specific steps of the web application fuzz testing method based on the multimodal large model assisted web crawler in this embodiment are as follows:

[0037] Step S1. Page information collection: Use LLM to collect page information of Web applications, and extract the association information between each Web page based on the page information of each Web page. Based on the association information between Web pages, a state machine is constructed to guide the order of crawling Web pages. Page information includes Web page functions and Web page URL information, etc. Association relationships include state relationships, interaction logic, dependency relationships, and navigation relationships, etc.

[0038] LLM can be used to efficiently collect information about web pages, and based on the information content of the pages, it can further extract associations between web pages. A user's actions on one page may affect the state or content of another page. A state relationship is a relationship where the state or content of the pages affects each other. User interactions between pages (such as form submissions) can trigger dynamic changes or page redirections. Interaction logic is the dynamic changes or page redirections caused by user interactions between pages. Certain pages or functions rely on the content or data of other pages. Dependency relationships are the dependencies between page content or data. Navigation relationships are the user's browsing paths formed by connecting different pages through links or buttons.

[0039] In this embodiment, the association information between web pages is used as a state machine, and then the navigation strategy of the next crawled page is determined based on the state machine. By guiding the order of crawling web pages based on the association information between web pages, the crawler efficiency can be effectively improved.

[0040] Generative large models, such as ChatGPT, and retrieval models, such as search engines, represent two distinct approaches to information acquisition. Generative large models store extensive knowledge in parameterized models and can generate answers based on user questions to meet user needs. However, the structure of LLM-generated answers is flexible. To effectively collect web page information and extract relevant information between web pages, this embodiment employs a method for fine-tuning prompt words, enabling LLM to answer questions in the desired format.

[0041] In this embodiment, the steps of using the LLM to collect web application page information and extract the web pages using the prompt word fine-tuning method include:

[0042] Step S101. Define a first prompt word template to prompt the LLM to collect web page functions, web page URL information, and extract association information between web pages;

[0043] Step S102. Conduct multiple rounds of dialogue using the first prompt word template. In each round of dialogue, the LLM is first prompted to output the web page function and web page URL information in a specified format, and then prompted to output the association information between the web pages in a specified format. During each round of dialogue, the answer to the previous round of dialogue is provided to the LLM, and the LLM conducts the next round of dialogue based on the answer to the previous round of dialogue. After completing multiple rounds of dialogue, the final web page function, web page URL information, and the association information between the web pages are obtained.

[0044] by Figure 2 Taking the first prompt word template shown as an example, the left side is the prompt word input and the right side is the model output. At the prompt word input end, the prompt word is first input to the model to ask "What is the function of [Web application] and its corresponding URL?", and two examples of pages, page functions, and URLs are given: [Page 1][Function][URL], [Page 2][Function][URL]. By communicating with the LLM, the page functions and URL information of each web application can be obtained from massive data. The page set of [Web application] is output at the model output end, and it is output in the following format: [Page 1][Function][URL], [Page 2][Function][URL]... Then the question word is input to the model:

[0045] “This is a collection of pages for [web app]:

[0046] [Place the above model output here]

[0047] The model then asks the model what the connections between pages are: "Please give the connections between these pages," and provides two well-formatted examples to avoid relying on a single example. The model outputs a set of connections between pages in the [web application] format: [page n][page m]: connection, which refers to the connection between page n and page m. During the multi-round dialogue, the answer from the previous round is provided to the LLM. After multiple rounds of dialogue with the LLM, the consistent answer is used to obtain the final extracted result.

[0048] It is understandable that the first prompt word template can of course adopt other definition formats according to actual needs. The key is to obtain the page function, URL and other page information of the Web interface and extract the relationship between pages to guide the crawler to select the next URL to crawl.

[0049] Step S2. Text input generation: Take a screenshot of the web page and obtain the HTML content of the web page, and provide the screenshot of the web page and the HTML content as multimodal information to MLLM. MLLM generates a natural language description based on the screenshot of the web page and the HTML content to describe the tasks that the user needs to complete on the corresponding web page and the operations required to complete the tasks.

[0050] In order to enable the multimodal large model to fully understand the function of each element, multimodal information is first provided. In this embodiment, puppeteer (an API for controlling Chromium) is used to take a screenshot of the current web page and obtain the HTML content. The page screenshot and HTML content are then provided to the multimodal large model MLLM as multimodal information. MLLM combines the screenshot and HTML content of the web page to generate a natural language description of the tasks and operations that need to be completed on the current web page.

[0051] MLLM can understand and analyze the structure of a web page, identify the elements in the page (such as buttons, text boxes, drop-down menus, etc.) and the location and attributes of each element. MLLM can also perform interactive operation analysis to understand the functions of various interactive elements on the page (such as clicking buttons, filling out forms, selecting options, etc.) and their possible impact. Data in different modalities usually contain complementary information. For example, text can provide semantics and contextual information, and images can provide visual details and spatial relationships. In this embodiment, by providing MLLM with multimodal information that complements screenshots of web pages and HTML content, MLLM can make full use of this complementary information to accurately understand the tasks and operation content on the web page, thereby effectively improving task performance and results.

[0052] In this embodiment, the steps of using MLLM to take a screenshot of a web page and obtain the HTML content of the web page include:

[0053] Step S201. Define a second prompt word template to prompt the MLLM to generate a natural language description of the task based on the HTML content of the web page;

[0054] Step S202: The second prompt word template first prompts MLLM to generate code to use puppeteer to simulate operations to complete the screenshot of the web page and obtain the HTML content, and then prompts it to output a natural language description of the task that the user needs to complete on the current web page and the operations required to complete the task.

[0055] To perform fuzz testing on a web application, it is necessary to obtain all the tasks that users need to complete on the web page and the operations required to complete the tasks, such as clicks and filling out forms. This embodiment prompts the MLLM to use Puppeteer to simulate operations to complete a screenshot of the web page and obtain the HTML content. The page screenshot and HTML content are then used to generate prompt words and provide them to the MLLM. The MLLM is then asked questions to generate a natural language description of the tasks and corresponding operations on the web page, that is, to generate text input.

[0056] The output of the large model is random. This embodiment uses the multimodal large model to output all tasks that can be completed on the page in natural language. Under each task, the operations required to complete the task are described in the desired format, such as Figure 1 The output format and operation template are shown in . Using MLLM, users can use natural language to describe the tasks and corresponding operations that can be completed on the current page. This fully utilizes the multimodal large model's ability to integrate multi-source information, improve reasoning capabilities, enrich contextual understanding, and possess the characteristics of cross-modal information transmission.

[0057] In this embodiment, the second prompt word template further includes adding an attribute element-id (element ID) to each element of the HTML content as a unique identifier of each element, so as to facilitate locating each element.

[0058] In this embodiment, after using Puppeteer to simulate operations to complete the screenshot of the web page and obtain the HTML content, it also includes removing invisible elements from the obtained HTML content, removing attributes in the elements that are not useful for understanding the elements, and extracting interactive elements. Attributes such as 'name', 'type', and 'aria-label' that are effective for inferring the function of the elements are retained to obtain the final HTML content. This can ensure that the large model can effectively focus its attention when processing complex data and extract more valuable HTML content.

[0059] Step S3. Operation sequence generation: Split the natural language description generated by MLLM according to tasks to form natural language descriptions of multiple tasks, use LLM to convert the natural language description of each task into test code in turn, and judge whether the generated test code successfully completes the corresponding task based on the state changes of the web page before and after executing the test code. If it is successfully completed, generate the form content and operation sequence corresponding to the task to generate a valid HTTP request.

[0060] Directly using a multimodal large model to generate corresponding code has poor actual effect. This embodiment splits the operation sequence generation into two steps. First, MLLM is used to generate a natural language description of the task that needs to be completed on the page. Then, LLM is used to convert the natural language description of each task into test code. The page status changes before and after execution are used to determine whether the task is successfully completed. For the test code that can successfully complete the task, corresponding form content and operation sequence are generated accordingly. This can give full play to the respective advantages of the multimodal large model and the large language model, use the large language model to understand and express complex semantic structures, and accurately capture the user's intentions and requirements based on the natural language description when generating code, avoiding the common semantic ambiguity problems in traditional programming, and effectively converting the natural language description generated by MLLM into corresponding test code.

[0061] The MLLM generated results contain multiple tasks. It is difficult to directly determine whether the generated code can accurately complete all tasks by processing it directly by LLM. This embodiment splits the code into individual tasks and conducts multiple rounds of dialogue with LLM to more accurately generate reasonable form content and effective operation sequences.

[0062] In this embodiment, by defining a third prompt word template, each task uses the third prompt word template to conduct multiple rounds of dialogue. In each round of dialogue, the LLM is first prompted to obtain a natural language description of a task and convert it into test code. Then, the LLM is prompted to determine whether the generated test code successfully completes the corresponding task based on the state changes of the Web pages before and after executing the test code. If successful, the LLM is prompted to output the form content and operation sequence corresponding to the task based on the association information between the Web pages, and continue to obtain the natural language description of the next task to generate the corresponding form content and operation sequence until all tasks are completed.

[0063] In this embodiment, judging whether the generated test code successfully completes the corresponding task based on the state change of the web page before and after executing the test code includes:

[0064] Step S301: Submit the page content before and after executing the generated test code to the LLM, and use the LLM to infer the page state changes;

[0065] Step S302: Determine whether the currently generated test code has successfully completed the task based on the page status change, wherein if the page has changed, it is determined that the task has been successfully completed; otherwise, it is determined that the task has not been successfully completed.

[0066] Furthermore, if it is determined that the currently generated test code successfully completes the task, the natural language description of the next task is continued to be obtained to generate the corresponding form content and operation sequence. If it is determined that the currently generated test code does not successfully complete the task, the LLM is asked again to generate new executable test code for the current task. When the number of questions reaches the preset number of questions, the current task is abandoned and the natural language description of the next task is continued to be selected to generate the corresponding form content and operation sequence.

[0067] Specifically, after generating test code for the current task, the page content before and after executing the generated test code is submitted to LLM. LLM infers the page state changes to determine whether the test code generated by LLM can complete the task. If LLM determines that it is successful, it generates code for the next task. If it is unsuccessful, it asks LLM again to generate code for execution. If this process occurs more than three times (configurable), the task is abandoned and the next one is selected.

[0068] by Figure 3Taking the prompt template for multi-round dialogue with a large language model as an example, in order to enable MLLM to better understand the user's intention, the system prompt word is set to "You are a browser automation assistant." Then, after prompting to output the HTML content of the current web page and the tasks and operations to be completed, the prompt word is set to "You only need to generate code and use Puppeteer to simulate the operation to complete the task. The selector is "element-id". Puppeteer uses the "element-id" attribute to locate the element to be operated, and provides the expected format for interacting with the web page. It also provides examples of using Puppeteer to complete click operations and fill in form operations, prompting the user to use Puppeteer to perform the tasks and operations required on the current web page and interact in the expected format.

[0069] This embodiment uses a large multimodal model to generate natural language descriptions of tasks that can be performed on a web page and the operations required. This allows for a precise understanding of page elements and interaction logic, enabling the generation of reasonable form inputs. The large multimodal model is then used to infer the state of the web page, and changes in the page state are used to determine the effectiveness of test code execution. This allows for the generation of valid HTTP requests, avoiding invalid operation sequences that would otherwise result in invalid HTTP requests, thereby significantly improving test efficiency. This embodiment, based on the large multimodal model, uses prompt word fine-tuning to guide the generation of operation sequences. By sharing model weights and computing resources, it can also save the overhead of multiple separate models. This allows for flexible application to different tasks and data modalities, enabling more comprehensive testing.

[0070] like Figure 1 As shown, during the fuzz testing of Web applications, the test cases generated by the above crawler method generate HTTP requests to the Web server, so as to perform corresponding tests on the Web application in the real-time environment of the Web application, find out the vulnerabilities therein, and realize vulnerability detection. Since the above-mentioned method of using a multimodal large model to assist the web crawler can accurately generate effective HTTP requests, the efficiency and coverage of vulnerability detection can be greatly improved, and as many vulnerabilities as possible can be detected.

[0071] In summary, this embodiment implements Web application fuzz testing by utilizing a multimodal large model to assist a web crawler. It first utilizes LLM to collect Web page information and extract the relationship between pages, and fully utilizes the information retrieval and reasoning capabilities of LLM to collect the page connections and URLs of Web applications. At the same time, it utilizes MLLM to generate a summary of tasks that can be performed on a certain Web page and the interactive operations required to complete the tasks described in natural language to form text input. LLM is then used to convert the generated text input into test code, and MLLM is used to infer the Web page status to determine whether the generated code successfully completes the task. If the task is completed, a corresponding operation sequence is generated, which can simulate the tester's interactive intention and fully utilize MLLM to assist in guiding the web crawler, thereby improving the effectiveness of requests captured in the crawler stage and avoiding the generation of invalid HTTP requests. This can greatly improve the efficiency and coverage of Web application fuzz testing.

[0072] This embodiment also provides a Web application fuzz testing system based on a multimodal large model assisted web crawler, comprising a processor and a memory connected to each other, the memory being used to store computer programs, and the processor being programmed or configured to execute the above-mentioned Web application fuzz testing method based on a multimodal large model assisted web crawler.

[0073] This embodiment also provides a computer-readable storage medium, which stores a computer program / instruction. The computer program / instruction is programmed or configured to execute the Web application fuzz testing method based on a multimodal large model assisted web crawler through a processor.

[0074] Those skilled in the art will appreciate that the above-mentioned embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including the instruction device, which implements the function specified in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. Therefore, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection of the technical solution of the present invention.

[0075] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A web application fuzz testing method based on a multimodal large model assisted web crawler, characterized in that: The following steps are involved: Using LLM to collect web application page information and extract association information between web pages, a state machine is constructed based on the association information between web pages to guide the order of crawling web pages. The page information includes web page functions and web page URL information. The association information includes any one or more of state relationships, interaction logic, dependency relationships, and navigation relationships. The state relationship refers to the relationship between pages that may be affected by state or content. The interaction logic refers to the dynamic change or page redirection caused by user interaction between pages. The dependency relationship refers to the dependency relationship between page content or data. The navigation relationship refers to the relationship of user browsing paths formed by connecting different pages. Take a screenshot of the web page and obtain the HTML content of the web page. Provide the screenshot and HTML content of the web page as multimodal information to the MLLM. The MLLM generates a natural language description of the task based on the screenshot and HTML content of the web page, which is used to describe the task that the user needs to complete on the web page and the operations required to complete the task. The natural language description generated by MLLM is split according to tasks to form natural language descriptions of multiple tasks. LLM is used to convert the natural language description of each task into test code in turn. The state changes of the web page before and after executing the test code are used to determine whether the generated test code successfully completes the corresponding task. If it is successfully completed, the form content and operation sequence corresponding to the task are generated to generate a valid HTTP request.

2. The method for fuzz testing of web applications based on a multimodal large model assisted web crawler according to claim 1, characterized in that: The method of using LLM to collect web application page information and extract association information between web pages includes: Define a first prompt word template to prompt the LLM to collect web page functions, web page URL information, and extract association information between web pages; Multiple rounds of dialogue are conducted using the first prompt word template. In each round of dialogue, the LLM is first prompted to output the web page function and web page URL information in a specified format, and then the LLM is prompted to output the association information between the web pages in a specified format. During each round of dialogue, the answer to the previous round of dialogue is provided to the LLM, and the LLM conducts the next round of dialogue based on the answer to the previous round of dialogue. After completing multiple rounds of dialogue, the final web page function, web page URL information, and association information between the web pages are obtained.

3. The method for fuzz testing of web applications based on a multimodal large model assisted web crawler according to claim 1, characterized in that: The method of taking a screenshot of a web page and obtaining the HTML content of the web page includes: A second prompt word template is defined to prompt the MLLM to generate a natural language description of the task based on the HTML content of the web page; In the second prompt word template, MLLM is first prompted to generate code to use puppeteer to simulate operations to complete the screenshot of the web page and obtain the HTML content, and then prompted to output natural language to describe the tasks that the user needs to complete on the current web page and the operations required to complete the tasks.

4. The method for fuzz testing of web applications based on a multimodal large model assisted web crawler according to claim 3, characterized in that: The second prompt word template also includes adding an attribute element-id to each element of the HTML content as a unique identifier of each element.

5. The method for fuzz testing of web applications based on a multimodal large model assisted web crawler according to claim 3, characterized in that: After using puppeteer to simulate operations to complete the screenshot of the web page and obtain the HTML content, the method also includes removing invisible elements from the obtained HTML content, removing attributes in the elements that are useless for understanding the elements, and extracting interactive elements, retaining attributes that are effective for inferring the functions of the elements, to obtain the final HTML content.

6. The method for fuzz testing of web applications based on a multimodal large model assisted web crawler according to any one of claims 1 to 5, characterized in that: By defining a third prompt word template, each task uses the third prompt word template to conduct multiple rounds of dialogue. In each round of dialogue, the LLM is first prompted to obtain a natural language description of a task and convert it into test code. Then, the LLM is prompted to judge whether the generated test code successfully completes the corresponding task based on the state changes of the web pages before and after executing the test code. If successful, the LLM is prompted to output the form content and operation sequence corresponding to the task based on the association information between the web pages, and continue to obtain the natural language description of the next task to generate the corresponding form content and operation sequence until all tasks are completed.

7. The method for fuzz testing of web applications based on a multimodal large model assisted web crawler according to claim 6, characterized in that: The step of judging whether the generated test code successfully completes the corresponding task based on the state change of the web page before and after the test code is executed includes: Submit the page content before and after executing the generated test code to LLM, and use LLM to infer page state changes; It is determined whether the currently generated test code has successfully completed the task according to the page status change, wherein if the page changes, it is determined that the task has been successfully completed; otherwise, it is determined that the task has not been successfully completed.

8. The method for fuzz testing of web applications based on a multimodal large model assisted web crawler according to claim 7, characterized in that: If it is determined that the currently generated test code successfully completes the task, the natural language description of the next task will be obtained to generate the corresponding form content and operation sequence. If it is determined that the currently generated test code does not successfully complete the task, the LLM will be asked again to generate new executable test code for the current task. When the number of questions reaches the preset number of questions, the current task will be abandoned and the natural language description of the next task will be selected to generate the corresponding form content and operation sequence.

9. A web application fuzz testing system based on a multimodal large model assisted web crawler, comprising a processor and a memory connected to each other, wherein the memory is used to store a computer program, characterized in that: The processor is programmed or configured to execute the Web application fuzz testing method based on a multimodal large model assisted web crawler as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program / instruction stored therein, characterized in that: The computer program / instruction is programmed or configured to execute the Web application fuzz testing method based on a multimodal large model assisted web crawler as described in any one of claims 1 to 8 through a processor.

Images