Web application fuzzy testing method based on multi-modal large model assisted web crawler

Abstract

The invention provides a Web application fuzzy testing method based on a multi-modal large model assisted web crawler, which comprises the following steps of: collecting page information of a Web application program by using an LLM (Logical Link Model), extracting associated information among Web pages, and constructing a state machine according to the associated information among the Web pages to guide a sequence of crawling the Web pages; performing screenshot on the Web page by using the MLLM, obtaining HTML (Hypertext Markup Language) content of the Web page, and generating a natural language description of a task so as to describe the task needing to be completed by a user on the page and an operation required for completing the task; and sequentially converting the natural language description into test codes by using LLM, judging whether the test codes successfully complete a task or not according to the state change of the Web page before and after execution, and if the test codes successfully complete the task, generating corresponding form content and an operation sequence to generate an effective HTTP request. The method has the advantages of simple implementation method, low cost, high test efficiency, high coverage rate and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of fuzz testing and automated testing, and in particular to a Web application fuzz testing method based on a multimodal large model assisted web crawler. Background Art

[0002] As the number of web applications continues to increase and the supported frameworks become more diverse, it is essential to automatically and efficiently implement web vulnerability detection technology. Fuzz testing is an automated testing technology that can automatically or semi-automatically generate random data according to certain rules, and then input these generated random data into the entrance of the dynamically running program under test, while monitoring whether the program under test has any abnormalities. Fuzz testing is currently widely used in detecting web application vulnerabilities. Fuzz testing of web applications is an automated testing method that uses typical web interfaces and a large number of HTTP requests to find web vulnerabilities. It can obtain prior knowledge by using crawlers to interact with running web application instances.

[0003] Vulnerability detection of web applications requires in-depth detection of each web page. Web crawlers can automatically access the pages of web applications and discover the URLs, HTML form fields, and other input points therein. The above information is crucial for further vulnerability detection. For example, for URLs that the crawler has not crawled, the test cases generated by the subsequent fuzz test cannot cover the content of this part of the URL. However, traditional web crawlers usually adopt a simple depth-first or breadth-first strategy, which can only obtain the content of the page, and cannot discover the status and association information (such as dependencies) between pages. Therefore, in the fuzz test of traditional web applications, the web page information crawled by the web crawler is directly used. The actual generated form content is not related to the page, which may result in the generation of a large number of invalid operation sequences, and then generate a large number of invalid HTTP requests, resulting in low test efficiency and coverage.

[0004] Some practitioners have proposed using gray-box coverage-guided mutation fuzz testing to detect SQL and command line injection vulnerabilities in web applications, that is, running black-box crawlers to automatically discover HTTP requests and parameters, and using the Gremlins testing tool to randomly trigger user input events (e.g., clicks, form filling, scrolling, and typing, etc.). However, the above test method uses a method of randomly triggering user input, that is, randomly filling in form content, so the generated form content is often unreasonable and has nothing to do with the page content. The randomly generated operation sequence will also generate a large number of repeated and useless requests or fail to trigger requests, and there will still be problems with low efficiency and coverage.

[0005] In addition, some practitioners have proposed a method that combines multiple machine learning methods to obtain "interaction intentions" that are consistent with human cognition through machines, and then use this information to simulate the test personnel's "understanding-operation-checking" test verification process for client products. By selecting three modal page information: image information, rendering tree information, and text information, the goal of understanding UI interaction intentions is achieved, and ultimately correct interactions are performed on many form pages. However, the above method is only applicable to mobile applications, not web applications, and needs to rely on multimodal self-attention deep learning. The integration of multimodal information requires complex model design and training processes, which greatly increases the difficulty of development and debugging, and requires a large amount of labeled data to support the training of multimodal deep learning models. Summary of the invention

[0006] The technical problem to be solved by the present invention is as follows: In view of the above-mentioned problems in the prior art, a method and system for fuzzy testing of Web applications based on a multimodal large model assisted web crawler is provided, which is efficient and easy to operate.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] A method for fuzz testing a web application based on a multimodal large model assisted web crawler comprises the following steps:

[0009] LLM (Large Language Model) is used to collect page information of Web applications and extract association information between Web pages. A state machine is constructed based on the association information between Web pages to guide the order of crawling Web pages. The page information includes Web page functions and Web page URL (uniform resource location) information. The association relationship includes any one or more of a state relationship, an interaction logic, a dependency relationship, and a navigation relationship. The state relationship is a relationship between pages that may be affected by state or content. The interaction logic is a relationship of dynamic change or page redirection caused by user interaction between pages. The dependency relationship is a dependency relationship between page content or data. The navigation relationship is a relationship of a user's browsing path formed by connecting different pages.

[0010] Use MLLM (Multimodal Large Language Model) to take a screenshot of a web page and obtain the HTML (hyper text markup language) content of the web page, and generate a natural language description of the task based on the obtained HTML content, so as to describe the task that the user needs to complete on the web page and the operations required to complete the task;

[0011] The natural language description generated by MLLM is split according to tasks to form natural language descriptions of multiple tasks. LLM is used to convert the natural language description of each task into test code in turn, and the generated test code is judged whether the corresponding task is successfully completed according to the state changes of the Web pages before and after the execution of the test code. If it is successfully completed, the form content and operation sequence corresponding to the task are generated according to the association information between the Web pages to guide the generation of valid HTTP requests.

[0012] Furthermore, the method of using LLM to collect web application page information and extract association information between web pages includes:

[0013] Define a first prompt word template to prompt the LLM to collect Web page functions, Web page URL information, and extract association information between each Web page;

[0014] The first prompt word template is used to conduct multiple rounds of dialogues. In each round of dialogue, the LLM is first prompted to output the web page function and web page URL information in a specified format, and then the LLM is prompted to output the association information between the web pages in a specified format; in each round of dialogue, the answer to the previous round of dialogue is provided to the LLM, and the LLM conducts the next round of dialogue based on the answer to the previous round of dialogue. After completing multiple rounds of dialogue, the final web page function, web page URL information and the association information between the web pages are obtained.

[0015] Furthermore, the method of taking a screenshot of a web page using MLLM and obtaining the HTML content of the web page includes:

[0016] A second prompt word template is defined to prompt the MLLM to generate a natural language description of the task according to the HTML content of the Web page;

[0017] In the second prompt word template, MLLM is first prompted to generate code to use puppeteer to simulate operations to complete the screenshot of the web page and obtain the HTML content, and then prompted to output natural language to describe the tasks that the user needs to complete on the current web page and the operations required to complete the tasks.

[0018] Furthermore, the second prompt word template also includes adding an attribute element-id to each element of the HTML content as a unique identifier of each element.

[0019] Furthermore, after the puppeteer simulation operation is used to complete the screenshot of the web page and obtain the HTML content, it also includes removing invisible elements from the obtained HTML content, removing attributes in the elements that are useless for understanding the elements, and extracting interactive elements, retaining attributes that are effective for inferring the functions of the elements, and obtaining the final HTML content.

[0020] Furthermore, by defining a third prompt word template, each task uses the third prompt word template to conduct multiple rounds of dialogues. In each round of dialogue, the LLM is first prompted to obtain a natural language description of a task and convert it into test code, and then the LLM is prompted to determine whether the generated test code successfully completes the corresponding task based on the state changes of the Web pages before and after the execution of the test code. If successful, the LLM is prompted to output the form content and operation sequence corresponding to the task based on the association information between the Web pages, and continue to obtain the natural language description of the next task to generate the corresponding form content and operation sequence until all tasks are completed.

[0021] Further, judging whether the generated test code successfully completes the corresponding task according to the state change of the web page before and after executing the test code includes:

[0022] Submit the page content before and after executing the generated test code to LLM, and use LLM to infer the page state changes;

[0023] It is determined whether the currently generated test code has successfully completed the task according to the page status change, wherein if the page changes, it is determined that the task has been successfully completed, otherwise it is determined that the task has not been successfully completed.

[0024] Furthermore, if it is determined that the currently generated test code successfully completes the task, the natural language description of the next task is continued to be obtained to generate the corresponding form content and operation sequence. If it is determined that the currently generated test code has not successfully completed the task, the LLM is asked again to generate new executable test code for the current task. When the number of questions reaches the preset number of questions, the current task is abandoned and the natural language description of the next task is continued to be selected to generate the corresponding form content and operation sequence.

[0025] A Web application fuzz testing system based on a multimodal large model assisted web crawler comprises a processor and a memory connected to each other, the memory is used to store a computer program, and the processor is programmed or configured to execute the above-mentioned Web application fuzz testing method based on a multimodal large model assisted web crawler.

[0026] A computer-readable storage medium stores a computer program / instruction, which is programmed or configured to execute the above-mentioned Web application fuzz testing method based on a multimodal large model assisted web crawler through a processor.

[0027] Compared with the prior art, the advantages of the present invention are: the present invention uses LLM to collect page functions, URLs and other page information of Web applications and extracts the association information between each Web page. In the process of web crawling, a state machine is constructed according to the association information between Web pages to guide the correct order of crawling Web pages. LLM can be fully utilized to assist in guiding the web crawling process, so that more parts of the Web page are covered. At the same time, MLLM is used to take screenshots of Web pages and obtain the HTML content of the Web page, and then a natural language description of the tasks that users need to complete on the Web page and the operations required to complete the tasks is generated according to the HTML content. LLM is then used to convert the natural language description of each task into test code in turn, and whether the test code can successfully complete the task is determined according to the state changes of the Web page before and after execution. Reasonable form content and effective operation sequence can be generated to avoid generating invalid HTTP requests, which can greatly improve the efficiency and coverage of Web application fuzz testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Schematic diagram of the implementation process of the Web application fuzz testing method based on multimodal large model assisted web crawler in this embodiment.

[0029] Figure 2 It is a schematic diagram of the effect of the first prompt word template defined in a specific application embodiment of the present invention.

[0030] Figure 3 It is a schematic diagram of the effect of the third prompt word template defined in a specific application embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] Vulnerability detection of web applications requires in-depth detection of each web page. In the process of fuzz testing to explore web application vulnerabilities, the key goal of the crawler is to collect all URLs as much as possible and obtain valid parameter information. However, there will be certain associations between pages, and web applications are complex and diverse, which may include different functions. Many pages may also require specific inputs. Different web application pages and different page designs of the same web application are also very different. Therefore, it is impossible to accurately generate specific inputs based only on the structure and style of static pages, and web pages are often dynamic. Traditional web crawlers based on simple depth-first or breadth-first strategies cannot discover the status and association between pages. Therefore, the generated form content is not associated with the page, which will generate a large number of invalid operation sequences, resulting in low efficiency and coverage of web application fuzz testing.

[0033] For example, when a page needs to be accessed after logging in or performing specific operations, the page is accessed directly without considering the prerequisites. Since traditional crawlers lack memory and state management of visited pages, they may repeatedly visit the same page or enter an infinite loop, and cannot effectively continue to explore other pages and paths, affecting the efficiency and coverage of Web application fuzz testing.

[0034] The large language model LLM can learn rich language patterns and structures through pre-training on a large amount of text data, enabling it to understand and process complex language phenomena. LLM can understand user needs through natural language processing capabilities and automatically generate crawler code or strategies to reduce manual intervention and maintenance costs. At the same time, it can accurately infer the association between Web pages. If the association between Web pages can be considered in the crawling process to determine the order of crawling, the efficiency of the crawler can be effectively improved and more parts of the Web application can be covered. The multimodal large language model MLLM combines natural language understanding and the processing capabilities of other media (such as images, videos, etc.), allowing the model to obtain information from multiple sensory inputs and conduct comprehensive analysis and understanding. MLLM can accurately generate natural language descriptions of tasks and operations on Web pages by combining page screenshots and page content. Combined with LLM, the association between pages can be used to guide web crawlers, which can generate reasonable form inputs and effective operation sequences. The generated form content and pages will be related, effectively improving the efficiency and coverage of the test.

[0035] The present invention is aimed at fuzzy testing of Web applications. By using LLM to collect page information such as page functions and URLs of Web applications and extracting association information between Web pages, a state machine is constructed according to the association information between Web pages during the web crawler process to guide the correct order of crawling Web pages. LLM can be fully utilized to assist in guiding the web crawler process, so that more parts of the Web pages are covered. At the same time, MLLM is used to take screenshots of Web pages and obtain the HTML content of Web pages, and then a natural language description of the tasks that users need to complete on the Web pages and the operations required to complete the tasks is generated according to the HTML content, and each task is then split into a single task. The natural language description of each task is converted into test code using LLM in turn, and whether the test code can successfully complete the task is determined according to the state changes of the Web page before and after execution. When the test code can successfully complete the task, the form content and operation sequence are generated. The web crawler process guided by LLM can generate reasonable form content and effective operation sequence, and then generate effective HTTP requests, thereby improving the effectiveness of the requests captured in the crawler stage and avoiding invalid HTTP requests caused by invalid operation sequences. Therefore, the efficiency of Web application fuzz testing can be greatly improved, and the coverage of Web application fuzz testing can also be effectively improved to discover more Web vulnerabilities.

[0036] like Figure 1 As shown, the specific steps of the Web application fuzz testing method based on the multimodal large model assisted web crawler in this embodiment are as follows:

[0037] Step S1. Page information collection: Use LLM to collect page information of Web applications, and extract the association information between each Web page based on the page information of each Web page. According to the association information between Web pages, a state machine is constructed to guide the order of crawling Web pages. The page information includes Web page functions and Web page URL information, etc. The association relationship includes state relationship, interaction logic, dependency relationship, and navigation relationship, etc.

[0038] LLM can be used to efficiently collect information about Web pages, and the associated information between Web pages can be further extracted based on the information content of the pages. The user's operation on one page may affect the state or content of another page. The state relationship is the relationship between pages that have state or content influences each other. User interaction between pages (such as form submission) may cause dynamic changes or page redirection. The interaction logic is the relationship between dynamic changes or page redirection caused by user interaction between pages. Some pages or functions depend on the content or data of other pages. The dependency relationship is the dependency relationship between the content or data of the pages. The navigation relationship is the relationship between the user's browsing path formed by connecting different pages through links or buttons.

[0039] In this embodiment, the association information between Web pages is used as a state machine, and then the navigation strategy for the next crawled page is determined according to the state machine. By guiding the order of crawling Web pages based on the association information between Web pages, the crawler efficiency can be effectively improved.

[0040] Generative big models represented by ChatGPT and retrieval models represented by search engines are two different ways of obtaining information. Generative big models store a lot of knowledge in parameterized models and can generate answers based on questions raised by users to meet user needs. However, the structure of the answers generated by LLM is flexible. In order to effectively collect page information of Web pages and extract related information between Web pages, this embodiment adopts a prompt word fine-tuning method so that LLM can answer questions in the desired format.

[0041] In this embodiment, the steps of using the prompt word fine-tuning method to use LLM to collect page information of Web applications and extract the web pages include:

[0042] Step S101. Define a first prompt word template to prompt LLM to collect Web page functions, Web page URL information, and extract association information between Web pages;

[0043] Step S102. Conduct multiple rounds of dialogue using the first prompt word template. In each round of dialogue, the LLM is first prompted to output the Web page function and Web page URL information in a specified format, and then prompted to output the association information between the Web pages in a specified format. In each round of dialogue, the answer to the previous round of dialogue is provided to the LLM, and the LLM conducts the next round of dialogue based on the answer to the previous round of dialogue. After completing multiple rounds of dialogue, the final Web page function, Web page URL information and the association information between the Web pages are obtained.

[0044] by Figure 2 Take the first prompt word template shown as an example. The left side is the prompt word input and the right side is the model output. At the prompt word input end, first input the prompt word to the model to ask "What are the functions of [Web application] and the corresponding URL?", and give two examples of pages and page functions and URLs: [Page 1][Function][URL], [Page 2][Function][URL]. By talking to LLM, the page functions and URL information of each Web application can be obtained on the massive data. The page set of [Web application] is output at the model output end, and it is output in the following format: [Page 1][Function][URL], [Page 2][Function][URL]...... Then input the question word to the model:

[0045] “This is a collection of pages in [the web app]:

[0046] [Place the above model output here]

[0047] Then ask the model what the connections between pages are: "Please give the connections between these pages", and give two examples with reasonable format to avoid relying on a single example; the model outputs the set of connections between pages of [Web application], and the output format is: [page n][page m]: connection, that is, the connection between page n and page m. In the multi-round dialogue process, the answer of the previous round is provided to LLM. After multiple rounds of dialogue with LLM, the consistent answer is taken to obtain the final extracted result.

[0048] It is understandable that the first prompt word template can of course adopt other definition formats according to actual needs. The key is to obtain the page function, URL and other page information of the Web interface and extract the relationship between pages to guide the crawler to select the next URL to crawl.

[0049] Step S2. Text input generation: Take a screenshot of the web page and obtain the HTML content of the web page, and provide the screenshot of the web page and the HTML content as multimodal information to MLLM. MLLM generates a natural language description based on the screenshot of the web page and the HTML content to describe the tasks that the user needs to complete on the corresponding web page and the operations required to complete the tasks.

[0050] In order to enable the multimodal large model to fully understand the function of each element, multimodal information is first provided. In this embodiment, puppeteer (an API used to control Chromium) is used to take a screenshot of the current web page and obtain the HTML content. The page screenshot and HTML content are then provided to the multimodal large model MLLM as multimodal information. MLLM combines the screenshot and HTML content of the web page to generate a natural language description of the tasks and operations that need to be completed on the current web page.

[0051] MLLM can understand and analyze the structure of a Web page, identify the elements in the page (such as buttons, text boxes, drop-down menus, etc.) and the location and attributes of each element. MLLM can also perform interactive operation analysis to understand the functions of various interactive elements on the page (such as clicking buttons, filling out forms, selecting options, etc.) and possible impacts. Data in different modalities usually contain complementary information. For example, text can provide semantics and contextual information, and images can provide visual details and spatial relationships. In this embodiment, by providing MLLM with multimodal information that complements the screenshots of Web pages and the HTML content, MLLM can make full use of this complementary information to accurately understand the tasks and operation content on the Web page, thereby effectively improving task performance and results.

[0052] In this embodiment, the steps of taking a screenshot of a web page using MLLM and obtaining the HTML content of the web page include:

[0053] Step S201. Define a second prompt word template to prompt the MLLM to generate a natural language description of the task according to the HTML content of the Web page;

[0054] Step S202. The second prompt word template first prompts MLLM to generate code to use puppeteer to simulate operations to complete the screenshot of the web page and obtain the HTML content, and then prompts to output a natural language description of the task that the user needs to complete on the current web page and the operations required to complete the task.

[0055] To perform fuzz testing of Web applications, it is necessary to obtain all the tasks that users need to complete on the Web page and the operations required to complete the tasks, such as clicking, filling out forms, etc. This embodiment prompts MLLM to use puppeteer to simulate operations to complete the screenshot of the Web page and obtain the HTML content, and then uses the page screenshot and HTML content to generate prompt words to provide to MLLM, and then asks MLLM questions to generate natural language descriptions of the tasks and corresponding operations on the Web page, that is, to generate text input.

[0056] The output of the large model is random. This embodiment uses the multimodal large model to output all tasks that can be completed on the page in natural language. Under each task, the operations required to complete the task are described in the expected format, such as Figure 1 The output format and operation template are shown in . Using MLLM to describe the tasks that users can complete on the current page and the corresponding operations in natural language can give full play to the multi-modal large model's ability to integrate multi-source information, improve reasoning capabilities, and enrich contextual understanding, with the characteristics of cross-modal information transmission.

[0057] In this embodiment, the second prompt word template also includes adding an attribute element-id (element ID) to each element of the HTML content as a unique identifier of each element, so as to facilitate locating each element.

[0058] In this embodiment, after using puppeteer to simulate the operation to complete the screenshot of the web page and obtain the HTML content, it also includes removing invisible elements from the obtained HTML content, removing attributes in the elements that are useless for understanding the elements, and extracting interactive elements, retaining attributes such as 'name', 'type', 'aria-label', etc. that are effective for inferring the function of the elements, and obtaining the final HTML content, which can ensure that the large model can effectively concentrate when processing complex data and extract more valuable HTML content.

[0059] Step S3. Operation sequence generation: Split the natural language description generated by MLLM according to tasks to form natural language descriptions of multiple tasks, use LLM to convert the natural language description of each task into test code in turn, and judge whether the generated test code successfully completes the corresponding task based on the state changes of the Web page before and after executing the test code. If it is successfully completed, generate the form content and operation sequence corresponding to the task to generate a valid HTTP request.

[0060] Directly using a multimodal large model to generate corresponding codes has poor actual effect. This embodiment splits the operation sequence generation into two steps. First, MLLM is used to generate a natural language description of the tasks that need to be completed on the page. Then, LLM is used to convert the natural language description of each task into test code. The page state changes before and after execution are used to determine whether the task is successfully completed. For the test code that can successfully complete the task, corresponding form content and operation sequence are generated accordingly. This can give full play to the respective advantages of the multimodal large model and the large language model, use the large language model to understand and express complex semantic structures, and accurately capture the user's intentions and requirements based on the natural language description when generating code, avoiding the common semantic ambiguity problems in traditional programming, and effectively converting the natural language description generated by MLLM into corresponding test code.

[0061] The MLLM generated result contains multiple tasks. It is difficult to directly determine whether the generated code can accurately complete all tasks by directly processing it by LLM. This embodiment can more accurately generate reasonable form content and effective operation sequences by splitting it into single tasks and conducting multiple rounds of dialogue with LLM.

[0062] In this embodiment, by defining a third prompt word template, each task uses the third prompt word template to conduct multiple rounds of dialogues. In each round of dialogue, the LLM is first prompted to obtain a natural language description of a task and convert it into a test code. Then, the LLM is prompted to determine whether the generated test code successfully completes the corresponding task based on the state changes of the Web pages before and after the execution of the test code. If successful, the LLM is prompted to output the form content and operation sequence corresponding to the task based on the association information between the Web pages, and continue to obtain the natural language description of the next task to generate the corresponding form content and operation sequence until all tasks are completed.

[0063] In this embodiment, judging whether the generated test code successfully completes the corresponding task according to the state change of the web page before and after the test code is executed includes:

[0064] Step S301. Submit the page content before executing the generated test code and the page content after executing the test code to LLM, and use LLM to infer the page state change;

[0065] Step S302: Determine whether the currently generated test code has successfully completed the task according to the page status change, wherein if the page changes, it is determined that the task has been successfully completed, otherwise it is determined that the task has not been successfully completed.

[0066] Furthermore, if it is determined that the currently generated test code successfully completes the task, the natural language description of the next task is continued to be obtained to generate the corresponding form content and operation sequence. If it is determined that the currently generated test code has not successfully completed the task, the LLM is asked again to generate new executable test code for the current task. When the number of questions reaches the preset number of questions, the current task is abandoned and the natural language description of the next task is continued to be selected to generate the corresponding form content and operation sequence.

[0067] Specifically, after generating the test code for the current task, the page content before and after the execution of the generated test code is submitted to LLM, and LLM infers the page status changes to determine whether the test code generated by LLM can complete the task. If LLM determines that it is successful, it generates code for the next task. If it is not successful, LLM is asked again to generate code for execution. If this process occurs more than three times (configurable), the task is abandoned and the next one is selected.

[0068] by Figure 3Taking the prompt template of the large language model for multi-round dialogue as an example, in order to make MLLM better understand the user's intention, the system prompt word is set to "You are a browser automation assistant." Then, after prompting to output the HTML content of the current web page and the tasks and operations that need to be completed, the prompt word is set to "You only need to generate code to use puppeteer to simulate the operation to complete the task. The selector is "element-id". Puppeteer can use the "element-id" attribute to locate the element to be operated, and provide the expected format for interacting with the web page and examples of using puppeteer to complete click operations and fill in form operations, so as to prompt the use of puppeteer to perform the tasks and operations that need to be completed on the current web page, and interact in the expected format.

[0069] This embodiment uses a multimodal large model to generate natural language descriptions of tasks that can be performed on a certain Web page and the operations required for them, thereby achieving an accurate understanding of page elements and interaction logic, and being able to generate reasonable form inputs. At the same time, the multimodal large model is used to infer the Web page state, and the page state changes are used to determine the execution effect of the test code, thereby being able to generate valid HTTP requests and avoid generating invalid operation sequences and generating invalid HTTP requests, thereby greatly improving the efficiency of the test. This embodiment uses a multimodal large model based on the use of prompt word fine-tuning to guide the generation of operation sequences. By sharing model weights and computing resources, it can also save the overhead of multiple separate models, and can be flexibly applied to different tasks and data modalities to achieve more comprehensive detection.

[0070] like Figure 1 As shown, in the process of fuzz testing of Web applications, the test cases generated by the above crawler method generate HTTP requests to the Web server, so as to perform corresponding tests on the Web application in the real-time environment of the Web application, find out the vulnerabilities therein, and realize vulnerability detection. Since the above-mentioned method of using a multimodal large model to assist the web crawler can accurately generate effective HTTP requests, the efficiency and coverage of vulnerability detection can be greatly improved, and as many vulnerabilities as possible can be detected.

[0071] In summary, this embodiment implements Web application fuzz testing by using a multimodal large model to assist a web crawler. First, LLM is used to collect Web page information and extract the relationship between pages. The information retrieval and reasoning capabilities of LLM are fully utilized to collect the page connections and URLs of Web applications. At the same time, MLLM is used to generate a summary of tasks that can be performed on a certain Web page and the interactive operations required to complete the tasks in natural language to form text input. LLM is then used to convert the generated text input into test code, and MLLM is used to infer the Web page status to determine whether the generated code successfully completes the task. If the task is completed, a corresponding operation sequence is generated, which can simulate the tester's interactive intention, fully utilize MLLM to assist in guiding the web crawler, improve the effectiveness of the requests captured in the crawler stage, avoid generating invalid HTTP requests, and greatly improve the efficiency and coverage of Web application fuzz testing.

[0072] This embodiment also provides a Web application fuzz testing system based on a multimodal large model assisted web crawler, including a processor and a memory connected to each other, the memory is used to store computer programs, and the processor is programmed or configured to execute the above-mentioned Web application fuzz testing method based on a multimodal large model assisted web crawler.

[0073] This embodiment also provides a computer-readable storage medium, in which a computer program / instruction is stored. The computer program / instruction is programmed or configured to execute the Web application fuzz testing method based on a multimodal large model assisted web crawler through a processor.

[0074] Those skilled in the art will appreciate that the above-mentioned embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The above are only preferred embodiments of the present invention and are not intended to limit the present invention in any form. Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection of the technical solution of the present invention.

[0075] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A Web application fuzz testing method based on a multimodal large model assisted web crawler, characterized in that: The following steps are involved: Using LLM to collect page information of Web applications and extract association information between each Web page, a state machine is constructed according to the association information between the Web pages to guide the order of crawling Web pages, wherein the page information includes Web page functions and Web page URL information, and the association relationship includes any one or more of a state relationship, an interaction logic, a dependency relationship, and a navigation relationship, wherein the state relationship is a relationship between pages that may be affected by state or content, the interaction logic is a relationship of dynamic changes or page redirection caused by user interaction between pages, the dependency relationship is a dependency relationship between page content or data, and the navigation relationship is a relationship of a user's browsing path formed by connecting different pages; Take a screenshot of the web page and obtain the HTML content of the web page, and provide the screenshot of the web page and the HTML content as multimodal information to the MLLM, which generates a natural language description of the task based on the screenshot of the web page and the HTML content, so as to describe the task that the user needs to complete on the web page and the operations required to complete the task; The natural language description generated by MLLM is split according to tasks to form natural language descriptions of multiple tasks. LLM is used to convert the natural language description of each task into test code in turn. The state changes of the web page before and after the execution of the test code are used to determine whether the generated test code successfully completes the corresponding task. If it is successfully completed, the form content and operation sequence corresponding to the task are generated to generate a valid HTTP request.

2. The method for fuzz testing of Web applications based on a multimodal large model assisted web crawler according to claim 1, characterized in that: The method of using LLM to collect web application page information and extract association information between web pages includes: Define a first prompt word template to prompt the LLM to collect Web page functions, Web page URL information, and extract association information between each Web page; The first prompt word template is used to conduct multiple rounds of dialogues. In each round of dialogue, the LLM is first prompted to output the web page function and web page URL information in a specified format, and then the LLM is prompted to output the association information between the web pages in a specified format; in each round of dialogue, the answer to the previous round of dialogue is provided to the LLM, and the LLM conducts the next round of dialogue based on the answer to the previous round of dialogue. After completing multiple rounds of dialogue, the final web page function, web page URL information and the association information between the web pages are obtained.

3. The method for fuzz testing of Web applications based on multimodal large model assisted web crawler according to claim 1, characterized in that: The method of using MLLM to take a screenshot of a web page and obtain the HTML content of the web page includes: A second prompt word template is defined to prompt the MLLM to generate a natural language description of the task according to the HTML content of the Web page; In the second prompt word template, MLLM is first prompted to generate code to use puppeteer to simulate operations to complete the screenshot of the web page and obtain the HTML content, and then prompted to output natural language to describe the tasks that the user needs to complete on the current web page and the operations required to complete the tasks.

4. The method for fuzz testing of Web applications based on multimodal large model assisted web crawler according to claim 3 is characterized in that: The second prompt word template also includes adding an attribute element-id to each element of the HTML content as a unique identifier of each element.

5. The method for fuzz testing of Web applications based on multimodal large model assisted web crawler according to claim 3 is characterized in that: After using puppeteer to simulate the operation to complete the screenshot of the web page and obtain the HTML content, it also includes removing invisible elements from the obtained HTML content, removing attributes in the elements that are useless for understanding the elements, and extracting interactive elements, retaining attributes that are effective for inferring the functions of the elements, and obtaining the final HTML content.

6. The method for fuzz testing of Web applications based on a multimodal large model assisted web crawler according to any one of claims 1 to 5, characterized in that: By defining a third prompt word template, each task uses the third prompt word template to conduct multiple rounds of dialogues. In each round of dialogue, the LLM is first prompted to obtain a natural language description of a task and convert it into test code. Then, the LLM is prompted to determine whether the generated test code successfully completes the corresponding task based on the state changes of the Web pages before and after the execution of the test code. If successful, the LLM is prompted to output the form content and operation sequence corresponding to the task based on the association information between the Web pages, and continue to obtain the natural language description of the next task to generate the corresponding form content and operation sequence until all tasks are completed.

7. The method for fuzz testing of Web applications based on multimodal large model assisted web crawler according to claim 6 is characterized in that: The step of judging whether the generated test code successfully completes the corresponding task according to the state change of the web page before and after the test code is executed includes: Submit the page content before and after executing the generated test code to LLM, and use LLM to infer the page state changes; It is determined whether the currently generated test code has successfully completed the task according to the page status change, wherein if the page changes, it is determined that the task has been successfully completed, otherwise it is determined that the task has not been successfully completed.

8. The method for fuzz testing of Web applications based on multimodal large model assisted web crawler according to claim 7, characterized in that: If it is determined that the currently generated test code successfully completes the task, then continue to obtain the natural language description of the next task to generate the corresponding form content and operation sequence. If it is determined that the currently generated test code does not successfully complete the task, then ask the LLM again to generate new executable test code for the current task. When the number of questions reaches the preset number of questions, the current task is abandoned and the natural language description of the next task is continued to be selected to generate the corresponding form content and operation sequence.

9. A Web application fuzz testing system based on a multimodal large model assisted web crawler, comprising a processor and a memory connected to each other, wherein the memory is used to store a computer program, characterized in that: The processor is programmed or configured to execute the Web application fuzz testing method based on a multimodal large model assisted web crawler as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program / instruction stored therein, characterized in that: The computer program / instruction is programmed or configured to execute the Web application fuzz testing method based on a multimodal large model assisted web crawler as described in any one of claims 1 to 8 through a processor.

Images