A Method and System for Oriented Fuzzy Testing of GUI Programs Based on Large Model Intelligent Agents

Abstract

This invention discloses a method and system for targeted fuzz testing of GUI programs based on a large-scale intelligent model. The method includes: receiving the source code of the GUI program to be tested and constructing an event flow graph through static analysis; obtaining target associations and dependency chains through reverse event flow analysis based on the event flow graph and the target code region specified by the user; generating an initial event sequence that conforms to the interaction logic through a multimodal large-scale model based on the real-time state and constraint information of the GUI interface; and performing targeted evolution on the initial event sequence based on the event constraint relationships to obtain the test results. This invention achieves a deep integration of the target-driven capability of targeted fuzz testing and the complex interaction adaptation capability of environmental awareness testing, solving the problems of insufficient depth and low efficiency in the deep vulnerability detection of GUI programs in existing technologies.

Description

Technical Field

[0001] This invention relates to the field of software vulnerability detection, specifically to a method and system for targeted fuzz testing of GUI programs based on large model intelligent agents. Background Technology

[0002] In the field of software vulnerability detection, fuzzing, as an important technical method, has been widely researched and applied in recent years. Traditional coverage-guided grey-box fuzzing, with AFL as a typical example, guides input mutation by monitoring improvements in code coverage, thereby exploring new program paths. This type of method performs excellently in breadth of exploration and can effectively discover diverse execution paths. However, due to its lack of focus on specific vulnerability locations, it often has low vulnerability triggering efficiency under deep logic or complex constraints.

[0003] To improve the targeting of vulnerability discovery, some researchers have proposed Directed Grey-box Fuzzing (DGF). The core idea is to focus fuzzing activities on specific target locations within a program that have security significance, such as patch modifications, functions in crash stacks, potentially vulnerable basic blocks, or sensitive functions and system call points. Directed fuzzing analyzes the distance or correlation between the input and the target location, allocating more energy to seeds that are more likely to reach the target, thus triggering potential vulnerabilities with a higher probability.

[0004] A typical example is Hawkeye, which constructs call graphs and control flow graphs based on static analysis. By calculating the distance from seed inputs to the target location (such as function call distance, basic block distance, etc.), it prioritizes scheduling seeds with closer distances during the fuzzing process, thereby accelerating the speed at which the target location is reached. Subsequently, DAFL introduces data dependency analysis, providing feedback only to data flow paths related to the target location, enabling the fuzzer to more accurately focus on inputs that can affect the target state. This type of method achieves targeted optimization in both path and data flow dimensions, significantly improving the efficiency of triggering deep vulnerabilities.

[0005] Building upon single-objective orientation, some studies have further developed multi-objective orientation fuzzing. For example, FishFuzz achieves parallel exploration of multiple target points through dynamic target ranking, energy redistribution, and a hybrid "coverage + target" scheduling strategy; Prospector proposes iterative prioritization and dynamic target redistribution mechanisms for large-scale target sets, enabling the fuzzer to balance exploration and focusing in multi-objective environments. Furthermore, FuzzGuard and BEACON further improve the efficiency of orientation strategies by avoiding execution of code regions unrelated to the target, through deep learning prediction and provable path pruning, respectively.

[0006] However, current research on targeted fuzzing generally fails to consider the diversity of execution environments. Most targeting methods still assume a single input source (such as standard input or a specific file), ignoring external dependencies such as configuration files, GUI resources, and window system events. This simplistic assumption prevents programs from entering parts of the state space or triggering specific logical paths, thus limiting the scope of vulnerability coverage.

[0007] In view of the above, this application is hereby submitted. Summary of the Invention

[0008] This invention provides a directional fuzz testing method for GUI programs based on large model intelligent agents, comprising:

[0009] S101, Receive the GUI program source code to be tested, and construct an event flow graph for the GUI program source code through static analysis. Based on the event flow graph and the target code area specified by the user, obtain the target association and dependency chain through reverse event flow analysis.

[0010] S102, based on the real-time status of the GUI interface and constraint information, an initial event sequence conforming to the interaction logic is generated through a multimodal large model; wherein, the constraint information includes target association and dependency chain, data dependency and control dependency defined by the event flow graph, and functional description of the target code region;

[0011] S103, based on the initial event sequence, the initial event sequence is subjected to directional evolution with event constraint relationships as the core to obtain test results; wherein, in the directional optimization process, without violating the data dependency and control dependency relationships in the event flow graph, new event sequences are generated by mutation, and the proximity of the new event sequences to the target code region is evaluated by event-level distance metric. Event sequences that are closer are regarded as better individuals and obtain higher mutation priority in subsequent iterations.

[0012] Preferably, in step S101, by abstracting the events in the identified GUI program source code into nodes of the event flow graph and abstracting the dependencies between events into directed edges of the event flow graph, a structured event logic model, namely the event flow graph, is formed.

[0013] Preferably, the dependencies include control dependencies and data dependencies; the control dependencies refer to the logical constraints of the order of event execution, which are determined by analyzing the calling relationship of event callback functions and program business logic; the data dependencies refer to the constraints of the source of input data required for event execution, which are determined by analyzing the parameter passing of event callback functions and global variable references.

[0014] Preferably, in step S101, the extraction process of target association and dependency chain is as follows:

[0015] Starting from the entry address of the target code region specified by the user, trace the call stack backward to find the event callback function that directly calls the target code region, and obtain the direct target event corresponding to the target code region;

[0016] Based on the dependencies in the event flow graph, the predecessor dependent events of the direct target event are recursively traced to form a complete dependency chain including the predecessor events, the direct target event, and the target code region dependencies.

[0017] Preferably, in step S102, the multimodal large model generates an initial event sequence through the following steps:

[0018] Receive screenshots of the interface and a dictionary of control attributes generated by the GUI automation tool based on the real-time status of the GUI interface, and identify the semantic functions of the controls through visual-text alignment learning;

[0019] Receive the target associations and dependency chains output by static analysis, and combine them with the understanding of the semantic functions of the controls to plan the event execution order that meets the constraints;

[0020] By combining the functional description of the target code region with the target-oriented event parameter selection in the event execution sequence, a natural language description of the event sequence is generated;

[0021] The natural language description of the event sequence is converted into an instruction format executable by a GUI automation tool to obtain the initial event sequence.

[0022] Preferably, step S103 specifically includes:

[0023] Receive an initial event sequence and mutate the initial event sequence using a dual mutation mechanism that combines event sequence structure mutation and event parameter mutation to obtain a seed sequence;

[0024] For seed sequences generated by mutation, their quality is determined by whether they can reach the target code region and the depth of their reach. Specifically, if no seed sequence has reached the target code region yet: the number of events of the target association and dependency chain extracted by static analysis covered by the seed sequence is counted. If the current seed sequence can execute new events of the target association and dependency chain, it is considered a high-quality seed sequence and is fed back to the next iteration to gradually approach the target code region through continuous iteration. If a seed sequence has already reached the target code region: the difference between the target path of the new seed sequence and the generated seed sequence is compared. If the new seed sequence can cover the target path that other seed sequences have not reached, it is marked that the seed sequence has reached the target code region through another undiscovered path.

[0025] Test results are generated based on the final high-quality seed sequence.

[0026] Preferably, the event sequence structure mutation is a mutation operation performed on the execution order and event type of the initial event sequence. The mutation operation strictly follows the control dependencies and data dependencies in the event flow graph, and the generation of invalid sequences that violate constraints is prohibited. Supported mutation operations include event insertion, event replacement, and event reordering.

[0027] Preferably, the event parameter mutation is performed on the input parameters contained in the event sequence to cover boundary values, outliers and special scenarios, thereby increasing the probability of vulnerability triggering; wherein, the mutated parameters include numerical parameters, text parameters and option parameters.

[0028] This invention also provides a GUI program-based fuzzy testing system for large model agents, comprising:

[0029] The static analysis module is used to receive the source code of the GUI program to be tested, and to construct an event flow graph through static analysis of the GUI program source code. Based on the event flow graph and the target code area specified by the user, the target association and dependency chain are obtained through reverse event flow analysis.

[0030] The multimodal agent module is used to generate an initial event sequence that conforms to the interaction logic based on the real-time status of the GUI interface and constraint information through a multimodal large model; wherein, the constraint information includes target associations and dependency chains, data dependency relationships and control dependency relationships defined by the event flow graph, and functional descriptions of the target code region;

[0031] The directional fuzz testing module is used to perform directional evolution on the initial event sequence generated by the multimodal agent module, with event constraints as the core, to obtain test results. In the directional optimization process, without violating the data dependencies and control dependencies in the event flow graph, new event sequences are generated through mutation. The proximity of the new event sequences to the target code region is evaluated by event-level distance metrics. Event sequences with closer proximity are regarded as better individuals and have higher mutation priority in subsequent iterations.

[0032] Compared to existing technologies, the embodiments of the present invention achieve the following beneficial effects in GUI program vulnerability detection through a collaborative architecture of "static analysis-intelligent generation-directed mutation":

[0033] (1) Improved target reach efficiency: Based on the dependency constraints of the Event Flow Graph (EFG) and the semantic understanding of the large model, an initial sequence that conforms to the GUI interaction logic is generated, which improves the efficiency of the first target reach during testing for complex GUI programs.

[0034] (2) Significantly enhanced vulnerability coverage:

[0035] Based on the initial event sequence, a targeted mutation strategy based on event dependency constraints is used to adjust the event combinations and parameters with high correlation. This enables the continuous discovery of potential vulnerabilities after reaching the target path, significantly improving the depth and breadth of vulnerability discovery. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating the directional fuzzy testing method for GUI programs based on large model intelligent agents provided in the first embodiment of the present invention;

[0037] Figure 2 This is a detailed flowchart of step S101;

[0038] Figure 3 This is a detailed flowchart of step S102;

[0039] Figure 4 This is a detailed flowchart of step S103;

[0040] Figure 5 This is a schematic diagram of the structure of the GUI program-based fuzzy testing system for large model intelligent agents provided in the second embodiment of the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] To better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0043] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0044] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0045] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0046] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0047] The use of "first" and "second" in the embodiments is merely to distinguish similar objects and does not represent a specific ordering of objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permitted. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0048] Please see Figure 1The first embodiment of the present invention provides a GUI program orientation fuzzing method based on a large model intelligent agent, which can be executed by a GUI program orientation fuzzing testing device based on a large model intelligent agent (hereinafter referred to as the testing device), specifically, by one or more processors within the testing device, to achieve the following steps:

[0049] S101: Receive the GUI program source code to be tested, and construct an event flow graph for the GUI program source code through static analysis. Based on the event flow graph and the target code area specified by the user, obtain the target association and dependency chain through reverse event flow analysis.

[0050] In this embodiment, GUI stands for Graphical User Interface, which refers to the interface form through which users interact with programs using visual controls such as windows, buttons, and input boxes. The GUI can be generated from GUI program source code.

[0051] In this embodiment, combined with Figure 2 An Event Flow Graph (EFG) is a structured model that abstracts interactive events that can be triggered in a GUI program into nodes, and the control dependencies and data dependencies between events into directed edges. It is used to extract target association constraints and guide the generation of effective event sequences.

[0052] Specifically, an event flow graph can be formed by abstracting the events in the parsed and identified GUI program source code into nodes of an event flow graph, and abstracting the dependencies between events into directed edges of the event flow graph.

[0053] In this embodiment, the dependencies in the event flow graph include control dependencies and data dependencies. Wherein:

[0054] The control dependency relationship refers to the logical constraints on the order of event execution, which is determined by analyzing the calling relationship of event callback functions and the program business logic.

[0055] For example, in the "user login" process, the "click login button" event (node ​​B) must be executed after the "enter account" event (node ​​A1) and the "enter password" event (node ​​A2). Therefore, directed edges A1→B and A2→B are constructed in EFG. If A1 or A2 is not executed and the B event is triggered directly, the program will pop up a "please complete account and password" prompt. Such invalid paths will be excluded by controlling dependencies.

[0056] The data dependency refers to the constraints on the source of input data required for event execution, which is determined by analyzing the parameter passing of the event callback function and the reference to global variables.

[0057] For example, in the "Transfer Amount Input" event (node ​​C), the input amount parameter needs to be passed to the callback function of the "Confirm Transfer" event (node ​​D), and it needs to meet the business rule of "amount > 0". Therefore, a directed edge from C to D is constructed in EFG, and the data transfer fields (such as transferAmount) and data validity constraints (transferAmount > 0) are labeled.

[0058] In this embodiment, target association and dependency chain extraction is performed based on the target code region specified by the user (such as the suspected vulnerability function divideCalc(), the key business logic block "transfer exception handling module"), and by locating the "target-related events" that can trigger the target region through reverse event flow analysis.

[0059] Specifically, starting from the entry address of the target code region specified by the user, the call stack is traced backward to find the event callback function that directly calls the region (i.e., the "direct target event"). Then, based on the dependency relationship of EFG, the predecessor dependent events (including control dependency and data dependency events) of the direct target event are recursively traced to form a complete dependency chain of "predecessor event 1 → predecessor event 2 → ... → direct target event → target code region".

[0060] For example, if the target code area is the divideCalc() function (division calculation logic) of the calculator program, tracing back reveals that it is triggered by the "click equals button" event (direct target event), which depends on the "click division button" event and two "input number" events, ultimately forming a target dependency chain of "input number 1 → click ÷ → input number 2 → click = → divideCalc()".

[0061] S102, based on the real-time status of the GUI interface and constraint information, an initial event sequence conforming to the interaction logic is generated through a multimodal large model; wherein, the constraint information includes target association and dependency chain, data dependency and control dependency defined by the event flow graph, and functional description of the target code region.

[0062] In this embodiment, as Figure 3 As shown, the core implementation of S102 includes:

[0063] Multimodal information input and preprocessing, wherein the multimodal information includes:

[0064] Constraint information consists of target associations and dependency chains output by static analysis (such as "input number 1 → click ÷ → input number 2 → click ="), event dependency rules in EFG (such as "input number" must precede "click operator") and functional descriptions of the target code area (such as "test division calculation logic, which must cover the abnormal scenario of division by 0").

[0065] Real-time GUI interface information: Screenshots of the interface captured in real time (uniform resolution 1920×1080, PNG format) and control attribute lists (including control ID, type, position coordinates, text description, and enabled status) are obtained through GUI automation tools (such as AutoIt on Windows and UIAutomator on Android). In the preprocessing stage, the control attribute lists are converted into structured dictionaries (e.g., {"control_id":"btn_div","type":"Button","text":"÷","position":"(500,300)"}), and the interface screenshots are lightweight compressed and feature-annotated (e.g., using bounding boxes to mark control positions).

[0066] In this embodiment, based on the information obtained from the above preprocessing, the multimodal large model (such as GPT-4V) can perform semantic reasoning and sequence planning, such as performing interface semantic recognition, control semantic mapping (control → event), and target code region understanding, to generate an initial event sequence that conforms to the interaction logic, i.e., a target-related event sequence, specifically:

[0067] Understanding Control Functionality: By inputting screenshots of the interface and a dictionary of control attributes into the model, the multimodal large model learns through visual-text alignment to identify the semantic functions of the controls. For example, based on the combined features of "text description '÷' + control type Button + located in the calculator numeric keypad area," the multimodal large model determines that the control is a "division operator button," which will trigger the division operation logic when clicked; based on "text description 'input box' + type TextBox + can receive numeric input," it determines that it is a "numeric input control," and the input content will be used as the calculation parameter.

[0068] Dependency constraint satisfaction: The multimodal large model receives the target associations and dependency chains (such as "enter number 1 → click ÷ → enter number 2 → click =") from the static analysis output, understands the results in conjunction with the control functions, and plans the event execution order that meets the constraints. For example, the multimodal large model specifies that "enter number 1" must correspond to the subsequence of "click the number input box → enter a number (such as 8)", and "click ÷" must correspond to the event of "click the division operator button". Moreover, the subsequences must strictly follow the dependency order of "enter number → click operator → enter number → click equal sign" to avoid generating invalid sequences such as "click ÷ first and then enter number".

[0069] Target scenario adaptation: Target association and dependency chain are combined with the functional description of the target code region (e.g., "covering the abnormal scenario of division by 0") to incorporate target-oriented event parameter selection into sequence planning, generating a natural language-described event sequence. For example, when generating the subsequence "input number 2", the target association and dependency chain preferentially selects "0" as the input parameter to trigger the division by 0 exception handling logic in the divideCalc() function, improving the target relevance of the initial sequence.

[0070] Sequence format conversion and output: Convert the natural language description event sequence generated by the multimodal large model (such as "click the number input box, enter 8, click the ÷ button, click the number input box, enter 0, click the = button") into an instruction format that can be executed by GUI automation tools, i.e., the initial event sequence.

[0071] S103, based on the initial event sequence, with event constraint relationships as the core, the initial event sequence is subjected to directional evolution to obtain test results.

[0072] Among them, such as Figure 4 As shown, step S103 specifically includes:

[0073] First, an initial event sequence is received, and the initial event sequence is mutated using a dual mutation mechanism that combines event sequence structure mutation and event parameter mutation to obtain a seed sequence.

[0074] Among them, event sequence structure mutation involves modifying the execution order and event types of the initial event sequence. The mutation operation strictly adheres to the control dependency and data dependency rules in EFG, prohibiting the generation of invalid sequences that violate constraints. Supported mutation operations include:

[0075] Event insertion: Insert relevant events at positions that match the dependencies. For example, between “Enter number 1 (8) → Click ÷”, insert the event “Click the sign toggle button (±)” to generate the sequence “Enter 8 → Click ± → Click ÷ → Enter 0 → Click =" to test the scenario of negative numbers participating in division operations;

[0076] Event replacement: Replace similar events while keeping dependencies unchanged. For example, replace "click the ÷ button" in the initial sequence with "click the × button" (if the target code area also contains multiplication logic), but ensure that the replaced event is still related to the target code;

[0077] Event reordering: Only events with no dependencies are reordered. For example, if the sequence contains two undependent events, "Enter remarks" and "Select transfer method", their execution order can be swapped, but dependent events such as "Enter transfer amount" and "Confirm transfer" cannot be reordered.

[0078] Event parameter mutation involves modifying input parameters (such as numeric input, text input, and option selection) within an event sequence, focusing on boundary values, outliers, and special scenarios to increase the probability of vulnerability triggering. Specific mutation rules include:

[0079] Numerical parameter mutation: For numeric inputs (such as amount, quantity), generate boundary values ​​(such as 0, maximum value 2). 31 -1, minimum value -2 31 ), outliers (such as NaN, Infinity), and values ​​in special formats (such as scientific notation 1e+20, hexadecimal 0xFF);

[0080] Text-based parameter mutation: For text input (such as account, remarks), generate extremely long strings (such as 1000 consecutive characters) and special character combinations (such as... <script>、.. / ）、空值与空格；

[0081] 选项型参数变异：针对下拉框、复选框等选择类控件，选择未被初始序列覆盖的选项（如初始选择 "储蓄卡”，变异为"信用卡”），或同时选择多个冲突选项（如复选框同时勾选 "退款”与"不退款”）。

[0082] 在本实施例中，对变异生成的种子序列，以序列是否能触达目标代码区域及触达深度为条件进行种子质量评估，判断其是否达到目标区域（即是否为优质种子序列）；其中，若当前尚未有种子序列到达目标代码区域：统计种子序列覆盖静态分析提取的目标依赖链事件数量，若当前种子序列能够执行到了新的目标依赖链事件，则视为优质种子，并反馈至下一轮迭代（即结合当前迭代的初始事件序列与历史优秀序列进行迭代），以通过持续迭代逐步向目标代码区域逼近；若当前已有种子序列到达目标代码区域：对比新种子序列与已生成种子序列的目标路径差异，若新种子序列能覆盖其他种子序列未触达的目标路径，即标记该种子序列以另一条未发现的路径到达目标代码区域。这种方式可避免种子同质化，确保测试能探索目标代码区域的多样化执行路径。

[0083] 最后，根据最终的优质种子序列生成测试结果。

[0084] 相较于现有技术，本发明实施例通过"静态分析-智能生成-定向变异” 的协同架构，在GUI程序漏洞检测中实现如下有益效果：

[0085] （1）目标触达效率提升：依托事件流图（EFG）的依赖约束与大模型语义理解，生成符合GUI交互逻辑的初始序列，针对复杂的GUI 程序，提高测试时首次触达目标的效率。

[0086] （2）漏洞覆盖能力显著增强：

[0087] 在初始事件序列基础上，通过基于事件依赖约束的定向变异策略，对事件组合与参数进行高相关性调整，能够在到达目标路径后持续挖掘潜在漏洞，显著提升漏洞发现的深度与广度。

[0088] 本发明实施例具有广泛的应用前景，例如，可应用如如下场景：

[0089] （1）软件安全测试工具：可转化为桌面端（Windows / macOS）、移动端（Android / iOS）GUI 程序专用测试平台，重点服务金融 APP（检测支付逻辑漏洞）、办公软件（检测权限控制漏洞）等，满足企业自动化漏洞检测需求。

[0090] （2）DevOps 流程集成：嵌入软件开发 "持续集成 - 持续部署（CI / CD）” 环节，代码提交后自动触发定向测试，提前拦截GUI功能新增 / 优化中的漏洞，降低软件测试与漏洞修复的成本。

[0091] （3）关键领域安全保障：适用于电力监控系统、政务服务 APP、工业控制GUI等场景，定向检测 "数据采集”"控制指令发送” 等核心模块漏洞，保障关键信息基础设施与公共服务软件安全。

[0092] 请参阅图5，本发明第二实施例还提供了一种基于大模型智能体的GUI程序定向模糊测试系统，其包括：

[0093] 静态分析模块210，用于接收待测试的GUI程序源码，并对所述GUI程序源码通过静态分析构建事件流图，基于所述事件流图以及用户指定的目标代码区域，通过反向事件流分析得到目标关联与依赖链；

[0094] 多模态Agent模块220，用于基于GUI界面实时状态以及约束信息，通过多模态大模型生成符合交互逻辑的初始事件序列；其中，所述约束信息包括目标关联与依赖链、由事件流图定义的数据依赖关系与控制依赖关系及目标代码区域的功能描述；

[0095] 定向模糊测试模块230，用于在多模态Agent模块生成的初始事件序列的基础上，以事件约束关系为核心，对初始事件序列进行定向进化，得到测试结果；其中，在定向优化过程中，在不违反事件流图中的数据依赖关系与控制依赖关系的前提下，变异生成新的事件序列，并通过事件级距离度量评估新的事件序列与目标代码区域的接近程度，将距离更近的事件序列视为更优个体，并在后续迭代中获得更高的变异优先级。

[0096] 以上所述是本发明的优选实施方式，应当指出，对于历史文档修复技术领域的普通技术人员来说，在不脱离本发明核心原理（如扩散模型架构、隐式神经表示增强和多尺度训练策略）的前提下，还可以做出若干改进和润饰，这些改进和润饰也视为本发明的保护范围。< / script>

Claims

1. A GUI program targeted fuzz testing method based on a large model agent, characterized in that, Comprise: S101, receive the GUI program source code to be tested, and construct an event flow graph by static analysis on the GUI program source code, based on the event flow graph and the target code region specified by the user, obtain the target association and dependency chain by reverse event flow analysis; S102, based on the real-time state of the GUI interface and the constraint information, generate an initial event sequence conforming to the interaction logic through a multi-modal large model; wherein the constraint information includes the target association and dependency chain, the data dependency relationship and the control dependency relationship defined by the event flow graph, and the functional description of the target code region; the multi-modal large model generates the initial event sequence by the following steps: receiving the interface screenshot and the control attribute dictionary generated by the GUI automation tool according to the real-time state of the GUI interface, identifying the semantic function of the control through visual-text alignment learning; receiving the target association and dependency chain output by static analysis, combining the understanding result of the semantic function of the control, and planning the event execution sequence conforming to the constraint; In combination with the functional description of the target code region, the event parameter selection of the target is integrated into the event execution sequence to generate a natural language description event sequence; the natural language description event sequence is converted into an instruction format executable by the GUI automation tool to obtain the initial event sequence; S103, on the basis of the initial event sequence, taking the event constraint relationship as the core, the initial event sequence is optimized to obtain the test result; wherein in the directional optimization process, without violating the data dependency relationship and the control dependency relationship in the event flow graph, a new event sequence is generated by mutation, and the proximity of the new event sequence to the target code region is evaluated by event-level distance measurement, the event sequence with closer distance is regarded as a better individual, and a higher mutation priority is obtained in the subsequent iteration; wherein S103 specifically comprises: receiving the initial event sequence, and using a double mutation mechanism combining event sequence structure mutation and event parameter mutation to mutate the initial event sequence to obtain a seed sequence; judging whether the seed sequence generated by mutation is a high-quality seed sequence or not according to whether the sequence can reach the target code region and the depth of reaching; wherein if there is no seed sequence reaching the target code region at present: count the number of events in the target association and dependency chain covered by the seed sequence extracted by static analysis, if the current seed sequence can execute to the events of the new target association and dependency chain, it is regarded as a high-quality seed sequence, and is fed back to the next iteration to gradually approach the target code region through continuous iteration; if there is a seed sequence reaching the target code region at present: compare the target path difference between the new seed sequence and the generated seed sequence, if the new seed sequence can cover the target path not reached by other seed sequences, mark the seed sequence to reach the target code region through another undiscovered path; generate the test result according to the final high-quality seed sequence.

2. The large model-based intelligent agent's GUI program oriented fuzz testing method of claim 1, wherein, In step S101, the events in the recognized GUI program source code are abstracted as nodes of the event flow graph, and the dependency relationship between events is abstracted as directed edges of the event flow graph, forming a structured event logic model, i.e. an event flow graph; The dependency relationship includes a control dependency relationship and a data dependency relationship; The control dependency relationship refers to a logical constraint of event execution in sequence, which is determined by analyzing the calling relationship of the event callback function and the program business logic; and the data dependency relationship refers to a constraint of the source of input data required by event execution, which is determined by analyzing the parameter passing and global variable reference of the event callback function.

3. The large model-based intelligent agent's GUI program oriented fuzz testing method of claim 1, wherein, In step S101, the extraction process of the target association and dependency chain is as follows: Taking the entry address of the target code region specified by the user as the starting point, the calling stack is traced back to find the event callback function that directly calls the target code region, and a direct target event corresponding to the target code region is obtained; Based on the dependency relationship of the event flow graph, the predecessor dependent events of the direct target event are recursively traced back to form a complete dependency chain including the predecessor events, the direct target event and the dependency relationship of the target code region.

4. The large model-based intelligent agent's GUI program oriented fuzz testing method of claim 1, wherein, The event sequence structure variation is a variation operation on the execution order and event type of the initial event sequence, and the variation operation follows the control dependency relationship and the data dependency relationship in the event flow graph, and invalid sequences that violate the control dependency relationship and the data dependency relationship are prohibited; the supported variation operations include event insertion, event replacement and event reordering.

5. The large model-based intelligent agent's GUI program oriented fuzz testing method of claim 1, wherein, The event parameter variation is a variation on the input parameters contained in the event sequence, which is used to cover boundary values, abnormal values and special scenarios to improve the vulnerability triggering probability; wherein the variation parameters include numerical parameters, text parameters and option parameters.

6. A large model-based intelligent agent GUI program directed fuzz testing system, characterized in that, It comprises: A static analysis module for receiving a GUI program source code to be tested, and constructing an event flow graph by static analysis on the GUI program source code; based on the event flow graph and a target code region specified by a user, a target association and a dependency chain are obtained by reverse event flow analysis; A multi-modal Agent module for generating an initial event sequence conforming to the interaction logic based on the real-time state of the GUI interface and constraint information; wherein the constraint information includes the target association and the dependency chain, the data dependency relationship and the control dependency relationship defined by the event flow graph, and the functional description of the target code region; the multi-modal large model generates the initial event sequence by the following steps: receiving the interface screenshot and the control attribute dictionary generated by the GUI automation tool according to the real-time state of the GUI interface, identifying the semantic function of the control through visual-text alignment learning; receiving the target association and the dependency chain output by the static analysis, combining the understanding result of the semantic function of the control, and planning an event execution sequence conforming to the constraint; combining the functional description of the target code region, integrating target-oriented event parameter selection in the event execution sequence, and generating a natural language description event sequence; converting the natural language description event sequence into an instruction format executable by the GUI automation tool to obtain the initial event sequence; The directional fuzzing test module is configured to generate a test result by performing directional evolution on an initial event sequence generated by the multi-modal agent module based on the event constraint relationship; in the directional optimization process, a new event sequence is generated without violating the data dependency relationship and the control dependency relationship in the event flow graph, and the proximity of the new event sequence to the target code region is evaluated by using an event-level distance measurement, the event sequence with a closer distance is regarded as a better individual, and a higher mutation priority is obtained in subsequent iterations; the directional fuzzing test module is specifically configured to: receive the initial event sequence, and perform mutation on the initial event sequence by using a double mutation mechanism combining event sequence structure mutation and event parameter mutation to obtain a seed sequence; the seed sequence generated by mutation is judged whether it is a high-quality seed sequence according to whether the sequence can reach the target code region and the depth of reaching; if no seed sequence has reached the target code region, the number of events of the target association and the dependency chain covered by the seed sequence is counted, if the current seed sequence can execute to the events of the new target association and the dependency chain, the seed sequence is regarded as a high-quality seed sequence, and is fed back to the next iteration to gradually approach the target code region by continuous iteration; if a seed sequence has reached the target code region, the target path difference between the new seed sequence and the generated seed sequence is compared, if the new seed sequence can cover the target path that has not been reached by other seed sequences, the seed sequence is marked to reach the target code region by another undiscovered path; and a test result is generated according to the final high-quality seed sequence.

Images