Large model-based automatically synthesizing method and system for distinguishable webgui tasks
Patent Information
- Application Number
- CN202611139668.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-29
- Publication Date
- 2026-10-30
AI Technical Summary
实践中，一个自动合成的判分器容易被多种情形误导而失效：例如仅凭访问某个网址即判为通过，从而被″只访问网址而未真正操作＂的轨迹骗过(判定过松)；又如把仅在本次会话中偶然出现的高熵随机令牌(如会话表单令牌form_key)也纳入断言，导致换一个被测智能体运行时、其令牌取值不同而被误判为失败(判定过严，过拟合于本次运行)；再如意图与最终页面状态在语义上并不一致，而判分器却判为通过
[0025]1.无需人工编写判分器即可产出可靠可判分的任务：通过从真实成功轨迹反推判分器，并施加多个自验证环节，在无人工标注答案的前提下获得可自动判分且经自验证的评测任务；这与仅产出任务或训练数据、不产出自验证判分器的现有方法存在本质区别。
Smart Images
Figure CN123011551A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of software automated testing and artificial intelligence technology, specifically to a method and system for automatically synthesizing scoreable Web GUI tasks based on a large model. This method generates executable, scoreable, and deduplicated multi-step graphical user interface (GUI) operation tasks and their corresponding automatic scorers for target Web applications without the need for manually writing scorers. Background Technology
[0002] The evaluation of graphical user interface (GUI) agents relies on large-scale operational tasks and their scoring logic: each task requires a natural language intent (e.g., "add an item to the shopping cart") and an evaluator that can automatically determine whether the agent under test has actually completed the intent. Currently, such tasks and evaluators are mainly built manually, which is costly, time-consuming, and difficult to scale with the emergence of new websites.
[0003] Existing automated task generation technologies suffer from two types of systemic defects:
[0004] Firstly, the structured exploration (web crawler-style traversal) method can mechanically traverse page links and controls, but the resulting action sequences lack semantics and are often meaningless, repetitive clicks and jumps. Furthermore, it does not include any scorer to determine whether the task is completed, and therefore cannot be directly used as an evaluation task.
[0005] Secondly, while methods based on direct generation from large language models can produce semantically meaningful task descriptions, their generation lacks systematicity. They tend to generate numerous redundant and repetitive tasks for the same function without deduplication, and they struggle to cover the deeper functionalities of a website, resulting in unsystematic coverage. More importantly, such methods (e.g., schemes that generate supervised fine-tuning training data) typically only generate tasks or trajectories, not automatic scorers. Even when scorers are generated, there is a lack of verification of their correctness.
[0006] The reliability of the scorer is crucial to the success of the evaluation task. In practice, an automatically generated scorer is easily misled and fails due to various situations: for example, judging a task as passed simply by visiting a URL, thus being fooled by the "only visiting the URL without actually performing any action" trajectory (overly lenient judgment); another example is including high-entropy random tokens (such as the session form token form_key) that only occasionally appear in the current session in the assertions, leading to a different token value when the tested agent runs the test, resulting in a false failure (overly strict judgment, overfitting to the current run); yet another example is when the intent and the final page state are semantically inconsistent, but the scorer still judges it as passed. If these problems are not resolved, the automatically generated scorer's judgments are unreliable, and the evaluation benchmark constructed based on it becomes meaningless.
[0007] Therefore, how to generate multi-step operation tasks and their scorers for target websites that are executable, reliable, scoreable, deduplicated, and cover the system without relying on manual annotation of answers has become an urgent technical problem to be solved. Summary of the Invention
[0008] To address the aforementioned problems, this invention provides a method and system for automatically synthesizing scoreable Web GUI tasks based on a large model. Its core concept is as follows: A visual language model guides the collection of real-world success trajectories; a scorer is derived from the network request side effects of these trajectories; and multiple self-verification checks are applied to the scorer for different failure types, thereby obtaining a reliable automatic scorer without manual annotation. Simultaneously, a shared set of covered functions drives frontier exploration, weighted guidance, and deduplication, achieving a systematic generation of coverage-guided functions. Furthermore, decisions involving page semantics are handled by a large model, ensuring that the core logic is free from hard-coded site information.
[0009] In this invention, a successful trajectory refers to a record of the trajectory that the browser determines as the final state of a task after performing a multi-step interaction on a target website. This trajectory includes at least the action sequence, network request logs, and the page states before and after the operation. Network request side effects refer to server-side requests triggered by interactive actions and their response results within the trajectory, particularly including non-read write requests, request body fields, and response status codes. Suspected volatile values refer to field values that occur only by chance in the current run, such as those found in scenarios involving session tokens, random identifiers, or one-time form fields. Stable evidence refers to request paths, request methods, non-volatile field values, response status codes, or page state evidence that still characterize the task completion state after excluding suspected volatile values. A scoring equivalence class refers to a set of tasks that are equivalent in scoring target, target request, key fields, final state description, or covered functionality.
[0010] The first aspect of this invention provides a method for automatically synthesizing discriminable Web GUI tasks based on a large model, comprising:
[0011] S1. Exploration and Collection Steps: The visual language model makes decisions on interactive actions step by step based on the screenshots of the target website, driving the browser to perform multiple operations on the website and collecting at least one successful trajectory. The trajectory includes the network request logs of this interaction and the page state before and after the operation.
[0012] S2. Intent synthesis step: Derive the natural language intent describing the task from the trajectory and determine the task type (at least including information retrieval and state change tasks, and optionally navigation tasks).
[0013] S3. Scorer synthesis steps: Without referring to any manually labeled answers, the scorer is derived from the network request side effects of the trajectory.
[0014] (1) For information retrieval tasks: determine the result pattern based on the shape of the retrieved data in the final response of the agent under test, and use the retrieved data as the expected value to assemble a response scorer; this process does not require the use of a large model.
[0015] (2) For state-change tasks: First, the code layer filters candidate write requests from the network request logs that are from the same origin as the target website and whose request methods are POST, PUT, PATCH, or DELETE, and restores the host and port to site-independent placeholders; then, based on the value length, the occurrence of uppercase and lowercase letters and numbers, and the character set, fields that appear to be in the form of random tokens are excluded from the fields presented to the large model; then, the large model selects the target request and key fields from the candidate write requests and the range of fields that have not been excluded; for target requests with volatile identifier segments in the path, the code layer collapses the identifier segment into a wildcard match; finally, the code layer assembles a combination of a response scorer and a network event scorer, in which the network event scorer asserts the target write request, its key fields, and the response status code. Throughout the process, the large model undertakes the semantic decision of "picking requests and selecting fields," and the code layer does not set any site keyword whitelist.
[0016] S4. Self-verification step: Apply multiple verification steps targeting different failure types to the scorer obtained in S3. Only after all steps pass can the data be written to disk. The verification steps include:
[0017] (1) Positive Example Gate: The sorting device must determine that the success trajectory from which it originates is passed, and is used to capture gross errors such as incorrect URL, incorrect method, and incorrect expected value;
[0018] (2) Cross-task gate: The scorer must reject the task on the trajectory of other tasks to catch overly lenient scoring ("let anyone pass");
[0019] (3) Generalization gate: Replace the variable identifier segment of the unified resource location path in the successful trajectory network request log with a different identifier segment to generate a twin trajectory. The scorer must still judge it as passing on the twin trajectory; otherwise, it is judged as overfitting to the identifier segment specific to this operation.
[0020] (4) Optional consistency determination: When the final state screenshot is available and the visual language model is enabled, the visual language model observes the final state screenshot after the operation to determine whether the task described in the intention has been truly completed, in order to identify the semantic misalignment between the intention and the final state.
[0021] S5. Repair steps: When the generalization gate fails, the code layer removes candidate requests containing volatile identifier segments from the Uniform Resource Locator (URL) path, and S3 reselects and resynthesizes the remaining candidate requests, and applies the verification step in S4 again; if the resynthesized scorer still fails the verification, the candidate task is discarded.
[0022] The second aspect of this invention provides a coverage-guided closed loop: maintaining a set of covered functions, and simultaneously performing frontier exploration (enumerating real but untriggered controls or unvisited function pages in the page document object model as frontier targets, with candidates coming from real controls rather than arbitrary model assumptions), weight reduction guidance (reducing the weight of covered areas), and deduplication (checking for duplicates by equivalence class, retaining only a predetermined number of different instances of the same function); and may further include progressive bootstrapping (generating sub-targets in-situ from the final state screenshot on deeper pages, extending coverage of deeper functions). Frontier exploration, weight reduction guidance, and deduplication share the same set of covered functions, and are two aspects of the same mechanism.
[0023] The third aspect of this invention provides a generalization guarantee: the core logic executing S1 to S5 does not contain any hard-coded constants for a specific website (including specific URL paths, interface text keywords, style selectors, domain target enumerations, etc.), and the decision involving page semantics is completed by the visual language model; the differences between websites are isolated in an independent adaptation layer (environment address configuration, login authentication, initial entry and running constraints), and the core logic does not need to be modified when changing the target website.
[0024] The beneficial effects of this invention include:
[0025] 1. Reliable and self-verifiable tasks can be generated without manually writing a scorer: By back-deriving the scorer from real success trajectories and applying multiple self-verification steps, an evaluation task that can be automatically scored and self-verified can be obtained without manual annotation of answers; this is fundamentally different from existing methods that only produce tasks or training data without producing a self-verifying scorer.
[0026] 2. Scoring reliability verification: Positive example gate and cross-task gate are used to detect loose scoring or structural inconsistencies. Value shape filtering, generalization gate and patching loop are used to detect overfitting to tokens specific to this run. When consistency judgment is enabled, semantic misalignment between intent and final state can also be identified.
[0027] 3. Coverage-driven and naturally deduplicated: The shared set of covered functions simultaneously drives frontier exploration and deduplication, enabling the generation to advance towards uncovered functional systems while naturally suppressing repetition, avoiding the two defects of "full of meaningless repetition" and "unsystematic coverage".
[0028] 4. Reduced site switching costs: The core logic is hard-coded on zero sites and uses a visual language model to handle semantic decisions. When switching websites, only the adaptation layer needs to be replaced, which has good portability and reproducibility.
[0029] 5. Reasonable division of mechanical and semantic functions: Constraints that can be verified by mechanical rules (judging and filtering suspected volatile values) are handled by the code layer, while judgments involving page semantics (picking evidence and determining completion) are handled by the large model, balancing reliability and generalization. Attached Figure Description
[0030] Figure 1 This is a schematic diagram of the overall process and system architecture of the method of the present invention, showing the five steps of exploration and acquisition, intent synthesis, scorer synthesis, self-verification, and repair, and their data flow.
[0031] Figure 2 This is a flowchart illustrating the self-verification and repair process and its arrangement (generation-verification-repair closed loop) of the present invention.
[0032] Figure 3 This is a schematic diagram illustrating the principle of the covered guidance closed loop of the present invention, showing how the same covered function set can simultaneously drive frontier exploration, weighted guidance, and deduplication;
[0033] Figure 4 This is a schematic diagram of a specific embodiment of the present invention for a state change type task synthesis scorer, illustrating the conversion process from network request logs to the final scorer.
[0034] Figure labeling explanations: 100 - Exploration and acquisition module; 110 - Visual language model; 120 - Browser; 130 - Success trajectory (including network request logs and previous and subsequent page states); 200 - Intent synthesis module; 300 - Scorer synthesis module; 310 - Candidate write request filtering unit; 320 - Value shape filtering unit; 330 - Large model request and field selection unit;
[0035] 340 - Discriminator assembly unit; 400 - Self-verification module; 401 - Positive example gate; 402 - Cross-task gate; 403 - Generalized gate;
[0036] 404 - Repair loop; 405 - Consistency check; 500 - Covered feature set; 510 - Frontier exploration; 520 - Demotion guidance;
[0037] 530 - Deduplication; 540 - Progressive Bootstrapping; 600 - Adaptation Layer. Detailed Implementation
[0038] The present invention will be further described below with reference to the accompanying drawings, using a web e-commerce website as an example. This embodiment is only for explaining the present invention and does not constitute a limitation on the scope of protection of the present invention. All equivalent substitutions and improvements made by those skilled in the art without departing from the concept of the present invention are within the scope of protection of the present invention.
[0039] 1. System Composition (see...) Figure 1 )
[0040] The system of this invention includes an exploration and acquisition module 100, an intent synthesis module 200, a scorer synthesis module 300, a self-verification module 400, and a repair module 404. It also maintains a cross-module shared set of covered functions 500, and an adaptation layer 600 decoupled from the core logic. The core logic (100 / 200 / 300 / 400 / 404) does not contain any site-specific constants; the adaptation layer 600 only carries site differences such as environment address, login authentication, initial entry point, account protection rules, and operational constraints, so that the core logic does not need to be modified when the system changes websites.
[0041] The scorer uses a structured data format, and its disk output includes: a task data file (containing fields such as task identifier, starting page identifier, intent, and scorer array), a side metadata file (containing action sequences, trajectory paths, target equivalence classes, and self-verifying evidence, used for research and tracing but not for scoring), and an environment configuration file. The starting page identifier uses a site-independent placeholder, without hardcoding the host and port.
[0042] 2. S1 Exploration and Data Collection (Module 100)
[0043] The visual language model 110 observes the screenshot of the current page in the browser 120, and combines it with the interactive element numbers marked in the page document object model to gradually decide the next interactive action (click, input, jump, etc.), driving the browser to execute multiple operations. The system records the network request log (HAR) of this interaction and the page state before and after the operation (screenshot, URL, title, etc.). When an operation achieves a certain functional goal, it is saved as a success trajectory 130.
[0044] During the exploration process, page semantic understanding is primarily handled by the visual language model, such as determining "whether the current page is a login page," "which element number should be clicked," and "what value should be entered in a certain field," rather than using conditional judgments based on fixed login terms in the page title or fixed copy keywords. To prevent exploration from disrupting the user's session, general security measures are implemented to prohibit users from modifying their account identity or logging out during the exploration period.
[0045] 3. S2 Intent Synthesis (Module 200)
[0046] Based on the action path and preceding and following states of the successful trajectory, the natural language intent describing the task is deduced, and the task type is determined. The task type should at least be distinguished as: information retrieval (reading and returning certain information, such as "querying my first order date"), state change (causing write side effects to the server state, such as "changing the delivery address to a certain address"); optionally, navigation tasks may also be included.
[0047] 4. S3 scorer synthesis (Module 300, see...) Figure 4 )
[0048] 4.1 Information Retrieval Category
[0049] The result pattern is determined based on the data shape of the retrieval result fields in the final response of the tested agent. For example, when the return value is a string array, the result pattern includes array type constraints and constraints that the array elements are strings. Then, this retrieval data is used as the expected value and assembled into a response scorer. This process does not use a large model. This class is used to illustrate that information retrieval tasks can be directly scored by structured results.
[0050] 4.2 Status Change Classes (Focus of this Invention)
[0051] For example, to change the delivery address to 654 Aspen Road, Boston, 02110:
[0052] Step ① (Candidate Write Request Filtering Unit 310, Executed at the Code Level): Filter requests from the network request log that originate from the target website and use the POST, PUT, PATCH, or DELETE method. Restore the host and port in the URL to site-independent placeholders and address paths. In this example, a POST write request is selected. Its fields include first name, last name, address, city, zip code, country, and session token; the corresponding values are Emma, Lopez, 654Aspen Road, Boston, 02110, US, and a random token, respectively.
[0053] Step ② (Value Shape Filtering Unit 320, executed at the code level): The value shape of candidate fields is judged. Fields exhibiting a suspected random token shape (e.g., random values in a session token field) are excluded from the fields displayed to the large model, while non-random string values such as postal codes and cities are retained. In one embodiment, field values with a length of at least 12, containing uppercase letters, lowercase letters, and numbers, and consisting only of letters, numbers, underscores, hyphens, plus signs, forward slashes, or equal signs, are judged as potentially volatile values. Values such as purely numeric postal codes, natural language addresses, city names, and country codes are not excluded as volatile fields. The above judgment is based on the field value shape, not the site-specific field name.
[0054] Step 3 (Large Model Request and Field Selection Unit 330): The intent and candidate request list (with volatile fields removed) are given to the large model, which will then return: which request to select, which key fields to assert, and whether to use exact matching for the URL or wildcard matching for obviously volatile identifier segments (default is exact). No site keyword whitelist is set at the code level; the complete candidate list is directly given to the large model.
[0055] Step ④ (Scorer Assembly Unit 340, Code Layer Execution): Assemble into a combination of two scorers—a response scorer and a network event scorer. The response scorer asserts that the agent under test returns a successful status for the state change task; the network event scorer asserts the request path, key fields and values, and successful response status code of the target POST request. The response status code assertion is used to eliminate false positives of "request sent but actually failed" and to identify successful requests when multiple POST requests occur on the same request path.
[0056] In a non-limiting embodiment, the scorer data structure may include the following table fields:
[0057]
[0058] The multiple scorers are logically ANDed, and a score is considered successful only if all scorers pass.
[0059] 5. S4 self-verification (Module 400, see...) Figure 2 )
[0060] Self-verification is driven by an orchestration process to generate a verification-repair closed loop, sequentially applying verification steps for different failure types; when the generalization gate fails, the S5 repair step is triggered, and data is only pushed to disk after all steps pass.
[0061] ① Positive Example Gate 401: Uses a synthesized scorer to determine the successful trajectory from which it originates. If it is determined to pass, it captures gross errors such as incorrect URL, incorrect method, or incorrect expected value.
[0062] ② Cross-task gate 402: This scorer judges the trajectory of other tasks and must reject them to capture overly lenient scoring. For example, if the scoring is too lenient, such as "as long as other POSTs occur, it is considered passed", it will be mistakenly judged as passed on the cross-task trajectory and thus exposed.
[0063] ③ Generalization Gate 403: Replaces the volatile identifier segments of the URL path in the successful trajectory network request log to obtain a twin trajectory. When generating the twin trajectory, the request method, non-volatile path segments, request body fields, response status codes, action sequences, and page state evidence remain unchanged; only the volatile identifier segments are replaced with identifier segments of the same length and character category but different specific characters. The scorer must still classify this twin trajectory as pass. If it fails, it means that the scorer asserted that the path identifier segment is valid only in this run, i.e., overfitting, and the generalization gate fails.
[0064] ④ Consistency Decision 405 (Visual Language Model): Observe the final state screenshot after the operation using the visual language model to determine whether the task described in the intention has been truly completed, and capture the "semantic misalignment between intention and final state" that may still be missed by the structural verification.
[0065] The aforementioned verification steps address different types of scoring failures: the positive example gate and cross-task gate address overly strict or loose scorer structure; the generalization gate addresses overfitting to the current path identifier segment; and the consistency judgment addresses semantic misalignment between the intent and the final state. Mechanically verifiable elements (value shape filtering and path identifier segment replacement) reside at the code level, while semantically relevant elements (S3 evidence selection, ④ completion judgment) reside at the large model level. The scorer's self-verification of its own correctness is the essence that distinguishes this invention from methods that only generate training data.
[0066] 6. S5 Repair Procedure (Module 404, see...) Figure 2 )
[0067] When the generalization gate fails, the code layer removes candidate requests containing volatile identifier segments from the Uniform Resource Locator (URL) path. S3 then reselects the target request and key fields from the remaining candidate requests, resynthesizes the scorer, and reapplies the validation in S4. This removal is performed by the code layer, rather than by modifying the prompts given to the large model, thus solidifying mechanically verifiable constraints at the code level. If the resynthesized scorer still fails validation, the candidate task is discarded.
[0068] 7. Coverage of the closed-loop boot process (Module 500, see [link]) Figure 3 )
[0069] The system maintains a set of 500 covered functions and executes them simultaneously based on the same set:
[0070] (1) Frontier Exploration 510: Enumerate the writable controls that actually exist in the current page document object model but have not yet been triggered, as well as the functional pages that have not yet been accessed, as the frontier targets that must be done; the candidates come from the actual controls on the page rather than the model's imagination, thus ensuring that the target can be achieved.
[0071] (2) Deweighting guidance 520: Deweight the targets in the area corresponding to the covered function set, so that the target selection is tilted towards the uncovered functions.
[0072] (3) Deduplication 530: Before the synthesis result is written to disk, deduplication is performed according to the scoring equivalence class (which can be based on the target state, scoring target, action sequence abstraction, final state description or covered function, etc.). Only a predetermined number of k different instances are retained for the same function.
[0073] (4) Progressive Bootstrapping 540: When exploration reaches a deep page that is not visible from the home page, the visual language model generates a new sub-target from the final state screenshot in place, extending the coverage to the deep functionality.
[0074] Frontier exploration, de-weighting guidance, and deduplication sharing of the same covered function set are two aspects of the same mechanism of "guiding coverage" and "suppressing duplication": the more fully the coverage is promoted, the more thoroughly duplication is suppressed.
[0075] 8. Standardized Support and Site Switching Implementation (Module 600)
[0076] The core logic executing S1 through S5 does not contain any site-specific constants. The following patterns are prohibited in the core logic, and alternatives are provided: Do not hardcode specific URL paths (inferring them from screenshots using the visual language model, or dynamically discovering them from the page DOM); Do not hardcode single-language interface text keywords (identifying interaction intent using the visual language model, or matching using accessibility tags, input types, roles, and autofill attributes in semantic attributes); Do not hardcode style selectors (determining them based on element numbers using the visual language model, or using semantic selectors); Do not hardcode domain target enumerations and business process scripts (dynamically generating them from screenshots using the visual language model).
[0077] Site differences are isolated in the adaptation layer 600, which only includes environment address configuration, login authentication, initial entry point, account protection rules, and operational constraints. Therefore, when changing the target website from an e-commerce site to another type of website (such as a document site), no changes to the core logic are required; only the adaptation layer needs to be replaced to ensure the entire workflow runs smoothly.
[0078] 9. Connection between plate placement and evaluation
[0079] The synthesized results are stored on disk in a structured data format, which can be read and scored by the scoring execution component. A side-mounted metadata file records the action sequence, trajectory path, target equivalence class, and self-verifying evidence (such as positive example acceptance, cross-task rejection, and final state consistency judgment results), for research retrospective purposes but not for scoring.
[0080] The above embodiments demonstrate that the present invention can generate executable, classifiable, deduplicated, and overridden multi-step GUI operation tasks for target Web applications and their self-verified reliable scorers without the need for manually writing scorers.
Claims
1. A method for automatically synthesizing discriminable Web GUI tasks based on a large model, characterized in that, Includes the following steps: S1. Exploration and collection steps: The visual language model makes decisions on interactive actions step by step based on the screenshots of the target website page, drives the browser to perform multiple operations on the website, and collects at least one successful trajectory. The trajectory includes the network request log of the interaction and the page state before and after the operation. S2, Intent Synthesis Step: Based on the trajectory, deduce the natural language intent describing the operation task, and determine the task type, which includes at least information retrieval and state change tasks; S3. Scorer Synthesis Step: Without referring to manually labeled answers, the automatic scorer for the task is derived by back-engineering the network request side effects of the trajectory; for state change type tasks, candidate write requests that are from the same source as the target website and whose request methods are POST, PUT, PATCH or DELETE are filtered from the network request log. The host and port in the candidate write requests are replaced with site-independent placeholders. The field values in the candidate write requests are identified as potentially volatile fields according to preset length and character category rules and excluded from the assertable fields shown to the large model. The large model selects the target request and key fields from the candidate write requests and the unexcluded fields, and assembles a network event scorer that includes at least the target request, key fields and response status code assertions. S4. Self-verification step: Apply multiple verification steps with different failure types to the scorer synthesized in S3 for self-verification. Only when all verification steps pass will the task and its scorer be stored as the synthesis result on disk. The verification steps include at least: positive example verification where the scorer is determined to pass on the successful trajectory, cross-task verification where the scorer is determined to reject on the trajectories of other tasks, and generalization verification where the variable identifier segment of the Uniform Resource Locator (URL) path in the successful trajectory is replaced to generate a twin trajectory, and the scorer still determines that it passes on the twin trajectory. S5. Repair step: When the generalization check fails, remove the candidate requests containing volatile identifier segments in the Uniform Resource Locator (URL) at the code level, so that S3 can reselect the target request and key fields within the remaining candidate request range and resynthesize the scorer, and then execute S4 again.
2. The method according to claim 1, characterized in that, The suspected volatile field identification in S3 includes: calculating the length of the field value, the occurrence of uppercase and lowercase letters and numbers, and the character set; when the length of the field value is not less than a preset length threshold, contains uppercase letters, lowercase letters and numbers, and consists only of a preset token character set, the field value is judged as a suspected volatile value, and the corresponding field is excluded from the assertable fields displayed to the large model, thereby avoiding the scorer asserting values that are only accidentally true in this session; the twin trajectory in S4 is generated in the following way: while keeping the request method, non-volatile path segment, request body field value, response status code and page status evidence of the network request unchanged, the volatile identifier segment of the Uniform Resource Locator (URL) path in the network request log is replaced with a different identifier segment that matches the length and character category of the original identifier segment; if the synthesized scorer determines rejection on the twin trajectory, it is determined that the scorer is overfitted to the identifier segment specific to this run, and the generalization verification fails.
3. The method according to claim 1, characterized in that, The verification step described in S4 also includes consistency determination: by observing the screenshot of the page state after the operation using a visual language model, it is determined whether the task described by the natural language intent is actually completed in the final state, in order to identify semantic misalignment between the intent and the final state.
4. The method according to claim 1, characterized in that, The network event scorer is used in combination with the response scorer that asserts the final response of the agent under test. The multiple scorers are logically ANDed, and the task is considered successful only if all of them pass. For state change type tasks, the network event scorer asserts the response status code of the target write request to eliminate false positives where the request has been sent but actually failed.
5. The method according to claim 1, characterized in that, It also includes an overlay guidance step: maintaining an overlaid feature set and simultaneously performing the following two operations based on the same overlaid feature set: When exploring target selection, enumerate interactive controls or function pages that actually exist in the current page's document object model but have not yet been triggered, as leading targets, and reduce the weight of targets in the areas corresponding to the covered function sets to guide the exploration towards the uncovered functions. Before the synthesis result is written to disk, the covered function set is checked for duplicates based on the equivalence class of the judgment. Only a predetermined number of different instances of the same function are retained to achieve deduplication. This allows frontier exploration, de-weighting guidance, and deduplication to share the same set of covered functions; the coverage guidance step also includes progressive bootstrapping: when exploration reaches a deep page that is not visible from the homepage, the visual language model generates a new sub-target on-site from the final state screenshot of the deep page, so as to extend the coverage to deep functions that cannot be directly reached from the homepage.
6. The method according to claim 1, characterized in that, S1 to S5 constitute a closed loop of generation-verification-repair. Constraints that can be verified by mechanical rules are handled by the code layer, including the judgment and filtering of suspected volatile values. Judgments involving page semantic understanding are handled by the large model, including the selection of target requests and key fields in S3 and the judgment of task completion in S4. The core logic of executing S1 to S5 does not contain any hard-coded constants for specific websites. The hard-coded constants include the Uniform Resource Locator (URL) path, interface text keywords, style selectors, and domain target enumerations for specific websites. Differences between websites are isolated in an adaptation layer independent of the core logic. The adaptation layer includes at least environment address configuration, login authentication, initial entry, and runtime constraints, so that the method does not need to modify the core logic when changing the target website.
7. The method according to claim 1, characterized in that, The synthesis result includes a task data structure and a scorer data structure; the task data structure includes at least a task identifier, a starting page identifier, and a natural language intent, and the scorer data structure includes at least one or more of the following: scorer type, target request request path, request method, key fields, and response status code.
8. An automatic synthesis system for discriminative Web GUI tasks based on a large model, characterized in that, It includes a processor, memory, browser control interface, network request acquisition interface, and visual language model calling interface; the memory stores a computer program, which includes: An exploration and acquisition module is used to perform step S1 as described in claim 1; An intent synthesis module is used to perform step S2 as described in claim 1; A scorer synthesis module is used to perform step S3 as described in claim 1; A self-verification module is used to perform step S4 as described in claim 1; A repair module is used to perform step S5 as described in claim 1; The processor is used to execute the computer program to implement the method of any one of claims 1 to 7.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.