Data construction method for terminal page navigation task training and related device
Generate simulation tasks and terminal pages through generative models, solving the problem of high construction cost of terminal page navigation task training data, and achieving the effect of low-cost and rapid generation of training data.
Patent Information
- Application Number
- CN202411867122.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-17
- Publication Date
- 2025-03-18
AI Technical Summary
In terminal page navigation task training, building the data used for training requires offline each online terminal page to the local environment, resulting in huge workload and high cost.
By obtaining initial training samples, including initial terminal pages and initial tasks, the simulation tasks and simulation terminal pages are generated using a generative model, and then these simulation tasks and terminal pages are added to the initial training set to form the target training set.
This method quickly generates a large number of different terminal page navigation tasks and terminal pages through a generative model, eliminating the development costs brought by offline terminal pages and realizing the low-cost construction of data for terminal page navigation task training.
Smart Images
Figure CN119647283A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a data construction method and related devices for terminal page navigation task training. Background Art
[0002] With the development of artificial intelligence, performing terminal page navigation tasks through terminal agents and simplifying and accelerating the completion of various tasks in human-computer interaction have become the key to improving user experience and efficiency.
[0003] Completing the terminal page navigation task requires the terminal agent to be able to understand the content and structure of the web page like a human, analyze the page elements, and simulate interactive operations such as mouse clicks and text input. When training the terminal page navigation task, a large number of training samples containing terminal pages and tasks need to be provided to quickly iterate the model and obtain a terminal agent that can complete the terminal page navigation task. In the process of constructing training samples for terminal page navigation task training, it is usually necessary to offline each online terminal page to simulate and run in the local environment. The construction process of training samples faces a huge workload and the cost of data construction is high.
[0004] Therefore, how to construct data for terminal page navigation task training at low cost becomes a problem that needs to be solved. Summary of the invention
[0005] Based on the above problems, the present application provides a data construction method and related devices for terminal page navigation task training, which can construct data for terminal page navigation task training at low cost.
[0006] The embodiments of the present application disclose the following technical solutions:
[0007] In a first aspect, an embodiment of the present application provides a data construction method for terminal page navigation task training, the method comprising:
[0008] Acquire an initial training sample; the initial training sample includes an initial terminal page and at least one initial task corresponding to the initial terminal page;
[0009] Based on the initial terminal page, generating a simulation task corresponding to the initial terminal page through a generative model;
[0010] Based on the initial task, generating a simulation terminal page corresponding to the initial task through a generative model;
[0011] The simulation task and the simulation terminal page are added to an initial training set including the initial training samples to obtain a target training set.
[0012] Optionally, the generating, based on the initial terminal page, a simulation task corresponding to the initial terminal page through a generative model includes:
[0013] Based on the initial terminal page and a first constraint on the number of task steps, a simulation task corresponding to the initial terminal page is generated through a generative model.
[0014] Optionally, the generating, based on the initial task, a simulation terminal page corresponding to the initial task through a generative model includes:
[0015] Based on the initial task and a second constraint on the number of terminal page operation elements, a simulation terminal page corresponding to the initial task is generated through a generative model.
[0016] Optionally, after generating a simulation terminal page corresponding to the initial task through a generative model based on the initial task, the method further includes:
[0017] Compiling the implementation code of the simulation terminal page;
[0018] The step of adding the simulation task and the simulation terminal page to the initial training set including the initial training sample to obtain the target training set includes:
[0019] If the implementation code of the simulation terminal page is compiled successfully, the simulation task and the simulation terminal page are added to the initial training set including the initial training samples to obtain a target training set.
[0020] Optionally, after compiling the implementation code of the simulation terminal page, the method further includes:
[0021] If the implementation code of the simulation terminal page compiles and reports an error, inputting the error prompt of the simulation terminal page into the generative model;
[0022] Based on the generative model and the error prompt, the implementation code of the simulation terminal page is updated.
[0023] Optionally, after adding the simulation task and the simulation terminal page to an initial training set including the initial training sample to obtain a target training set, the method further includes:
[0024] Using the simulation task and the simulation page as supplementary training samples;
[0025] Generate an extended training sample based on the supplementary training sample through a generative model;
[0026] The extended training samples are added to the target training set.
[0027] In a second aspect, an embodiment of the present application provides a data construction device for terminal page navigation task training, the device comprising: an acquisition module, a generation module and an addition module;
[0028] The acquisition module is used to acquire an initial training sample; the initial training sample includes an initial terminal page and at least one initial task corresponding to the initial terminal page;
[0029] The generation module is used to generate a simulation task corresponding to the initial terminal page through a generative model based on the initial terminal page; based on the initial task, generate a simulation terminal page corresponding to the initial task through a generative model;
[0030] The adding module is used to add the simulation task and the simulation terminal page to the initial training set containing the initial training samples to obtain a target training set.
[0031] Optionally, the device further comprises: a compiling module;
[0032] The compiling module is used to compile the implementation code of the simulation terminal page;
[0033] The adding module is specifically used to: if the implementation code of the simulation terminal page is compiled successfully, then add the simulation task and the simulation terminal page to the initial training set containing the initial training samples to obtain a target training set.
[0034] In a third aspect, an embodiment of the present application provides a data construction device for terminal page navigation task training, the device comprising: a memory and a processor;
[0035] The memory is used to store program code and transmit the program code to the processor;
[0036] The processor is used to execute the steps of the data construction method for terminal page navigation task training described in any implementation of the first aspect according to the program code.
[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is run on a data construction device for terminal page navigation task training, the data construction device for terminal page navigation task training performs the steps of the data construction method for terminal page navigation task training as described in any implementation method in the first aspect.
[0038] Compared with the prior art, this application has the following beneficial effects:
[0039] The embodiment of the present application provides a data construction method for terminal page navigation task training, in which, first, an initial training sample is obtained; the initial training sample includes an initial terminal page and at least one initial task corresponding to the initial terminal page; then, based on the initial terminal page, a simulation task corresponding to the initial terminal page is generated through a generative model; based on the initial task, a simulation terminal page corresponding to the initial task is generated through a generative model; finally, the simulation task and the simulation terminal page are added to the initial training set containing the initial training sample to obtain a target training set. Thus, a large number of different terminal page navigation tasks and terminal pages are quickly generated through a generative model, eliminating the development cost caused by the offline terminal page, and data for terminal page navigation task training can be constructed at a low cost. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0041] Figure 1 A flow chart of a data construction method for terminal page navigation task training provided in an embodiment of the present application;
[0042] Figure 2 A flow chart of another data construction method for terminal page navigation task training provided in an embodiment of the present application;
[0043] Figure 3 A schematic diagram of a data construction device for terminal page navigation task training provided in an embodiment of the present application;
[0044] Figure 4 A structural diagram of a data construction device for terminal page navigation task training provided in an embodiment of the present application. DETAILED DESCRIPTION
[0045] The data construction method and related devices for terminal page navigation task training provided in the present application can be used in the field of artificial intelligence. The above is only an example and does not limit the application field of the data construction method and related devices for terminal page navigation task training provided in the present application.
[0046] The terms "first", "second", "third" and "fourth" etc. in the specification, claims and drawings of this application are used to distinguish different objects rather than to limit a specific order.
[0047] In the embodiments of the present application, the words "as an example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "as an example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the words "as an example" or "for example" is intended to present the relevant concepts in a specific way.
[0048] The terms used in the implementation section of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.
[0049] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0050] See also Figure 1 , which is a flow chart of a data construction method for terminal page navigation task training provided by an embodiment of the present application, the method comprising:
[0051] S101: Obtain initial training samples.
[0052] As an example, an initial training set with a small amount of data can be constructed in advance by offlineizing an online terminal page, and the initial training set includes at least one initial training sample, wherein the initial training sample includes at least one initial terminal page and an initial task corresponding to the initial terminal page.
[0053] The terminal page navigation task is used to support the terminal agent to realize navigation and operation on the terminal page such as the web page, client or mobile terminal to complete specific tasks. Tasks may include but are not limited to searching for information on search engines based on given keywords, formatting documents in office software, posting on social media or forum websites, browsing and extracting target information on web pages, filling in and submitting forms as required, registering or logging in on web pages or applications, and finding products on shopping websites and adding them to shopping carts.
[0054] S102: Based on the initial terminal page, generate a simulation task corresponding to the initial terminal page through a generative model.
[0055] S103: Based on the initial task, generate a simulation terminal page corresponding to the initial task through a generative model.
[0056] Specifically, the generative model can be a generative large language model represented by ChatGPT, which can generate text that meets the Prompt requirements based on different Prompt (prompt words) inputs.
[0057] At least one initial terminal page and an initial task corresponding to the initial terminal page are collected from the initial training samples. Through the generative model, data expansion can be performed based on the initial training samples from the task dimension and the terminal page dimension to generate new simulation tasks and simulation terminal pages.
[0058] In the task dimension, based on the same initial terminal page, different terminal page navigation tasks (i.e., simulation tasks) can be generated through the generative model; in the terminal page dimension, based on the same initial task, the implementation code of different terminal pages (i.e., simulation terminal pages) can be generated through the generative model.
[0059] As an example, the implementation code of the initial terminal page and a Prompt requirement such as "Generate a terminal page operation task based on the above code, and the operation steps must be controlled within 5 steps" are input into the generative model. The generative model can then output a simulation task that meets the requirements and can be implemented based on the initial terminal page, "Find an Android flagship phone released in the second half of 2024"; the initial task and a Prompt requirement such as "Generate a terminal page based on the initial task, and the operation elements of the terminal page are controlled within 30" are input into the generative model. The generative model can then output the implementation code of the terminal page that can implement the initial task and meet the requirement on the number of operation elements, and obtain a simulated terminal page.
[0060] It is understandable that the embodiment of the present application does not limit the execution order of steps S102 and S103. For example, step S102 may be executed first, and then step S103; step S103 may be executed first, and then step S102; or steps S102 and S103 may be executed simultaneously.
[0061] S104: Add the simulation task and the simulation terminal page to the initial training set including the initial training samples to obtain a target training set.
[0062] Specifically, a large number of generated simulation tasks and simulation terminal pages can be added to an initial training set containing initial training samples to form a target training set that can meet the training requirements of the terminal page navigation task.
[0063] Optionally, after adding the simulation task and the simulation terminal page to the initial training set containing the initial training samples, the simulation task and the simulation terminal page can be used as new initial training samples (supplementary training samples), and the supplementary training samples can be used as the input of the generative model. Through the generative model, extended training samples are generated based on the supplementary training samples, and the extended training samples are added to the target training set, so as to further expand the target training set, so that the task boundaries and terminal page boundaries continue to expand, the coverage of the training samples continues to expand, and a large number of training samples are obtained quickly and at low cost to form a target training set.
[0064] On the one hand, the terminal page navigation task needs to be constructed based on the functions of the specific terminal page and the scope of supported operations. There is a certain threshold for manually browsing, understanding the terminal page and constructing qualified tasks. At the same time, it is limited by each person's knowledge scope, and the scope of manually constructed tasks is limited, resulting in a relatively limited coverage of tasks in the training samples. On the other hand, the similarity of terminal pages at the code level is difficult to capture manually. Even if the visually presented terminal page content and layout are different, the underlying code development framework may be similar. Manually constructing terminal pages for training may cause the underlying development frameworks of each terminal page to be similar, the distinction between terminal pages is low, and the coverage of terminal pages in the training samples is insufficient.
[0065] In the embodiment of the present application, an iterative data construction method is adopted to gradually expand the coverage of simulation tasks and simulation terminal pages, significantly improving the richness of training data. Based on a large number of training samples with wide coverage, the terminal agent can fully learn various terminal page scenarios and terminal page navigation tasks, greatly improving the generalization ability of the terminal agent.
[0066] In the embodiment of the present application, first, an initial training sample is obtained; the initial training sample includes an initial terminal page and at least one initial task corresponding to the initial terminal page; then, based on the initial terminal page, a simulation task corresponding to the initial terminal page is generated through a generative model; based on the initial task, a simulation terminal page corresponding to the initial task is generated through a generative model; finally, the simulation task and the simulation terminal page are added to the initial training set containing the initial training sample to obtain a target training set. Thus, a large number of different terminal page navigation tasks and terminal pages are quickly generated through a generative model, eliminating the development cost caused by the offline terminal page, and data for terminal page navigation task training can be constructed at a low cost.
[0067] See also Figure 2 , which is a flow chart of another data construction method for terminal page navigation task training provided by an embodiment of the present application, the method comprising:
[0068] S201: Obtain initial training samples.
[0069] The initial training sample at least includes an initial terminal page and an initial task corresponding to the initial terminal page.
[0070] S202: Based on the initial terminal page and a first constraint condition on the number of task steps, a simulation task corresponding to the initial terminal page is generated through a generative model.
[0071] As an example, the first constraint condition can be used to constrain the steps of the generated simulation task, for example, the number of task steps can be constrained to be "within 5 operation steps" or "more than 8 operation steps", etc., so as to control the difficulty of the simulation task, so that the terminal pages in the target training set obtained in the end are difficult to balance, which facilitates the terminal agent to complete complex tasks on the terminal page. The more task steps there are, the more difficult the simulation task is.
[0072] Optionally, the first constraint condition may also include a similarity constraint condition between the simulation task and the initial task, so that the generated simulation task can have a large difference from the initial task, avoid task homogeneity, and expand the coverage of tasks, so as to obtain a terminal agent with strong generalization ability after using the target training set to perform terminal page navigation task training.
[0073] S203: Based on the initial task and a second constraint on the number of terminal page operation elements, a simulation terminal page corresponding to the initial task is generated through a generative model.
[0074] As an example, the second constraint condition can be used to constrain the number of operation elements contained in the generated simulated terminal page, for example, constraining the number of terminal page operation elements to be "within 30 terminal page operation elements" or "more than 10 terminal page operation elements", etc., so as to control the difficulty of the simulated terminal page, so that the terminal pages in the target training set obtained in the end are difficult to balance, which facilitates the terminal agent to complete tasks on complex terminal pages. The more operation elements a terminal page contains, the higher the difficulty of the simulated terminal page.
[0075] Optionally, the second constraint condition may also include a similarity constraint condition between the simulated terminal page and the initial terminal page, so that the generated simulated terminal page can have a large difference from the initial terminal page, avoiding the homogenization of terminal pages and expanding the coverage of terminal pages, so as to obtain a terminal agent with strong generalization ability after using the target training set to perform terminal page navigation task training.
[0076] S204: compile the implementation code of the simulation terminal page. If the implementation code of the simulation terminal page compiles successfully, execute step S205; if the implementation code of the simulation terminal page compiles with an error, execute step S206.
[0077] Specifically, the implementation code of the simulation terminal page can be compiled by a compiler. If the implementation code of the simulation terminal page is compiled successfully, it indicates that the simulation terminal page can run normally, and step S205 is executed; if the implementation code of the simulation terminal page is compiled with an error, it indicates that the simulation terminal page cannot run normally, and step S206 is executed.
[0078] S205: Add the simulation task and the simulation terminal page to the initial training set including the initial training samples.
[0079] As an example, after adding the simulation task and the simulation terminal page to the initial training set, the simulation task and the simulation terminal page can be used as supplementary training samples, and then a training sample is randomly obtained from the training set containing the supplementary training samples, and steps S201 to 205 are repeated.
[0080] S206: Input the error message of the simulation terminal page to the generative model.
[0081] By inputting the error prompts of the simulation terminal page into the generative model, the generative model can learn the pattern and structure of the implementation code that cannot be run, so that the generative model can avoid the same error in the newly output implementation code.
[0082] S207: Based on the generative model and the error prompt, update the implementation code of the simulation terminal page.
[0083] Specifically, the generative model can output new implementation code corresponding to the simulation terminal page based on the error prompt to update the error implementation code, and then execute step S204 to compile it again until an executable implementation code is obtained, thereby ensuring that the generated simulation terminal page can run normally.
[0084] See also Figure 3 , this figure is a schematic diagram of a data construction device for terminal page navigation task training provided in an embodiment of the present application, and the device includes: an acquisition module 301, a generation module 302 and an addition module 303.
[0085] The acquisition module 301 is used to acquire an initial training sample; the initial training sample includes an initial terminal page and at least one initial task corresponding to the initial terminal page;
[0086] A generation module 302 is used to generate a simulation task corresponding to the initial terminal page through a generative model based on the initial terminal page; based on the initial task, generate a simulation terminal page corresponding to the initial task through a generative model;
[0087] The adding module 303 is used to add the simulation task and the simulation terminal page to the initial training set including the initial training samples to obtain the target training set.
[0088] Therefore, a large number of different terminal page navigation tasks and terminal pages are quickly generated through the generative model, which eliminates the development costs caused by the offline terminal pages and enables the construction of data for terminal page navigation task training at low cost.
[0089] Optionally, the generating module 302 is specifically configured to generate a simulation task corresponding to the initial terminal page through a generative model based on the initial terminal page and a first constraint condition on the number of task steps.
[0090] Optionally, the generating module 302 is specifically configured to generate a simulation terminal page corresponding to the initial task through a generative model based on the initial task and a second constraint condition on the number of terminal page operation elements.
[0091] Optionally, another data construction device for terminal page navigation task training provided by an embodiment of the present application also includes: a compilation module, used to compile the implementation code of the simulation terminal page; an adding module 303, specifically used to: if the implementation code of the simulation terminal page is compiled, then the simulation task and the simulation terminal page are added to the initial training set containing the initial training samples to obtain the target training set.
[0092] Optionally, another data construction device for terminal page navigation task training provided by an embodiment of the present application also includes: an error reporting module, which is used to input an error prompt of the simulation terminal page into the generative model when an error occurs in the compilation of the implementation code of the simulation terminal page; and update the implementation code of the simulation terminal page based on the generative model and the error prompt.
[0093] Optionally, another data construction device for terminal page navigation task training provided by an embodiment of the present application also includes: a loop generation module, used to use simulation tasks and simulation pages as supplementary training samples; generate extended training samples based on the supplementary training samples through a generative model; and add the extended training samples to the target training set.
[0094] See also Figure 4 , this figure is a structural diagram of a data construction device for terminal page navigation task training provided in an embodiment of the present application, and the device includes: a memory 401 and a processor 402.
[0095] Memory 401: used to store program codes and transmit the program codes to the processor.
[0096] Processor 402: used to execute the steps of the above-mentioned data construction method for terminal page navigation task training according to the instructions in the program code.
[0097] In addition, the present application also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are run on a data construction device for terminal page navigation task training, the data construction device for terminal page navigation task training performs the steps of the above-mentioned data construction method for terminal page navigation task training.
[0098] It should be noted that each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components indicated as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0099] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A data construction method for terminal page navigation task training, characterized in that: The method comprises: Acquire an initial training sample; the initial training sample includes an initial terminal page and at least one initial task corresponding to the initial terminal page; Based on the initial terminal page, generating a simulation task corresponding to the initial terminal page through a generative model; Based on the initial task, generating a simulation terminal page corresponding to the initial task through a generative model; The simulation task and the simulation terminal page are added to an initial training set including the initial training samples to obtain a target training set.
2. The method according to claim 1, characterized in that The generating a simulation task corresponding to the initial terminal page through a generative model based on the initial terminal page includes: Based on the initial terminal page and a first constraint on the number of task steps, a simulation task corresponding to the initial terminal page is generated through a generative model.
3. The method according to claim 1, characterized in that: The step of generating a simulation terminal page corresponding to the initial task through a generative model based on the initial task includes: Based on the initial task and a second constraint on the number of terminal page operation elements, a simulation terminal page corresponding to the initial task is generated through a generative model.
4. The method according to claim 1, characterized in that After generating a simulation terminal page corresponding to the initial task through a generative model based on the initial task, the method further includes: Compiling the implementation code of the simulation terminal page; The step of adding the simulation task and the simulation terminal page to the initial training set including the initial training sample to obtain the target training set includes: If the implementation code of the simulation terminal page is compiled successfully, the simulation task and the simulation terminal page are added to the initial training set including the initial training samples to obtain a target training set.
5. The method according to claim 4, characterized in that After compiling the implementation code of the simulation terminal page, the method further includes: If the implementation code of the simulation terminal page compiles and reports an error, inputting the error prompt of the simulation terminal page into the generative model; Based on the generative model and the error prompt, the implementation code of the simulation terminal page is updated.
6. The method according to claim 1, characterized in that After adding the simulation task and the simulation terminal page to the initial training set including the initial training sample to obtain the target training set, the method further includes: Using the simulation task and the simulation page as supplementary training samples; Generate an extended training sample based on the supplementary training sample through a generative model; The extended training samples are added to the target training set.
7. A data construction device for terminal page navigation task training, characterized in that: The device comprises: an acquisition module, a generation module and an addition module; The acquisition module is used to acquire an initial training sample; the initial training sample includes an initial terminal page and at least one initial task corresponding to the initial terminal page; The generation module is used to generate a simulation task corresponding to the initial terminal page through a generative model based on the initial terminal page; based on the initial task, generate a simulation terminal page corresponding to the initial task through a generative model; The adding module is used to add the simulation task and the simulation terminal page to the initial training set containing the initial training samples to obtain a target training set.
8. The device according to claim 7, characterized in that The device also includes: a compiling module; The compiling module is used to compile the implementation code of the simulation terminal page; The adding module is specifically used to: if the implementation code of the simulation terminal page is compiled successfully, then add the simulation task and the simulation terminal page to the initial training set containing the initial training samples to obtain a target training set.
9. A data construction device for terminal page navigation task training, characterized in that: The device comprises: a memory and a processor; The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the steps of the data construction method for terminal page navigation task training according to any one of claims 1 to 6 according to the program code.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program. When the computer program runs on a data construction device for terminal page navigation task training, the data construction device for terminal page navigation task training executes the steps of the data construction method for terminal page navigation task training as described in any one of claims 1-6.