Training data generation device, training data generation method, and training data generation program

Abstract

The present invention provides a training data generation device that can increase the efficiency of the process of generating training data for a generative model, which is used to train a generative model and generates actions to accomplish an input task. [Solution] The training data generation device comprises: an acquisition unit that acquires data; a generation unit that generates executable actions for the data; an identification unit that identifies tasks that can potentially be achieved by executing the actions generated by the generation unit; and an output unit that outputs pairs of tasks identified by the identification unit and actions generated by the generation unit as training data used to train a generative model that generates actions to achieve the input tasks.

Description

Technical Field

[0001] The present disclosure relates to a training data generation device, a training data generation method, and a training data generation program.

Background Art

[0002] There is known a generation model that generates actions for achieving an input task. Also, there is known a technique for generating training data for training such a generation model. For example, Non-Patent Document 1 discloses a self-evolving online curriculum reinforcement learning framework (WebRL) designed to train a high-performance Web agent using an open LLM. In the technique described in Non-Patent Document 1, reinforcement learning is performed to optimize actions while considering the overall impact on the task.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional technology, actions for achieving a task may not be appropriately generated, and as a result, generation of training data may fail.

[0005] This disclosure has been made in view of the above-mentioned problems, and one exemplary purpose thereof is to provide a technique that can improve the efficiency of generating training data for a generative model, which is used to train a generative model and generates actions to accomplish an input task. [Means for solving the problem]

[0006] A training data generation device relating to an exemplary aspect of this disclosure comprises: acquisition means for acquiring data; generation means for generating executable actions for the data; identification means for identifying tasks that can potentially be achieved by executing the actions generated by the generation means; and output means for outputting pairs of tasks identified by the identification means and actions generated by the generation means as training data used to train a generative model that generates actions to achieve an input task.

[0007] A training data generation method relating to an illustrative aspect of this disclosure includes: an acquisition process in which at least one processor acquires data; a generation process in which the at least one processor generates executable actions for the data; an identification process in which the at least one processor uses the actions generated in the generation process to identify tasks that can be achieved by executing those actions; and an output process in which the at least one processor outputs pairs of tasks identified in the identification process and actions generated in the generation process as training data to be used to train a generative model that generates actions to achieve an input task.

[0008] An exemplary aspect of the present disclosure is a training data generation program for causing a computer to function as a training data generation device that generates training data used to train a generative model that generates actions to accomplish an input task, wherein the computer functions as an acquisition means for acquiring data, a generation means for generating executable actions on the data, an identification means for identifying tasks that can potentially be achieved by executing the actions generated by the generation means, and an output means for outputting pairs of tasks identified by the identification means and actions generated by the generation means as training data. [Effects of the Invention]

[0009] According to an illustrative aspect of this disclosure, one exemplary effect is that it can provide a technique that improves the efficiency of the generation process for generating training data for a generative model, which is used to train a generative model and generates actions to accomplish an input task. [Brief explanation of the drawing]

[0010] [Figure 1] This block diagram shows the configuration of the training data generation device related to this disclosure. [Figure 2] This flowchart shows the flow of the training data generation method related to this disclosure. [Figure 3] This is a block diagram showing the configuration of the information processing device related to this disclosure. [Figure 4] This flowchart shows the flow of the training data generation method related to this disclosure. [Figure 5] This figure shows a specific example of the training data generation process related to this disclosure. [Figure 6] This is a block diagram showing the configuration of a computer that functions as the device related to this disclosure. [Modes for carrying out the invention]

[0011] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining some or all of the technologies (things or methods) employed in each of the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, embodiments obtained by appropriately omitting some of the technologies employed in each of the exemplary embodiments shown below may also be included in the scope of the present invention. In addition, the effects mentioned in each of the exemplary embodiments shown below are examples of effects that can be expected in that exemplary embodiment and do not define the scope of the present invention. That is, embodiments that do not produce the effects mentioned in each of the exemplary embodiments shown below may also be included in the scope of the present invention.

[0012] [First Exemplary Embodiment] A first exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is the basic form for each of the exemplary embodiments described later. The scope of application of each technology adopted in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology adopted in this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems occur. Furthermore, each technology shown in the drawings referenced to explain this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems occur.

[0013] (Configuration of the training data generation device) The configuration of the training data generation device 1 will be described with reference to Figure 1. Figure 1 is a block diagram showing the configuration of the training data generation device 1. As shown in Figure 1, the training data generation device 1 comprises an acquisition unit 11, a generation unit 12, a specification unit 13, and an output unit 14.

[0014] The acquisition unit 11 acquires data. The generation unit 12 generates executable actions for the data. The identification unit 13 uses the actions generated by the generation unit 12 to identify tasks that can potentially be achieved by executing those actions. The output unit 14 outputs pairs of tasks identified by the identification unit 13 and actions generated by the generation unit 12 as training data used to train a generative model that generates actions to achieve the input tasks.

[0015] (Effectiveness of the training data generation device) As described above, the training data generation device 1 employs a configuration comprising: an acquisition unit 11 that acquires data; a generation unit 12 that generates executable actions for the data; an identification unit 13 that identifies tasks that can potentially be achieved by executing the actions generated by the generation unit 12; and an output unit 14 that outputs pairs of tasks identified by the identification unit 13 and actions generated by the generation unit 12 as training data used to train a generative model that generates actions to achieve the input task. Therefore, the training data generation device 1 has the effect of increasing the efficiency of the generation process for training data used to train a generative model, which is training data for a generative model that generates actions to achieve the input task.

[0016] (Workflow for generating training data) The flow of the training data generation method S1 will be explained with reference to Figure 2. Figure 2 is a flowchart showing the flow of the training data generation method S1. As shown in Figure 2, the training data generation method S1 includes an acquisition process S11, a generation process S12, a identification process S13, and an output process S14.

[0017] In the acquisition process S11, at least one processor acquires data. In the generation process S12, the at least one processor generates an action executable on the data. In the identification process S13, the at least one processor uses the action generated in the generation process S12 to identify a task that may be achieved by executing the action. In the output process S14, the at least one processor outputs a pair of the task identified in the identification process S13 and the action generated in the generation process S12 as training data used for training a generation model that generates an action for achieving an input task.

[0018] (Effect of the training data generation method) As described above, in the training data generation method S1, a configuration including an acquisition process S11 in which at least one processor acquires data, a generation process S12 in which the at least one processor generates an action executable on the data, an identification process S13 in which the at least one processor uses the action generated in the generation process S12 to identify a task that may be achieved by executing the action, and an output process S14 in which the at least one processor outputs a pair of the task identified in the identification process S13 and the action generated in the generation process S12 as training data used for training a generation model that generates an action for achieving an input task is adopted. Therefore, according to the training data generation method S1, an effect that the generation efficiency of the generation process of the training data used for training a generation model that generates an action for achieving an input task can be increased is obtained.

[0019] [Second exemplary embodiment] A second exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiments are denoted by the same reference numerals, and the description thereof will be omitted as appropriate. Note that the scope of application of each technique employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technique employed in this exemplary embodiment can be employed in other exemplary embodiments included in the present disclosure as long as there are no particular technical obstacles. In addition, each technique shown in each drawing referred to for explaining this exemplary embodiment can be employed in other exemplary embodiments included in the present disclosure as long as there are no particular technical obstacles.

[0020] (Overview of the information processing apparatus) The information processing apparatus 1A according to the present disclosure generates training data for a generation model that generates actions for achieving an input task. The generation model is, for example, a large language model (LLM). For example, when a task such as "purchase XX" is given by a user, as an example, the generation model generates a series of actions including an action to access a website on a web browser, an action to add a product to a cart on the website, and an action to click a button to purchase the product, as actions for achieving the task.

[0021] The training data for training the generation model includes a plurality of pairs of a task and actions that can achieve the task. For example, when the task is the text "purchase XX", the actions corresponding to the task are, as an example, a series of actions including an action to access a website on a web browser, an action to add a product to a cart on the website, and an action to click a button to purchase the product.

[0022] (Configuration of the information processing apparatus) The configuration of the information processing device 1A will be described with reference to Figure 3. Figure 3 is a block diagram showing the configuration of the information processing device 1A. The information processing device 1A comprises a control unit 10A, a storage unit 20A, a communication unit 30A, an input unit 40A, and an output unit 50A.

[0023] (Communications Department) The communication unit 30A communicates with external devices of the information processing device 1A via a communication line N. The specific configuration of the communication line N is not limited to this exemplary embodiment, but examples of communication line N include a wireless LAN (Local Area Network), a wired LAN, a WAN (Wide Area Network), a public telephone network, a mobile data communication network, or a combination thereof. The communication unit 30A transmits data supplied from the control unit 10A to other devices and supplies data received from other devices to the control unit 10A.

[0024] (Input section) The input unit 40A is configured to receive input to the information processing device 1A, and may include, for example, an input device such as a keyboard, mouse, touch panel, camera, or microphone. Alternatively, the input unit 40A may be configured to receive data from the input device via an interface such as USB (Universal Serial Bus).

[0025] (Output section) The output unit 50A is configured to output from the information processing device 1A, and may include, for example, an output device such as a display, printer, touch panel, or speaker. The output unit 50A may also be configured to have an interface such as USB, and to output data to the output device via this interface.

[0026] (Storage part) The memory unit 20A stores instructions for the computer program executed by the control unit 10A. The memory unit 20A also stores various types of information that the control unit 10A references.

[0027] (Control Unit) As shown in Figure 3, the control unit 10A includes an acquisition unit 111A, an action generation unit 112A, an action execution unit 113A, a task identification unit 114A, a determination unit 115A, an output control unit 116A, and a training unit 117A. The acquisition unit 111A, action generation unit 112A, action execution unit 113A, task identification unit 114A, determination unit 115A, output control unit 116A, and training unit 117A are examples of the acquisition means, generation means, identification means, determination means, output means, and training means according to this disclosure.

[0028] (Acquisition Department) The acquisition unit 111A acquires data. The data is the data to which the action will be performed. Examples of data include, but are not limited to, web pages, image data, audio data, video data, PDF files, or combinations thereof.

[0029] As an example, the acquisition unit 111A may acquire data by reading data from a storage location specified by the user of the information processing device 1A (which may be a storage device within the information processing device 1A or a storage device outside the information processing device 1A). Alternatively, the acquisition unit 111A may acquire data by receiving data from another device via the communication unit 30A. Furthermore, the acquisition unit 111A may acquire data input to the input unit 40A.

[0030] (Action generation unit) The action generation unit 112A generates executable actions for the data acquired by the acquisition unit 111A. For example, the action generation unit 112A generates actions using an action generator. The action generator may be a model generated by machine learning, a rule-based model, or a combination of these, but is not limited to these. For example, the action generation unit 112A may randomly select one of several executable actions for the above data. The action generation unit 112A may generate one action or multiple actions.

[0031] (Action generator) When an action generator is generated by machine learning, the machine learning method used for the action generator is not limited. For example, decision tree-based, linear regression, or neural network methods may be used, or two or more of these methods may be used. Examples of decision tree-based methods include LightGBM (Light Gradient Boosting Machine), Random Forest, and XGBoost. Examples of linear regression methods include Bayesian regression, Support Vector Regression, Ridge Regression, Lasso Regression, and ElasticNet. Examples of neural networks include deep learning.

[0032] More specifically, an action generator is, for example, a generative model generated by machine learning, consisting of an artificial neural network with numerous parameters. Examples of generative models include, but are not limited to, LLMs such as ChatGPT (Chat Generative Pre-trained Transformer) and GPT-4 (Generative Pre-trained Transformer 4), or LLMs that have been fine-tuned with data related to task execution.

[0033] The action generator may be stored in the memory unit 20A, or it may be stored in a device other than the information processing device 1A. Note that when we say the action generator is stored in a memory device, we mean that the parameters defining the action generator are stored in the memory unit. If the action generator is stored in a device other than the information processing device 1A, the action generation unit 112A transmits input data to the device where the action generator is stored and receives output data transmitted from that device as a response to the transmitted input data.

[0034] (Input to the action generator) The input data that the action generation unit 112A inputs to the action generator includes, as an example, data acquired by the acquisition unit 111A (e.g., a web page) and a prompt instructing the generation of an action. The input data may also include, for example, a provisional task set by the user of the information processing device 1A. The provisional task is a task different from the task included in the training data. The input data may also include the provisional task and the reason for determining whether the provisional task is achievable. In this case, the action generation unit 112A can generate the above action using the provisional task, or the provisional task and the reason for determining whether the provisional task is achievable. The provisional task and the reason for determining whether it is achievable are set by the user of the information processing device 1A, as an example.

[0035] Furthermore, the input data may include tasks identified by the task identification unit 114A, which will be described later. In this case, the prompt input to the action generator may, for example, be the text "[I want to solve the task output by the action rationalizer]. Please generate an appropriate action." The input data may also include the reason for determining whether the task identified by the task identification unit 114A, which will be described later, is achievable (why the task cannot be achieved).

[0036] Furthermore, the input data may include actions generated by the action generation unit 112A in previous generation processes. In this case, the action generation unit 112A can also generate actions using the sequence of actions generated so far.

[0037] (Output of the action generator) The output of an action generator is an action. The output from an action generator may also include a description of the output (the reason why that particular action was generated).

[0038] (Action Execution Department) The action execution unit 113A executes the action generated by the action generation unit 112A. For example, if the data acquired by the acquisition unit 111A is a web page and the action generated by the action generation unit 112A is "Click[beds]", the action execution unit 113A executes the action of clicking [beds] on the web page. The execution of this action causes the user to transition to a different web page than the one acquired by the acquisition unit 111A.

[0039] (Task Identification Department) The task identification unit 114A uses the actions generated by the action generation unit 112A to identify tasks that can potentially be achieved by executing those actions. As an example, the task identification unit 114A identifies tasks using an action rationalizer.

[0040] (Action Rationalizer) The action rationalizer is a generative model generated by machine learning. In this case, the task identification unit 114A can also identify tasks obtained by inputting the actions generated by the action generation unit 112A into the generative model.

[0041] An action rationalizer is, for example, a generative model generated by machine learning, consisting of an artificial neural network with numerous parameters. Examples of generative models include, but are not limited to, LLMs such as ChatGPT and GPT-4, or LLMs that have been fine-tuned with data related to task execution. The action generator and action rationalizer may be the same generative model, or they may be different generative models.

[0042] The action rationalizer may be stored in the memory unit 20A, or it may be stored in a device other than the information processing device 1A. Note that when the action rationalizer is stored in the memory unit, it means that the parameters defining the action rationalizer are stored in the memory unit. If the action rationalizer is stored in a device other than the information processing device 1A, the task identification unit 114A transmits input data to the device where the action rationalizer is stored and receives output data transmitted from that device as a response to the transmitted input data.

[0043] (Input to the action rationalizer) The input data that the task identification unit 114A inputs to the action rationalizer includes, as an example, actions or action sequences generated by the action generation unit 112A. If the input to the action rationalizer is an action sequence, the task identification unit 114A can also use the action sequence generated by the action generation unit 112A to identify tasks that can potentially be achieved by executing that action sequence.

[0044] Furthermore, the input data may include a description of the action generated by the action generation unit 112A (the reason why that action was generated). The input data may also include a prompt instructing the generation of a task. For example, the prompt may be text such as, "The output action of the action generator is as follows. Please think of a task that can be achieved by this action." The prompt may also include text instructing the output of the reason for determining whether the task is achievable.

[0045] Furthermore, the input data may include data acquired by the acquisition unit 111A. The input data may also include data after a transition resulting from the action execution unit 113A executing an action. In this case, the task identification unit 114A can use the data corresponding to the state after the transition resulting from the execution of the action to identify tasks that may be achieved by the execution of that action.

[0046] (Output of the action rationalizer) The output of the action rationalizer includes the task you currently want to accomplish (or are likely to accomplish). The output may also include information indicating whether the input action makes the task achievable. Furthermore, the output may include the task name. Finally, the output may include the reason for determining whether the task is achievable (why the task cannot be accomplished).

[0047] (Judgment Department) The determination unit 115A determines whether the task identified by the task identification unit 114A can be achieved by executing the action generated by the action generation unit 112A. For example, the determination unit 115A makes a determination based on information output from the action rationalizer indicating whether the task can be achieved.

[0048] (Output control unit) The output control unit 116A outputs pairs of tasks identified by the task identification unit 114A and actions generated by the action generation unit 112A as training data to be used to train a generative model that generates actions to achieve the input task. In other words, the output control unit 116A can also output tasks that the determination unit 115A has determined to be achievable, and actions that can achieve those tasks. Alternatively, the output control unit 116A may output the reason for the determination unit 115A's determination of whether the task is achievable.

[0049] The output control unit 116A may output the data by writing it to a storage location specified by the user of the information processing device 1A (which may be a storage device within the information processing device 1A or a storage device outside the information processing device 1A). Alternatively, the output control unit 116A may output the data to an output device such as a display or printer. Alternatively, the output control unit 116A may output the data by transmitting it to another device connected via the communication unit 30A.

[0050] (Training Department) The training unit 117A trains the generative model using the training data output by the output control unit 116A. As an example, the training unit 117A may also fine-tune the generative model using the training data.

[0051] (Workflow for generating training data) Figure 4 is a flowchart showing the flow of the training data generation method related to this disclosure. Figure 4 illustrates an example of generating action and task pairs for a web page. In step S101, the acquisition unit 111A acquires the web page.

[0052] In step S102, the action generation unit 112A generates an action that can be executed on the web page. For example, the action generation unit 112A generates an action by inputting input data, including the web page and prompts acquired by the acquisition unit 111A, into the action generator. At this time, the action generation unit 112A may include a temporary task in the input data.

[0053] Furthermore, in the second and subsequent executions of step S102 when the processes from step S102 to step S107 are repeatedly performed, the action generation unit 112A may include the task identified by the task identification unit 114A in the input data to the action generator. In this case, the prompt may be, for example, the text: "I want to solve [the task output by the action rationalizer]. Please generate an appropriate action."

[0054] In step S103, the action execution unit 113A executes the action generated by the action generation unit 112A. The action execution unit 113A executes the action, causing the web page to change. In step S104, the task identification unit 114A obtains the web page that was accessed after the action execution unit 113A executed the action.

[0055] In step S105, the task identification unit 114A uses the action generated by the action generation unit 112A to identify tasks that can potentially be achieved by executing that action. For example, the task identification unit 114A generates a task by inputting input data, including the action generated by the action generation unit 112A, the subsequent web page, and a prompt instructing the generation of a task, into the action rationalizer. For example, the prompt may be the text, "The output action from the action generator is as follows. Please reconsider the tasks that can be achieved by this action."

[0056] In step S106, the determination unit 115A determines whether the task identified by the task identification unit 114A can be achieved with the action generated by the action generation unit 102. For example, the determination unit 115A makes the determination based on information indicating whether the task can be achieved, which is included in the output of the action rationalizer. If the determination unit 115A determines that the task can be achieved (step S106: YES), it proceeds to the process in step S108. On the other hand, if the determination unit 115A determines that the task cannot be achieved (step S106: NO), it proceeds to the process in step S107.

[0057] In step S107, the determination unit 115A determines whether the number of repetitions of the series of processes in steps S102 to S107 is equal to or greater than a preset value. If the number of repetitions is equal to or greater than the preset value (step S107: YES), the determination unit 115A proceeds to the process in step S109. On the other hand, if the number of repetitions is less than the preset value (step S107: NO), the determination unit 115A returns to the process in step S102.

[0058] As shown in Figure 4, the information processing device 1A repeatedly executes the processes in steps S102 to S107 until an achievable task is generated. In the second and subsequent executions of step S102, the action generation unit 112A, for example, inputs input data to the action generator that includes the task identified by the task identification unit 114A and the actions generated by the action generation unit 112A in the previous steps S102, thereby executing the action generation process again. In other words, the action generation unit 112A can also execute the generation of an achievable action again using the task that the determination unit 115A has determined to be unachievable and the action used by the task identification unit 114A to identify that task. In this way, the action generator and the action rationalizer exchange information with each other.

[0059] In step S108, the output control unit 116A assigns a task name to the task determined to be achievable in step S106 and outputs the task-action pair as training data. At this time, the output control unit 116A may, for example, assign a task name obtained by inputting a prompt instructing the output of the action and task name into a generation model such as an action rationalizer. In this case, the prompt may, for example, be the text: "The output action history of the action generator is as follows. Please assign a task name to it."

[0060] In step S109, the output control unit 116A terminates processing because it was unable to acquire the task. At this time, the output control unit 116A may output a message indicating that it was unable to acquire the task.

[0061] Figure 5 shows a specific example of the training data generation process. In the example in Figure 5, the information processing device 1A repeats steps S102 to S107 in Figure 4 three times. In this example, the data acquired by the acquisition unit 111A is the web page "abcde.com".

[0062] In the first iteration, the action generation unit 112A inputs input data to the action generator, including the prompt "Select an appropriate action on this page" and the web page "abcde.com". As a result, the action generator outputs the action "click[beds]". The action execution unit 113A executes this action, causing the web page to transition to "abcde.com / beds".

[0063] Furthermore, in the first iteration, the task identification unit 114A inputs the prompt "Consider a task such that the action is successful," along with input data including the action output from the action generator and the web page after the transition, into the action rationalizer. The action rationalizer outputs the task and the reason for determining task success, such as "A task such as purchasing a bed is possible, which is an action that moves to the beds page. The task has not been completed yet as only the page has been transitioned."

[0064] In the second iteration, the action generation unit 112A inputs input data to the action generator that includes the prompt "I want to solve the task of buying a bed. Select the appropriate action" and the web page "abcde.com / beds" (i.e., the web page to be redirected). This prompt contains the task identified by the task identification unit 114A, "buy a bed". Upon receiving this input data, the action generator outputs the action "click[**bed]". The action execution unit 113A executes this action, causing the web page to transition to "abcde.com / beds / **bed".

[0065] Furthermore, in the second iteration, the task identification unit 114A inputs the prompt "Consider a task so that the action is successful," along with input data including the action output from the action generator and the web page to which the user transitioned, into the action rationalizer. The action rationalizer outputs the task and the reason for determining task success, stating, "Therefore, a task can be considered that involves moving to the **beds page**, which is the action **purchase a bed**. However, since the user has only transitioned to the page, the task has not yet been completed."

[0066] Furthermore, in the third iteration, the action generation unit 112A inputs input data to the action generator, including the prompt "I want to complete the task of purchasing a bed. Please select the appropriate action" and the web page "abcde.com / beds / **bed" (i.e., the web page to which the user is redirected). As a result, the action generator outputs the action "click[add to cart]". The action execution unit 113A then executes this action on the web page.

[0067] Furthermore, in the third iteration, the task identification unit 114A receives input data including the prompt "Consider a task so that the action is successful," the action output from the action generator, and the web page "abcde.com / beds / **bed". The action rationalizer outputs the task and the reason for determining task completion, stating, "A task to purchase a **bed by clicking add to cart is considered. As a result, the task to purchase a **bed has been completed."

[0068] The three iterations described above yield training data that includes the task "**buy a bed" and action columns "click[beds]", "click[**bed]", and "click[add to cart]".

[0069] (Effects of information processing equipment) Incidentally, as a technique for generating training data to train a generative model that generates actions to accomplish an input task, for example, the technique described in Non-Patent Document 1 can be cited. In the technique described in Non-Patent Document 1, the task is generated first, and then actions to solve that task are generated. Therefore, if an appropriate action cannot be generated, the generation process up to that point is wasted, resulting in poor generation efficiency. In contrast, according to the information processing device 1A of this disclosure, instead of setting the task first, actions are generated first, and then the task is identified using the generated actions. By generating actions prior to identifying the task in this way, the success rate of the training data generation process can be increased.

[0070] Furthermore, the information processing device 1A includes a determination unit 115A that determines whether the task identified by the task identification unit 114A can be achieved by executing the action generated by the action generation unit 112A, and the output control unit 116A is configured to output the task determined to be achievable by the determination unit 115A and the action that makes it possible to achieve that task. By outputting the achievable task and action, the information processing device 1A can achieve the effect of increasing the generation efficiency of the training data generation process.

[0071] Furthermore, in the information processing device 1A, the output control unit 116A is configured to output the reason for the determination by the determination unit 115A regarding whether the above task can be accomplished. Users of the information processing device 1A can easily determine whether the generated training data is useful by checking the outputted reason for determination. This allows them to select, for example, more appropriate data as training data to be used to train a generative model.

[0072] Furthermore, in the information processing device 1A, the action generation unit 112A is configured to generate executable actions again using the tasks that the determination unit 115A has determined to be unachievable and the actions that the task identification unit 114A has used to identify those tasks. By repeatedly generating actions and identifying tasks in this way, the information processing device 1A can generate more appropriate data as training data for training the generation model.

[0073] Furthermore, in the information processing device 1A, the action generation unit 112A is configured to generate the above action using a provisional task, or the provisional task and the reason for determining whether the provisional task is achievable. By using the provisional task and / or the reason for determining the provisional task in generating the action, the information processing device 1A can generate more appropriate data as training data to be used to train the generation model.

[0074] Furthermore, in the information processing device 1A, the task identification unit 114A is configured to identify tasks obtained by inputting the actions generated by the action generation unit 112A into the generation model. By identifying tasks using the generation model, the information processing device 1A can increase the efficiency of the generation process for training the training data used to train the generation model.

[0075] Furthermore, in the information processing device 1A, the task identification unit 114A uses a sequence of actions generated by the action generation unit 112A to identify tasks that can potentially be achieved by executing that sequence of actions. By using a sequence of actions obtained by repeatedly performing the action generation process, the information processing device 1A can increase the efficiency of generating achievable tasks.

[0076] Furthermore, the information processing device 1A includes an action execution unit 113A that executes the actions generated by the action generation unit 112A, and a task identification unit 114A uses data corresponding to the state after the transition resulting from the execution of the action to identify tasks that can potentially be achieved by the execution of the action.

[0077] When tasks are generated using only the data from the previous state transition, the generated tasks may be limited. For example, only unrealistic tasks that can be completed on a single web page may be generated, or, for example, even though the website only sells three-drawer chests, a task such as "Purchase a five-drawer chest" may be generated using only the data from the previous web page. In contrast, the information processing device 1A can prevent the generated tasks from being limited by using both the data from the previous state transition and the data from the current state transition.

[0078] Furthermore, the information processing device 1A is configured to include a training unit 117A that trains a generative model using training data. This allows the information processing device 1A to train a generative model that generates actions to accomplish an input task.

[0079] [Examples of implementation using software] Some or all of the functions of the training data generation device and the information processing device 1A (hereinafter also referred to as "the above devices") may be implemented by hardware such as integrated circuits (IC chips) or by software.

[0080] In the latter case, each of the above devices is implemented, for example, by a computer that executes instructions for a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as Computer C) is shown in Figure 6. Figure 6 is a block diagram showing the hardware configuration of Computer C, which functions as each of the above devices.

[0081] Computer C comprises at least one processor C1 and at least one memory C2. Memory C2 stores a program P that causes computer C to operate as each of the above-mentioned devices. In computer C, processor C1 reads program P from memory C2 and executes it, thereby realizing each of the above-mentioned devices.

[0082] For processor C1, for example, a CPU (Central Processing Unit), GPU (Graphic Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating Point Number Processing Unit), PPU (Physics Processing Unit), TPU (Tensor Processing Unit), quantum processor, microcontroller, or a combination thereof can be used. For memory C2, for example, flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), or a combination thereof can be used.

[0083] Computer C may also be equipped with RAM (Random Access Memory) for loading program P at runtime and for temporarily storing various data. Furthermore, computer C may be equipped with communication interfaces for sending and receiving data with other devices. Additionally, computer C may be equipped with input / output interfaces for connecting input / output devices such as keyboards, mice, displays, and printers.

[0084] Furthermore, program P can be recorded on a non-temporary, tangible recording medium M that is readable by computer C. Such a recording medium M could be, for example, tape, disk, card, semiconductor memory, or programmable logic circuitry. Computer C can acquire program P via such a recording medium M. Program P can also be transmitted via a transmission medium. Such a transmission medium could be, for example, a communication network or broadcast waves. Computer C can also acquire program P via such a transmission medium.

[0085] Furthermore, each of the above functions of each of the above devices may be implemented by a single processor in a single computer, by multiple processors in a single computer working together, or by multiple processors in each of multiple computers working together. In addition, the programs for implementing each of the above functions in each of the above devices may be stored in a single memory in a single computer, distributed and stored in multiple memories in a single computer, or distributed and stored in multiple memories in each of multiple computers.

[0086] [Additional Note A] This disclosure includes the technologies described in the following appendices. However, the present invention is not limited to the technologies described in the following appendices, and various modifications are possible within the scope of the claims.

[0087] (Note A1) A means of acquiring data, A generation means for generating executable actions for the aforementioned data, A means for identifying tasks that can be achieved by executing the actions generated by the generation means, An output means outputs a pair of a task identified by the identification means and an action generated by the generation means as training data used to train a generative model that generates actions to accomplish an input task. A training data generation device equipped with the following features.

[0088] (Appendix A2) The system further comprises a determination means for determining whether the task identified by the identification means can be achieved by executing the action generated by the generation means, The output means outputs a task that has been determined to be achievable by the determination means and an action that can achieve that task. The training data generation device described in Appendix A1.

[0089] (Note A3) The output means outputs the reason for the determination by the determination means whether the task can be accomplished. The training data generation device described in Appendix A2.

[0090] (Note A4) The generation means then uses the task that the determination means determined to be unachievable and the action used by the identification means to identify the task to re-generate an action that can be executed. The training data generation device described in Appendix A2 or A3.

[0091] (Note A5) The generation means generates the action using a provisional task, or the provisional task and the reason for determining whether the provisional task is achievable. A training data generation device as described in one of the appendices A1 to A4.

[0092] (Note A6) The identifying means identifies a task obtained by inputting the action generated by the generating means into a generation model. A training data generation device as described in one of the appendices A1 to A5.

[0093] (Note A7) The identifying means uses the sequence of actions generated by the generating means to identify tasks that can be achieved by executing the sequence of actions. The training data generation device described in Appendix A4.

[0094] (Note A8) The system further comprises an execution means for executing the action generated by the generation means, The identifying means uses data corresponding to the state after the transition resulting from the execution of the action to identify tasks that can potentially be achieved by the execution of the action. A training data generation device as described in any one of the appendices A1 to A7.

[0095] (Note A9) The system further comprises training means for training the generative model using the aforementioned training data. A training data generation device as described in any one of the appendices A1 to A8.

[0096] [Additional Note B] This disclosure includes the technologies described in the following appendices. However, the present invention is not limited to the technologies described in the following appendices, and various modifications are possible within the scope of the claims.

[0097] (Note B1) At least one processor handles the data acquisition process, The at least one processor performs a generation process that generates an action that can be performed on the data, The at least one processor performs a specific process that uses the actions generated in the generation process to identify tasks that can potentially be achieved by executing those actions, The at least one processor outputs a pair of tasks identified in the specific process and actions generated in the generation process as training data to be used to train a generative model that generates actions to accomplish an input task; A method for generating training data that includes this.

[0098] (Note B2) The at least one processor further includes a determination process that determines whether the task identified in the specific process can be achieved by executing the action generated in the generation process, In the output process, the at least one processor outputs a task that has been determined to be achievable by the determination process and an action that can achieve that task. The training data generation method described in Appendix B1.

[0099] (Note B3) In the output processing, the at least one processor outputs the reason for the determination by the determination process whether the task can be accomplished. The training data generation method described in Appendix B2.

[0100] (Note B4) In the generation process, the at least one processor shall again generate executable actions using the task that was determined to be unachievable by the determination process and the action used to identify the task in the identification process. The training data generation method described in Appendix B2 or B3.

[0101] (Note B5) In the generation process, the at least one processor generates the action using a provisional task, or the provisional task and the reason for determining whether the provisional task is achievable. The training data generation method described in any one of the appendices B1 to B4.

[0102] (Note B6) In the aforementioned specific process, the at least one processor identifies a task obtained by inputting the actions generated in the generation process into the generation model. The training data generation method described in one of the appendices B1 to B5.

[0103] (Note B7) In the aforementioned specific processing, the at least one processor uses the column of actions generated in the generation processing to identify tasks that can be achieved by executing the column of actions. The training data generation method described in Appendix B4.

[0104] (Note B8) The at least one processor further includes an execution process that performs the actions generated in the generation process, In the specified process, the at least one processor uses data corresponding to the state after the execution of the action to identify a task that may be achieved by the execution of the action. The training data generation method described in any one of the appendices B1 to B7.

[0105] (Note B9) The at least one processor further includes a training process for training the generative model using the training data, The training data generation method described in any one of the appendices B1 to B8.

[0106] [Additional Note C] This disclosure includes the technologies described in the following appendices. However, the present invention is not limited to the technologies described in the following appendices, and various modifications are possible within the scope of the claims.

[0107] (Note C1) A program for causing a computer to function as a training data generation device, wherein the computer, A means of acquiring data, A generation means for generating executable actions for the aforementioned data, A means for identifying tasks that can be achieved by executing the actions generated by the generation means, An output means outputs a pair of a task identified by the identification means and an action generated by the generation means as training data used to train a generative model that generates actions to accomplish an input task. A training data generation program to enable it to function as such.

[0108] (Note C2) The aforementioned computer, The generation means further functions as a determination means that determines whether the task identified by the identification means can be achieved by executing the action generated by the generation means, The output means outputs a task that has been determined to be achievable by the determination means and an action that can achieve that task. The training data generation program described in Appendix C1.

[0109] (Note C3) The output means outputs the reason for the determination by the determination means whether the task can be accomplished. The training data generation program described in Appendix C2.

[0110] (Note C4) The generation means then uses the task that the determination means determined to be unachievable and the action used by the identification means to identify the task to re-generate an action that can be executed. The training data generation program described in Appendix C2 or C3.

[0111] (Note C5) The generation means generates the action using a provisional task, or the provisional task and the reason for determining whether the provisional task is achievable. The training data generation program described in any one of the appendices C1 to C4.

[0112] (Appendix C6) The identifying means identifies a task obtained by inputting the action generated by the generating means into a generation model. The training data generation program described in any one of the appendices C1 to C5.

[0113] (Note C7) The identifying means uses the sequence of actions generated by the generating means to identify tasks that can be achieved by executing the sequence of actions. The training data generation program described in Appendix C4.

[0114] (Note C8) The aforementioned computer, The aforementioned generation means is further configured to function as an execution means that executes the action generated by the generation means, The identifying means uses data corresponding to the state after the transition resulting from the execution of the action to identify tasks that can potentially be achieved by the execution of the action. The training data generation program described in any one of the appendices C1 to C7.

[0115] (Note C9) The aforementioned computer, Further functioning as a training means for training the generative model using the aforementioned training data, The training data generation program described in any one of the appendices C1 to C8.

[0116] [Additional Note D] This disclosure includes the technologies described in the following appendices. However, the present invention is not limited to the technologies described in the following appendices, and various modifications are possible within the scope of the claims.

[0117] (Note D1) It comprises at least one processor, and the at least one processor is The data acquisition process, A generation process that generates executable actions for the aforementioned data, A specific process that uses the actions generated in the generation process to identify tasks that can potentially be achieved by executing those actions, An output process that outputs pairs of tasks identified in the aforementioned specific process and actions generated in the aforementioned generation process as training data to be used to train a generative model that generates actions to accomplish the input task, A training data generation device that performs the following actions.

[0118] The training data generation device may also include memory. Furthermore, the memory may store a program for causing at least one processor to perform each of the aforementioned processes.

[0119] (Note D2) The aforementioned at least one processor, A determination process is further executed to determine whether the task identified in the specific process can be achieved by executing the action generated in the generation process, In the output process, the at least one processor outputs a task that has been determined to be achievable by the determination process and an action that can achieve that task. The training data generation device described in Appendix D1.

[0120] (Note D3) In the output processing, the at least one processor outputs the reason for the determination by the determination process whether the task can be accomplished. The training data generation device described in Appendix D2.

[0121] (Note D4) In the generation process, the at least one processor shall again generate executable actions using the task that was determined to be unachievable by the determination process and the action used to identify the task in the identification process. The training data generation device described in Appendix D2 or D3.

[0122] (Note D5) In the generation process, the at least one processor generates the action using a provisional task, or the provisional task and the reason for determining whether the provisional task is achievable. A training data generation device as described in any one of the appendices D1 to D4.

[0123] (Note D6) In the aforementioned specific process, the at least one processor identifies a task obtained by inputting the actions generated in the generation process into the generation model. A training data generation device as described in any one of the appendices D1 to D5.

[0124] (Note D7) In the aforementioned specific processing, the at least one processor uses the column of actions generated in the generation processing to identify tasks that can be achieved by executing the column of actions. The training data generation device described in Appendix D4.

[0125] (Note D8) The aforementioned at least one processor, The execution process that executes the actions generated in the aforementioned generation process is further executed, In the specified process, the at least one processor uses data corresponding to the state after the execution of the action to identify a task that may be achieved by the execution of the action. A training data generation device as described in any one of the notes D1 to D7.

[0126] (Note D9) The aforementioned at least one processor, Further, a training process is performed to train the generative model using the aforementioned training data. A training data generation device as described in any one of the appendices D1 to D8.

[0127] [Additional Note E] This disclosure includes the technologies described in the following appendices. However, the present invention is not limited to the technologies described in the following appendices, and various modifications are possible within the scope of the claims.

[0128] (Note E1) A program for causing a computer to function as a training data generation device, wherein the computer, The data acquisition process, A generation process that generates executable actions for the aforementioned data, A specific process that uses the actions generated in the generation process to identify tasks that can potentially be achieved by executing those actions, An output process that outputs pairs of tasks identified in the aforementioned specific process and actions generated in the aforementioned generation process as training data to be used to train a generative model that generates actions to accomplish the input task, A non-temporary recording medium containing a training data generation program for executing [the program]. [Explanation of symbols]

[0129] 1. Training data generation device 1A Information Processing Device 11, 111A Acquisition Department 12 Generation part 13 Specific section 14. 50A output section 112A Action Generation Unit 113A Action Execution Unit 114A Task Identification Unit 115A Judgment section 116A Output Control Unit 117A Training Department

Claims

1. A means of acquiring data, A generation means for generating executable actions for the aforementioned data, A means for identifying tasks that can be achieved by executing the actions generated by the generation means, An output means outputs a pair of a task identified by the identification means and an action generated by the generation means as training data used to train a generative model that generates actions to accomplish an input task. A training data generation device equipped with the following features.

2. The system further comprises a determination means for determining whether the task identified by the identification means can be achieved by executing the action generated by the generation means, The output means outputs a task that has been determined to be achievable by the determination means and an action that can achieve that task. The training data generation device according to claim 1.

3. The output means outputs the reason for the determination by the determination means whether the task can be accomplished. The training data generation device according to claim 2.

4. The generation means then uses the task that the determination means determined to be unachievable and the action used by the identification means to identify the task to re-execute the generation of executable actions. The training data generation device according to claim 2 or 3.

5. The generation means generates the action using a provisional task, or the provisional task and the reason for determining whether the provisional task is achievable. The training data generation device according to claim 1 or 2.

6. The identifying means identifies a task obtained by inputting the action generated by the generating means into a generation model. The training data generation device according to claim 1 or 2.

7. The identifying means uses the sequence of actions generated by the generating means to identify tasks that can be achieved by executing the sequence of actions. The training data generation device according to claim 4.

8. The system further comprises an execution means for executing the action generated by the generation means, The identifying means uses data corresponding to the state after the transition resulting from the execution of the action to identify tasks that can potentially be achieved by the execution of the action. The training data generation device according to claim 1 or 2.

9. At least one processor handles the data acquisition process, The at least one processor performs a generation process that generates an action that can be performed on the data, The at least one processor performs a specific process that uses the actions generated in the generation process to identify tasks that can potentially be achieved by executing those actions, The at least one processor outputs a pair of tasks identified in the specific process and actions generated in the generation process as training data to be used to train a generative model that generates actions to accomplish an input task, A method for generating training data that includes this.

10. A training data generation program for causing a computer to function as a training data generation device that generates training data used to train a generative model that generates actions to accomplish an input task, wherein the computer is... A means of acquiring data, A generation means for generating executable actions for the aforementioned data, A means for identifying tasks that can be achieved by executing the actions generated by the generation means, An output means that outputs a pair of a task identified by the identification means and an action generated by the generation means as training data, A training data generation program to enable it to function as such.

Images