Information processing device

Abstract

The information processing device of the present disclosure includes: a generation unit that generates action information representing an action of an agent on the basis of an environment graph representing a state of an environment as a graph and task information representing a target task; and an execution unit that causes the agent to execute, in the environment, an action based on the action information, and collects, as training data used when performing machine learning of a model for predicting a cost for the action information, the action information, the environment graph representing states of the environment corresponding respectively to before and after the executed action, and an actual cost incurred by execution of the action based on the action information.

Description

Information processing device

[0001] This disclosure relates to an information processing device.

[0002] The introduction of autonomous systems in which robots act independently at work sites is being considered. For example, Patent Document 1 describes a method in which a target action prediction model is used to determine a future target action in order to minimize a cost function representing free energy, and an action generation model is used to generate actions according to the target action.

[0003] Japanese Patent Publication No. 2024-1984

[0004] However, in the technology described in Patent Document 1, it is unclear whether the cost of the generated action is appropriate to the actual cost of the action. As a result, the problem arises that it is difficult to accurately predict the cost of the robot's actions.

[0005] Therefore, one of the purposes of this disclosure is to solve the aforementioned problem, which is the difficulty in accurately predicting the cost of robot actions.

[0006] An information processing device, as one embodiment of the present disclosure, has the following configuration: a generation unit that generates action information representing the actions of an agent based on an environment graph that represents the state of the environment in a graph and task information that represents a target task; and an execution unit that causes the agent to perform an action based on the action information within the environment, and collects the action information, the environment graph that represents the state of the environment before and after the performed action, and the actual cost incurred by performing the action based on the action information, as training data used when machine learning a model to predict the cost of the action information. An information processing method, as one embodiment of the present disclosure, has the following configuration: an information processing device that generates action information representing the actions of an agent based on an environment graph that represents the state of the environment in a graph and task information that represents a target task, causes the agent to perform an action based on the action information within the environment, and collects the action information, the environment graph that represents the state of the environment before and after the performed action, and the actual cost incurred by performing the action based on the action information, as training data used when machine learning a model to predict the cost of the action information. Furthermore, a program in one form of this disclosure has the following configuration: it generates action information representing the actions of an agent based on an environment graph that represents the state of the environment in a graph and task information that represents a target task; it causes the agent to perform an action based on the action information within the environment; and it collects the action information, the environment graph that represents the state of the environment before and after the performed action, and the actual cost incurred by performing the action based on the action information as training data to be used when machine learning a model to predict the cost of the action information.

[0007] This disclosure, configured as described above, enables accurate prediction of the cost of a robot's actions.

[0008] This is a block diagram showing an example of the configuration of the information processing device related to this disclosure. This is a flowchart showing an example of the processing operation by the information processing device related to this disclosure. This is a diagram showing an example of the processing by the information processing device related to this disclosure. This is a diagram showing an example of the processing by the information processing device related to this disclosure. This is a diagram showing an example of the processing by the information processing device related to this disclosure. This is a block diagram showing an example of the configuration of the information processing device related to this disclosure. This is a flowchart showing an example of the processing operation by the information processing device related to this disclosure. This is a block diagram showing an example of the configuration of the information processing device related to this disclosure. This is a block diagram showing an example of the processing operation by the information processing device related to this disclosure. This is a block diagram showing an example of the hardware configuration of the information processing device related to this disclosure. This is a block diagram showing an example of the configuration of the information processing device related to this disclosure. This is a flowchart showing an example of the processing operation by the information processing device related to this disclosure.

[0009] <First Embodiment> A first embodiment of the present disclosure will be described with reference to the drawings. The drawings may be relevant to any embodiment.

[0010] The information processing device disclosed herein is used, for example, to collect training data used in machine learning a predictive model for predicting the cost of robot actions. The information processing device is also used to machine learn the predictive model using the collected training data. In this embodiment, the cost to be predicted is the time required for a robot to move, pick up packages, and place items autonomously in a warehouse. However, the information processing device disclosed herein is not limited to robots operating in a warehouse, but may be used for robots performing any action in any location, such as robots assembling products in a manufacturing plant or robots providing customer service. Furthermore, the cost to be predicted is not limited to time, but may be any parameter, such as power consumption or expenses incurred during the action.

[0011] The following describes an example of the configuration and operation of the information processing device in this embodiment. The information processing device consists of one or more information processing devices equipped with an arithmetic unit and a memory device. As shown in Figure 1, the information processing device includes a work planning unit 11, a plan result selection unit 12, a plan result execution unit 13, a graph cost prediction model learning unit 14, and a graph cost prediction unit 15. The functions of the work planning unit 11, the plan result selection unit 12, the plan result execution unit 13, the graph cost prediction model learning unit 14, and the graph cost prediction unit 15 can be realized by the arithmetic unit executing a program for realizing each function stored in the memory device. The information processing device 10 also includes a training data storage unit 16 realized in the memory device.

[0012] The work planning unit 11 (generation unit) generates a problem for work planning from the environment graph 21 and task information 22. The environment graph 21 represents the state of the environment in which the robot agent acts, using a graph consisting of nodes and edges. An example of an environment is shown in Figure 3. The environment shown in Figure 3 is a warehouse, and there are various locations P within the warehouse, an agent A which is the robot that acts, objects O such as goods that agent A transports, shelves E where the objects are stored, and shipping shelves F where the objects are placed for shipment. The environment graph represents the state of this environment using a graph consisting of nodes and edges. For example, as shown in Figure 4, agent A, object O, shelves E and F, and location P are represented as nodes, and the relationships between each node are connected by edges to represent the environment graph. In this situation, agent A is capable of performing three types of actions: move, pick, and place, and the time required to perform each action will differ depending on the state. For example, the longer the distance traveled, the longer the time required, and the picking time may differ depending on the object. Furthermore, depending on the situation, it may be impossible to perform an action, such as when the aisles between shelves are too narrow for two agents A to pass each other.

[0013] Furthermore, task information 22 represents the target task (work) for agent A within the environment. For example, task information 22 may represent the target task of picking object O from a specific shelf E in the warehouse and placing it on the shipping shelf F.

[0014] Specifically, the work planning unit 11 first initializes the graph cost prediction model (step S1 in Figure 2), starts a learning loop (step S2 in Figure 2), and creates a work planning problem from the current environment graph and task information (step S3 in Figure 2). At this time, the work planning unit 11 creates the work planning problem using a planning algorithm such as A-star. For example, as shown in equation 1, while checking the cost f(x) of each state x in the state space X to be planned, the initial state x ｓ From target state x ｆ The system searches for the most cost-optimal path to the destination and generates an action sequence. In this embodiment, cost is defined as time. Here, g(x) in equation 1 represents the initial state x ｓ This is the actual cost to reach state x from state x' (for example, the sum of the actual costs along the path). Also, in equation 1, h(x') is the distance from state x' to the target state x ｆ This is the predicted cost to reach a certain state (for example, a heuristic value). And Δg(x, x') in equation 1 is the actual cost from state x to state x'. For example, the graph representations of states x and x' and the information of the actions taken at each state are used as queries, and a graph cost prediction model is used to predict the predicted cost value and its uncertainty information (variance, etc.), and the predicted cost value is used as Δg(x, x').

[0015] More specifically, the work planning unit 11 performs an action column set search loop process consisting of the following steps (steps S4 to S13 in Figure 2) in order to calculate multiple action columns that can reach the target state of the same task information. 1. Define the problem setting for work planning from the environment graph and task information. 2. Initialize the action column set Da and the predicted cost column set Dc. Repeat steps 3 and 4 N times to start the action column set search loop. 3. Execute the planning algorithm to calculate the action column (equation 2) and the corresponding predicted cost column (predicted cost value and uncertainty) (equation 3). 4. The calculated optimal action column is added to the action column set, and the corresponding predicted cost is added to the predicted cost column set (Equation 4). 5. Output the action column set Da and the predicted cost column set Dc.

[0016] As described above, when calculating the predicted cost value Δg(x, x') and its uncertainty u during the search, the graph cost prediction unit 15 uses a machine learning-trained graph cost prediction model to perform the calculation (step S8 in Figure 2). At this time, the graph cost prediction unit 15 inputs the action information during the search and the environment graphs before and after the action based on the action information into the machine learning-trained graph cost prediction model. Specifically, the action information includes, for example, an action ID that identifies the action, an agent ID that identifies the agent performing the action, and a target cost as needed. The environment graphs before and after the action are an environment graph representing the situation before agent A performs the action and an environment graph representing the situation after the action is performed. For example, the environment graphs before and after the action can be represented as shown in Figures 4 and 5. Figure 4 shows, as an example, the environment graph before and after agent A's movement action, and Figure 5 shows, as an example, the environment graph before and after agent A's pick action of object O.

[0017] Then, the graph cost prediction unit 15 calculates a set of predicted cost values using the graph cost prediction model. At this time, as shown in Equation (5), the set of predicted cost values consists of the upper limit value and the lower limit value of the predicted cost value. Using these, the graph cost prediction unit 15 can calculate the final predicted cost value Δg and its uncertainty u. For example, Equation (6) shows an example of calculating the predicted cost based on the average of the set of predicted cost values and calculating the uncertainty based on the variance of the set of predicted cost values. Also, Equation (7) shows an example of calculating the predicted cost using the upper limit value of the predicted cost and calculating the uncertainty based on the difference between the upper and lower limit values of the predicted cost value.

[0018] Also, during the search for the action sequence, when the action sequence being searched already includes an action registered in the action sequence set, the work plan unit 11 adds a penalty to the cost and conducts the search. For example, as shown in Equation (8), a penalty function φ is added to the cost.

[0019] In the above manner, the work plan unit 11 can obtain the action sequence set and the predicted cost sequence set, and outputs them to the plan result selection unit 12.

[0020] The plan result selection unit 12 (generation unit) selects an execution action sequence to be executed in the environment based on the predicted cost among the above-described action sequence set. Specifically, the plan result selection unit 12 statistically processes each element C ｉ of the predicted cost sequence set, and determines a score Sc ｉ for each action sequence a ｉ (step S14 in FIG. 2). Here, the score Sc ｉ is calculated based on, for example, the predicted cost and its uncertainty as shown in Equation (9).

[0021] Then, the plan result selection unit 12 sorts the elements of the action sequence set according to the value of the score Sc ｉ and selects the action sequence with the smallest score as the execution action sequence, and outputs it to the plan result execution unit 13 (step S15 in FIG. 2).

[0022] The planning result execution unit 13 (execution unit) uses the selected execution action sequence (action information) as a control command to execute the execution action sequence within the target environment. At this time, the target environment may be a real environment or a simulated environment (virtual environment). The planning result execution unit 13 can then obtain an actual cost sequence, which is the actual cost incurred by executing the action sequence. After executing the execution action sequence, the planning result execution unit 13 collects a dataset consisting of the execution action sequence, an environment graph sequence representing the state of the environment before and after the execution of the action, and the obtained actual cost sequence as training data, and stores it in the training data storage unit 16 (step S16 in Figure 2).

[0023] The graph cost prediction model learning unit 14 (learning unit) uses the training data dataset stored in the training data storage unit 16 to machine-learn the graph cost prediction model (step S17 in Figure 2). Specifically, the graph cost prediction model learning unit 14 takes the execution action column and the environment graph columns before and after the execution of the action as explanatory variables and machine-learns the graph cost prediction model to output the actual cost as the objective variable. When the machine-learning by the graph cost prediction model learning unit 14 is completed (step S18 in Figure 2), the machine-learned graph cost prediction model can output the predicted cost by taking the action information generated by the work planning unit 11 and the environment graphs before and after the action as input, as described above.

[0024] As described above, in this embodiment, a graph cost prediction model is trained using machine learning with environmental graphs before and after the generated action (behavior). This improves the accuracy of the predicted cost by the prediction model for an action.

[0025] The information processing device in this disclosure may further include a control unit. The control unit has the function of, for example, using a created graph cost prediction model to calculate the cost between the environment graph before the operation and the environment graph after the operation for each of the agent's multiple operations, selecting the operation with the smallest calculated cost from among the multiple operations, and controlling the agent's operation to perform the selected operation.

[0026] <Second Embodiment> Next, a second embodiment of the present disclosure will be described with reference to the drawings. Note that the drawings may be relevant to either embodiment.

[0027] The information processing device in this embodiment has the same configuration as Embodiment 1 described above. In addition, the information processing device has the following configuration. The following will mainly describe the configuration that differs from the above.

[0028] As shown in Figure 6, the information processing device in this embodiment further includes an action column decomposition unit 17, and accordingly, the work planning unit 11 has the following functions. Specifically, the work planning unit 11 first executes the planning algorithm as described above and calculates the action column shown in equation 2 and the corresponding predicted cost column shown in equation 3 (steps S1 to S7 in Figure 7). Then, when searching for an action column, unlike in Embodiment 1, the work planning unit 11 does not use prediction by a cost prediction model, but uses a predefined constant or a value determined by heuristics for the predicted cost Δg(x, x') (step S21 in Figure 7). If an action that is already registered in the action column set is selected during the search, a penalty is added to the cost as described above and the search is performed (step S22 in Figure 7). After that (YES in step S23 in Figure 7), the work planning unit 11 outputs the action column after the search is completed and the corresponding environment graph column before and after the action to the action column decomposition unit 17, and obtains the predicted cost column corresponding to the action column through prediction by the graph cost prediction unit 15.

[0029] Specifically, the action column decomposition unit 17 (generation unit) decomposes the action column output from the work planning unit 11 and the environment graph column before and after the action into individual action units as shown in equation 10 (step S24 in Figure 7). Then, the action column decomposition unit 17 processes each action a ｉ，ｊ The graph cost prediction unit 15 outputs the corresponding environment graphs before and after the action. As a result, the graph cost prediction unit 15 calculates the corresponding predicted cost (predicted cost value and uncertainty) for each action using the graph cost prediction model, as shown in equation 11 (steps S25 and S26 in Figure 7). The action column decomposition unit 17 reconstructs the action column and predicted cost column from the predicted costs obtained from the graph cost prediction unit 14 (step S27 in Figure 7) and outputs them to the work planning unit 11.

[0030] The work planning unit 11 adds the action column and predicted cost column obtained from the action column decomposition unit 17 to each set (steps S12 and S13 in Figure 7) and outputs them to the planning result selection unit 12. From there, the process is the same as in Embodiment 1, and the execution action column is selected based on the predicted cost and executed in the environment, thereby collecting training data (steps S14 to S18 in Figure 7).

[0031] As described above, in this embodiment, the process of searching for action columns and calculating predicted costs are separated. This makes it possible to reduce the time required to search for action information, which is the action column.

[0032] <Third Embodiment> Next, a third embodiment of the present disclosure will be described with reference to the drawings. Note that the drawings may be relevant to any of the embodiments.

[0033] The information processing device in this embodiment has the same configuration as Embodiment 1 described above. In addition, the information processing device has the following configuration. The following will mainly describe the configuration that differs from the above.

[0034] Figure 8 shows the configuration of the graph cost prediction unit 15 of the information processing apparatus in the present embodiment. As shown in this figure, the graph cost prediction unit 15 in the present embodiment includes a local environment graph extraction unit 18.

[0035] The local environment graph extraction unit 18 (generation unit) generates a local environment graph by extracting a part around the target agent A from the environment graphs corresponding to before and after the above-described action (behavior). At this time, the local environment graph extraction unit 18 extracts nodes by restricting the depth N from the nodes of the location P to which the target agent A belongs. In the example of FIG. 9, the right figure shows an example in which nodes up to a depth of 2 nodes are extracted from the nodes of the location P to which the target agent A belongs. Further, the local environment graph extraction unit 18 may perform node extraction not only by a method based on the depth of the nodes but also by other methods. For example, by referring to the position information of the nodes of the location P to which the agent A belongs and the surrounding nodes, nodes existing within a radius r from the nodes of the location P may be extracted to generate a local environment graph.

[0036] The local environment graph extraction unit 18 performs node extraction in the environment graphs corresponding to before and after the action, and generates local environment graphs corresponding to before and after the action respectively. Then, the local environment graph extraction unit 18 can obtain a predicted cost by inputting action information (action ID, agent ID, target cost as necessary), and the local environment graphs before and after the action, into the graph cost prediction model.

[0037] Incidentally, along with the above, the local environment graph extraction unit 18 (learning unit) may also be provided in the graph cost prediction model learning unit 14. In this case, the local environment graph extraction unit 18 generates a local environment graph by extracting a part around the target agent A from the environment graphs corresponding to before and after the action (behavior) collected as training data, in the same manner as described above. Then, the graph cost prediction model learning unit 14 inputs the action information (action ID, agent ID, target cost as required) that is training data, and the local environment graphs before and after the action extracted from the training data, as explanatory variables into the graph cost prediction model, and performs machine learning on the graph cost prediction model using the actual cost that is the training data as the objective variable. Incidentally, the local environment graph may be generated by extracting nodes at the time of collecting the training data and stored in the training data storage unit 16.

[0038] As described above, in the present embodiment, since the data amount of the environment graph input when calculating the predicted cost can be reduced, the prediction of the predicted cost can be performed with higher accuracy.

[0039] <Fourth Embodiment> Next, a fourth embodiment of the present disclosure will be described with reference to the drawings. In the present embodiment, an outline of an information processing apparatus and the like described in the above-described embodiment is shown. Note that the drawings may be relevant to any of the embodiments.

[0040] First, the hardware configuration of the information processing device 100 in this disclosure will be described. The information processing device 100 is configured as a general information processing device, and as an example, as shown in Figure 10, it is equipped with the following hardware configuration: • CPU (Central Processing Unit) 101 (arithmetic unit) • ROM (Read Only Memory) 102 (storage device) • RAM (Random Access Memory) 103 (storage device) • Program group 104 loaded into RAM 103 • Storage device 105 storing the program group 104 • Drive device 106 for reading and writing to external storage medium 110 • Communication interface 107 connecting to an external communication network 111 • Input / output interface 108 for data input / output • Bus 109 connecting each component

[0041] Figure 10 shows an example of the hardware configuration of the information processing device 100, and the hardware configuration of the information processing device is not limited to the case described above. For example, the information processing device may consist of only a part of the above configuration, such as not having a drive device 106. In addition, the information processing device may use a GPU (Graphics Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating Point Number Processing Unit), PPU (Physics Processing Unit), TPU (Tensor Processing Unit), quantum processor, microcontroller, or a combination thereof instead of the CPU described above.

[0042] The information processing device 100 can be equipped with the generation unit 121 and execution unit 122 shown in Figure 11 by having the CPU 101 acquire the program group 104 and execute it. The program group 104 is, for example, stored in advance in the storage device 105 or ROM 102, and the CPU 101 loads it into the RAM 103 and executes it as needed. The program group 104 may also be supplied to the CPU 101 via the communication network 111, or it may be stored in advance in the storage medium 110, and the drive device 106 reads the program and supplies it to the CPU 101. However, the generation unit 121 and execution unit 122 described above may be constructed with dedicated electronic circuits to realize such means.

[0043] The generation unit 121 generates action information representing the agent's actions based on an environment graph that represents the state of the environment in a graph and task information that represents the target task (step S101 in Figure 12). The execution unit 122 has the agent perform an action based on the action information within the environment and collects the action information, an environment graph that represents the corresponding state of the environment before and after the performed action, and the actual cost incurred by performing the action based on the action information as training data to be used when machine learning a model to predict the cost of the action information (step S102 in Figure 12).

[0044] In the above configuration, the information processing device 100 first generates agent action information and executes this action information within the environment. Then, it collects a dataset of the executed action information, the environment graphs corresponding to the time before and after the executed action, and the actual cost obtained from the execution as training data. Furthermore, by performing machine learning using the training data, a model can be generated to predict the cost of the action. In this way, by using the environment graphs before and after the action, the cost can be predicted with greater accuracy.

[0045] Furthermore, at least one of the functions of the generation unit 121 and the execution unit 122 described above may be executed on an information processing device installed and connected to any location on the network, that is, it may be executed using so-called cloud computing.

[0046] Furthermore, the programs described above can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memory (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). Programs may also be supplied to a computer using various types of transient computer-readable media. Examples of transient computer-readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer-readable media can be supplied to a computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.

[0047] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure are possible, as can be understood by those skilled in the art within the scope of the present disclosure. Furthermore, each of the embodiments described above can be combined with other embodiments as appropriate.

[0048] <Notes> Some or all of the above embodiments may also be described as shown in the following notes. The outline of the configuration of the information processing apparatus, information processing method, and program in this disclosure will be described below. However, this disclosure is not limited to the configurations described in the following notes. Furthermore, some or all of the configurations and functions of the configurations described in Notes 2 to 8, which are dependent on Note 1 below, may also be dependent on the other Notes 9 and 10 in the same way as Notes 2 to 8. Moreover, not limited to Notes 1, 9, and 10, some or all of the configurations and functions of the configurations described as notes may also be dependent on similar hardware, software, various recording means for recording software, or systems, without departing from the above embodiments. (Note 1) An information processing device comprising: a generation unit that generates action information representing the actions of an agent based on an environment graph that represents the state of the environment in a graph and task information that represents a target task; and an execution unit that causes the agent to perform an action based on the action information within the environment, and collects the action information, the environment graph that represents the state of the environment before and after the performed action, and the actual cost incurred by performing the action based on the action information as training data to be used when machine learning a model that predicts the cost of the action information. (Note 2) An information processing device according to Note 1, comprising a learning unit that machine learning the model using the action information and the environment graph, which are the training data, as explanatory variables and the actual cost, which is the training data, as the objective variable. (Note 3) An information processing device as described in Note 1, wherein the generation unit predicts costs by inputting the generated behavior information and the environment graphs corresponding to the before and after of the behavior included in the behavior information into the model, selects the behavior information to be executed in the environment from the generated behavior information based on the predicted costs, and the execution unit executes the behavior based on the selected behavior information in the environment and collects the training data.(Note 4) An information processing device as described in Note 3, wherein the generation unit uses the model to predict the cost and uncertainty of the cost corresponding to the generated behavior information, and selects the behavior information to be executed within the environment from among the generated behavior information based on at least one of the values ​​of the cost and uncertainty. (Note 5) An information processing device as described in Note 1, wherein the generation unit decomposes the behavior included in the generated behavior information, inputs the decomposed behavior and the environment graph corresponding to the before and after of the behavior into the model to predict the cost for each behavior, predicts the cost of the behavior information based on the predicted cost for each behavior, and selects the behavior information to be executed within the environment from among the behavior information based on the predicted cost. (Note 6) An information processing device as described in Note 3, wherein the generation unit predicts the cost by inputting the generated behavior information and a part of the area around the agent in the environment graph corresponding to the before and after of the behavior included in the behavior information into the model. (Note 7) An information processing device as described in Note 2, wherein the learning unit uses the behavior information and a portion of the environment graph surrounding the agent corresponding to the behavior before and after the behavior included in the behavior information as explanatory variables, and uses the actual cost from the training data as the objective variable to machine learn the model. (Note 8) An information processing device as described in Note 3, wherein the generation unit predicts the cost of the generated behavior information by adding a penalty if the generated behavior information is already included.(Note 9) An information processing method comprising: an information processing device generating action information representing the actions of an agent based on an environment graph representing the state of the environment in a graph and task information representing a target task; causing the agent to perform an action based on the action information within the environment; and collecting the action information, the environment graph representing the state of the environment before and after the performed action, and the actual cost incurred by performing the action based on the action information as training data to be used when machine learning a model to predict the cost of the action information. (Note 10) A program causing an information processing device to execute a process that generates action information representing the actions of an agent based on an environment graph representing the state of the environment in a graph and task information representing a target task; causing the agent to perform an action based on the action information within the environment; and collecting the action information, the environment graph representing the state of the environment before and after the performed action, and the actual cost incurred by performing the action based on the action information as training data to be used when machine learning a model to predict the cost of the action information.

[0049] 10 Information Processing Unit 11 Work Planning Unit 12 Planning Result Selection Unit 13 Planning Result Execution Unit 14 Graph Cost Prediction Model Learning Unit 15 Graph Cost Prediction Unit 16 Training Data Storage Unit 17 Action Column Splitting Unit 18 Local Environment Graph Extraction Unit 21 Environment Graph 22 Task Information 100 Information Processing Unit 101 CPU 102 ROM 103 RAM 104 Program Group 105 Storage Device 106 Drive Device 107 Communication Interface 108 Input / Output Interface 109 Bus 110 Storage Medium 111 Communication Network 121 Generation Unit 122 Execution Unit

Claims

1. An information processing device comprising: a generation unit that generates action information representing the actions of an agent based on an environment graph that represents the state of the environment in a graph and task information that represents a target task; and an execution unit that causes the agent to perform an action based on the action information within the environment, and collects the action information, the environment graph that represents the state of the environment before and after the performed action, and the actual cost incurred by performing the action based on the action information as training data to be used when machine learning a model to predict the cost of the action information.

2. An information processing device according to claim 1, comprising a learning unit that performs machine learning on the model using the behavioral information and the environmental graph, which are training data, as explanatory variables, and the actual cost, which is training data, as the objective variable.

3. An information processing device according to claim 1, wherein the generation unit predicts costs by inputting the generated behavior information and the environment graphs corresponding to the before and after of the behavior included in the behavior information into the model, selects the behavior information to be executed in the environment from the generated behavior information based on the predicted costs, and the execution unit executes the behavior based on the selected behavior information in the environment and collects the training data.

4. An information processing device according to claim 3, wherein the generation unit predicts the cost and uncertainty of the cost corresponding to the generated behavioral information using the model, and selects the behavioral information to be executed in the environment from among the generated behavioral information based on at least one of the values ​​of the cost and uncertainty.

5. An information processing device according to claim 1, wherein the generation unit decomposes the actions included in the generated action information, inputs the decomposed actions and the environment graphs corresponding to the before and after of the actions into the model to predict the cost of each action, predicts the cost of the action information based on the predicted cost for each action, and selects the action information to be executed within the environment from among the action information based on the predicted cost.

6. An information processing device according to claim 3, wherein the generation unit predicts cost by inputting the generated behavior information and a portion of the area surrounding the agent in the environment graph corresponding to the time before and after the behavior included in the behavior information into the model.

7. An information processing device according to claim 2, wherein the learning unit uses the behavior information and a portion of the environment graph surrounding the agent corresponding to the behavior before and after the behavior included in the behavior information as explanatory variables, and uses the actual cost from the training data as the objective variable, to machine-learn the model.

8. An information processing device according to claim 3, wherein the generation unit predicts the cost of the generated behavioral information by adding a penalty if the generated behavioral information includes behavioral information that has already been generated.

9. An information processing method comprising: an information processing device that generates action information representing the actions of an agent based on an environment graph representing the state of the environment in a graph and task information representing a target task; having the agent perform an action based on the action information within the environment; and collecting the action information, the environment graph representing the state of the environment before and after the performed action, and the actual cost incurred by performing the action based on the action information as training data to be used when machine learning a model to predict the cost of the action information.

10. A program that causes an information processing device to generate action information representing the actions of an agent based on an environment graph representing the state of the environment and task information representing a target task, to cause the agent to perform an action based on the action information within the environment, and to collect the action information, the environment graph representing the state of the environment before and after the performed action, and the actual cost incurred by performing the action based on the action information as training data to be used when machine learning a model to predict the cost of the action information.

Images