GENERATION DEVICE, GENERATION METHOD, AND GENERATION PROGRAM

Abstract

The generating device (100) has a generating unit (140) that generates a graph in which a plurality of node groups corresponding to a plurality of component groups arranged in parallel are connected in series, or a graph in which a first node corresponding to a first component belonging to a first component group among the plurality of component groups and a second node corresponding to a second component belonging to a second component group among the plurality of component groups are connected via an edge.

Description

[Technical field]

[0001] The present disclosure relates to a generating device, a generating method, and a generating program. [Background technology]

[0002] A robot is composed of multiple parts. The configuration of a robot may be expressed as a graph. The nodes included in the graph represent the parts. The connections between the parts are represented by edges. For example, graphs are described in Non-Patent Document 1. Non-Patent Document 1 also describes reinforcement learning. An agent can perform reinforcement learning by using the graph. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Wenlong Huang et al. “One Policy to Control Them All:Shared Modular Policies for Agent-Agnostic Control”, ICML, 2020 Summary of the Invention [Problem to be solved by the invention]

[0004] Messages are transmitted within the graph. Messages include messages for making multiple parts move in the same or similar manner (hereafter referred to as "equivalent messages"), messages for making each part move independently (hereafter referred to as "independent messages"), and messages for making multiple parts move in a coordinated manner (hereafter referred to as "cooperative messages"). Incidentally, the graph is generated so as to represent the hardware configuration of the robot. For example, if multiple parts are arranged in parallel, multiple nodes included in the graph are arranged in parallel. Since multiple nodes are arranged in parallel, an equality message is transmitted to multiple nodes. However, since multiple nodes are arranged in parallel, the agent cannot learn the movements indicated by messages other than equality messages.

[0005] The objective of this disclosure is to generate graphs for learning the behavior exhibited by messages other than equivalent messages. [Means for solving the problem]

[0006] According to one aspect of the present disclosure, there is provided a generating device, which includes a generating unit configured to generate a graph in which a plurality of node groups corresponding to a plurality of component groups arranged in parallel are connected in series, or a graph in which a first node corresponding to a first component belonging to a first component group among the plurality of component groups is connected via an edge to a second node corresponding to a second component belonging to a second component group among the plurality of component groups. Effect of the Invention

[0007] According to the present disclosure, it is possible to generate graphs for learning the behavior exhibited by messages other than equivalent messages. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a diagram illustrating hardware included in a generating device. [Diagram 2] FIG. 2 is a block diagram showing the functions of a generating device. [Diagram 3] FIG. 13 is a diagram showing a comparative example of a graph. [Figure 4] FIG. 11 is a diagram showing a specific example (part 1) of graph generation processing. [Diagram 5] FIG. 13 is a diagram showing a specific example (part 2) of the graph generation process. [Figure 6]13 is a diagram showing a specific example of a graph generation process according to the first modified example. FIG. [Figure 7] FIG. 13 is a diagram showing a specific example of a graph generation process according to the second modification. [Figure 8] FIG. 13 is a diagram showing a specific example of a graph generation process according to the third modification. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment will be described with reference to the drawings. The following embodiment is merely an example, and various modifications are possible within the scope of the present disclosure.

[0010] Embodiment 1 is a diagram showing hardware included in a generating device 100. The generating device 100 is a computer. The generating device 100 is a device that executes a generating method. The generation device 100 generates a graph to be included in the agent. The graph is composed of a plurality of nodes and a plurality of edges. The agent performs reinforcement learning using the generated graph.

[0011] The generating device 100 includes a processor 101 , a volatile storage device 102 , and a non-volatile storage device 103 .

[0012] The processor 101 controls the entire generating device 100. For example, the processor 101 is a central processing unit (CPU) or a field programmable gate array (FPGA). The processor 101 may be a multiprocessor. Furthermore, the generating device 100 may include a processing circuit.

[0013] The volatile storage device 102 is a main storage device of the generating device 100. For example, the volatile storage device 102 is a random access memory (RAM). The non-volatile storage device 103 is an auxiliary storage device of the generating device 100. For example, the non-volatile storage device 103 is a hard disk drive (HDD) or a solid state drive (SSD).

[0014] Next, the functions of the generating device 100 will be described. 2 is a block diagram showing the functions of the generating device 100. The generating device 100 includes a storage unit 110, an acquisition unit 120, a modification unit 130, and a generating unit 140.

[0015] The storage unit 110 may be realized as a storage area secured in the volatile storage device 102 or the non-volatile storage device 103 . A part or all of the acquiring unit 120, the changing unit 130, and the generating unit 140 may be realized by a processing circuit. Also, a part or all of the acquiring unit 120, the changing unit 130, and the generating unit 140 may be realized as a module of a program executed by the processor 101. For example, the program executed by the processor 101 is also called a generating program. For example, the generating program is recorded on a recording medium.

[0016] The storage unit 110 stores various information. The acquisition unit 120 acquires hardware configuration information of the robot. For example, the acquisition unit 120 acquires the hardware configuration information from the storage unit 110. Also, for example, the acquisition unit 120 acquires the hardware configuration information from an external device. Note that the external device is a device that exists outside the generating device 100. For example, the external device is a cloud server. Note that the illustration of the external device is omitted.

[0017] The hardware configuration information indicates that a plurality of groups of components are arranged in parallel. This sentence may be expressed as follows: The hardware configuration information indicates that a plurality of groups are arranged in parallel. Note that a group of components includes one or more components.

[0018] The change unit 130 identifies a plurality of component groups that are arranged in parallel based on the hardware configuration information, and changes the identified plurality of component groups to be arranged in series. The generating unit 140 generates a graph based on the changed hardware configuration information.

[0019] Next, the graph generation process will be described using a concrete example. First, a case where a graph is generated so as to represent the hardware configuration of a robot will be described.

[0020] FIG. 3 is a diagram showing a comparative example of a graph. The hardware configuration information indicates that a plurality of component groups are arranged in parallel. The left diagram of FIG. 3 shows component groups 10a, 10b, and 10c. A component group includes one or more components. Components a1, a2, and a3 belong to component group 10a. Components b1, b2, and b3 belong to component group 10b. Components c1, c2, and c3 belong to component group 10c.

[0021] It is assumed that a graph is generated so as to represent the hardware configuration. The right diagram in Fig. 3 shows the generated graph. Component groups 10a, 10b, and 10c correspond to node groups 20a, 20b, and 20c. Component groups a1 to a3 correspond to nodes A1 to A3. Component groups b1 to b3 correspond to nodes B1 to B3. Component groups c1 to c3 correspond to nodes C1 to C3. As shown in the right diagram of Figure 3, when multiple node groups are placed in parallel, the agent cannot learn the behavior indicated by messages other than equivalent messages (e.g., independent messages).

[0022] Therefore, the generating device 100 generates a graph as follows. 4 is a diagram showing a specific example (part 1) of the graph generation process. The change unit 130 identifies the component groups 10a, 10b, and 10c that are arranged in parallel based on the hardware configuration information. The change unit 130 changes the component groups 10a, 10b, and 10c to be arranged in series. The generating unit 140 generates a graph based on the changed hardware configuration information. Specifically, the generating unit 140 generates a graph in which the node groups 20a, 20b, and 20c corresponding to the component groups 10a, 10b, and 10c are connected in series. By connecting the node groups 20a, 20b, and 20c in series, the agent can learn the actions indicated by the independent messages.

[0023] Furthermore, the generating device 100 may execute the following process. The process will be described using a specific example. FIG. 5 is a diagram showing a specific example (part 2) of the graph generation process. The change unit 130 identifies a plurality of component groups arranged in parallel based on the hardware configuration information. For example, the change unit 130 identifies component groups 10a, 10b, and 10c arranged in parallel based on the hardware configuration information. The change unit 130 changes the hardware configuration information so that a first component belonging to a first component group among the identified plurality of component groups is electrically connected to a second component belonging to a second component group among the plurality of component groups. For example, the change unit 130 changes the hardware configuration information so that a component a3 belonging to the component group 10a among the component groups 10a, 10b, and 10c is electrically connected to a component b1 belonging to the component group 10b among the component groups 10a, 10b, and 10c.

[0024] The generating unit 140 generates a graph based on the changed hardware configuration information. In detail, the generating unit 140 generates a graph in which a first node corresponding to a first part belonging to a first part group among the multiple part groups is connected via an edge to a second node corresponding to a second part belonging to a second part group among the multiple part groups. For example, the generating unit 140 generates a graph in which a node A3 corresponding to a part a3 belonging to the part group 10a is connected via an edge to a node B1 corresponding to a part b1 belonging to the part group 10b.

[0025] By connecting the node A3 and the node B1, the cooperation message is transmitted. For example, the cooperation message is transmitted from the node A3 to the node B1. This allows the agent to learn the case where the node A3 and the node B1 cooperate with each other.

[0026] As described above, the agent can learn the behavior indicated by the independent message or the cooperative message by using the graph. Therefore, according to the embodiment, the generation device 100 can generate a graph for learning the behavior indicated by the message other than the equivalent message.

[0027] Variation of embodiment 1. The generating device 100 may execute the following processes. The processes will be described using a specific example. 6 is a diagram showing a specific example of the graph generation process of Modification 1. The change unit 130 identifies a plurality of component groups arranged in parallel based on the hardware configuration information. For example, the change unit 130 identifies component groups 10a, 10b, and 10c arranged in parallel based on the hardware configuration information.

[0028] The change unit 130 changes two or more of the identified multiple component groups to be connected in series. For example, the change unit 130 changes the component groups 10a and 10b of the component groups 10a, 10b, and 10c to be connected in series.

[0029] The generating unit 140 generates a graph based on the changed hardware configuration information. In detail, the generating unit 140 generates a graph in which two or more node groups corresponding to two or more component groups among the multiple component groups are connected in series. For example, the generating unit 140 generates a graph in which the node groups 20a and 20b corresponding to the component groups 10a and 10b among the component groups 10a, 10b, and 10c are connected in series. In addition, the remaining component groups (for example, the component group 10c) among the multiple component groups are not changed by the changing unit 130. Therefore, the two or more node groups connected in series and the node groups corresponding to the remaining component groups among the multiple component groups are connected in parallel. For example, the node groups 20a and 20b and the node group 20c are connected in parallel.

[0030] Connected in this way, the agents can learn the behaviors implied by equivalent and independent messages.

[0031] According to the first modification, the generating device 100 can generate a graph for learning the behaviors exhibited by equivalent messages and independent messages.

[0032] Modification of embodiment 2. The generating device 100 may execute the following processes. The processes will be described using a specific example. 7 is a diagram showing a specific example of the graph generation process of Modification 2. The change unit 130 identifies a plurality of component groups arranged in parallel based on the hardware configuration information. For example, the change unit 130 identifies component groups 10a, 10b, and 10c arranged in parallel based on the hardware configuration information.

[0033] The change unit 130 changes the identified multiple component groups to be connected in series. For example, the change unit 130 changes the component groups 10a, 10b, and 10c to be connected in series. The change unit 130 changes the hardware configuration information so that a first component belonging to a first component group among the identified multiple component groups and a second component belonging to a second component group among the identified multiple component groups are electrically connected. For example, the change unit 130 changes the hardware configuration information so that a component a3 belonging to the component group 10a and a component c1 belonging to the component group 10c are electrically connected.

[0034] The generating unit 140 generates a graph based on the changed hardware configuration information. In detail, the generating unit 140 generates a graph in which a plurality of node groups corresponding to a plurality of component groups are connected in series. The generating unit 140 also generates a graph in which a first node corresponding to a first component belonging to a first component group among the plurality of component groups is connected via an edge to a second node corresponding to a second component belonging to a second component group among the plurality of component groups. For example, the generating unit 140 generates a graph in which node groups 20a, 20b, and 20c corresponding to component groups 10a, 10b, and 10c are connected in series. The generating unit 140 generates a graph in which a node A3 corresponding to a component a3 belonging to the component group 10a is connected via an edge to a node C1 corresponding to a component c1 belonging to the component group 10c.

[0035] Connected in this way, the agents can learn the behaviors implied by independent and cooperative messages.

[0036] According to the second modification, the generating device 100 can generate a graph for learning the behaviors indicated by the independent messages and the cooperative messages.

[0037] Modification of embodiment 3. The generating device 100 may execute the following processes. The processes will be described using a specific example. 8 is a diagram showing a specific example of the graph generation process of Modification 3. The change unit 130 identifies a plurality of component groups arranged in parallel based on the hardware configuration information. For example, the change unit 130 identifies component groups 10a, 10b, and 10c arranged in parallel based on the hardware configuration information.

[0038] The change unit 130 changes two or more of the identified multiple component groups to be connected in series. For example, the change unit 130 changes the component groups 10a and 10b of the component groups 10a, 10b, and 10c to be connected in series. The change unit 130 changes the hardware configuration information so that a first component belonging to the first component group of the two or more component groups changed to be connected in series is electrically connected to a second component belonging to the unchanged component group. For example, the change unit 130 changes the hardware configuration information so that a component a3 belonging to the component group 10a of the component groups 10a and 10b is electrically connected to a component c3 belonging to the component group 10c.

[0039] The generating unit 140 generates a graph based on the changed hardware configuration information. In detail, the generating unit 140 generates a graph in which two or more node groups corresponding to two or more component groups among the multiple component groups are connected in series. For example, the generating unit 140 generates a graph in which the node groups 20a and 20b corresponding to the component groups 10a and 10b are connected in series. Furthermore, the remaining component groups (for example, the component group 10c) among the multiple component groups are not changed by the changing unit 130. Therefore, for example, the node groups 20a and 20b and the node group 20c are connected in parallel. Furthermore, the generating unit 140 generates a graph in which a node belonging to a first node group among the two or more node groups connected in series and a node belonging to a second node group connected in parallel to the two or more node groups are connected via an edge. For example, the generating unit 140 generates a graph in which a node A3 belonging to the node group 20a among the node groups 20a and 20b and a node C3 belonging to the node group 20c connected in parallel to the node groups 20a and 20b are connected via an edge.

[0040] Connected in this way, agents can learn the behaviors implied by equivalent, independent, and cooperative messages.

[0041] According to the third modification, the generating device 100 can generate a graph for learning the behaviors exhibited by equivalent messages, independent messages, and cooperative messages.

[0042] In the above, a case has been described in which the processes are performed in the following order: The acquisition unit 120 acquires hardware configuration information. The change unit 130 identifies a group of multiple components to be arranged in parallel based on the hardware configuration information. The change unit 130 changes the hardware configuration information. The generation unit 140 generates a graph based on the changed hardware configuration information.

[0043] The generating device 100 may generate a graph in the following order: The acquiring unit 120 acquires hardware configuration information. The generating unit 140 generates a graph based on the hardware configuration information. The modifying unit 130 identifies a plurality of node groups corresponding to a plurality of component groups arranged in parallel based on the graph. The modifying unit 130 modifies the plurality of node groups. In this way, the generating device 100 can generate the graphs described in the embodiment and the first to third modifications.

[0044] Furthermore, the generating device 100 may generate a graph in the following order: The acquiring unit 120 acquires a graph including a plurality of node groups corresponding to a plurality of component groups arranged in parallel from the storage unit 110 or an external device. The modifying unit 130 identifies a plurality of node groups corresponding to a plurality of component groups arranged in parallel based on the graph. The modifying unit 130 modifies the plurality of node groups. This enables the generating device 100 to generate the graphs described in the embodiment and the first to third modifications. [Explanation of symbols]

[0045] Component groups 10a, 10b, 10c; generation device 100; processor 101; volatile storage device 102; non-volatile storage device 103; storage unit 110; acquisition unit 120; modification unit 130; generation unit 140.

Claims

1. a generating unit that generates a graph in which a plurality of node groups corresponding to a plurality of component groups arranged in parallel are connected in series, or a graph in which a first node corresponding to a first component belonging to a first component group among the plurality of component groups and a second node corresponding to a second component belonging to a second component group among the plurality of component groups are connected via an edge; A generating device having the following:

2. the generation unit generates a graph in which two or more node groups corresponding to two or more component groups among the plurality of component groups are connected in series; the two or more node groups and node groups corresponding to the remaining component groups among the plurality of component groups are connected in parallel; The generating device of claim 1 .

3. the generation unit generates a graph in which the plurality of node groups corresponding to the plurality of component groups are connected in series and the first node and the second node are connected via an edge. The generating device of claim 1 .

4. the generation unit generates a graph in which two or more node groups corresponding to two or more of the multiple component groups are connected in series, and in which nodes belonging to a first node group among the two or more node groups are connected via edges to nodes belonging to a second node group connected in parallel to the two or more node groups. The generating device of claim 1 .

5. The generating device, generating a graph in which a plurality of node groups corresponding to a plurality of component groups arranged in parallel are connected in series, or a graph in which a first node corresponding to a first component belonging to a first component group among the plurality of component groups and a second node corresponding to a second component belonging to a second component group among the plurality of component groups are connected via an edge; Generation method.

6. On the computer, generating a graph in which a plurality of node groups corresponding to a plurality of component groups arranged in parallel are connected in series, or a graph in which a first node corresponding to a first component belonging to a first component group among the plurality of component groups and a second node corresponding to a second component belonging to a second component group among the plurality of component groups are connected via an edge; The generating program that causes the processing to occur.

Images