Control system and control method

Abstract

To control a device group for executing the same task with a simpler configuration.SOLUTION: The control apparatus 1 performs adversarial training of the generative model including the generator 121 that generates pseudo action data similar to true action data as the true action data for actions sequentially executed by each device in the device group to transition from one state to the next state at each time, and the discriminator 122 that discriminates between the pseudo action data generated by the generator 121 and the true action data, in the trained recurrent neural network constructed by the training of the first learning unit 11.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a control device, a device, and a control method, and more particularly to a control technique for a group of devices. [Background technology]

[0002] In recent years, there has been an increasing demand for autonomous AI systems and autonomous AI agents for robots, vehicles, etc. For example, Patent Document 1 discloses a technology in which a system for a single agent that controls a moving object employs a reinforcement learning model and guides the moving object to a destination point using an action value function that represents the expected cumulative value of future rewards that will be obtained if an action indicating the next course of action is taken based on the current position of the moving object.

[0003] When single-agent learning is applied to non-cooperative multi-agent systems where each agent independently performs the same task, each agent independently learns using methods such as reinforcement learning. In this case, the amount of learning for the entire system increases, resulting in a heavier computational load. Furthermore, when each agent independently learns similar information, the learning efficiency may decrease due to the duplication of redundant information about similar environments. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7407329 Summary of the Invention [Problem to be solved by the invention]

[0005] With conventional technology, it has been difficult to control a group of devices that execute the same task with a simpler configuration.

[0006] The present invention has been made to solve the above-mentioned problems, and has an object to control a group of devices that execute the same task with a simpler configuration. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, a control device according to the present invention is a control device for controlling a group of devices, each of which performs the same task by sequentially executing actions, and includes: a first learning unit configured to learn a recurrent neural network that uses actions sequentially performed by a first device of the group of devices to transition from a state to the next state over time as training data, inputs the state of the first device, and outputs actions sequentially performed by the first device to transition from the state to the next state over time; a second learning unit configured to perform adversarial learning of a generative model having a generator that generates pseudo action data similar to true action data for actions sequentially performed by each of the group of devices to transition from a state to the next state over time, using the trained recurrent neural network constructed by the training by the first learning unit; and a classifier that distinguishes between the pseudo action data generated by the generator and the true action data; and a presentation unit configured to present the trained generator obtained by the adversarial learning of the second learning unit as control information for the group of devices to execute the task.

[0008] In addition, the control device of the present invention may further include a first acquisition unit configured to acquire actions that the first device sequentially performs to transition from one state to the next state over time, which are used as the teaching data.

[0009] In the control device according to the present invention, the pseudo action data may have a data distribution that has a minimum statistical distance from the data distribution of the true action data.

[0010] In order to solve the above-mentioned problems, the device of the present invention comprises a second acquisition unit configured to acquire the state of the device itself, a generation unit configured to generate the pseudo action data indicating actions to be executed sequentially to transition from the acquired state of the device itself to the next state at each time using the learned generator included in the control information presented by the above-mentioned control device, and an execution unit configured to sequentially execute the actions indicated by the pseudo action data generated by the generation unit and execute a set task.

[0011] In order to solve the above-mentioned problems, a control method according to the present invention is a control method for controlling a group of devices that perform the same task by each performing an action sequentially, the control method including: a first learning step of training a recurrent neural network that uses actions sequentially performed by a first device of the group of devices to transition from one state to the next state over time as training data, inputs the state of the first device, and outputs actions sequentially performed by the first device to transition from the state to the next state over time; a second learning step of performing adversarial learning of the trained recurrent neural network constructed in the first learning step as true action data for actions sequentially performed by each device of the group of devices to transition from the state to the next state over time, using the trained recurrent neural network constructed in the first learning step as true action data for the actions sequentially performed by each device of the group of devices to transition from the state to the next state over time, a generator that generates pseudo action data similar to the true action data, and a classifier that distinguishes between the pseudo action data generated by the generator and the true action data; and a presentation step of presenting the trained generator obtained by the adversarial learning in the second learning step as control information for the group of devices to perform the task.

[0012] In addition, the control method of the present invention may further include a first acquisition step of acquiring actions that the first device has sequentially performed to transition from one state to the next state at each time, which are used as the teaching data. [Effects of the Invention]

[0013] According to the present invention, the trained recurrent neural network constructed by the training by the first training unit is subjected to adversarial training of a generative model having a generator that generates pseudo action data similar to true action data as true action data for actions that each of a group of devices sequentially executes to transition from one state to the next over time, and a classifier that distinguishes between the pseudo action data generated by the generator and the true action data. As a result, a group of devices performing the same task can be controlled with a simpler configuration. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a block diagram showing the configuration of a control system including a control device and multiple UAVs according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing the configuration of a UAV according to this embodiment. [Figure 3] FIG. 3 is a block diagram showing the hardware configuration of the UAV according to this embodiment. [Figure 4] FIG. 4 is a diagram for explaining the first learning unit included in the control device according to the present embodiment. [Figure 5] FIG. 5 is a block diagram showing the configuration of the second learning unit included in the control device according to this embodiment. [Figure 6] FIG. 6 is a block diagram showing the configuration of the second learning unit included in the control device according to this embodiment. [Figure 7] FIG. 7 is a diagram for explaining the second learning unit included in the control device according to the present embodiment. [Figure 8] FIG. 8 is a block diagram showing the hardware configuration of the control device according to this embodiment. [Figure 9] FIG. 9 shows the operation sequence of the control system according to this embodiment. [Figure 10] FIG. 10 is a flowchart showing the operation of the control device according to this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will now be described in detail with reference to FIGS.

[0016] [Control system configuration] First, referring to FIG. 1, an overview of a control system including a control device 1 and multiple UAVs 2 according to an embodiment of the present invention will be described. In the following description, a group of UAVs 2 will be used as an example of a group of devices that perform the same task by sequentially executing actions. The same task will be described as a group of UAVs 2 collectively reaching a set destination. Furthermore, the action performed at each time to perform the task will be described as each UAV 2 moving sequentially in a predetermined direction to move from its current location to the next location at each time. It should be understood that the present invention is not limited to collective movement control using a group of UAVs 2, as long as the group of devices collectively perform a single task by sequentially executing actions in response to events that occur at each time.

[0017] The control system according to this embodiment includes a control device 1 and multiple UAVs 2, which are communicatively connected to each other via a wireless communication network NW that conforms to a predetermined communication standard such as LTE / 4G, 5G, or 6G. The control system controls the multiple UAVs 2 to perform a task of moving collectively to a set destination position in a mobile space A. As shown in FIG. 1, the mobile space A in which the UAVs 2 fly is capable of communication using, for example, a 5G wireless communication system.

[0018] The UAV2 is a mobile object such as a drone or unmanned aerial vehicle, and is capable of autonomous flight. The autonomous flight of the UAV2 is controlled by a flight controller that processes information from a sensor 208 (described later) and controls the rotation speed of a motor 209 and the angle of a propeller 210. In this embodiment, the UAV2 controls its autonomous flight to a destination point based on route control information stored in an auxiliary storage device 205. Each of the multiple UAV2 has the same configuration, and when there is no need to distinguish between them, they are collectively referred to as UAV2. Details of the functional blocks and hardware configuration of the UAV2 will be described later. In this embodiment, of the multiple UAV2, for example, a UAV2 that has flown first through the movement space A can be selected as UAV2a (first device).

[0019] As shown in Figure 1, mobile space A, in which multiple UAVs 2 fly, is a three-dimensional matrix-like space composed of multiple unit spaces divided into spaces. Each unit space that makes up mobile space A has the same volume. Furthermore, each unit space has a node ID, and each unit space is represented by a single representative position (x, y, z). The location information can be three-dimensional GPS position coordinates consisting of latitude, longitude, and altitude. For example, a representative value such as the center position of the unit space can be used as the position of the unit space.

[0020] As shown in Figure 1, multiple UAVs 2 use each unit space as a waypoint to move from a unit space position corresponding to the current position S of each UAV 2 to a unit space position of a destination point G. The location of the destination point G of each UAV 2 is set arbitrarily and is a common destination point for multiple UAVs 2. The unit space has a distance that the UAV 2 can travel at each time, i.e., in one time step, and the UAV 2 can reach adjacent unit spaces at each time.

[0021] The control system according to this embodiment uses a recurrent neural network (RNN) to learn the actions to be executed sequentially to move from one state to the next state, e.g., the UAV 2a flying ahead, selected from among the multiple UAVs 2. That is, the direction of travel for moving from the current unit space position to the next unit space position. Furthermore, the trained RNN is used as true data for the sequential direction of travel of each of the multiple UAVs 2 to move through the unit space to a destination point, i.e., true action data. A generator is constructed by adversarial learning that generates pseudo-action data similar to the true action data. The control system presents the trained generator 121' to the multiple UAVs 2 as control information for the multiple UAVs 2 to execute the task of moving collectively to the destination point. Furthermore, each UAV 2 determines the sequential direction of travel based on its own current unit space position based on the control information, and performs flight control to the destination point.

[0022] Each UAV 2 generates pseudo action data using the trained generator 121' included in the control information, and changes its direction of travel by maneuvering its own aircraft, as shown by the arrows in Figure 1, based on the direction of travel indicated by the generated pseudo action data from the current unit space position of each UAV 2. The direction of travel indicated by the pseudo action data can include various directions of travel as the action to be executed. The direction of travel specifies which adjacent unit space the UAV 2 will move to next from its current unit space position. Although Figure 1 illustrates the movement space A as a two-dimensional plane, the direction of travel of the UAV 2 can also be a three-dimensional direction of travel. Using the control information presented by the control device 1, multiple UAVs 2 can collectively complete the task of arriving at the unit space of the destination point G.

[0023] [Controller function block] As shown in FIG. 1, the control device 1 includes a first acquisition unit 10, a first learning unit 11, a second learning unit 12, a first storage unit 13, and a presentation unit 14.

[0024] The first acquisition unit 10 acquires actions sequentially performed by a UAV 2a selected from among multiple UAVs 2 to transition from one state to the next state over time. A state is a position in a unit space in which the UAV 2 exists. The time-series sequence history of actions acquired by the first acquisition unit 10 is used as training data for RNN learning in the first learning unit 11. More specifically, the first acquisition unit 10 acquires a time-series sequence history related to the direction of travel of the UAV 2a from the GPS position in the unit space at the start point, with each unit space to the destination being used as a waypoint. As described above, the UAV 2a is a UAV 2 selected as a representative from among the multiple UAVs 2.

[0025] The first learning unit 11 uses as training data the actions that are executed sequentially to transition from one state to the next state at each time of the UAV 2a among the multiple UAVs 2, and learns an RNN that outputs the actions that are executed sequentially to transition from that state to the next state at each time, using as input the state of the UAV 2a at each time. Specifically, the first learning unit 11 uses as training data for the RNN a time series sequence in which the GPS position value of the unit space in which the UAV 2a exists, acquired by the first acquisition unit 10, is assigned as a correct answer label the value of the direction of travel that is followed sequentially to move from that GPS position to a position in the next unit space.

[0026] 4 is a schematic diagram showing the network structure of the RNN used in the learning of the first learning unit 11. As shown in FIG. 4, the RNN is a neural network consisting of an input layer X, a hidden layer H which is a memory cell, and an output layer Y, and input time series data x (t) Predict time series data y (t)In this embodiment, the input time series data is the GPS position of the unit space where UAV2a exists at each time. The predicted time series data is the direction of travel for moving from the position of the unit space where UAV2a exists to the position of the next unit space at each time. In an RNN, information is transmitted from the hidden layer at a certain time to the hidden layer at the next time along the time series. Therefore, the hidden layer at time t receives input from the hidden layer at time t and the hidden layer at the immediately preceding time t-1.

[0027] Given time series data x (t) On the other hand, RNN is defined by the following equations (1) and (2). h (t) =σ(U×x (t) +W×h (t-1) +b) (1) y (t) =V×h (t) +c (2)

[0028] The above equation (1) represents the propagation from the input layer to the hidden layer, and x (t) is the input vector at time t. h (t) is the output of the hidden layer at time t. U is the weight parameter from the input to the hidden layer, W is the weight parameter from the hidden state at the previous time t-1 to the hidden state at the current time t. b is the bias vector of the hidden layer, and σ is the activation function.

[0029] The above equation (2) represents the propagation from the hidden layer to the output layer, and y (t) is the output vector at time t. V is the weight parameter from the hidden layer to the output layer, and c is the bias vector for the output layer.

[0030] The first learning unit 11 uses a backpropagation method or the like to compare the obtained predicted output value with the output value of the training data for a given input value, introduces an objective function to minimize the error, examines the error of the weight parameters and propagates it backward, and finally determines the weight parameters U, W, V and biases b and c. Through this learning process, the first learning unit 11 constructs a trained RNN. The trained RNN constructed by the first learning unit 11 is used as training data for adversarial learning of a generative model by the second learning unit 12.

[0031] The second learning unit 12 performs adversarial learning of a generative model that uses the trained RNN obtained by learning by the first learning unit 11 as true action data for actions that multiple UAVs 2 perform sequentially to transition from one state to the next state at each time, and has a generator 121 that generates pseudo action data similar to the true action data, and a discriminator 122 that distinguishes between the pseudo action data generated by the generator 121 and the true action data.

[0032] As shown in FIG. 5, the second learning unit 12 adversarially learns a GAN (Generative Adversarial Network) having a generator 121 and a classifier 122. A trained generator 121' is constructed by the learning of the second learning unit 12. As described above, in this embodiment, the optimized action for the selected traveling direction of the UAV 2a indicated by the trained RNN constructed by the learning of the first learning unit 11 is considered as true action data of the actions of multiple UAVs 2. Furthermore, the true action data is used as training data to train the generator 121 that generates pseudo action data similar to the action for the traveling direction of the UAV 2a.

[0033] 6 and 7 are diagrams schematically illustrating the neural network structures of the generator 121 and the classifier 122 of the GAN used by the second learning unit 12. As shown in FIG. 6, the generator 121 is configured as a neural network having an input layer, a hidden layer, and an output layer. The generator 121 generates a neural network at each time t1 to t D 5 and 6, the generator 121 is configured with D generators 121_1 to 121_D corresponding to GPS_1 to GPS_D, which are GPS positions of the unit space in which the UAV 2 exists. The generator 121 provides the GPS position s of the sequential unit space as an input to each of the generators 121_1 to 121_D, and generates pseudo action data a1 to a D The GPS position s input to the generator 121 is the GPS route that the UAV 2a actually flew to the destination point.

[0034] Each of the generators 121_1 to 121_D corresponds to a memory cell of the trained RNN at each time used in training by the first learning unit 11. For example, the generator 121_1 indicated by G1 has a neural network structure capable of training pseudo action data similar to the parameters of a trained RNN that receives GPS_1, which is the GPS position in the unit space at time t1, as input and outputs a traveling direction a1 for moving from the position of GPS_1 to a position in the next unit space. A CNN or a ResNet can be used as the neural network constituting the generator 121.

[0035] The generator 121 outputs an output G(s) after performing a product-sum operation on the input and weight parameters and threshold processing using an activation function. For example, the output G1(s) from the generator 121_1 is pseudo-action data similar to the action a1 output when GPS_1 at time t1 is given as input to the trained RNN. Similarly, at each time t2 to t D For the generators 121_2 to 121_D corresponding to the states, the actions a2 to a DIn this way, the generator 121 according to this embodiment receives the actual GPS position of the UAV 2a at each time as an input and outputs a random function.

[0036] The classifier 122 shown in FIG. 7(a) is configured with a neural network having an input layer, a hidden layer, and an output layer. In the example of FIG. 7(a), as the input of the training data, true action data is given, which is output from the trained RNN obtained by the first learning unit 11, and indicates the direction of travel taken to move from the position in the unit space where the UAV 2a exists to the position in the next unit space at each time. FIG. 7(b) is a diagram for explaining the input data given to the classifier 122. In the example of FIG. 7(b), training data related to the true action data is illustrated. That is, the true action data is output from the trained RNN obtained by the first learning unit 11, and indicates the direction of travel taken to move from the position in the unit space where the UAV 2a exists to the position in the next unit space at each time. D When UAV2 is located in the unit space of GPS_1 to GPS_D, the action a1 to a2 is the direction of travel of UAV2 to reach the position of the adjacent unit space at the next time. D is given to the classifier 122 as training data.

[0037] The classifier 122 outputs a binary value of 1 or 0 after performing a product-sum operation on the input and weight parameters and threshold processing using an activation function. The classifier 122 outputs an output y=1 when it correctly identifies the training data related to the input true action data as true action data. On the other hand, it outputs an output y=0 when it correctly identifies the training data related to the input pseudo action data as pseudo action data. In this way, the classifier 122 is a model that distinguishes the model distribution generated by the generator 121 from the data distribution of the training data, which is the true distribution. The pseudo action data has a data distribution that has the smallest statistical distance from the data distribution of the true action data. A CNN can be used as the neural network that constitutes the classifier 122.

[0038] FIG. 5 is a block diagram for explaining the adversarial learning of GAN by the second learning unit 12. The generator 121 of the GAN adopted by the second learning unit 12 is represented as function G, and the discriminator 122 is represented as function D. Note that, hereinafter, the functions G1 to GD of the generators 121_1 to 121_D are collectively referred to as function G. Furthermore, true action data is represented as x, the predicted value output by the discriminator 122 is represented as y, and the correct label is represented as t. The correct label t is set to 1 for true action data and 0 for pseudo action data generated by the generator 121. At this time, the discriminator 122 calculates the cross entropy E CE It can be expressed as:

[0039]

number

[0040] The first term in the brace of the above equation (3) represents t n lny n In this case, the predicted value y n is the correct label of the true action data, t n = 1. On the other hand, the second term in the braces represents (1-t n )ln(1-y n ), the predicted value y n is the correct label value (1-t n ) = 0. In this way, the cross entropy E CE is the maximum value when the predicted value matches the correct label value.

[0041] Here, the generator 121 that constitutes the GAN has parameters w G ,θ G and the function G(w G ,θ G ) The classifier 122 also uses the parameter w D ,θ D and function D(w D ,θ D ) The cross entropy E in the above equation (3) CEThe objective function E of the GAN including the generator 121 and the discriminator 122 based on the above can be expressed by the following equation (4).

number

[0042] The first term in the above equation (4) represents E D(x)=1 lnD(w D ,θ D ) is the expected value that the classifier 122 will classify the true action data as the true action data. D(x)=0 ln(1-D(G(w G ,θ G ),w D ,θ D )) is the expected value at which the classifier 122 classifies the pseudo action data generated by the generator 121 as pseudo action data. In GAN learning, the generator 121 and the classifier 122 are trained adversarially through min-max optimization of the objective function E. Therefore, the generator 121 is trained to be able to generate pseudo action data that deceives the classifier 122, and the classifier 122 is trained to classify the pseudo action data generated by the generator 121 as pseudo action data.

[0043] In learning of the classifier 122, when true action data is given, the classifier 122 outputs an output close to y=1, thereby maximizing the first term of the objective function E in the above equation (4). On the other hand, when pseudo action data is given, the classifier 122 learns to output an output close to y=0, thereby maximizing the second term of the objective function E.

[0044] In the learning of the generator 121, D(G(w G ,θ G ),w D ,θ D ) (G1(s)~GD(s) in Figure 5) is close to 1. G ,θ G) (G1(s) to GD(s) in FIG. 5), thereby minimizing the objective function E. The second learning unit 12 uses a learning procedure that alternately updates the parameters of the generator 121 and the parameters of the classifier 122. Details of the learning procedure of the generator 121 and the classifier 122 by the second learning unit 12 will be described later.

[0045] The first storage unit 13 stores the parameters of the trained RNN and the trained generator 121'. Specifically, the first storage unit 13 stores trained generators 121_1' to 121_D' corresponding to states at each time.

[0046] The presentation unit 14 presents the trained generator 121' obtained by the adversarial learning of the second learning unit 12 as control information for the multiple UAVs 2 to execute a task. More specifically, the presentation unit 14 transmits the trained generators 121_1' to 121_D' to the multiple UAVs 2 via the network NW as control information for the multiple UAVs 2 to execute a task of reaching a set destination point. Note that for the UAV 2a, a trained RNN constructed by the first learning unit 11 can be presented as control information.

[0047] [Control device hardware configuration] Next, an example of a hardware configuration for realizing the control device 1 having the above-described functions will be described with reference to FIG.

[0048] As shown in FIG. 8, the control device 1 can be realized, for example, by a computer having a processor 102, a main memory device 103, a communication interface 104, an auxiliary memory device 105, and an input / output (I / O) 106 connected via a bus 101, and a program that controls these hardware resources.

[0049] The processor 102 is realized by a CPU, a GPU, an FPGA, an ASIC, or the like.

[0050] The main memory device 103 pre-stores programs for the processor 102 to perform various controls and calculations. The processor 102 and the main memory device 103 implement the functions of the control device 1, such as the first acquisition unit 10, the first learning unit 11, the second learning unit 12, and the presentation unit 14 shown in FIG.

[0051] The communication interface 104 is an interface circuit for connecting the control device 1 to various external electronic devices via a network.

[0052] The auxiliary storage device 105 is composed of a readable / writable storage medium and a drive for reading and writing various information such as programs and data from and to the storage medium. The auxiliary storage device 105 can use a semiconductor memory such as a hard disk or flash memory as the storage medium.

[0053] The auxiliary storage device 105 has a program storage area for storing the control program executed by the control device 1. The auxiliary storage device 105 also has an area for storing a first learning program for RNN learning and a second learning program for adversarial learning using GAN. The auxiliary storage device 105 realizes the first storage unit 13 described in FIG. 1. The auxiliary storage device 105 also has an area for storing map data including the position coordinates of the movement space A, as well as the position coordinates and node IDs of the unit spaces that make up the movement space A in association with each other. The auxiliary storage device 105 also has an area for storing identification information of the UAV 2. Furthermore, the auxiliary storage device 105 may have, for example, a backup area for backing up the above-mentioned data, programs, etc.

[0054] The input / output I / O 106 is an input / output device that inputs signals from external devices and outputs signals to external devices.

[0055] The display device 107 is configured with an organic EL display, a liquid crystal display, etc. The display device 107 can display a map display of the moving space A, the current position of the UAV 2, the flight route, and the position information of the destination point.

[0056] [UAV functional blocks] Next, the functional blocks of the UAV 2 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the configuration of the UAV 2. As described above, the multiple UAVs 2 include a UAV 2a selected by the control device 1 and multiple UAVs 2 other than the UAV 2a, and each UAV 2 can be configured with the same functional blocks.

[0057] The UAV 2 includes a second storage unit 20, a second acquisition unit 21, a generation unit 22, a third storage unit 23, and an execution unit 24. Each of the multiple UAVs 2 determines the next direction of travel for the UAV based on the route control information set by the control device 1, and controls the flight of the UAV to the destination position.

[0058] The second storage unit 20 stores control information presented by the presentation unit 14 of the control device 1. The control information includes a trained generator 121' constructed by adversarial learning by the second learning unit 12 of the control device 1. More specifically, the second storage unit 20 stores trained generators 121_1' to 121_D'.

[0059] The second acquisition unit 21 acquires the status of the aircraft itself. More specifically, the second acquisition unit 21 acquires the GPS position of the unit space in which the aircraft exists at each time from the start of flight. The second acquisition unit 21 can refer to the third storage unit 23 and acquire the GPS position received by the GPS receiver 207 as the position of the aircraft itself in the unit space.

[0060] The generation unit 22 uses the learned generator 121' included in the control information to generate pseudo action data indicating actions to be executed sequentially to transition from the current state of the UAV 2 to the next state. More specifically, the generation unit 22 inputs the GPS position of the UAV 2 in the current unit space acquired by the second acquisition unit 21 into the learned generators 121_1' to 121_D', and performs calculations on the learned generators 121_1' to 121_D' to generate pseudo action data indicating the sequential direction of travel to move from the current unit space position of the UAV 2 to the next unit space position. The sequence of pseudo action data generated by each of the learned generators 121_1' to 121_D' is used as information to guide the flight of the UAV 2 to the destination point.

[0061] The third storage unit 23 stores map data including the position coordinates of the movement space A, and information associating the position coordinates of the unit spaces constituting the movement space A with the node IDs of the unit spaces. The third storage unit 23 also stores the position information of the destination point.

[0062] The execution unit 24 sequentially executes actions indicated by the pseudo action data generated by the generation unit 22 to execute the set task. Specifically, the execution unit 24 controls the flight of the UAV in accordance with the pseudo action data generated by the generation unit 22, which indicates the sequential direction of travel to move from the position of the unit space where the UAV 2 exists to the position of the next unit space at each time. The execution unit 24 can calculate a control command for the next direction of travel from the current GPS position of the UAV 2 and transmit the control command value to the motor 209. The execution unit 24 flies sequentially through the unit spaces that are used as waypoints, causing the UAV 2 to reach the set destination point.

[0063] In addition, with regard to UAV2a, the trained RNN constructed by the first learning unit 11 of the control device 1 can be configured to calculate the direction of travel for moving from the position in the unit space where UAV2a exists to the position in the next unit space at each time, and to guide the flight to the destination point.

[0064] [UAV hardware configuration] Next, an example of a hardware configuration for realizing the UAV2 having the above-described functions will be described with reference to FIG.

[0065] 3, the UAV 2 can be realized by, for example, a microcomputer including a processor 202, a main memory device 203, a communication interface 204, an auxiliary memory device 205, and an input / output (I / O) 206, all connected via a bus 201, a program for controlling these hardware resources, a GPS receiver 207, a sensor 208, a motor 209, a propeller 210, and a battery 211. A flight controller that controls the autonomous flight of the UAV 2 is realized by a computer such as a microcomputer and the program. The UAV 2 can mount these components on a frame of a predetermined shape.

[0066] The main memory device 203 pre-stores programs for the processor 202 to perform flight control and calculations. The processor 202 and the main memory device 203 implement the functions of the UAV 2, such as the second acquisition unit 21, the generation unit 22, and the execution unit 24, shown in FIG.

[0067] The communication interface 204 is an interface circuit for network connection between the UAV 2 and the control device 1.

[0068] The auxiliary storage device 205 is composed of a readable / writable storage medium and a drive for reading and writing various information such as programs and data from and to the storage medium. The auxiliary storage device 205 can use a semiconductor memory such as a hard disk or flash memory as the storage medium.

[0069] The auxiliary storage device 205 has a program storage area for storing a flight control program executed by the UAV 2. The auxiliary storage device 205 also has an area for storing a calculation program for determining the sequential direction of travel for moving from the unit space position where the UAV 2 is located to the next unit space position at each time based on the learned generator 121' included in the control information. The auxiliary storage device 205 realizes the second storage unit 20 and the third storage unit 23 described in FIG. 2. The auxiliary storage device 205 also has an area for storing identification information for the UAV 2. Furthermore, the auxiliary storage device 205 may have, for example, a backup area for backing up the above-mentioned data and programs.

[0070] The input / output I / O 206 is an input / output device that inputs signals from external devices and outputs signals to external devices.

[0071] The GPS receiver 207 has an antenna for receiving GPS signals built in. The GPS receiver 207 implements the second acquisition unit 21 in FIG.

[0072] The sensor 208 is composed of various sensors such as an altitude sensor, an attitude sensor, a camera, LiDAR, and RADAR. In addition to the GPS receiver 207, the altitude sensor realizes the second acquisition unit 21 in Fig. 2. Furthermore, the flight controller performs flight control of the UAV 2 based on the various sensor data measured by the sensor 208.

[0073] The motor 209 rotates by the rotation drive, and rotates the propeller 210 attached to the rotation shaft of the motor 209 .

[0074] The battery 211 is a lithium ion battery or the like, and supplies power to the components of the UAV2.

[0075] The UAV 2 may include a SIM and have the SIM's International Mobile Subscriber Identity (IMSI). The UAV 2 may also be configured as an IoT terminal with an IP address. These IMSI and IP address may be used as identification information unique to the UAV 2.

[0076] [Control system operation] Next, the operation of the control system including the control device 1 and multiple UAVs 2 configured as described above will be described with reference to the sequence diagram of FIG.

[0077] First, a destination point to which multiple UAVs 2 are to move collectively in the movement space A is set in advance. A UAV 2a selected from the multiple UAVs 2 transmits to the control device 1 a history of the direction of travel from its initial position in the movement space A to the destination point. The history of the direction of travel is a history of the direction of travel that the UAV 2a has traveled at each time point as it moves sequentially from the unit space position of its initial position to positions in adjacent unit spaces. The control device 1 acquires the history of the direction of travel transmitted from the UAV 2a as training data to be used in learning the RNN (step S1).

[0078] Next, the first learning unit 11 performs a first learning process to learn the RNN (step S2). More specifically, the first learning unit 11 uses training data in which the GPS position value of each unit space until the UAV 2a reaches the destination point is assigned, as a correct answer label, the value of the direction of travel actually taken from the GPS position of that unit space. Using the training data, the first learning unit 11 trains an RNN that inputs the GPS position of the unit space in which the UAV 2a exists and outputs the direction of travel taken sequentially to move from the GPS position of that unit space to the position of the next unit space at each time.

[0079] The first learning unit 11 uses a backpropagation method or the like to compare the predicted output value obtained for a given input value with the output value of the training data, introduces an objective function that minimizes the error, checks the error of each weight parameter, and propagates it backward, ultimately determining the weight parameters U, W, V and biases b and c in the above equations (1) and (2). The first storage unit 13 stores the trained RNN constructed in the first learning process of step S3 (step S3).

[0080] Next, the second learning unit 12 performs a second learning process (step S4). More specifically, the second learning unit 12 performs adversarial learning of a GAN having a generator 121 that generates pseudo action data similar to the true action data using the trained RNN obtained in the first learning process of step S2 as training data, i.e., true action data, and a classifier 122 that distinguishes between the pseudo action data generated by the generator 121 and the true action data. The true action data represents the traveling direction in which each of the multiple UAVs 2 moves sequentially from a position in the unit space where it exists to a position in the next unit space. Details of the second learning process of step S5 will be described later.

[0081] Next, the first storage unit 13 stores the trained generator 121' constructed in step S5 (step S4). Specifically, the trained generators 121_1' to 121_D' corresponding to the states at each time are stored in the first storage unit 13. Subsequently, the presentation unit 14 presents the trained generator 121' (trained generators 121_1' to 121_D') constructed in step S5 to the multiple UAVs 2 as control information for executing a task in which the multiple UAVs 2 collectively reach a destination point (step S6).

[0082] After the UAV2 receives the control information via the wireless communication network NW, the second acquisition unit 21 acquires the GPS position of the unit space in which the UAV2 is currently located (step S7). Next, the generation unit 22 provides the GPS position of the unit space in which the UAV2 is located at the time acquired in step S7 as input to the learned generator 121' included in the control information, and generates pseudo action data (step S8). The pseudo action data generated in step S8 indicates the direction in which the UAV2 will travel from the GPS position of the unit space in which the UAV2 is located at a certain time.

[0083] Furthermore, the execution unit 24 performs flight control of the UAV2 along the traveling direction indicated by the pseudo action data generated in step S8 (step S9). Subsequently, the UAV2 repeats the processes from step S7 to step S9 until it reaches the unit space of the destination point (step S10: NO). After that, the UAV2 completes the task by reaching the destination point (step S10: YES). Each of the multiple UAV2 performs the processes from step S7 to step S11 based on the GPS position of the unit space existing at each time. Note that the trained RNN constructed in the first learning process in step S2 is set for UAV2a of the multiple UAV2, and flight control to the destination point can be performed.

[0084] Next, the second learning process by the second learning unit 12 of the control device 1 will be described with reference to Fig. 10. First, the second learning unit 12 acquires the learned parameters, i.e., the learned RNN, obtained in the first learning process by the first learning unit 11 in step S2 of Fig. 9 as true action data related to the traveling directions in which the multiple UAVs 2 will move sequentially until they reach the destination point (step S30).

[0085] 5, the true action data collected in step S30 is used as training data 124 input when training the classifier 122. The training data related to the true action data input to the classifier 122 is a time-series sequence of the traveling direction of the UAV 2 obtained by the trained RNN. Furthermore, a set of n time-series sequences of the traveling directions can be used as training data.

[0086] Next, the second learning unit 12 inputs the true action data to the classifier 122 as training data 124, and adjusts the parameter w of the classifier 122 so that the true action data is distinguished from the true action data (y=1). D ,θ D (Step S31). In Step S31, the second learning unit 12 can cause the classifier 122 to learn the true action data using, for example, an error backpropagation method. By Step S31, the classifier 122 that can distinguish the true action data from the true action data is constructed in advance.

[0087] Next, the second learning unit 12 provides the GPS position of the unit space for each time as an input to the generator 121 (step S32). More specifically, as shown in FIG. 5, the GPS position of the unit space where the UAV 2a exists at time t1, GPS_1, is input to the generator 121_1. Similarly, from time t2 to t D For each of the UAVs 1a and 1b, the corresponding GPS positions (GPS_2 to GPS_D) are input to the respective generators 121_2 to 121_D. That is, the GPS route actually flown by the UAV 2a is given as an input value to the generator 121.

[0088] The generator 121 then generates a vector of input s and weight parameters w based on the given GPS location. G ,θ Gand threshold processing using an activation function to generate pseudo action data G(s) (step S33). More specifically, the generators 121_1 to 121_D generate pseudo action data G1(s) to GD(s), respectively.

[0089] Next, the second learning unit 12 learns the classifier 122. The learning of the classifier 122 is performed by using the parameter w D ,θ D First, the second learning unit 12 provides the training data 124 of the true action data acquired in step S30 as input to the classifier 122, and calculates the parameter w by backpropagation or the like so that the objective function E in the above formula (4) is maximized. D ,θ D (Step S34). The label of the training data 124 is set to 1 (true action data).

[0090] Next, in step S33, the second learning unit 12 provides the pseudo action data generated by the generator 121 to the classifier 122 as an input, and calculates the parameter w by backpropagation or the like so that the objective function E in the above equation (4) is maximized. D ,θ D That is, in steps S34 and S35, in order to maximize the objective function E in the above equation (4), the first term is updated as D(w D ,θ D )=1 is output, and the second term is D(G(w G ,θ G ),w D ,θ D )=0. Note that the label 0 (pseudo action data) is set in the training data 124. In step S35, the pseudo action data G1(s) to GD(s) generated by the generators 121_1 to 121_D corresponding to each time are provided to the classifier 122.

[0091] The training of the classifier 122 in steps S34 and S35 corresponds to the dashed arrows in FIG. 5, which indicate that a classifier error is calculated in block 125 of the objective function E based on the output 123 from the classifier 122, and then the error is backpropagated to the classifier 122.

[0092] Next, the second learning unit 12 trains the generator 121. The training of the generator 121 is performed with the parameters of the discriminator 122 fixed. The second learning unit 12 trains the generator 121 so that pseudo action data is generated when the GPS position of the unit space for each time is given to the generator 121. Specifically, the second learning unit 12 trains the parameter w by backpropagation or the like in order to minimize the objective function E in the above equation (4). G ,θ G is updated (step S36).

[0093] The learning in step S36 corresponds to the dashed arrows shown in Fig. 5, which indicate backpropagation of errors to the generators 121. That is, step S36 corresponds to the dashed arrows in which the pseudo action data G1(s) to GD(s) generated by the generators 121_1 to 121_D in Fig. 5 are input to the discriminator 122, a generator error is calculated from the output 123 in a block 125 of the objective function E, and the error is further backpropagated to the generators 121_1 to 121_D. In this way, in step S36, the generators 121_1 to 121_D provided corresponding to the states at each time are trained separately.

[0094] Thereafter, learning of the discriminator 122 and the generator 121 (generators 121_1 to 121_D) from step S33 to step S36 is repeated until the value of the objective function E reaches a Nash equilibrium and converges (step S37: NO). On the other hand, if the value of the objective function E has converged (step S37: YES), the processing from step S31 to step S37 is repeated for n sets of true action data until learning of the generator 121 (generators 121_1 to 121_D) and the discriminator 122 is performed (step S38: NO).

[0095] Then, all n pieces of true action data are taken at times t1 to t D When the generator 121 (generators 121_1 to 121_D) and the discriminator 122 have been trained using training data of the GPS positions (GPS_1 to GPS_D) of the unit space for (step S38: YES), the second learning unit 12 stores the trained generator 121′ (trained generators 121_1′ to 121_D′) in the first storage unit 13 (step S5). The trained generator 121′ (trained generators 121_1′ to 121_D′) is constructed by the processes from step S30 to step S38.

[0096] As described above, according to the control device 1 of this embodiment, the RNN is trained using the history of the direction of travel of a UAV 2a selected from multiple UAVs 2, sequentially traveled in each unit space until it reached the destination, as training data. Then, using the trained RNN as true action data, a trained generator 121' is constructed by GAN adversarial learning and transmitted as control information to multiple UAVs 2. Therefore, with a simpler configuration, it is possible to control UAVs 2, which are a group of devices that perform the same task.

[0097] Furthermore, according to the control device 1 of this embodiment, latent parameters such as the hidden state of the RNN at each time, memory cell values, and weight parameters between units are learned through GAN learning, so that control information applicable to a large number of devices can be generated with a simpler configuration.

[0098] Furthermore, the UAV 2 according to this embodiment uses the trained generator 121′ to generate pseudo action data indicating the sequential directions to follow until reaching the destination point based on the position of the UAV at each time. This allows the same task to be executed with a simpler configuration.

[0099] In the embodiment described above, collective movement control of UAVs 2 was exemplified as a group of devices that perform the same task by sequentially executing actions, but as mentioned above, the group of devices is not limited to UAVs 2 as long as they collectively perform a task by sequentially executing actions in response to events that occur at different times. Examples include traffic management of autonomous vehicles, management of a sensor network in which a group of IoT devices such as sensors collect and process sensor data for monitoring, and control of assembly and logistics in a factory using a group of robots.

[0100] Furthermore, in the embodiment described above, the first learning unit 11 learns an RNN, but the first learning unit 11 can also perform learning using an LSTM (Long Short-Term Memory).

[0101] The above describes embodiments of the control device, device, and control method of the present invention, but the present invention is not limited to the described embodiments, and various modifications that a person skilled in the art can make within the scope of the invention described in the claims are possible. [Explanation of symbols]

[0102] 1...control device, 2, 2a...UAV, 10...first acquisition unit, 11...first learning unit, 12...second learning unit, 13...first memory unit, 14...presentation unit, 20...second memory unit, 21...second acquisition unit, 22...generation unit, 23...third memory unit, 24...execution unit, 201, 101...bus, 202, 102...processor, 203, 103...main memory unit, 204, 104...communication interface, 205, 105...auxiliary memory unit, 20 6, 106...input / output I / O, 107...display device, 121, 121_1 to 121_D...generator, 121', 121_1' to 121_D'...trained generator, 122...discriminator, 123...output, 124...training data, 125...objective function E block, 207...GPS receiver, 208...sensor, 209...motor, 210...propeller, 211...battery, A...mobile space, NW...wireless communication network.

Claims

1. A control device that controls a group of devices that each perform an identical task by sequentially executing an action, a first learning unit configured to learn a recurrent neural network that uses actions that a first device among the device group has executed sequentially to transition from a state to a next state at each time as training data, receives a state of the first device as an input, and outputs actions that the first device will execute sequentially to transition from the state to the next state at each time; a second learning unit configured to perform adversarial learning of a generative model including a generator that generates pseudo action data similar to true action data for actions that each of the device group sequentially executes to transition from a state to a next state at each time, using the trained recurrent neural network constructed by the training by the first learning unit, and a classifier that distinguishes between the pseudo action data generated by the generator and the true action data; a presentation unit configured to present a trained generator obtained by the adversarial learning of the second learning unit as control information for the device group to execute the task; A control device comprising:

2. 2. The control device according to claim 1, The control device further includes a first acquisition unit configured to acquire the actions performed sequentially by the first device to transition from a state to the next state at each time, which are used as the teacher data.

3. 2. The control device according to claim 1, The pseudo action data has a data distribution that has a minimum statistical distance from the data distribution of the true action data. A control device characterized by:

4. a second acquisition unit configured to acquire a status of the device itself; a generation unit configured to generate the pseudo action data indicating actions to be executed sequentially to transition from an acquired state of the own device to a next state at each time point, using the learned generator included in the control information presented by the control device according to any one of claims 1 to 3; an execution unit configured to sequentially execute actions indicated by the pseudo action data generated by the generation unit and execute a set task; 1. A device comprising:

5. 1. A control method for controlling a group of devices each performing an identical task by sequentially executing an action, comprising: a first learning step of learning a recurrent neural network that uses actions that a first device among the device group has executed sequentially to transition from a state to a next state at each time as training data, inputs a state of the first device, and outputs actions that the first device will execute sequentially to transition from the state to the next state at each time; a second learning step of performing adversarial learning of a generative model using the trained recurrent neural network constructed in the first learning step, the generative model having a generator that generates pseudo action data similar to true action data regarding actions that each of the device group sequentially executes to transition from a state to a next state at each time, and a classifier that distinguishes between the pseudo action data generated by the generator and the true action data; a presentation step of presenting the trained generator obtained by the adversarial learning of the second learning step as control information for the group of devices to execute the task; A control method comprising:

6. 6. The control method according to claim 5, The control method further includes a first acquisition step of acquiring actions that the first device sequentially performs to transition from one state to the next state at each time, which are used as the teaching data.

Images