Control system and control method

Abstract

The object is to control a group of devices that execute the same task with a simpler configuration. [Solution] The control device 1 performs adversarial learning of a generative model having a generator 121 that generates pseudo action data similar to true action data for actions that each of a group of devices sequentially executes to transition from a state to the next state at each time, using the learned recurrent neural network constructed by learning by the first learning unit 11, and a discriminator 122 that discriminates between the pseudo action data generated by the generator 121 and the true action data.

Description

[Technical field]

[0001] The present invention relates to a control device, a device, and a control method, and in particular to a control technique for a group of devices. [Background technology]

[0002] In recent years, there has been an increasing demand for autonomous AI systems and autonomous AI agents for robots, vehicles, etc. For example, Patent Document 1 discloses a technology that employs a reinforcement learning model in a system targeted at a single agent that controls a moving object, and guides the moving object to a destination point using an action value function that represents an expected value of a cumulative value of future rewards obtained when an action indicating a next course to be taken based on the current position of the moving object.

[0003] When single-agent learning is applied to non-cooperative multi-agent systems in which each agent performs the same task independently, each agent learns independently using methods such as reinforcement learning. In this case, the amount of learning for the entire system increases, resulting in an increased computational load. In addition, when each agent learns similar information independently, the efficiency of learning may decrease due to the overlapping of learning redundant information about similar environments. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7407329 Summary of the Invention [Problem to be solved by the invention]

[0005] With conventional technology, it has been difficult to control a group of devices that execute the same task with a simpler configuration.

[0006] The present invention has been made to solve the above-mentioned problems, and has an object to control a group of devices that execute the same task with a simpler configuration. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, a control device according to the present invention is a control device that controls a group of devices each performing the same task by sequentially executing actions, and includes: a first learning unit configured to learn a recurrent neural network that uses actions sequentially executed by a first device of the group of devices to transition from a state to a next state at each time as teacher data, inputs a state of the first device, and outputs actions sequentially executed by the first device to transition from the state to the next state at each time; a second learning unit configured to perform adversarial learning of a generative model having a generator that generates pseudo action data similar to true action data for actions sequentially executed by each of the group of devices to transition from a state to the next state at each time, using the trained recurrent neural network constructed by the training by the first learning unit, and a discriminator that discriminates between the pseudo action data generated by the generator and the true action data; and a presentation unit configured to present the trained generator obtained by the adversarial learning of the second learning unit as control information for the group of devices to execute the task.

[0008] In addition, the control device of the present invention may further include a first acquisition unit configured to acquire actions performed sequentially by the first device to transition from a state to the next state at each time, which are used as the teacher data.

[0009] In the control device according to the present invention, the pseudo action data may have a data distribution that has a minimum statistical distance from a data distribution of the true action data.

[0010] In order to solve the above-mentioned problems, the device of the present invention comprises a second acquisition unit configured to acquire a state of the device itself, a generation unit configured to generate the pseudo action data indicating actions to be executed sequentially to transition from the acquired state of the device itself to a next state at each time using the learned generator included in the control information presented by the above-mentioned control device, and an execution unit configured to sequentially execute the actions indicated by the pseudo action data generated by the generation unit and execute a set task.

[0011] In order to solve the above-mentioned problems, a control method according to the present invention is a control method for controlling a group of devices each performing the same task by sequentially executing an action, the control method including: a first learning step of learning a recurrent neural network that uses actions sequentially executed by a first device of the group of devices to transition from a state to a next state at each time as teacher data, inputs a state of the first device, and outputs actions sequentially executed by the first device to transition from the state to the next state at each time; a second learning step of performing adversarial learning of a generative model having a generator that generates pseudo action data similar to true action data for actions sequentially executed by each of the group of devices to transition from a state to the next state at each time, and a discriminator that discriminates between the pseudo action data generated by the generator and the true action data, and a presentation step of presenting the trained generator obtained by the adversarial learning of the second learning step as control information for the group of devices to execute the task.

[0012] In addition, the control method of the present invention may further include a first acquisition step of acquiring actions performed sequentially by the first device to transition from a state to a next state at each time, which are used as the teaching data. Effect of the Invention

[0013] According to the present invention, the trained recurrent neural network constructed by the training by the first training unit is subjected to adversarial training of a generative model having a generator that generates pseudo action data similar to true action data for actions that each of a device group sequentially executes to transition from a state to the next state at each time, and a discriminator that discriminates between the pseudo action data generated by the generator and the true action data. Therefore, a device group that executes the same task can be controlled with a simpler configuration. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a block diagram showing a configuration of a control system including a control device and multiple UAVs according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing the configuration of a UAV according to this embodiment. [Diagram 3] FIG. 3 is a block diagram showing the hardware configuration of a UAV according to this embodiment. [Figure 4] FIG. 4 is a diagram for explaining the first learning unit included in the control device according to the present embodiment. [Diagram 5] FIG. 5 is a block diagram showing the configuration of a second learning unit included in the control device according to the present embodiment. [Figure 6] FIG. 6 is a block diagram showing the configuration of a second learning unit included in the control device according to the present embodiment. [Figure 7] FIG. 7 is a diagram for explaining the second learning unit included in the control device according to the present embodiment. [Figure 8] FIG. 8 is a block diagram showing a hardware configuration of the control device according to the present embodiment. [Figure 9] FIG. 9 shows an operation sequence of the control system according to the present embodiment. [Figure 10] FIG. 10 is a flowchart showing the operation of the control device according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will now be described in detail with reference to FIGS.

[0016] [Control system configuration] First, referring to FIG. 1, an overview of a control system including a control device 1 and a plurality of UAVs 2 according to an embodiment of the present invention will be described. In the following description, a group of UAVs 2 will be taken as an example of a group of devices that execute the same task by sequentially executing actions. The same task will be described as a group of UAVs 2 reaching a destination point set as a group. Furthermore, the action executed at each time to execute the task will be described as an example of each UAV 2 moving in a predetermined direction in order to move from the position where each UAV 2 is located to the next position at each time. It should be understood that the present invention is not limited to collective movement control by a group of UAVs 2, as long as the group of devices executes one task collectively by sequentially executing actions in response to events that occur at each time.

[0017] The control system according to the present embodiment includes a control device 1 and a plurality of UAVs 2, which are communicatively connected to each other via a wireless communication network NW conforming to a predetermined communication standard such as LTE / 4G, 5G, or 6G. The control system controls the plurality of UAVs 2 to execute a task of moving collectively to a set destination position in a mobile space A. As shown in FIG. 1, the mobile space A in which the UAVs 2 fly is capable of communication by, for example, a 5G wireless communication method.

[0018] The UAV2 is a moving body such as a drone or an unmanned aerial vehicle, and can perform autonomous flight. The UAV2 controls its autonomous flight by a flight controller that processes information from a sensor 208 (described later) and controls the rotation speed of a motor 209 and the angle of a propeller 210. In this embodiment, the UAV2 controls its autonomous flight to a destination point based on route control information stored in the auxiliary storage device 205. Each of the multiple UAV2 has the same configuration, and when there is no need to distinguish between them, they are collectively referred to as UAV2. The functional blocks and hardware configuration of the UAV2 will be described in detail later. In this embodiment, of the multiple UAV2, for example, the UAV2 that has flown in the moving space A first can be selected as the UAV2a (first device).

[0019] As shown in FIG. 1, the mobile space A in which multiple UAVs 2 fly is a three-dimensional matrix-like space composed of multiple unit spaces divided into spaces. Each unit space that composes the mobile space A has the same volume. Furthermore, each unit space has a node ID, and each unit space is represented by a single representative position (x, y, z). The location information can be three-dimensional GPS position coordinates consisting of latitude, longitude, and altitude. For example, a representative value such as the center position of the unit space can be used as the position of the unit space.

[0020] Also, as shown in FIG. 1, multiple UAV2 move from a position in the unit space corresponding to the current position S of each UAV2 to a unit space position of a destination point G, with each unit space being used as a waypoint. The position of the destination point G of the UAV2 is set arbitrarily and is a destination point common to multiple UAV2. Also, the unit space has a distance that the UAV2 can travel at each time, i.e., in one time step, and the UAV2 can reach an adjacent unit space at each time.

[0021] The control system according to the present embodiment learns, using a recurrent neural network (RNN), the actions to be executed sequentially to move from a certain state to the next state for each time of, for example, a UAV2a flying ahead of the UAV2 selected from among the multiple UAV2, i.e., the traveling direction for moving from the current unit space position to the next unit space position for each time. Furthermore, the learned RNN is used as true data of the traveling direction in which each of the multiple UAV2 moves sequentially to move through the unit space to the destination point, i.e., true action data, and a generator that generates pseudo action data similar to the true action data is constructed by adversarial learning. The control system presents the learned generator 121' to the multiple UAV2 as control information for executing the task of multiple UAV2 moving collectively to the destination point. Furthermore, each UAV2 determines the traveling direction to move sequentially based on the current unit space position of the aircraft based on the control information, and performs flight control to the destination point.

[0022] Each UAV2 generates pseudo action data using the learned generator 121' included in the control information, and changes its direction of travel by maneuvering its own aircraft, as shown by the arrow in FIG. 1, based on the direction of travel in which each UAV2 advances sequentially from the current unit space position of each UAV2 indicated by the generated pseudo action data. The direction of travel indicated by the pseudo action data may include various directions of travel as the action to be executed. The direction of travel specifies which unit space the UAV2 will move to next from the position of the unit space in which it is currently located, among the adjacent unit spaces. In FIG. 1, the movement space A is described as a two-dimensional plane, but the direction of travel of the UAV2 can be a three-dimensional direction of travel. The control information presented by the control device 1 allows the multiple UAV2 to complete the task of collectively reaching the unit space of the destination point G.

[0023] [Controller function block] As shown in FIG. 1, the control device 1 includes a first acquisition unit 10, a first learning unit 11, a second learning unit 12, a first storage unit 13, and a presentation unit .

[0024] The first acquisition unit 10 acquires actions that a UAV2a selected from among a plurality of UAV2s performs sequentially to transition from a certain state to a next state at each time. The state is the position of the unit space in which the UAV2 exists. The history of the time series sequence of the actions acquired by the first acquisition unit 10 is used as teacher data when the first learning unit 11 learns the RNN. More specifically, the first acquisition unit 10 acquires the history of the time series sequence related to the traveling direction in which the UAV2a advances sequentially from the GPS position of the unit space at the start point, with each unit space to the destination point as a waypoint. As described above, the UAV2a is a UAV2 selected as a representative from among a plurality of UAV2s.

[0025] The first learning unit 11 learns an RNN that uses actions executed sequentially to transition from a state to the next state at each time as training data of the UAV2a among the multiple UAVs 2, inputs the state of the UAV2a at each time, and outputs actions executed sequentially to transition from that state to the next state at each time. Specifically, the first learning unit 11 uses a time series sequence in which the value of the GPS position of the unit space in which the UAV2a exists, acquired by the first acquisition unit 10, is assigned as a correct answer label the value of the traveling direction that has been sequentially advanced to move from the GPS position to the position of the next unit space, as training data for the RNN.

[0026] 4 is a schematic diagram showing the network structure of the RNN used in the learning of the first learning unit 11. As shown in FIG. 4, the RNN is a neural network consisting of an input layer X, a hidden layer H which is a memory cell, and an output layer Y. The input time series data x (t) Predict time series data from y (t)In this embodiment, the input time series data is the GPS position of the unit space where the UAV2a exists at each time. The predicted time series data is the direction of travel for moving from the position of the unit space where the UAV2a exists to the position of the next unit space at each time. In the RNN, information is transmitted from the hidden layer at a certain time to the hidden layer at the next time along the time series. Therefore, the hidden layer at time t receives input from the hidden layer at time t and the hidden layer at the immediately preceding time t-1.

[0027] Given a time series data x (t) On the other hand, the RNN is defined by the following equations (1) and (2). h (t) =σ(U×x (t) +W×h (t-1) +b) (1) y (t) = V × h (t) +c...(2)

[0028] The above equation (1) represents the propagation from the input layer to the hidden layer, and x (t) is the input vector at time t. (t) is the output of the hidden layer at time t. U is the weight parameter from the input to the hidden layer, W is the weight parameter from the hidden state at the previous time t-1 to the hidden state at the current time t. b is the bias vector of the hidden layer, and σ is the activation function.

[0029] The above equation (2) represents the propagation from the hidden layer to the output layer, and y (t) is the output vector at time t. V is the weight parameter from the hidden layer to the output layer, and c is the bias vector for the output layer.

[0030] The first learning unit 11 uses an error backpropagation method or the like to compare the obtained predicted output value with the output value of the teacher data for a given input value, introduces an objective function that minimizes the error, checks the error of the weight parameters and propagates it backward, and finally determines the weight parameters U, W, V and biases b and c. Through such a learning process, the first learning unit 11 constructs a trained RNN. The trained RNN constructed by the first learning unit 11 is used as training data in the adversarial learning of the generative model by the second learning unit 12.

[0031] The second learning unit 12 performs adversarial learning of a generative model having a generator 121 that generates pseudo action data similar to true action data for actions that multiple UAVs 2 execute sequentially to transition from a certain state to the next state at each time, using the trained RNN obtained by learning by the first learning unit 11, and a discriminator 122 that discriminates between the pseudo action data generated by the generator 121 and the true action data.

[0032] As shown in FIG. 5, the second learning unit 12 adversarially learns a GAN (Generative Adversarial Network) having a generator 121 and a classifier 122. A trained generator 121' is constructed by the learning of the second learning unit 12. Thus, in this embodiment, the optimized action for the selected traveling direction of the UAV2a indicated by the trained RNN constructed by the learning by the first learning unit 11 is considered as true action data of the actions of the multiple UAV2. Furthermore, the true action data is used as training data to train a generator 121 that generates pseudo action data similar to the action in the traveling direction of the UAV2a.

[0033] 6 and 7 are diagrams that show the neural network structures of the generator 121 and the classifier 122 of the GAN used by the second learning unit 12. As shown in FIG. 6, the generator 121 is configured as a neural network having an input layer, a hidden layer, and an output layer. The generator 121 generates a neural network for each of times t1 to t D The generator 121 is composed of D generators 121_1 to 121_D corresponding to GPS positions GPS_1 to GPS_D of the unit space in which the UAV2 exists. As shown in FIG. 5 and FIG. 6, the generator 121 provides GPS positions s of sequential unit spaces as inputs to the generators 121_1 to 121_D, and generates pseudo action data a1 to a D The GPS position s input to the generator 121 is the GPS route that the UAV 2a actually flew to the destination point.

[0034] Each of the generators 121_1 to 121_D corresponds to a memory cell of the trained RNN at each time used in the training by the first training unit 11. For example, the generator 121_1 indicated by G1 has a neural network structure capable of training pseudo action data similar to the parameters of the trained RNN that receives GPS_1, which is the GPS position of the unit space at time t1, as an input and outputs a travel direction a1 for moving from the position of GPS_1 to a position of the next unit space. CNN or ResNet can be used as the neural network constituting the generator 121.

[0035] The generator 121 outputs an output G(s) through a multiplication and addition operation of the input and weight parameters and a threshold process using an activation function. For example, the output G1(s) from the generator 121_1 is pseudo action data similar to the action a1 output when the GPS_1 at time t1 is given as an input to the trained RNN. Similarly, at each time t2 to t D For the generators 121_2 to 121_D corresponding to the states, the actions a2 to a DIn this way, the generator 121 according to this embodiment receives the actual GPS position of the UAV 2a at each time as an input and outputs a random function.

[0036] The classifier 122 shown in FIG. 7(a) is configured with a neural network having an input layer, a hidden layer, and an output layer. In the example of FIG. 7(a), as an input of training data, true action data is provided, which is output from the trained RNN obtained by the first learning unit 11, and indicates the traveling direction traveled to move from the position of the unit space where the UAV2a exists to the position of the next unit space at each time. FIG. 7(b) is a diagram for explaining input data provided to the classifier 122. In the example of FIG. 7(b), training data related to the true action data is illustrated. That is, the true action data is output from the trained RNN obtained by the first learning unit 11, and indicates the traveling direction traveled to move from the position of the unit space where the UAV2a exists to the position of the next unit space at each time. D When the UAV2 is located in the unit space of GPS_1 to GPS_D, the action a1 to a2 are the direction of travel of the UAV2 to reach the position of the adjacent unit space at the next time. D is provided to the classifier 122 as training data.

[0037] The classifier 122 outputs a binary output of 1 or 0 through a product-sum operation of the input and weight parameters and a threshold process using an activation function. When the classifier 122 correctly identifies the training data related to the input true action data as true action data, it outputs an output y=1. On the other hand, when the classifier 122 correctly identifies the training data related to the input pseudo action data as pseudo action data, it outputs an output y=0. In this way, the classifier 122 is a model that distinguishes the model distribution generated by the generator 121 from the data distribution of the training data, which is the true distribution. The pseudo action data has a data distribution that has a minimum statistical distance from the data distribution of the true action data. A CNN can be used as the neural network that constitutes the classifier 122.

[0038] FIG. 5 is a block diagram for explaining the adversarial learning of GAN by the second learning unit 12. The generator 121 of the GAN adopted by the second learning unit 12 is represented as a function G, and the discriminator 122 is represented as a function D. Note that, hereinafter, the functions G1 to GD of the generators 121_1 to 121_D are collectively referred to as function G. Furthermore, true action data is represented as x, the predicted value output by the discriminator 122 is represented as y, and the correct label is represented as t. The correct label t is set to 1 for true action data and 0 for pseudo action data generated by the generator 121. At this time, the discriminator 122 calculates the cross entropy E CE It can be expressed as:

[0039]

number

[0040] The first term in the brace of the above equation (3) represents t n lny n In the above, the predicted value y n is the correct label for the true action data, t n = 1. On the other hand, the second term in the braces represents (1-t n )ln(1-y n ), the predicted value y n is the correct label value (1-t n ) = 0. In this way, the cross entropy E CE is maximized when the predicted value matches the correct label value.

[0041] Here, the generator 121 constituting the GAN has parameters w G ,θ G and function G(w G ,θ G ) The discriminator 122 also uses the parameter w D ,θ D and function D(w D ,θ D ) The cross entropy E CEThe objective function E of the GAN including the generator 121 and the discriminator 122 based on the above can be expressed by the following equation (4).

number

[0042] The first term of the above formula (4) represents E D(x)=1 lnD(w D ,θ D ) is the expected value that the classifier 122 classifies the true action data as the true action data. D(x)=0 ln(1-D(G(w G ,θ G ),w D ,θ D )) is an expected value at which the classifier 122 classifies the pseudo action data generated by the generator 121 as pseudo action data. In GAN learning, the generator 121 and the classifier 122 are adversarially trained by min-max optimization of the objective function E. Therefore, the generator 121 is trained so as to generate pseudo action data that deceives the classifier 122, and the classifier 122 is trained so as to classify the pseudo action data generated by the generator 121 as pseudo action data.

[0043] In learning of the classifier 122, when true action data is given, the classifier 122 maximizes the first term of the objective function E in the above formula (4) by outputting an output close to y=1. On the other hand, when pseudo action data is given, the classifier 122 learns to maximize the second term of the objective function E by outputting an output close to y=0.

[0044] In the learning of the generator 121, D(G(w G ,θ G ),w D ,θ D ) (G1(s)~GD(s) in Fig. 5) are close to 1. G ,θ G) (G1(s) to GD(s) in FIG. 5 ), thereby minimizing the objective function E. The second learning unit 12 uses a learning procedure in which the parameters of the generator 121 and the classifier 122 are updated alternately. Details of the learning procedure of the generator 121 and the classifier 122 by the second learning unit 12 will be described later.

[0045] The first storage unit 13 stores the parameters of the trained RNN and the trained generator 121'. Specifically, the first storage unit 13 stores trained generators 121_1' to 121_D' corresponding to states at each time.

[0046] The presentation unit 14 presents the trained generator 121' obtained by the adversarial learning of the second learning unit 12 as control information for the multiple UAVs 2 to execute a task. More specifically, the presentation unit 14 transmits the trained generators 121_1' to 121_D' to the multiple UAVs 2 via the network NW as control information for the multiple UAVs 2 to execute a task of reaching a set destination point. Note that, for the UAV 2a, a configuration may be adopted in which the trained RNN constructed by the first learning unit 11 is presented as control information.

[0047] [Control device hardware configuration] Next, an example of a hardware configuration for realizing the control device 1 having the above-mentioned functions will be described with reference to FIG.

[0048] As shown in FIG. 8, the control device 1 can be realized, for example, by a computer having a processor 102, a main memory device 103, a communication interface 104, an auxiliary memory device 105, and an input / output (I / O) 106 connected via a bus 101, and a program that controls these hardware resources.

[0049] The processor 102 is realized by a CPU, a GPU, an FPGA, an ASIC, or the like.

[0050] The main memory device 103 stores in advance programs for the processor 102 to perform various controls and calculations. The processor 102 and the main memory device 103 realize the functions of the control device 1, such as the first acquisition unit 10, the first learning unit 11, the second learning unit 12, and the presentation unit 14 shown in FIG.

[0051] The communication interface 104 is an interface circuit for network connection between the control device 1 and various external electronic devices.

[0052] The auxiliary storage device 105 is composed of a readable / writable storage medium and a drive for reading and writing various information such as programs and data from and to the storage medium. The auxiliary storage device 105 can use a semiconductor memory such as a hard disk or a flash memory as the storage medium.

[0053] The auxiliary storage device 105 has a program storage area for storing the control program executed by the control device 1. The auxiliary storage device 105 also has an area for storing a first learning program for learning the RNN and a second learning program for performing adversarial learning using the GAN. The first storage unit 13 described in FIG. 1 is realized by the auxiliary storage device 105. The auxiliary storage device 105 also has an area for storing map data including the position coordinates of the movement space A, as well as the position coordinates of the unit space constituting the movement space A and the node ID of the unit space in association with each other. Furthermore, the auxiliary storage device 105 has an area for storing the identification information of the UAV2. Furthermore, for example, the auxiliary storage device 105 may have a backup area for backing up the above-mentioned data, programs, etc.

[0054] The input / output I / O 106 is an input / output device that inputs signals from an external device and outputs signals to an external device.

[0055] The display device 107 is configured with an organic EL display, a liquid crystal display, etc. The display device 107 can display a map display of the moving space A, the current position of the UAV 2, the flight route, and the position information of the destination point.

[0056] [UAV functional blocks] Next, the functional blocks of the UAV 2 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the configuration of the UAV 2. As described above, the multiple UAVs 2 include the UAV 2a selected by the control device 1 and multiple UAVs 2 other than the UAV 2a, and each UAV 2 can be configured with the same functional blocks.

[0057] The UAV 2 includes a second storage unit 20, a second acquisition unit 21, a generation unit 22, a third storage unit 23, and an execution unit 24. Each of the multiple UAVs 2 determines the next direction in which the UAV should proceed based on the route control information set by the control device 1, and controls the flight of the UAV to the destination position.

[0058] The second storage unit 20 stores control information presented by the presentation unit 14 of the control device 1. The control information includes a trained generator 121' constructed by adversarial learning by the second learning unit 12 of the control device 1. More specifically, the second storage unit 20 stores trained generators 121_1' to 121_D'.

[0059] The second acquisition unit 21 acquires the state of the own device. More specifically, the second acquisition unit 21 acquires the GPS position of the unit space in which the own device exists at each time from the start of flight. The second acquisition unit 21 can refer to the third storage unit 23 and acquire the GPS position received by the GPS receiver 207 as the unit space position of the own device.

[0060] The generating unit 22 uses the trained generator 121' included in the control information to generate pseudo action data indicating actions to be executed sequentially to move from the current state of the device to the next state. More specifically, the generating unit 22 inputs the GPS position of the current unit space of the device acquired by the second acquiring unit 21 to the trained generators 121_1' to 121_D', and performs calculations of the trained generators 121_1' to 121_D' to generate pseudo action data indicating a traveling direction to move from the current unit space position of the device to the next unit space position. The sequence of the pseudo action data generated by each trained generator 121_1' to 121_D' is used as information to guide the flight of the UAV2 to the destination point.

[0061] The third storage unit 23 stores map data including the position coordinates of the movement space A, and information associating the position coordinates of the unit spaces constituting the movement space A with the node IDs of the unit spaces. The third storage unit 23 also stores the position information of the destination point.

[0062] The execution unit 24 sequentially executes the actions indicated by the pseudo action data generated by the generation unit 22, and executes the set task. Specifically, the execution unit 24 controls the flight of the aircraft according to the pseudo action data generated by the generation unit 22, which indicates the sequential travel direction for moving from the position of the unit space where the UAV2 exists to the position of the next unit space at each time. The execution unit 24 can calculate a control command for the next travel direction from the current GPS position of the UAV2, and transmit the control command value to the motor 209. The execution unit 24 flies in order through the unit spaces set as waypoints, so that the UAV2 reaches the set destination point.

[0063] In addition, with regard to UAV2a, the trained RNN constructed by the first learning unit 11 of the control device 1 can be configured to calculate the direction of travel for moving from the position in the unit space where UAV2a is located to the position in the next unit space at each time, and to guide the flight to the destination.

[0064] [UAV hardware configuration] Next, an example of a hardware configuration for realizing a UAV2 having the above-mentioned functions will be described with reference to FIG.

[0065] 3, the UAV2 can be realized by, for example, a microcomputer including a processor 202, a main memory 203, a communication interface 204, an auxiliary memory 205, and an input / output I / O 206 connected via a bus 201, a program for controlling these hardware resources, a GPS receiver 207, a sensor 208, a motor 209, a propeller 210, and a battery 211. A flight controller for controlling the autonomous flight of the UAV2 is realized by a computer such as a microcomputer and a program. The UAV2 can mount these components on a frame of a predetermined shape.

[0066] The main memory device 203 stores in advance a program for the processor 202 to perform flight control and calculations. The processor 202 and the main memory device 203 realize the functions of the UAV 2, such as the second acquisition unit 21, the generation unit 22, and the execution unit 24, shown in FIG.

[0067] The communication interface 204 is an interface circuit for network connection between the UAV 2 and the control device 1.

[0068] The auxiliary storage device 205 is composed of a readable / writable storage medium and a drive for reading and writing various information such as programs and data from the storage medium. The auxiliary storage device 205 can use a semiconductor memory such as a hard disk or a flash memory as the storage medium.

[0069] The auxiliary storage device 205 has a program storage area for storing a flight control program executed by the UAV2. The auxiliary storage device 205 also has an area for storing a calculation program for determining a direction of travel to move from the position of the unit space where the UAV2 is located to the position of the next unit space at each time based on the learned generator 121' included in the control information. The auxiliary storage device 205 realizes the second storage unit 20 and the third storage unit 23 described in FIG. 2. The auxiliary storage device 205 also has an area for storing identification information of the UAV2. Furthermore, for example, the auxiliary storage device 205 may have a backup area for backing up the above-mentioned data and programs.

[0070] The input / output I / O 206 is an input / output device that inputs signals from an external device and outputs signals to an external device.

[0071] The GPS receiver 207 has an antenna for receiving GPS signals built in. The GPS receiver 207 realizes the second acquisition unit 21 in FIG.

[0072] The sensor 208 is composed of various sensors such as an altitude sensor, an attitude sensor, a camera, LiDAR, and RADAR. In addition to the GPS receiver 207, the altitude sensor realizes the second acquisition unit 21 in Fig. 2. Furthermore, the flight controller performs flight control of the UAV 2 based on various sensor data measured by the sensor 208.

[0073] The motor 209 rotates by the rotary drive, and rotates a propeller 210 attached to the rotary shaft of the motor 209 .

[0074] Battery 211 is a lithium ion battery or the like, and supplies power to the components of UAV2.

[0075] The UAV2 may include a SIM and have an International Mobile Subscriber Identity (IMSI) of the SIM. The UAV2 may be configured as an IoT terminal having an IP address. These IMSI and IP address may be used as identification information unique to the UAV2.

[0076] [Control system operation] Next, the operation of the control system including the control device 1 and multiple UAVs 2 having the above-described configuration will be described with reference to the sequence diagram of FIG.

[0077] First, a destination point to which multiple UAVs 2 are moved collectively in the movement space A is set in advance. A UAV 2a selected from the multiple UAVs 2 transmits to the control device 1 a history of the direction of travel from an initial position in the movement space A to the destination point. The history of the direction of travel is a history of the direction of travel that the UAV 2a has traveled at each time point to move sequentially from the position of the unit space of the initial position to the position of an adjacent unit space. The control device 1 acquires the history of the direction of travel transmitted from the UAV 2a as teacher data to be used in learning the RNN (step S1).

[0078] Next, the first learning unit 11 performs a first learning process to learn the RNN (step S2). More specifically, the first learning unit 11 uses training data in which the GPS position value of each unit space until the UAV2a reaches the destination point is given as a correct answer label the value of the direction of travel that actually proceeded from the GPS position of the unit space. Using the training data, the first learning unit 11 learns an RNN that inputs the GPS position of the unit space in which the UAV2a exists and outputs the direction of travel that proceeds sequentially to move from the GPS position of the unit space to the position of the next unit space at each time.

[0079] The first learning unit 11 uses an error backpropagation method or the like to compare the obtained predicted output value with the output value of the teacher data for a given input value, introduces an objective function that minimizes the error, checks the error of each weight parameter and propagates it backward, and finally determines the weight parameters U, W, V and biases b and c in the above equations (1) and (2). The first storage unit 13 stores the trained RNN constructed in the first learning process in step S3 (step S3).

[0080] Next, the second learning unit 12 performs a second learning process (step S4). More specifically, the second learning unit 12 performs adversarial learning of a GAN having a generator 121 that generates pseudo action data similar to the true action data using the trained RNN obtained in the first learning process in step S2 as training data, i.e., true action data, and a discriminator 122 that discriminates between the pseudo action data generated by the generator 121 and the true action data. The true action data represents the traveling direction in which each of the multiple UAVs 2 moves from the position of the unit space in which it exists to the position of the next unit space. Details of the second learning process in step S5 will be described later.

[0081] Next, the first storage unit 13 stores the trained generator 121' constructed in step S5 (step S4). Specifically, the trained generators 121_1' to 121_D' corresponding to the states at each time are stored in the first storage unit 13. Next, the presentation unit 14 presents the trained generator 121' (trained generators 121_1' to 121_D') constructed in step S5 to the multiple UAVs 2 as control information for executing a task of the multiple UAVs 2 reaching a destination point collectively (step S6).

[0082] After the UAV2 receives the control information via the wireless communication network NW, the second acquisition unit 21 acquires the GPS position of the unit space in which the UAV2 is currently located (step S7). Next, the generation unit 22 provides the GPS position of the unit space in which the UAV2 is located at the time acquired in step S7 as an input to the learned generator 121' included in the control information, and generates pseudo action data (step S8). The pseudo action data generated in step S8 indicates in which direction the UAV2 will proceed from the GPS position of the unit space in which the UAV2 is located at a certain time.

[0083] Furthermore, the execution unit 24 performs flight control of the UAV2 along the direction of travel indicated by the pseudo action data generated in step S8 (step S9). Next, the UAV2 repeats the processes from step S7 to step S9 until it reaches the unit space of the destination point (step S10: NO). After that, the UAV2 completes the task by reaching the destination point (step S10: YES). Each of the multiple UAV2 performs the processes from step S7 to step S11 based on the GPS position of the unit space existing at each time. Note that for the UAV2a of the multiple UAV2, the trained RNN constructed in the first learning process in step S2 can be set to perform flight control to the destination point.

[0084] Next, the second learning process by the second learning unit 12 of the control device 1 will be described with reference to Fig. 10. First, the second learning unit 12 acquires the learned parameters, i.e., the learned RNN, obtained in the first learning process by the first learning unit 11 in step S2 of Fig. 9 as true action data related to the traveling directions in which the multiple UAVs 2 will move sequentially until they reach the destination point (step S30).

[0085] Here, as shown in the learning process of the GAN by the second learning unit 12 in Fig. 5, the true action data collected in step S30 is used as training data 124 input when learning the classifier 122. In addition, the training data related to the true action data input to the classifier 122 is a time series sequence of the traveling direction of the UAV2 obtained by the trained RNN. Furthermore, a set of n time series sequences of the traveling directions can be used as the training data.

[0086] Next, the second learning unit 12 inputs the true action data to the classifier 122 as training data 124, and adjusts the parameter w of the classifier 122 so as to classify the true action data as true action data (y=1). D ,θ D (Step S31). In Step S31, the second learning unit 12 can cause the classifier 122 to learn the true action data by using, for example, an error backpropagation method. By Step S31, the classifier 122 that can distinguish the true action data from the true action data is constructed in advance.

[0087] Next, the second learning unit 12 provides the GPS position of the unit space for each time as an input to the generator 121 (step S32). More specifically, as shown in FIG. 5, the GPS position of the unit space in which the UAV 2a exists at time t1, GPS_1, is input to the generator 121_1. Similarly, from time t2 to t D For each of the UAVs 121, the corresponding GPS positions (GPS_2 to GPS_D) are input to the generators 121_2 to 121_D. That is, the GPS route that the UAV 2a actually flew is provided as an input value to the generator 121.

[0088] The generator 121 then generates a vector of input s and weight parameters w based on the given GPS position. G ,θ Gand threshold processing using an activation function to generate pseudo action data G(s) (step S33). More specifically, the generators 121_1 to 121_D generate pseudo action data G1(s) to GD(s), respectively.

[0089] Next, the second learning unit 12 trains the discriminator 122. The training of the discriminator 122 is performed by using the parameter w D ,θ D First, the second learning unit 12 provides the training data 124 of the true action data acquired in step S30 as an input to the classifier 122, and calculates the parameter w by backpropagation or the like so that the objective function E in the above formula (4) is maximized. D ,θ D (Step S34). Note that the label of the training data 124 is set to 1 (true action data).

[0090] Next, the second learning unit 12 provides the pseudo action data generated by the generator 121 in step S33 to the discriminator 122, and calculates the parameter w by using the backpropagation method or the like so that the objective function E in the above formula (4) is maximized. D ,θ D That is, in steps S34 and S35, in order to maximize the objective function E in the above formula (4), the first term is updated as D(w D ,θ D )=1 is output, and the second term is D(G(w G ,θ G ),w D ,θ D )=0. In addition, the label 0 (pseudo action data) is set in the training data 124. In addition, in step S35, the pseudo action data G1(s) to GD(s) generated by the generators 121_1 to 121_D corresponding to each time are provided to the classifier 122.

[0091] The training of the classifier 122 in steps S34 and S35 corresponds to the dashed arrows in FIG. 5 which indicate that a classifier error is calculated in block 125 of the objective function E based on the output 123 from the classifier 122, and then the error is back-propagated to the classifier 122.

[0092] Next, the second learning unit 12 trains the generator 121. The training of the generator 121 is performed with the parameters of the discriminator 122 fixed. The second learning unit 12 trains the generator 121 so that pseudo action data is generated when the GPS position of the unit space for each time is given to the generator 121. Specifically, the second learning unit 12 trains the parameter w G ,θ G is updated (step S36).

[0093] The learning in step S36 corresponds to the dashed arrows indicating backpropagation of errors to the generators 121 shown in Fig. 5. That is, step S36 corresponds to the dashed arrows in which the pseudo action data G1(s)-GD(s) generated by each of the generators 121_1-121_D in Fig. 5 are input to the discriminator 122, a generator error is calculated from the output 123 in the block 125 of the objective function E, and the error is backpropagated to the generators 121_1-121_D. In this way, in step S36, the generators 121_1-121_D provided corresponding to the states at each time are trained separately.

[0094] Thereafter, until the value of the objective function E reaches a Nash equilibrium and converges (step S37: NO), the learning of the discriminator 122 and the generator 121 (generators 121_1 to 121_D) from step S33 to step S36 is repeated. On the other hand, if the value of the objective function E has converged (step S37: YES), the processing from step S31 to step S37 is repeated for n sets of true action data until the learning of the generator 121 (generators 121_1 to 121_D) and the discriminator 122 is performed (step S38: NO).

[0095] Then, for all n true action data, times t1 to t D When the generator 121 (generators 121_1 to 121_D) and the discriminator 122 are trained using training data of the GPS positions (GPS_1 to GPS_D) of the unit space for (step S38: YES), the second training unit 12 stores the trained generator 121' (trained generators 121_1' to 121_D') in the first storage unit 13 (step S5). The trained generator 121' (trained generators 121_1' to 121_D') is constructed by the above-mentioned processes from step S30 to step S38.

[0096] As described above, according to the control device 1 of the present embodiment, the RNN is trained using the history of the travel direction that the UAV2a selected from among the multiple UAV2s has sequentially traveled in each unit space until it reaches the destination point as teacher data. Then, using the trained RNN as true action data, a trained generator 121' is constructed by adversarial learning of the GAN, and is transmitted to the multiple UAV2s as control information. Therefore, with a simpler configuration, it is possible to control the UAV2s, which are a group of devices that perform the same task.

[0097] In addition, according to the control device 1 of this embodiment, latent parameters such as the hidden state of the RNN at each time, memory cell values, and weight parameters between units are learned through GAN learning, so that control information applicable to a large number of devices can be generated with a simpler configuration.

[0098] In addition, the UAV2 according to the present embodiment uses the learned generator 121' to generate pseudo action data indicating the sequential directions to follow until the UAV reaches the destination point based on the UAV's position at each time. Therefore, the same task can be executed with a simpler configuration.

[0099] In the embodiment described above, collective movement control of UAV2 is exemplified as a group of devices that execute the same task by sequentially executing actions, but as described above, the group of devices is not limited to UAV2 as long as they execute one task collectively by sequentially executing actions in response to events that occur at each time. For example, this includes traffic management of autonomous vehicles, management of a sensor network in which a group of IoT devices such as sensors collect and process sensor data for monitoring, and control of assembly and logistics in a factory of a group of robots.

[0100] Furthermore, in the embodiment described above, the first learning unit 11 learns an RNN, but the first learning unit 11 can perform learning using an LSTM (Long short-term memory).

[0101] The above describes embodiments of the control device, device, and control method of the present invention. However, the present invention is not limited to the described embodiments, and various modifications that may be envisioned by a person skilled in the art can be made within the scope of the invention described in the claims. [Explanation of symbols]

[0102] 1...control device, 2, 2a...UAV, 10...first acquisition unit, 11...first learning unit, 12...second learning unit, 13...first memory unit, 14...presentation unit, 20...second memory unit, 21...second acquisition unit, 22...generation unit, 23...third memory unit, 24...execution unit, 201, 101...bus, 202, 102...processor, 203, 103...main memory unit, 204, 104...communication interface, 205, 105...auxiliary memory unit, 20 6, 106...input / output I / O, 107...display device, 121, 121_1 to 121_D...generator, 121', 121_1' to 121_D'...trained generator, 122...discriminator, 123...output, 124...training data, 125...objective function E block, 207...GPS receiver, 208...sensor, 209...motor, 210...propeller, 211...battery, A...moving space, NW...wireless communication network.

Claims

1. A control system comprising: a group of devices each performing a same task by sequentially executing actions; and a control device for controlling the group of devices, The control device includes: a first learning unit configured to learn a recurrent neural network that uses actions sequentially executed by a first device of the device group to transition from a state that is geographical location information of the first device to a next state at each time as teacher data, inputs a state of the first device, and outputs actions sequentially executed by the first device to transition from the state to the next state at each time; a second learning unit configured to perform adversarial learning of a generative model having a generator that generates pseudo action data similar to true action data for actions that each of the device group sequentially executes to transition from a state to a next state at each time, using the trained recurrent neural network constructed by the training by the first training unit, and a classifier that discriminates between the pseudo action data generated by the generator and the true action data; A storage unit configured to store an International Mobile Subscriber Identity (IMSI), which is identification information of each device in the device group to be controlled; a presentation unit configured to present a trained generator obtained by the adversarial learning of the second learning unit to each of the devices of the device group identified by the IMSI via a wireless communication network as control information for the device group to execute the task; Equipped with the generator generates the pseudo action data based on a state of each of the devices at each time given as an input, the input not including a random number vector; Each of the devices in the group of devices comprises: A second acquisition unit configured to acquire a status of the device itself; a generation unit configured to generate the pseudo action data indicating actions to be executed sequentially to transition from an acquired state of the own device to a next state at each time, using the learned generator included in the control information presented by the presentation unit via the wireless communication network; an execution unit configured to sequentially execute actions indicated by the pseudo action data generated by the generation unit and execute a set task; Equipped with Furthermore, the device includes a Subscriber Identity Module (SIM) that stores the IMSI. A control system comprising:

2. 2. The control system of claim 1, The control device includes: The control system further includes a first acquisition unit configured to acquire actions performed sequentially by the first device to transition from a state to the next state at each time, which are used as the teacher data.

3. 2. The control system of claim 1, The pseudo action data has a data distribution that has a minimum statistical distance from the data distribution of the true action data. A control system comprising:

4. 1. A method for controlling a group of devices each performing a same task by sequentially executing actions, comprising: a first learning step of learning a recurrent neural network that uses actions sequentially executed by a first device among the device group to transition from a state that is geographical location information of the first device to a next state at each time as training data, inputs a state of the first device, and outputs actions sequentially executed by the first device to transition from the state to the next state at each time; a second learning step of performing adversarial learning of a generative model having a generator that generates pseudo action data similar to true action data for actions that each of the device group sequentially executes to transition from a state to a next state at each time, using the trained recurrent neural network constructed in the first learning step, and a classifier that discriminates between the pseudo action data generated by the generator and the true action data; a storage step of storing an IMSI (International Mobile Subscriber Identity) which is identification information of each device of the device group to be controlled in a storage unit; a presentation step of presenting the trained generator obtained by the adversarial learning of the second learning step to each of the devices of the device group identified by the IMSI via a wireless communication network as control information for the device group to execute the task; Equipped with the generator generates the pseudo action data based on a state of each of the devices at each time given as an input, the input not including a random number vector; Further, each of the devices in the group of devices A second acquisition step of acquiring a status of the own device; a generating step of generating the pseudo action data indicating actions to be executed sequentially to transition from the acquired state of the own device to a next state at each time, using the learned generator included in the control information presented via the wireless communication network in the presenting step; an execution step of sequentially executing the actions indicated by the pseudo action data generated in the generation step to execute a set task; Equipped with The device stores the IMSI in a Subscriber Identity Module (SIM). A control method comprising:

5. 5. The control method according to claim 4, The control method further includes a first acquisition step of acquiring actions sequentially performed by the first device to transition from a state to the next state at each time, which are used as the teacher data.

Images