Flight guidance device and flight guidance method

Abstract

To more reliably guide an aircraft into a communication area of a specified wireless communication method.SOLUTION: A flight guidance device 1 for controlling a course of an aircraft 2 from a position of an initial point to a position of a communication area of a designated first wireless communication method in a movement space of the aircraft 2 in which communication areas A1 to A5 of heterogeneous wireless communication methods are superimposed, comprises: a first memory unit 10 that stores identification information of a unit space obtained by dividing the movement space into multiple spaces, a position of the unit space, and information indicating wireless communication methods of communication areas covering the unit space in association with each other; a first acquisition unit 11 that acquires a position of the unit space where the aircraft 2 is currently located as a current position of the aircraft 2; a determination unit 14 that determines a course that the aircraft 2 should proceed next from the current position of the unit space acquired by the first acquisition unit 11 based on a measure of the course that the aircraft 2 should proceed sequentially from a position of each unit space, which is learned using a learning model; and a flight guidance unit 15 that instructs the aircraft 2 about the course that the aircraft 2 should proceed next.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a flight guidance device and a flight guidance method. [Background technology]

[0002] A technology has been known for controlling flight guidance of drones and other flying objects, in which position information from the flying object's GPS or other sources is received by a ground system, a flight point to which the flying object should advance is found based on a flight route set using the flying object's position information and a map database of terrain and other information, and control information for advancing to that point is then transmitted to the flying object (see Patent Document 1).

[0003] In recent years, drones and other flying objects have been used in a variety of fields, including surveying, logistics, agriculture, and disaster investigation. In order to accommodate the communications required for each application, drones are also required to communicate using a variety of wireless methods.

[0004] On the other hand, heterogeneous networks, which are wireless communication networks that use different wireless communication methods and repeaters with different specifications, have been known for some time. In a mobile space where communication areas of different wireless communication methods are overlapped, in order for a drone to communicate using a specific wireless communication method, it is necessary to guide the drone to a communication area where the radio waves of that wireless communication method can be received.

[0005] However, conventional flight guidance technologies do not know the location of communication areas where different wireless communication methods overlap, even when using GPS location information and a map database of terrain, etc. acquired by the drone. Therefore, it was necessary to control the drone to fly on a random course, or to use remote flight control or autonomous flight control to guide the drone to an approximate location, and to determine whether or not the drone actually receives radio waves of the specified wireless communication method at the location where the drone moved. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2002-211494 A Summary of the Invention [Problem to be solved by the invention]

[0007] As described above, with conventional technology, it has been difficult to reliably guide an aircraft into the communication area of ​​a specified wireless communication method.

[0008] The present invention has been made to solve the above-mentioned problems, and aims to more reliably guide an aircraft into a communication area of ​​a specified wireless communication method. [Means for solving the problem]

[0009] In order to solve the above-mentioned problems, the flight guidance device of the present invention is a flight guidance device that controls the course of an aircraft from a position of an initial point to a position of a communication area of ​​a specified first wireless communication method in a movement space of the aircraft in which communication areas of different wireless communication methods are superimposed, and is equipped with a first memory unit configured to store identification information of a unit space obtained by dividing the movement space into a plurality of spaces, the position of the unit space, and information indicating the wireless communication method of the communication area covering the unit space in association with each other, a first acquisition unit configured to acquire the position of the unit space in which the aircraft is currently located as the current position of the aircraft, a determination unit configured to determine a course to be taken by the aircraft from the position of the current unit space acquired as the current position of the aircraft by the first acquisition unit based on a course plan for the aircraft to take sequentially from the position of each unit space learned using a learning model, and a flight guidance unit configured to instruct the aircraft of the course to be taken by the aircraft next determined by the determination unit.

[0010] In addition, the flight guidance device of the present invention may further include a learning unit configured to apply a reward function to an estimated result of calculating a course that the aircraft should take sequentially from the position of each unit space until it reaches a position in a first unit space corresponding to the position of the communication area of ​​the first wireless communication method from the current position of the unit space, and update the course so as to maximize the reward for the aircraft to reach the position in the first unit space, and to learn a course plan for the aircraft to take sequentially from the position of each unit space using the learning model, and a second memory unit configured to store the course plan learned by the learning unit, and the determination unit may read out the course plan from the second memory unit and determine the next course to be taken by the aircraft.

[0011] In addition, in the flight guidance device of the present invention, the reward function may include, as a variable, a degree of arrival at a position in the first unit space corresponding to the position of the communication area of ​​the first wireless communication method in the aircraft.

[0012] In addition, in the flight guidance device of the present invention, the learning model is a neural network model including an input layer, a hidden layer, and an output layer, and the learning unit provides the current unit space position as an input to the neural network model, performs calculations on the neural network model, and outputs a first estimate of an action value function representing an expected value of the future cumulative value of the reward to be obtained when the aircraft takes each action related to moving in a predetermined direction relative to the traveling direction as a course to be followed next from the current unit space position, the learning unit further provides the next unit space position reached by the aircraft as an input to the neural network model, performs calculations on the neural network model, and outputs a second estimate of the action value function, the learning unit learns weight parameters of the neural network model so that the first estimate becomes a target value calculated from the second estimate, and the second memory unit stores the learned weight parameters.

[0013] In addition, in the flight guidance device of the present invention, the first acquisition unit may acquire the position of the unit space in which the aircraft is currently located via a core network of a second wireless communication method that covers the widest range of the movement space among the different wireless communication methods.

[0014] In addition, in the flight guidance device of the present invention, the flight guidance unit may transmit instructions regarding the next course to the aircraft via a user plane function included in the core network of the second wireless communication method.

[0015] In order to solve the above-mentioned problems, the flight guidance method of the present invention is a flight guidance method for controlling the path of an aircraft from a position of an initial point to a position of a communication area of ​​a specified first wireless communication method in a movement space of the aircraft in which communication areas of different wireless communication methods are superimposed, and includes a first storage step of correlating and storing in a first memory unit identification information of a unit space obtained by dividing the movement space into a plurality of spaces, the position of the unit space, and information indicating the wireless communication method of the communication area covering the unit space, a first acquisition step of acquiring the position of the unit space in which the aircraft is currently located as the current position of the aircraft, a determination step of determining a next path to be taken by the aircraft from the position of the current unit space acquired as the current position of the aircraft in the first acquisition step based on a path strategy for the aircraft to sequentially take from the position of each unit space learned using a learning model, and a flight guidance step of instructing the aircraft on the next path to be taken by the aircraft determined in the determination step.

[0016] In addition, the flight guidance method of the present invention further includes a learning step of applying a reward function to an estimated result of calculating a course that the aircraft should take sequentially from the position of each unit space until the aircraft reaches a position in a first unit space corresponding to the position of the communication area of ​​the first wireless communication method from the current position of the unit space, and updating the course so as to maximize the reward for the aircraft to reach the position in the first unit space, and learning a course plan for the aircraft to take sequentially from the position of each unit space using the learning model, and a second storage step of storing the course plan learned in the learning step in a second storage unit, and the determination step may read out the course plan from the second storage unit and determine the next course that the aircraft should take.

[0017] In addition, in the flight guidance method according to the present invention, the reward function may include, as a variable, a degree of arrival of the aircraft to a position in the first unit space corresponding to the position of the communication area of ​​the first wireless communication method.

[0018] In addition, in the flight guidance method according to the present invention, the learning model is a neural network model including an input layer, a hidden layer, and an output layer, and the learning step may include providing the current unit space position as an input to the neural network model, performing calculations on the neural network model, and outputting a first estimate of an action value function representing an expected value of the future cumulative value of the reward to be obtained when the aircraft takes each action relating to movement in a predetermined direction relative to the traveling direction as a course to be followed next from the current unit space position, the learning step may further provide the next unit space position reached by the aircraft as an input to the neural network model, performing calculations on the neural network model, and outputting a second estimate of the action value function, the learning step may learn weight parameters of the neural network model so that the first estimate becomes a target value calculated from the second estimate, and the second storage step may store the learned weight parameters in the second storage unit.

[0019] In addition, in the flight guidance method of the present invention, the first acquisition step may acquire the position of the unit space in which the aircraft is currently located via a core network of a second wireless communication system that covers the widest range of the movement space among the heterogeneous wireless communication systems.

[0020] In addition, in the flight guidance method of the present invention, the flight guidance step may include transmitting instructions regarding the next course to the aircraft via a user plane function included in the core network of the second wireless communication system. Effect of the Invention

[0021] According to the present invention, a first storage unit is provided that stores, in association with each other, identification information of a unit space obtained by dividing a moving space into a plurality of spaces, the position of the unit space, and information indicating a wireless communication method of a communication area that covers the unit space, and based on a course plan for the aircraft to take from the position of each unit space, learned using a learning model, the next course for the aircraft to take is determined from the current position of the unit space acquired as the current position of the aircraft. Therefore, the aircraft can be more reliably guided to the communication area of ​​the designated wireless communication method. [Brief description of the drawings]

[0022] [Figure 1] FIG. 1 is a block diagram showing the configuration of a flight guidance system including a flight guidance device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram for explaining an overview of the flight guidance system according to the present embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of a data structure of the first storage unit according to the present embodiment. [Figure 4] FIG. 4 is a diagram for explaining the learning process performed by the learning unit according to the present embodiment. [Diagram 5] FIG. 5 is a block diagram showing a configuration of a learning unit according to the present embodiment. [Figure 6]FIG. 6 is a block diagram showing the hardware configuration of the flight guidance device according to this embodiment. [Figure 7] FIG. 7 is a flowchart showing the learning process of the flight guidance device according to the present embodiment. [Figure 8] FIG. 8 is a flowchart showing the learning process of the flight guidance device according to this embodiment. [Figure 9] FIG. 9 is a flowchart showing the flight guidance process of the flight guidance device according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will now be described in detail with reference to FIGS.

[0024] [Flight Guidance System Configuration] First, an overview of a flight guidance system including a flight guidance device 1 according to an embodiment of the present invention will be described. Fig. 1 is a block diagram showing the configuration of a flight guidance system including a flight guidance device 1 according to an embodiment of the present invention.

[0025] The flight guidance system according to the present embodiment controls the route of the aircraft 2 from the position of the initial point to a position corresponding to the communication areas A1-A5 of a specified wireless communication method (first wireless communication method) in a moving space of the aircraft 2 configured as a heterogeneous network in which communication areas A1-A5 of different wireless communication methods are superimposed. As shown in Fig. 1, the moving space in which the aircraft 2 flies includes, for example, a communication area A1 of a public 5G wireless communication method, a communication area A2 of a local 5G wireless communication method, a communication area A3 of a public LTE wireless communication method, a communication area A4 of a local LTE wireless communication method, and a communication area A5 of a WiFi wireless communication method.

[0026] The communication area A1 of the public 5G wireless communication system is provided by a communication carrier and covers a wider communication range. In the present embodiment, as an example, the communication area A1 of the public 5G wireless communication system covers the entire moving space. Therefore, in the communication areas A2 to A5 of other wireless communication systems, communication by the public 5G wireless communication system is possible.

[0027] The communication area A2 of the local 5G wireless communication system is, for example, a communication area provided by a local government or a business owner. In the present embodiment, a communication area A3 of the public LTE wireless communication system provided by a communication carrier is arranged so as to be superimposed on the communication area A1 of the public 5G wireless communication system. Furthermore, a communication area A4 of the local LTE wireless communication system arranged as a hotspot is arranged so as to be superimposed on the communication area A1. Also, a communication area A5 of the WiFi wireless communication system, which is narrower than the other communication areas A1 to A4 used in units such as homes, schools, and office buildings, is arranged so as to be superimposed on at least the communication area A1.

[0028] In this embodiment, the communication area A1 indicates a cell covered by a plurality of base stations 30. On the other hand, the communication area A2 refers to a cell covered by one or a plurality of base stations 31. The other communication areas A3 to A5 can be configured similarly to the communication area A2. The communication areas A1 to A5 of the wireless communication method arranged in the moving space are not limited to the communication areas A1 to A5 of the public 5G wireless communication method, the local 5G wireless communication method, the public LTE wireless communication method, the local LTE wireless communication method, and the WiFi wireless communication method. In the following, when the communication areas A1 to A5 are not distinguished from each other, they may be collectively referred to as the communication area A.

[0029] As shown in Fig. 1, the flight guidance system includes a flight guidance device 1, an aircraft 2, and base stations 31 to 34 corresponding to a public 5G wireless communication system, a local 5G wireless communication system, a public LTE wireless communication system, a local LTE wireless communication system, and a WiFi wireless communication system. Furthermore, the base station 30 of the public 5G wireless communication system is connected to a UPF (User Plane Function) 40, which is a node in the U-plane of a core network corresponding to the public 5G wireless communication system. The base station 31 of the local 5G wireless communication system is connected to a UPF 41 of the core network corresponding to the same wireless communication system.

[0030] The base station 32 of the public LTE wireless communication system is connected to a P-GW (Packet Data Network Gateway) 42, which is a node in the U-plane of a core network that supports the public LTE wireless communication system. Furthermore, the base station 33 that supports the local LTE wireless communication system is connected to a P-GW 43 of the core network of the same wireless communication system. Moreover, the base station 34 of the WiFi wireless communication system is configured as a so-called wireless access point (Wireless Access Point: WAP).

[0031] In the present embodiment, the UPFs 40 and 41 and the P-GWs 42 and 43 included in the core networks corresponding to the respective wireless communication methods are provided with communication interfaces 40a, 41a, 42a, and 43a for communicating with the flight guidance device 1. In contrast, the base station 34 of the WiFi wireless communication method and the flight guidance device 1 are directly connected.

[0032] The flight guidance device 1, UPFs 40, 41, P-GWs 42, 43 included in the core networks corresponding to the respective wireless communication methods, and the base station 34 for the WiFi wireless communication method are connected via a network NW such as a LAN or WAN. In addition, each of the base stations 30 to 33 constituting the wireless access network is connected to the UPFs 40, 41, and P-GWs 42, 43 of the core network via a network L such as a backhaul link.

[0033] The flying object 2 is an unmanned flying object such as a drone, and can perform remote radio-controlled flight. The flying object 2 includes a processor, a main memory device, an auxiliary memory device, and a communication interface. In the present embodiment, as an example, the flying object 2 includes a communication interface compatible with all of the public 5G wireless communication method, the local 5G wireless communication method, the public LTE wireless communication method, the local LTE wireless communication method, and the WiFi wireless communication method. However, the flying object 2 only needs to include a communication interface compatible with the public 5G wireless communication method and at least any other wireless communication method. For example, the flying object 2 can include a communication interface for a specific wireless communication method required depending on the use of the flying object 2. The flying object 2 can communicate with the flight guidance device 1 via the communication interface.

[0034] The aircraft 2 includes a SIM 20 and is uniquely identified by an International Mobile Subscriber Identity (IMSI) of the SIM 20. The aircraft 2 is also configured as an IoT terminal having an IP address.

[0035] Furthermore, the flying object 2 includes a battery, a motor, a flight controller, a sensor including a GPS receiver 21, a camera, and the like. The flying object 2 can mount these components on a frame of a predetermined shape. The flying object 2 transmits the current position information received by the GPS receiver 21 to the flight guidance device 1 via the UPF 40 included in the core network of the public 5G wireless communication system. The GPS receiver 21 includes an antenna.

[0036] FIG. 2 is a schematic diagram showing a moving space in which a flight guidance system guides the course of an aircraft 2. As shown in FIG. 2, a communication area A1 of the public 5G wireless communication system covers the entire moving space of the aircraft 2. Communication areas A2 to A5 of other wireless communication systems are arranged to overlap the communication area A1. The moving space is a three-dimensional matrix-like space composed of unit spaces divided into a plurality of spaces.

[0037] Moreover, each unit space constituting the travel space has the same volume. Furthermore, each unit space has a node ID, and each unit space is represented by one position (x, y, z). The position information can be three-dimensional GPS position coordinates consisting of latitude, longitude, and altitude. For example, a representative value such as the center position of the unit space can be used as the position of the unit space. In the example of FIG. 2, the node ID of the unit space of the initial point S is #1,9, and the node ID of the unit space of the destination point G is #9,3.

[0038] As shown in FIG. 2, the flying object 2 moves from a position in the unit space corresponding to the position of the initial point S to a unit space of a destination point G where communication is possible using a designated type of wireless communication method, with each unit space being used as a waypoint. The position of the destination point G of the flying object 2 is set by the flight guidance device 1 with reference to a map of the movement space stored in the first storage unit 10 (FIG. 2) described later, according to a wireless communication method designated in advance in the flying object 2. The wireless communication method designated in the flying object 2 is, for example, a specific wireless communication method that is set to be used preferentially in the flying object 2 among multiple types of wireless communication methods in the movement space. In addition, the flight guidance device 1 identifies the position of the communication area A of the designated wireless communication method in the movement space and sets it as the destination point G. The flying object 2 is guided to the position of the destination point G set by the flight guidance device 1, and is thereby enabled to communicate using the designated wireless communication method.

[0039] For example, in the aircraft 2 shown in Fig. 2, the local LTE wireless communication system is designated in advance as the type of wireless communication system to be preferentially used within the moving space. Therefore, the position of a unit space included in a communication area A4 of the local LTE wireless communication system within the moving space is set as a destination point G. The flight guidance device 1 can arbitrarily select the position of any one of the multiple unit spaces included in the communication area A4 and set it as the destination point G.

[0040] The flight guidance device 1 instructs the flying object 2 to move in a predetermined direction for each unit space as a course indicated by an arrow in relation to the direction of travel. The course instructed by the flight guidance device 1 can include various courses, i.e., movement directions, depending on the design. In FIG. 2, the movement space is described as a two-dimensional plane, but the course of the flying object 2 can be a three-dimensional course. When the flying object 2 reaches the unit space of the destination point G in accordance with the course instruction from the flight guidance device 1, it can start communication by a specified wireless communication method.

[0041] [Flight Guidance System Functional Blocks] 1, the flight guidance device 1 includes a first storage unit 10, a first acquisition unit 11, a learning unit 12, a learning model storage unit (second storage unit) 13, a determination unit 14, and a flight guidance unit 15. The flight guidance device 1 determines the next course that the aircraft 2 should take from the unit space in which it is currently located, based on the current GPS position received from the aircraft 2, and instructs the aircraft 2 on the determined course.

[0042] The first storage unit 10 stores, in association with each other, identification information of a unit space obtained by dividing a moving space into a plurality of spaces, the position of the unit space, and information indicating the wireless communication method of the communication area A that covers the unit space. FIG. 3 shows a table T1 that shows the data structure of the first storage unit 10. In the table T1, the node ID of each unit space is associated with the type of wireless communication method of the communication area A that covers the unit space, and the position coordinates (x, y, z) of the unit space. The first storage unit 10 can store information for all of the node IDs of a plurality of unit spaces.

[0043] The first acquisition unit 11 acquires the position of the unit space in which the aircraft 2 is currently located as the current position of the aircraft 2. Specifically, the first acquisition unit 11 acquires the position of the unit space in which the aircraft 2 is currently located via the UPF 40 provided in the core network of a public 5G wireless communication system (second wireless communication system) that covers the widest range of the moving space among different wireless communication systems. In addition, in this embodiment, the first acquisition unit 11 acquires the current position of the aircraft 2 for each set time step. In addition, when acquiring the current unit space position of the aircraft 2 based on the GPS position of the aircraft 2, the first acquisition unit 11 can acquire it by referring to the first storage unit 10.

[0044] More specifically, the first acquisition unit 11 can acquire a position in the unit space corresponding to the GPS position received by the GPS receiver 21 of the aircraft 2 as the current position of the aircraft 2. Note that there are cases where a plurality of position coordinates are included in the unit space. In such a case, the first acquisition unit 11 can acquire a position in the unit space that is closest to the GPS position received by the GPS receiver 21 of the aircraft 2 as the current position of the aircraft 2.

[0045] The learning unit 12 applies a reward function to the estimated results of calculating the course that the aircraft 2 should take sequentially from the position of each unit space until it reaches the position of the unit space of the destination point (first unit space) from the position of the unit space of the initial point, and updates it so as to maximize the reward for the aircraft 2 to reach the position of the unit space of the destination point, and learns the course strategy that the aircraft 2 should take sequentially from each unit space using the learning model.

[0046] In this embodiment, as a course of action for the aircraft 2 to move sequentially from the position of each unit space, the aircraft 2 may take actions a, b, c, and d in a predetermined number of directions (n ​​is a positive integer) relative to the traveling direction. n The travel direction is based on the position of the aircraft 2 in the unit space immediately prior to its arrival.

[0047] The learning unit 12 uses a neural network model including an input layer, a hidden layer, and an output layer as shown in Fig. 4 as a learning model. t , and all action values ​​Q(s t ,a1), Q(s t ,a2), Q(s t ,a3), ,Q(s t ,a n-1 ), Q(s t ,a n We adopt Deep Q-Network (DQN), a neural network that outputs

[0048] More specifically, the learning unit 12 inputs the current position of the aircraft 2 in the unit space, which indicates the current position of the aircraft 2, as an input to the neural network model, performs calculations on the neural network model, and determines actions a related to each movement in n directions as a course to be taken by the aircraft 2 from the current position in the unit space. n The first estimate Q1 of the action value function that represents the expected cumulative value of future rewards obtained if

[0049] The reward is a state s indicating the current position of the aircraft 2, and an action a for the aircraft 2 to move in a given direction. n , and a reward function r=r(s, a, s') for the next position of the aircraft 2, i.e., the next state s'. In this embodiment, the reward function includes as a variable the degree of arrival of the aircraft 2 at a position in the unit space related to the destination point. In addition, it may include as a variable the degree of arrival at a position in the unit space corresponding to a space containing obstacles such as high-rise buildings or steel towers. For example, when an action related to the movement of the aircraft 2 in a specified direction brings the aircraft 2 closer to the destination point or when the aircraft 2 reaches the destination point in the shortest distance, the reward, which is a scalar quantity, is set to a larger value.

[0050] On the other hand, if the aircraft 2 moves away from the destination point or reaches a unit space in which an obstacle exists, a negative reward value (e.g., r=-1) can be given. In this way, by setting the reward for a unit space in which an obstacle exists as a negative value, the aircraft 2 can reach the destination point while avoiding these points.

[0051] Furthermore, the learning unit 12 provides the next position in the unit space reached by the flying object 2 as an input to the neural network model, performs calculations on the neural network model, and outputs a second estimated value Q2 of the action value function. The learning unit 12 learns weight parameters of the neural network model so that the first estimated value Q1 becomes a target value calculated from the second estimated value Q2.

[0052] If the weight parameter of the neural network model is θ and the action value function is Q(s, a; θ), the minimization loss function for learning is given by the following equation (1). L(θ)=1 / 2{r+γmax a’ Q(s',a';θ)-Q(s,a;θ)} 2 (1)

[0053] In the above formula (1), r is the reward (immediate reward) and γ is the discount rate. Q(s, a; θ) corresponds to the first estimated value Q1, and Q(s', a'; θ) corresponds to the action value at the state s' one step forward, that is, the second estimated value Q2. The target value is r+γmax a’ It is expressed as Q(s',a';θ).

[0054] The learning unit 12 can update the weight parameters of the neural network model by backpropagating the gradient of the loss function given by the above formula (1).

[0055] More specifically, the learning unit 12 may employ a Fixed Target Q-Network that uses two neural networks, a main QN 121 and a target QN 123, as shown in FIG. 5. The main QN 121 selects an optimal action and updates an action-value function Q. Meanwhile, the target QN 123 estimates and evaluates the value of an action a' to be taken in the next state s' of the result of the action. The main QN 121 and the target QN 123 have neural networks with the same layer structure, but the parameter of the main QN 121 is "θ" and the parameter of the target QN 123 is "θ". - " is given by

[0056] The main QN 121 receives the current position of the flying object 2 as a state s from the environment 120. The environment 120 is a system of a moving space in which the flying object 2 is placed, and in this environment 120, the flying object 2 moves to another unit space by taking an action a related to movement in a predetermined direction, and at the same time, transitions to the next state s' and acquires a reward r from the environment 120.

[0057] The learning unit 12 inputs the state s relating to the current position of the aircraft 2 to the main QN 121 and obtains the action value function Q(s, a; θ). The learning unit 12 calculates the action a using, for example, the ε-greedy method, or calculates the optimal action argmax a Q(s, a; θ) is calculated. In the environment 120, the flying object 2 takes the action argmax a Q(s, a; θ). The environment 120 is a space where the flying object 2 takes action argmax a As a result of performing Q(s, a; θ), the position in the unit space to which the robot has moved is observed as the next state s', and a reward r is output. The experience data 124 stores the experience (s, a, r, s') output from the environment 120.

[0058] The learning unit 12 calculates a loss function L in a DQN loss calculation 122, and updates the weight of the main QN 121 with the gradient of the loss function L.

[0059] The learning unit 12 periodically copies the weights of the main QN 121 to the target QN 123 to synchronize them. The synchronization of the target QN 123 is performed less frequently than the update frequency of the weights of the main QN 121. The learning unit 12 extracts experience from the experience data 124, inputs past states to the target QN 123, and calculates the estimated value max a’ Q(s', a';θ - The learning unit 12 outputs the estimated value max a’ Q(s', a';θ - ) based on the target value r+γmax a’ Q(s', a';θ - ) is used to learn the weights of the main QN 121 in the DQN loss calculation 122.

[0060] Returning to FIG. 1, the learning model storage unit 13 stores the weights of the trained neural network model.

[0061] The determination unit 14 determines the next course to be taken by the aircraft 2 from the position of the current unit space acquired by the first acquisition unit 11, based on the course plan for the aircraft 2 to take sequentially from the position of each unit space, learned using the learning model. The determination unit 14 reads out the learned weights stored in the learning model storage unit 13, provides the position of the current unit space as an input to the trained neural network model, performs calculations on the trained neural network model, and determines the optimal action among actions related to movement in a predetermined n directions relative to the traveling direction of the aircraft 2 as the next course to be taken.

[0062] The flight guidance unit 15 instructs the flying object 2 on the next route that the flying object 2 should take, which has been determined by the determination unit 14. Specifically, the flight guidance unit 15 transmits instructions regarding the next route to the flying object 2 via the UPF 40 included in the core network of the public 5G wireless communication system. The flight guidance unit 15 instructs the flying object 2 on the route until the flying object 2 reaches the unit space of the destination point. Note that the flight guidance unit 15 may transmit instructions regarding the route by a wireless communication system other than the public 5G wireless communication system that can receive radio waves in the unit space in which the flying object 2 exists.

[0063] [Hardware configuration of flight guidance system] Next, an example of a hardware configuration for realizing the flight guidance device 1 having the above-mentioned functions will be described with reference to FIG.

[0064] As shown in Figure 5, the flight guidance device 1 can be realized, for example, by a computer including a processor 102, a main memory device 103, a communication interface 104, an auxiliary memory device 105, and an input / output (I / O) 106 connected via a bus 101, and a program that controls these hardware resources.

[0065] The main memory device 103 stores in advance programs for the processor 102 to perform various controls and calculations. The processor 102 and the main memory device 103 realize the functions of the flight guidance device 1, such as the first acquisition unit 11, the learning unit 12, the determination unit 14, and the flight guidance unit 15 shown in FIG.

[0066] The communication interface 104 is an interface circuit for network connection between the flight guidance device 1 and various external electronic devices.

[0067] The auxiliary storage device 105 is composed of a readable / writable storage medium and a drive for reading and writing various information such as programs and data from and to the storage medium. The auxiliary storage device 105 can use a semiconductor memory such as a hard disk or a flash memory as the storage medium.

[0068] The auxiliary storage device 105 has a program storage area for storing a flight guidance program executed by the flight guidance device 1. The auxiliary storage device 105 also has an area for storing a learning program for learning a neural network model. The first storage unit 10 and the learning model storage unit 13 described in FIG. 1 are realized by the auxiliary storage device 105. The auxiliary storage device 105 also has an area for storing an initial point and a destination point when guiding the course of the flying object 2. The auxiliary storage device 105 also has an area for storing position coordinates of the movement space and position coordinates of the unit space. The auxiliary storage device 105 also has an area for storing identification information of the flying object 2. Furthermore, for example, the auxiliary storage device 105 may have a backup area for backing up the above-mentioned data and programs.

[0069] The input / output I / O 106 is an input / output device that inputs signals from an external device and outputs signals to an external device.

[0070] [Flight guidance system operation] Next, the operation of the flight guidance device 1 having the above-mentioned configuration will be described with reference to the flowcharts of FIGS.

[0071] First, the learning process by the flight guidance device 1 will be described with reference to Fig. 7. First, the flight guidance device 1 acquires setting information of the initial point and destination point of the flying object 2 (step S1). For example, the flight guidance device 1 identifies the position of a communication area A of a specified type of wireless communication method that is set in advance in the flying object 2 within the movement space, and sets it as the destination point.

[0072] More specifically, in step S1, the flight guidance device 1 can refer to the first storage unit 10 and set the position of a unit space selected from among the unit spaces included in the communication area A of the specified wireless communication method as the destination point. As for the initial point, the flight guidance device 1 can receive the current GPS position received by the GPS receiver 21 of the flying object 2, and refer to the first storage unit 10 to set the position of the unit space corresponding to the GPS position as the initial point.

[0073] Next, the first acquisition unit 11 acquires the position of the unit space in which the aircraft 2 is currently located as the current position of the aircraft 2 (step S2). Specifically, the first acquisition unit 11 acquires the position of the unit space in which the aircraft 2 is currently located for each time step based on the GPS position of the aircraft 2 received via the base station 30 of the public 5G wireless communication system and the UPF 40 of the core network. Furthermore, the first acquisition unit 11 can refer to the first storage unit 10 and acquire the position of the unit space corresponding to the current GPS position received by the GPS receiver 21 of the aircraft 2 as the current position of the aircraft 2.

[0074] Next, the learning unit 12 provides as input to the neural network model the current state of the aircraft 2 obtained in step S2, i.e., the position in the unit space where the aircraft 2 is currently located, and performs calculations on the neural network model to output a first estimate Q1 of an action value function that represents the expected value of the cumulative value of future rewards to be obtained when the aircraft 2 takes each action related to moving in a specified direction relative to the direction of travel as the next course to be taken from the current position in the unit space (step S3).

[0075] Furthermore, the learning unit 12 provides the next position in the unit space where the aircraft 2 arrives as an input to the neural network model, performs calculations on the neural network model, and outputs a second estimated value Q2 of the action value function (step S5). The next position in the unit space where the aircraft 2 arrives is determined based on the GPS position of the aircraft 2 acquired by the first acquisition unit 11 for each time step.

[0076] Next, the learning unit 12 calculates a target value from the second estimated value Q2 (step S6). Then, the learning unit 12 learns the weight parameters of the neural network model so that the first estimated value Q1 becomes the target value calculated from the second estimated value Q2 (step S7). Specifically, the learning unit 12 updates the weight parameters of the neural network model so as to minimize the loss function of the above formula (1).

[0077] The learning model storage unit 13 stores the learned weights obtained in step S7 (step S8).

[0078] Next, the learning process by the learning unit 12 in the case where a Fixed Target Q-Network using two neural networks, a main QN 121 and a target QN 123, is adopted will be described with reference to FIG.

[0079] The processes from step S1 to step S3 are the same as the steps of the learning process described in Fig. 7. After that, the learning unit 12 provides the main QN121 with the current unit space position of the aircraft 2 acquired in step S3 as an input, performs neural network calculations, outputs an action value function Q, and calculates the next course a to be taken (step S120).

[0080] Next, the learning unit 12 returns the behavior of the aircraft 2 on the course a obtained in step S120 to the environment 120, and obtains the next state s' of the aircraft 2, that is, the position in the unit space to which the aircraft 2 has advanced, and the reward r (step S121).

[0081] The learning unit 12 stores the experience (s, a, r, a') obtained in step S121 in the experience data 124 (step S122). Next, the learning unit 12 calculates a loss function L in the DQN loss calculation 122, and updates the weight of the main QN 121 with the gradient of the loss function L (step S123). The learning unit 12 repeats the processes from step S120 to step S123 a set number of times.

[0082] Thereafter, the learning unit 12 periodically copies the weights of the main QN 121 to the target QN 123 for synchronization (step S124). The synchronization of the target QN 123 is performed less frequently than the update frequency of the weights of the main QN 121. Next, the learning unit 12 extracts experience from the experience data 124, inputs past states to the target QN 123, and calculates the estimated value max a’ Q(s', a';θ - ) is output (step S126).

[0083] Next, the learning unit 12 calculates the estimated value max a’ Q(s', a';θ - ) based on the target value r+γmax a’ Q(s', a';θ - ) (step S127). Next, the learning unit 12 uses the target value calculated in step S127 to calculate a loss function L in the DQN loss calculation 122 (step S128). Next, the learning unit 12 learns the weights of the main QN 121 so as to minimize the loss given by the loss function L (step S129). After that, the learned weights are stored in the learning model storage unit 13 (step S8).

[0084] Next, the flight guidance process by the flight guidance device 1 will be described with reference to Fig. 9. First, the determination unit 14 loads a trained neural network model from the learning model storage unit 13 (step S40). In this embodiment, the determination unit 14 loads a trained DQN. Next, the first acquisition unit 11 acquires the position of the unit space in which the aircraft 2 is currently located, which is the current position of the aircraft 2 (step S41).

[0085] Next, the determination unit 14 determines the next course that the aircraft 2 should take from the current position in the unit space based on the trained neural network model loaded in step S40, i.e., the course plan for the aircraft 2 to take from the position of each unit space (step S42). Specifically, the determination unit 14 inputs the current unit space position acquired in step S41 to the trained neural network model, performs calculations on the trained neural network model, and determines one of the predetermined n moving directions relative to the traveling direction, which is the course that the aircraft 2 should take next. The determination unit 14 selects the action with the highest probability value from the action value functions Q of the n actions output from the trained neural network model, and determines it as the course.

[0086] Thereafter, the flight guidance unit 15 instructs the flying object 2 on the path of the flying object 2 determined in step S42 (step S43). More specifically, the flight guidance unit 15 transmits the determined path to the flying object 2 via the UPF 40 and the base station 30 provided in the core network of the public 5G wireless communication system. Alternatively, the flight guidance unit 15 can transmit the path instruction via a core network of a wireless communication system with which the flying object 2 can currently communicate other than the public 5G wireless communication system. Upon receiving the path instruction, the flight controller provided in the flying object 2 controls the motor according to the path instruction and outputs a control command such as the number of revolutions of the propeller of the flying object 2, thereby moving to the next unit space. Whether the flying object 2 has moved to the next unit space is determined by comparing the GPS position from the flying object 2 with the position of the unit space stored in the first storage unit 10.

[0087] Next, if the aircraft 2 has reached the communication area A of the wireless communication method of the destination point, the process ends (step S44: YES). On the other hand, if the aircraft 2 has not reached the destination point (step S44: NO), the process from step S41 to step S43 is repeated. For example, whether the aircraft 2 has reached the destination point can be determined based on whether the position of the unit space to which the aircraft 2 has moved in accordance with the route guidance instruction matches the position of the unit space in the communication area A of the wireless communication method set as the destination point.

[0088] As described above, the flight guidance device 1 according to the present embodiment includes a first storage unit 10 that stores, in association with each other, identification information of a unit space obtained by dividing a movement space into a plurality of spaces, the position of the unit space, and information indicating the wireless communication method of the communication area A that covers the unit space. Furthermore, based on the current position of the unit space in the flying object 2, the next course to be taken is determined based on the course plan acquired by the trained neural network. Therefore, the flying object can be more reliably guided to the communication area A of the designated wireless communication method. As a result, more reliable communication of the flying object 2 is ensured.

[0089] Furthermore, according to the flight guidance device 1 of the present embodiment, since DQN is adopted as a learning model, the next course to be followed is determined based on the current position of the flying object 2. Therefore, flight guidance processing can be realized with less sensor data.

[0090] In the embodiment described above, the case where the communication area A of each wireless communication method is arranged one by one in the moving space has been described. However, the communication area A of the same wireless communication method can exist in a plurality of positions apart from each other in the moving space. In this case, the flight guidance device 1 can set, as the destination point when starting flight guidance, the position of the unit space corresponding to the communication area A that is closest to the initial point of the flying object 2 among the communication areas A arranged at different positions related to the same wireless communication method. Alternatively, the position of the unit space of the communication area A that is closest to the initial point according to the flight plan of the flying object 2 can be set as the destination point.

[0091] The above describes embodiments of the flight guidance device and flight guidance method of the present invention. However, the present invention is not limited to the described embodiments, and various modifications that can be envisioned by a person skilled in the art can be made within the scope of the invention described in the claims. [Explanation of symbols]

[0092] 1...flight guidance device, 10...first memory unit, 11...first acquisition unit, 12...learning unit, 13...learning model memory unit, 14...decision unit, 15...flight guidance unit, 2...aircraft, 20...SIM, 21...GPS receiver, 30-34...base station, 40, 41...UPF, 42, 43...P-GW, 101...bus, 102...processor, 103...main memory unit, 40a-43a, 104...communication interface, 105...auxiliary memory unit, 106...input / output I / O, 120...environment, 121...main QN, 122...DQN loss calculation, 123...target QN, 124...empirical data, A, A1-A5...communication area, L, NW...network.

Claims

1. A flight guidance device that controls a course of an aircraft from a position of an initial point to a position of a communication area of ​​a specified first wireless communication system in a moving space of the aircraft in which communication areas of different wireless communication systems are superimposed, a first storage unit configured to store, in association with each other, identification information of a unit space obtained by dividing the movement space into a plurality of spaces, a position of the unit space, and information indicating a wireless communication method of a communication area covering the unit space; A first acquisition unit configured to acquire a position of the unit space where the aircraft is currently located as a current position of the aircraft; A determination unit configured to determine a next course to be taken by the aircraft from the current unit space position acquired as the current position of the aircraft by the first acquisition unit, based on a course plan for the aircraft to take from each unit space position, learned using a learning model; and a flight guidance unit configured to instruct the flying object on the next course determined by the determination unit; A flight guidance device comprising:

2. 2. The flight guidance device according to claim 1, a learning unit configured to apply a reward function to an estimation result of a course that the aircraft should take from each unit space position to a first unit space position corresponding to the communication area position of the first wireless communication method from the current unit space position, and to update the course so as to maximize a reward for the aircraft to reach the first unit space position, and to learn a course plan for the aircraft to take from each unit space position using the learning model; a second storage unit configured to store the course strategy learned by the learning unit; Equipped with The determination unit reads out the course plan from the second storage unit and determines the course that the aircraft should take next. A flight guidance device characterized by:

3. The flight guidance device according to claim 2, The reward function includes, as a variable, a degree of arrival of the aircraft to a position in the first unit space corresponding to a position of the communication area of ​​the first wireless communication method. A flight guidance device characterized by:

4. The flight guidance device according to claim 3, The learning model is a neural network model including an input layer, a hidden layer, and an output layer; the learning unit provides the current position in the unit space as an input to the neural network model, performs calculations on the neural network model, and outputs a first estimate of an action value function that represents an expectation of a cumulative value of the future reward to be obtained when the aircraft takes each action related to movement in a predetermined direction relative to a traveling direction as a course to be followed next from the current position in the unit space; The learning unit further provides a next position in the unit space reached by the aircraft as an input to the neural network model, performs calculations on the neural network model, and outputs a second estimated value of the action-value function; the learning unit learns weight parameters of the neural network model so that the first estimated value becomes a target value calculated from the second estimated value; The second storage unit stores the learned weight parameters. A flight guidance device characterized by:

5. The flight guidance device according to claim 1, The first acquisition unit acquires the position of the unit space in which the aircraft is currently located via a core network of a second wireless communication system that covers the widest range of the movement space among the different wireless communication systems. A flight guidance device characterized by:

6. The flight guidance device according to claim 5, The flight guidance unit transmits instructions regarding the next course to the aircraft via a user plane function included in the core network of the second wireless communication system. A flight guidance device characterized by:

7. A flight guidance method for controlling a course of an aircraft from an initial position to a position of a communication area of ​​a specified first wireless communication system in a moving space of the aircraft in which communication areas of different wireless communication systems are superimposed, the method comprising: a first storage step of storing, in a first storage unit, identification information of a unit space obtained by dividing the movement space into a plurality of spaces, a position of the unit space, and information indicating a wireless communication method of a communication area covering the unit space in association with each other; A first acquisition step of acquiring a position of the unit space where the aircraft is currently located as a current position of the aircraft; a determination step of determining a next course to be taken by the aircraft from the current unit space position acquired as the current position of the aircraft in the first acquisition step, based on a course plan for the aircraft to take sequentially from each unit space position, learned using a learning model; a flight guidance step of instructing the flying object on the course to be followed next, the course being determined in the determination step; A flight guidance method comprising:

8. The flight guidance method according to claim 7, a learning step of applying a reward function to an estimation result of a course that the aircraft should take from each unit space position to a first unit space position corresponding to the position of the communication area of ​​the first wireless communication method from the current unit space position, updating the course so as to maximize a reward for the aircraft to reach the first unit space position, and learning a course plan for the aircraft to take from each unit space position using the learning model; a second storage step of storing the course plan learned in the learning step in a second storage unit; Equipped with The determining step reads out the course plan from the second storage unit and determines the course that the aircraft should take next. A flight guidance method comprising:

9. The flight guidance method according to claim 8, The reward function includes, as a variable, a degree of arrival of the aircraft to a position in the first unit space corresponding to a position of the communication area of ​​the first wireless communication method. A flight guidance method comprising:

10. The flight guidance method according to claim 9, The learning model is a neural network model including an input layer, a hidden layer, and an output layer; The learning step includes providing the current position in the unit space as an input to the neural network model, performing calculations on the neural network model, and outputting a first estimate of an action value function that represents an expectation of a cumulative value of the future reward to be obtained when the aircraft takes each action related to movement in a predetermined direction relative to a traveling direction as a course to be followed next from the current position in the unit space, The learning step further includes providing a next unit space position of the aircraft as an input to the neural network model, performing a calculation on the neural network model, and outputting a second estimated value of the action-value function; the learning step includes learning weight parameters of the neural network model so that the first estimated value becomes a target value calculated from the second estimated value; The second storage step stores the learned weight parameters in the second storage unit. A flight guidance method comprising:

11. The flight guidance method according to claim 7, The first acquisition step acquires the position of the unit space in which the flying object is currently located via a core network of a second wireless communication system that covers the widest range of the moving space among the different wireless communication systems. A flight guidance method comprising:

12. The flight guidance method according to claim 11, The flight guidance step includes transmitting instructions regarding the next course to the aircraft via a user plane function included in the core network of the second wireless communication system. A flight guidance method comprising:

Images