Monitoring plan creation device, monitoring system, monitoring plan creation method, and program

Abstract

To provide a monitoring plan creation device that creates action plan information for an unmanned aircraft that enables continuous monitoring by the unmanned aircraft.SOLUTION: A monitoring plan creation device divides monitoring map information corresponding to a monitoring area indicating the entire region to be monitored by an unmanned aircraft into small areas, sets initial values of time intervals for monitoring respective small areas obtained, counts the remaining time relative to the set initial values of the time intervals according to the elapse of time, creates action plan information that determines a movement route of the unmanned aircraft so that small areas in which the remaining time is shorter are prioritized for monitoring, and sets the initial values to the remaining time in small areas corresponding to the region monitored by the unmanned aircraft when the unmanned aircraft has moved based on the action plan information.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a monitoring plan creation device, a monitoring system, a monitoring plan creation method, and a program.

Background Art

[0002] In recent years, opportunities to perform operations such as monitoring and inspection using drones such as so-called drones have been increasing. By using drones, it is possible to automate operations, and there is an advantage that operations can be performed in dangerous places or places where humans cannot enter. For example, in infrastructure facilities such as power plants, there is a need to continuously monitor a wide area of the facility day and night, and autonomous constant monitoring by drones is required.

[0003] Patent Document 1 discloses a technique in which a monitoring target area is divided into a plurality of small areas, each small area is monitored by a drone, a high priority is given to an unmonitored small area, and the drone is moved in the order of the small areas with high priority to perform monitoring. Patent Document 2 discloses control for changing the importance of actions according to changes in the situation and causing each drone to change its actions according to the situation when performing monitoring or the like with a plurality of drones. Patent Document 3 discloses control in which each drone autonomously selects an operation that optimizes the whole by the coordinated operation of a plurality of drones.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0005] Patent Documents 1 to 3 propose monitoring techniques using drones and methods for controlling multiple drones, but do not disclose any techniques for continuously monitoring a target area using drones.

[0006] The present disclosure provides a monitoring plan creation device, a monitoring system, a monitoring plan creation method, and a program that can solve the above-mentioned problems. [Means for solving the problem]

[0007] According to another aspect of the present disclosure, the monitoring plan creation device divides the monitoring map information, which is a virtual area corresponding to a monitoring area indicating the entire area to be monitored by the unmanned aerial vehicle, into small areas, and creates a monitoring plan for each small area. The time interval in the case of repeatedly monitoring at a predetermined time interval and, as time passes, the initial value of the time interval is set. Obtained by subtracting the elapsed time from a priority setting unit that counts the remaining time; Set a high priority for the areas within the monitoring area corresponding to the small areas in ascending order of the remaining time, and move to the areas within the monitoring area where the priority is set in descending order of the priority and an action plan creation unit that creates action plan information that determines the movement route of the drone, and when the drone moves based on the action plan information and monitors a certain area within the monitoring area, the priority setting unit sets the initial value to the remaining time of the small area corresponding to that area.

[0008] According to another aspect of the present disclosure, a monitoring system includes the above-described monitoring plan creation device and one or more unmanned aerial vehicles that act based on the action plan information.

[0009] According to another aspect of the present disclosure, a monitoring plan creation method includes: A monitoring plan creation method executed by a computer, comprising For surveillance map information, which is a virtual area corresponding to a surveillance area showing the entire area to be monitored by the unmanned aerial vehicle, the surveillance map information is divided into small areas, and each small area is The time interval in the case of repeatedly monitoring at a predetermined time interval and, as time passes, the initial value of the time interval is set. Obtained by subtracting the elapsed time from counting the remaining time; Set a high priority for the areas within the monitoring area corresponding to the small areas in ascending order of the remaining time, and move to the areas within the monitoring area where the priority is set in descending order of the priorityA step of creating action plan information that defines the movement route of the drone, and a step of setting the initial value to the remaining time of the small area corresponding to the area when a certain area within the monitoring area is monitored by the drone when the drone moves based on the action plan information.

[0010] Also, according to one aspect of the present disclosure, the program causes a computer to divide the monitoring map information, which is virtual area information corresponding to the monitoring area to be monitored by the drone, for each small area formed by the division, and set an initial value for the small area, and count the remaining time according to the passage of time for the initial value of the time interval The time interval in the case of repeatedly monitoring at a predetermined time interval of the time interval, Obtained by subtracting the elapsed time from a step of counting the remaining time; Set a high priority for the areas within the monitoring area corresponding to the small areas in ascending order of the remaining time, and move to the areas within the monitoring area where the priority is set in descending order of the priority a step of creating action plan information that defines the movement route of the drone, and a step of setting the initial value to the remaining time of the small area corresponding to the area when a certain area within the monitoring area is monitored by the drone when the drone moves based on the action plan information.

Advantages of the Invention

[0011] According to the above-described monitoring plan creation device, monitoring system, monitoring plan creation method, and program, continuous monitoring by a drone can be performed.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3A

Figure 3B

Figure 4

Figure 5A

Figure 5B

Figure 6

Figure 7A

Figure 7B

Figure 8

Figure 9

Mode for Carrying Out the Invention

[0013] <Embodiment> Hereinafter, the method for controlling a drone of the present disclosure will be described with reference to FIGS. 1 to 9. (Configuration) FIG. 1 is a diagram showing an example of a monitoring system according to an embodiment. The monitoring system 1 includes a monitoring plan creation device 10 and n drones 20-1 to 20-N. The monitoring plan creation device 10 plans a monitoring area (monitoring route) for each of the drones 20-1 to 20-N. The monitoring plan creation device 10 instructs the planned monitoring route (action plan information) to the drones 20-1 to 20-N. The drones 20-1 to 20-N move along the instructed monitoring route and perform monitoring using a camera or the like.

[0014] The drone 20-1 is equipped with a control device 21-1, a monitoring sensor 22-1, a positioning sensor 23-1, and a communication device 24-1. The control device 21-1 controls the overall operation of the drone 20-1. The communication device 24-1 receives the action plan information instructed by the monitoring plan creation device 10. The control device 21-1 moves the drone 20-1 based on the action plan information and performs monitoring using the monitoring sensor 22-1. The monitoring sensor 22-1 is a sensor such as a camera or a temperature sensor, which is used to collect information on the area under surveillance. The communication device 24-1 transmits the information detected by the monitoring sensor 22-1 to the monitoring plan creation device 10. The positioning sensor 23-1 is, for example, a receiver of a satellite positioning system such as GPS (Global Positioning System), an acceleration sensor, a gyro sensor, etc. The communication device 24-1 transmits the position information measured by the positioning sensor 23-1, or information indicating the speed used for calculating the position information and the moving direction of the drone 20-1, etc. to the monitoring plan creation device 10.

[0015] Note that the control device 21-1 is equipped with a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), and the information detected by the monitoring sensor 22-1 and the positioning sensor 23-1 may be processed and lightweighted by this CPU or the like. Also, when the positioning sensor 23-1 is an acceleration sensor or the like, the CPU or the like may calculate the position information, speed movement, moving direction, etc. of the drone 20-1, and transmit this calculation result to the monitoring plan creation device 10 through the communication device 24-1.

[0016] Similarly, the drone 20-2 includes a control device 21-2, a monitoring sensor 22-2, a positioning sensor 23-2, and a communication device 24-2. The drone 20-N includes a control device 21-N, a monitoring sensor 22-N, a positioning sensor 23-N, and a communication device 24-N. The drones 20-1 to 20-N may be different types of drones. For example, the drone 20-1 may be a UAV (Unmanned Aerial Vehicle), the drone 20-2 may be an AUV (Autonomous Underwater Vehicle), the drone 20-3 may be a drone that travels or walks on the ground, and so on. Also, the types of the monitoring sensors 22-1 to 22-N and the types of the positioning sensors 23-N included in the drones 20-1 to 20-N may be different. Hereinafter, when there is no particular need for distinction, they may be described as the drone 20, the control device 21, the monitoring sensor 22, the positioning sensor 23, and the communication device 24. There is no limit to the number of drones 20, and it may be one or a plurality of them.

[0017] Next, the functions and configurations of the monitoring plan creation device 10 will be described. The monitoring plan creation device 10 includes an input reception unit 11, a sensor information acquisition unit 12, a monitored area identification unit 13, a priority setting unit 14, an action plan creation unit 15, an instruction unit 16, a monitoring map creation unit 17, an output unit 18, a storage unit 19, and a timer 1A. The input reception unit 11 receives an instruction operation by the user and input of setting information. The sensor information acquisition unit 12 acquires information detected by the monitoring sensor 22 or the positioning sensor 23 from the drone 20, or information processed and calculated by the control device 21 based on them.

[0018] The monitored area specifying unit 13 specifies the area monitored by the drone 20. For example, the monitored area specifying unit 13 may determine the area through which the drone 20 has passed as the monitored area, or analyze the image captured by the camera (monitoring sensor 22). For example, when the monitoring cannot be sufficiently captured due to being blocked by a structure or the like, it is determined that the area shown in the image is not a monitored area, and the area excluding the area determined not to be monitored from the area through which the drone 20 has passed may be specified as the monitored area. Alternatively, based on the map information of the monitoring area 100, a route suitable for monitoring may be defined in advance (for example, a route where the shooting of the monitoring target is not hindered by the camera due to topographical conditions such as mountains and buildings). Even if the drone is moving along the monitoring route generally defined in the action plan information, if it deviates from the route suitable for monitoring, it is determined that the area is not a monitored area, and the area excluding that area from the area through which the drone 20 has passed may be specified as the monitored area.

[0019] The priority setting unit 14 sets the priority for each small area when the entire area to be monitored is divided into a plurality of small areas. Specifically, the priority setting unit 14 sets a monitoring time interval for each small area as the monitoring priority. Here, refer to FIG. 2. On the left side of FIG. 2, a monitoring area 100 showing the area to be actually monitored is shown, and on the right side of FIG. 2, a monitoring map 200 obtained by modeling the monitoring area 100 to constantly monitor the monitoring area 100 is shown. The monitoring area 100 is a place to be monitored, such as a plant facility, an urban area, the sea, a mountain, a river, or an area including a plurality of them, for monitoring, security, inspection, search, etc. The monitoring map 200 is digital data of a virtual area corresponding to the area of the monitoring area 100, which simulates the shape of the monitoring area 100, and is divided into small areas as shown in the figure. In the example of FIG. 2, it is divided into a grid so that the area of each small area is equal, but it is not limited to this, and a plurality of small areas are provided so as to cover all the areas to be monitored for any shape and any area. For example, the non-monitoring target area 101 within the monitoring area 100 is an area to be excluded from the monitoring target. In such a case, the corresponding area in the monitoring map 200 is also excluded from the monitoring target (the shaded part 201). The monitoring map 200 may be created by the user and input to the monitoring plan creation device 10, or may be created by the monitoring map creation unit 17 as described later. The priority setting unit 14 sets a monitoring time interval (monitoring priority) for each small area of the monitoring map 200.

[0020] Here, one small area shown in FIG. 2 is called a cell, and the cell in row A and column 1 is described as cell (1, A). Then, the number "10" in cell (1, A) indicates that the area corresponding to cell (1, A) within the monitoring area 100 needs to be monitored every 10 minutes. Similarly, the number "5" in cell (2, B) indicates that the area within the monitoring area 100 corresponding to this cell needs to be monitored every 5 minutes, and the number "2" in cell (3, B) indicates that the area corresponding to this cell within the monitoring area 100 needs to be monitored every 2 minutes. That is, the smaller the number in each cell, the more frequently it needs to be monitored, indicating a higher monitoring priority. For example, the user inputs information on the monitoring time interval set for each cell into the monitoring plan creation device 10 together with the monitoring map 200. The priority setting unit 14 sets an initial value of the monitoring time interval for each cell of the monitoring map 200 according to the information on the monitoring time interval input by the user. The monitoring map 200 on the right side of FIG. 2 shows a state where the initial value of the monitoring time interval is set for each cell.

[0021] Next, refer to FIG. 3A. The priority setting unit 14 updates the monitoring time interval set in the monitoring map 200 according to the passage of time. FIG. 3A shows the monitoring time interval of each cell 1 minute after the start of monitoring. When 1 minute has passed since the start of monitoring, the priority setting unit 14 subtracts "1" from the monitoring time interval of each cell. This corresponds to, for example, the fact that cell (1, A) needs to be monitored every 10 minutes, and the remaining time until the time limit is 9 minutes. Similarly, the priority setting unit 14 updates the time interval of each cell with the monitoring time interval set to 10 minutes to "9". Also, the priority setting unit 14 updates the monitoring time interval of each cell with the initial value of the monitoring time interval set to 5 minutes to "4". The priority setting unit 14 updates the monitoring time interval of each cell with the initial value of the monitoring time interval set to 2 minutes to "1".

[0022] The priority setting unit 14 counts down the monitoring time interval as time elapses, and resets the monitoring time interval of the small area monitored by the drone 20. For example, assume that the area 202 is monitored by one or more drones 20 between 1 minute and 2 minutes after the start of monitoring. Then, the priority setting unit 14 resets the monitoring time interval of each cell included in the area 202 to the initial value. Specifically, for example, since the initial values of the cell (4, B) and the cell (5, F) are "2" and "5" respectively, the priority setting unit 14 updates the monitoring time interval of the cell (4, B) to "2" and the monitoring time interval of the cell (5, F) to "5" regardless of the passage of time. This corresponds to the situation where, for example, if an area that must be monitored once every 5 minutes is monitored once, the requirement is satisfied if it is monitored again by 5 minutes after that. The monitoring map 200 at 2 minutes after the start of monitoring is shown in FIG. 3B. As described above, the value of the monitoring time interval of the area 202 is reset to the initial value, and the values of the monitoring time intervals of the other cells are 1 less than the values in FIG. 3A. In this way, the priority setting unit 14 continuously performs the process of counting down the value of the monitoring time interval set for each cell according to the passage of time and resetting it to the initial value when monitoring is performed.

[0023] The action plan creation unit 15 calculates, for each drone 20, the area of responsibility up to a predetermined time in the future for which that drone 20 is responsible for monitoring. Based on the position of the drone 20, the value of the monitoring time interval for each cell, and the moving direction and moving speed of the drone 20, the action plan creation unit 15 calculates the area (monitoring route) for which each drone 20 is responsible for monitoring. Fig. 4 shows the situation where three drones 20-1 to 20-4 are monitoring. The arrows attached to the drones 20-1 to 20-4 indicate the moving direction and moving speed at that time. The moving direction and moving speed of each drone 20 can be calculated, for example, from the position information of the drones 20-1 to 20-4 at each moment obtained by the sensor information acquisition unit 12. The action plan creation unit 15, for example, (1) determines a goal (for example, any one of the cells with the smallest value of the monitoring time interval) based on the monitoring priority, (2) calculates the route to that goal by means of a graph search algorithm or the like, and while changing the goal and the route, repeats (1) and (2) to create an action plan for each drone 20. The action plan creation unit 15, for example, evaluates the searched route based on (a) the priority (value of the monitoring time interval) set for each cell and (b) the movement cost of the drone 20, and selects the route with a high evaluation.

[0024] (a) The priority set for each cell refers to the latest monitoring time interval of each cell. For example, when comparing a cell (1, B) with an initial value of "10" and a cell (2, B) with an initial value of "5", initially, the priority of cell (2, B) is higher. However, as time passes, if monitoring is performed on cell (2, B) but not on cell (1, B), a situation may occur where, as shown in FIG. 4, the value of cell (1, B) becomes "2" and the value of cell (2, B) becomes "4". In this situation, it means that the priority of cell (1, B) is higher than that of cell (2, B). The action plan creation unit 15 calculates a goal and the path (monitoring path) to that goal so that cells with higher priority can be monitored preferentially. For example, if there are three cells with high priority lined up immediately near the moving direction of the drone 20-4, the action plan creation unit 15 calculates these three cells as the monitoring path of the drone 20-N. More specifically, the action plan creation unit 15 calculates waypoints (way point: target passing points) WP1 to WP8 such that the drone 20-4 passes through these three areas. When the waypoints WP1 to WP8 are given, the control device 21-4 moves the drone 20-4 along these waypoints. During this movement, the camera (monitoring sensor 22-N) takes pictures of the surrounding conditions, and the communication device 24-4 transmits the captured images to the monitoring plan creation device 10, thereby monitoring these three cells. That is, when the monitoring path is calculated, that monitoring path directly becomes the area in charge of monitoring.

[0025] (b) The movement cost of the drone 20 is a value obtained by quantifying one or more of the distance from the current position to the target cell, the time required for movement, the energy consumed for movement, the ease of movement, etc. For example, in the case of the drone 20-1 in Fig. 4, there are cells (6, D) with a high priority (remaining time = 1 minute) in the adjacent cell in the movement direction as viewed from the current position. Also, there is a cell (5, D) with a high priority adjacent to it. In such a case, the action plan generation unit 15 calculates the cost when moving the drone 20-1 from the cell (6, E) to the cells (6, D) and (5, D) in this order using a predetermined calculation formula. The action plan generation unit 15 also calculates the movement cost when the drone 20-1 moves along other monitoring routes. Then, the action plan generation unit 15 calculates, for example, (a) the points based on the priority set for each cell and (b) the weighted sum of the points based on the movement cost of the drone 20, and selects a good monitoring route based on this value.

[0026] (a) The points based on the priority set for each cell are, for example, values calculated by assigning points according to the priority. For example, (a) the points based on the priority set for each cell are calculated by adding 100 points for each cell with a monitoring time interval value of "0" monitored, adding 50 points for each cell with a time interval value of "1" monitored, ···, and not adding points even if cells with a time interval value of "5" or more are monitored. Alternatively, in addition to such a method, a penalty (for example, -50 points for each cell with a value of "0") may be imposed according to the number of remaining cells with a monitoring time interval value of "1" or "0" when monitoring is performed along the monitoring route, and the points may be calculated.

[0027] (b) The points based on the movement cost of the drone 20 are values obtained by converting, according to a predetermined rule, for example, the movement time to the goal, the energy consumption required for movement considering the influence of tidal currents, wind, etc. For example, the greater the movement time or the energy consumption required for movement, the smaller the points it is converted to, and the smaller the movement time or the energy consumption required for movement, the larger the points it is converted to. For example, in the exact opposite direction of the movement direction of the drone 20-2 in FIG. 4, there are cells (5, D) with a monitoring time interval of "1". Although the straight-line distance between the drone 20-2 and the current cell (4, D) is short, if there are constraints such as a large cost required for the drone 20-2 to turn or it cannot turn directly backward at all, the points based on the movement cost of the drone 20 when moving the drone 20-2 from cell (4, D) to cell (5, D) will be a small value (large movement cost). Also, for example, in the case of the drone 20-3 shown in FIG. 4, there is a cell with a monitoring time interval of "1" in the diagonally adjacent cell (3, B), but considering the movement direction and movement speed of the drone 20-3, it is necessary to brake and decelerate to move from the current cell (2, A) to cell (3, B). The action plan creation unit 15 may give a penalty for this deceleration and calculate the points based on the movement cost of the drone 20. As a result of calculating the points considering the brake penalty, for example, instead of cell (3, B) (remaining time "1"), a cell (6, B) (remaining time "1") at a farther position may be targeted, and for example, a monitoring path that moves in the order of the current cell (2, A) to cell (3, A), cell (4, A), cell (5, B), cell (6, B) may be calculated as a path that can obtain good points.

[0028] For example, the action plan creation unit 15 may calculate P for each drone 20 by the following formula (1) and determine the movement path with the best P as the monitoring path. α and β are weighting coefficients and are values that can be arbitrarily set by the user. P = α × "Points based on the priority set for each cell" + β × "Points based on the movement cost of the drone 20"... (1) Alternatively, the action plan generation unit 15 may calculate the movement routes of each drone 20 when the total of P calculated for each drone 20 is optimal as the overall optimal solution, and determine the movement route calculated at this time as the monitoring route. Note that the length of the monitoring route calculated at one time (the number of cells to move) may change each time, or may be different for each drone 20.

[0029] Also, for example, when the monitoring area 100 straddles land and sea, and the sea is monitored by an AUV, and the land area is monitored by a robot or UAV that travels on the ground, the action plan generation unit 15 divides all cells into sea and land, calculates the monitoring route by the AUV within the range of cells corresponding to the sea, and calculates the monitoring route by the UAV within the range of cells corresponding to the land. Further, the action plan generation unit 15 may calculate the monitoring route based on the map information corresponding to the monitoring area 100. For example, when monitoring from the air by a UAV, if there is a position where the monitoring target cannot be photographed due to the influence of a structure or the like, the monitoring route is corrected so that the positional relationship between the drone 20 and the monitoring target becomes a positional relationship suitable for monitoring (photographing), and the corrected monitoring route may be used as the final monitoring route. When the action plan generation unit 15 determines the monitoring route using the monitoring map 200, it converts the monitoring route into action plan information (for example, waypoints indicating the monitoring route) for actually instructing the drone 20.

[0030] The instruction unit 16 transmits the action plan information (for example, waypoints indicating the monitoring route) calculated by the action plan generation unit 15 for each drone 20 to the corresponding drone 20.

[0031] The monitoring map creation unit 17 creates a monitoring map 200. For example, the user inputs map information of the monitoring area 100 (e.g., a map of an urban area or a facility), information indicating where to be monitored (e.g., a road), and the size of one cell or the number of cells (number of divisions) into the monitoring plan creation device 10. Then, the monitoring map creation unit 17 generates map information of a planar image simulating the shape of the monitoring area 100 based on the input map information, divides the area into the specified number of cells, and creates the monitoring map 200. Also, the monitoring map creation unit 17 classifies the area to be monitored into a road and other parts, and sets the parts other than the road as non-monitoring target areas.

[0032] Also, the monitoring map creation unit 17 may receive a designation of the range of the monitoring area from the user and create the monitoring map information for the received range. For example, a setting such as "monitor an area 3 km square 15 km ahead" is input into the monitoring plan device 10 with respect to the position where a certain drone 20 currently exists. Then, the monitoring map creation unit 17 generates map information corresponding to an area 3 km square 15 km ahead in the moving direction of the drone 20, for example, from the current position of the drone 20, divides the area into the specified number of cells, and creates the monitoring map 200. Also, the monitoring map creation unit 17 calculates the position information of, for example, the four corners of the created monitoring map 200 based on the position information of the current position of the drone 20 and the instruction information of "an area 3 km square 15 km ahead in the moving direction from the current position", associates the positional relationship between the monitoring map 200 and the actual monitoring area 100, and records it in the storage unit 19.

[0033] Alternatively, instead of dividing the cells based on the number of divisions input by the user, the monitoring map creation unit 17 may divide the monitoring area into cells of a size suitable for monitoring. For example, in the case of a road, cells may be created according to the road width. An example of the monitoring map 200 created by the monitoring map creation unit 17 is shown in FIG. 5A. The shaded area of the monitoring map 200 is, for example, a non-monitoring symmetric area where buildings or the like are built. Also, for the roads to be monitored, wide roads are divided into two cells, and narrow roads are divided corresponding to one cell. The output unit 18 outputs the monitoring map 200 to a display device. The user can set the monitoring time interval for each cell by looking at the displayed monitoring map 200.

[0034] In addition, the monitoring map creation unit 17 may set the monitoring time interval for each cell. For example, the user inputs the setting conditions for the monitoring time interval together with map information and information on the monitoring target (road). In the example of FIG. 5A, the user inputs by associating the scale of the road (road width or number of lanes) with the monitoring time interval, or by associating the road name, town name, etc. with the monitoring time interval. Then, the monitoring map creation unit 17 sets the monitoring time interval for the corresponding cell. FIG. 5B shows an example of the case where the monitoring time interval "5" is set for a wide road and the monitoring time interval "10" is set for a narrow road. Note that the setting of the monitoring time interval may be made changeable according to the time zone. For example, the user inputs setting information to the monitoring plan creation device 10 to set the monitoring time interval in a time zone with no human traffic such as at night to be shorter than that in the daytime, and the monitoring map creation unit 17 may create the monitoring map 200 separately for day and night based on this setting information.

[0035] The output unit 18 outputs various information such as the monitoring map 200 to a display device or an electronic file. The storage unit 19 stores various information such as the monitoring map 200. The timer 1A measures time. The priority setting unit 14 updates the monitoring time interval for each cell based on the time measured by the timer 1A.

[0036] Note that the monitoring plan creation device 10 may be configured by a plurality of computers. For example, the instruction unit 16 may be implemented on another computer as a repeater (gateway), and when transmitting the action plan information to the unmanned aircraft 20, this computer may be responsible for sorting out various information transmitted and received with the unmanned aircraft 20.

[0037] (Operation) Next, the flow of continuous monitoring processing by the monitoring system 1 will be described. FIG. 6 is a flowchart showing an example of the operation of the monitoring system according to the embodiment. First, a monitoring map 200 is created (step S1). Next, a monitoring time interval for each cell of the monitoring map 200 is set (step S2). For example, the user creates a monitoring map 200 corresponding to the monitoring area 100 using spreadsheet software or the like, and inputs information in which the monitoring time interval is set for each cell into the monitoring plan creation device 10. The input reception unit 11 receives this input and records the monitoring map 200 with the monitoring time interval set in the storage unit 19. Alternatively, after the user inputs the monitoring map 200 corresponding to the monitoring area 100 into the monitoring plan creation device 10, the output unit 18 displays the input monitoring map 200 on the display device. Then, the user sets the monitoring time interval for each cell of the displayed monitoring map 200. The input reception unit 11 acquires the set monitoring time interval, and the priority setting unit 14 sets the input monitoring time interval to the corresponding cell, and records the monitoring time interval for each cell in the storage unit 19. Alternatively, as described with reference to FIGS. 5A and 5B, the monitoring map creation unit 17 may create the monitoring map 200 and record the created monitoring map 200 in the storage unit 19.

[0038] Next, the user sets the information of the unmanned aircraft (step S3). For example, the number of unmanned aircraft 20 used for monitoring, the type, and the setting of the area where monitoring is possible are performed. For example, 2 UAVs, 1 AUV, and for the AUV, a range such as the sea or a river is specified. The input reception unit 11 receives the input of the set information and records this information in the storage unit 19.

[0039] Next, upon the user's instruction, the monitoring system 1 starts monitoring (step S4). The monitoring system 1 performs the process A in step S5 described below and the process B in steps S6 to S10 in parallel.

[0040] The priority setting unit 14 counts down the monitoring time intervals of the cells of the monitoring map 200 at a predetermined time interval (step S5). For example, the priority setting unit 14 subtracts 1 from the value of the monitoring time interval of each cell every minute based on the time measured by the timer 1A. The countdown interval does not have to be every minute. For example, the priority setting unit 14 may subtract 10 seconds from the value of the monitoring time interval of each cell every 10 seconds. The priority setting unit 14 continuously executes this operation during monitoring (process A).

[0041] On the other hand, in process B, the action plan creation unit 15 calculates the action plan for each drone 20 (S6). For example, the action plan creation unit 15 sets the cell with a high priority as the goal based on the monitoring time interval of each cell. The action plan creation unit 15 searches for various paths to reach the goal using a graph search algorithm or the like. Then, the action plan creation unit 15 selects the monitoring path with the maximum value of P using, for example, the above formula (1).

[0042] Next, the instruction unit 16 transmits the action plan information for each drone 20 calculated in step S6 to the corresponding drone 20 and instructs the next monitoring path (step S7). In the drone 20, the communication device 24 acquires the action plan information, and the control device 21 controls the movement of the drone 20 based on the action plan information. In the moving drone 20, the monitoring sensor 22 detects the information used for monitoring. Also, the positioning sensor 23 detects the position information during movement. The communication device 24 transmits the information detected by these sensors and the detection time to the monitoring plan creation device 10.

[0043] In the monitoring plan creation device 10, the sensor information acquisition unit 12 acquires sensor information (step S8). The sensor information is, for example, an image captured by a camera (monitoring sensor 22) or the position information of the drone 20 detected by a GPS receiver (positioning sensor 23). The sensor information acquisition unit 12 records the acquired sensor information in the storage unit 19 together with its detection time.

[0044] Next, the monitored area specifying unit 13 specifies the monitored area (step S9). For example, the monitored area specifying unit 13 estimates the actual movement route of the drone 20 from the time-series position information of the drone 20 acquired in step S8, and specifies as the monitored area the cells (including the virtual movement route cells) on the monitoring map 200 that overlap with the virtual movement route corresponding to the estimated movement route. Here, refer to FIG. 7A. The dashed arrow 203 in FIG. 7A is an example of the action plan information of the drone 20-1 calculated by the action plan creation unit 15. The solid arrow 204 in FIG. 7A is the actual movement route of the drone 20-1. The monitored area specifying unit 13 specifies the cells (6, D), cell (5, C), cell (4, C), and cell (3, C) through which the solid arrow 204 passes, rather than the dashed arrow 203, as the monitored area.

[0045] Further, for example, in addition to the position information, the monitored area specifying unit 13 determines whether the state of the area to be monitored is shown in the image acquired in step S18 based on the image, and excludes from the monitored area the cells in which a predetermined number or more of images in which the information necessary for monitoring is not shown are captured from among the cells on the actual movement path. For example, when images in which the object to be monitored cannot be captured because an obstacle blocks the camera's field of view, or the object to be monitored is covered by a structure or the like and cannot be captured, are captured at a predetermined ratio or more during the movement in a certain area, the monitored area specifying unit 13 excludes the cell corresponding to that area from the monitored area. Referring to FIG. 7A again. The dashed arrow 205 in FIG. 7A is the monitoring path of the drone 20-2 calculated by the action plan generation unit 15. The solid arrow 206 in FIG. 7A is the actual movement path of the drone 20-2. In this example, the drone 20-2 is moving as per the initial monitoring path. However, assuming that an obstacle occurs in the cell (4, B) and an image with the required quality cannot be captured. Then, the monitored area specifying unit 13 specifies the cells (3, B), (5, B), and (6, B) excluding the cell (4, B) as the monitored area.

[0046] Further, for example, when the difference (for example, the distance between the dashed arrow 205 and the solid arrow 206) between the monitoring path (for example, the dashed arrow 205) indicated by the action plan information and the actual movement path (for example, the solid arrow 206) of the drone 20-2 is within the allowable range, the cell corresponding to the actual movement path may be specified as the monitored area. This method can be used, for example, when the monitoring path is calculated as a path suitable for shooting in consideration of blind spots of the camera and the like.

[0047] Next, the priority setting unit 14 resets the monitoring time interval of the monitored area (step S10). The priority setting unit 14 resets the monitoring time interval of the monitored area specified in step S9 to the initial value. Refer to FIG. 7B. With respect to the movement of the drone 20-1 shown by the solid arrow 204 in FIG. 7A, the priority setting unit 14 initializes the values of the cells (6, D), (5, C), (4, C), and (3, C) to "2". Also, with respect to the movement of the drone 20-2 shown by the solid arrow 206 in FIG. 7A, the priority setting unit 14 initializes the values of the cells (3, B), (5, B), and (6, B) to "2". Note that the values of the monitoring time intervals of the cells (4, B) and (5, D) that are not subject to reset have been counted down to "0" by the concurrently executed process A (step S5).

[0048] Next, the monitoring plan creation device 10 determines whether to continue monitoring (step S11). For example, when the user inputs an instruction to end the monitoring to the monitoring plan creation device 10, the monitoring plan creation device 10 ends the monitoring. If there is no instruction to end the monitoring by the user, the monitoring plan creation device 10 continues the monitoring. The monitoring plan creation device 10 repeats the processes of steps S5 to S10.

[0049] As described above, according to the present embodiment, a monitoring time interval is set as the monitoring priority for each cell of the monitoring map 200, this value is counted down as time passes, and the drone 20 is controlled to monitor the area corresponding to the cell until this value becomes "0". Thereby, it is possible to preferentially monitor areas with a high monitoring priority. Also, once an area has been monitored, the monitoring time interval of the cell corresponding to that area is set to the initial value. Thereby, even if monitoring has been completed once, there is an obligation to monitor the area again within the set monitoring time interval, and by repeating this, continuous monitoring is automatically executed. Also, the entire monitoring area 100 is divided into small areas on the monitoring map 200, and a monitoring time interval is set for each small area. Then, for each small area, by continuing the control of counting down and resetting the monitoring time interval, and moving the drone 20 to execute monitoring until the monitoring time interval becomes "0", it is possible to perform continuous and non-leaky constant monitoring of the entire monitoring area 100.

[0050] Also, when monitoring is performed by a plurality of drones 20, the monitored areas of the plurality of drones 20 and the monitoring time intervals of each cell are managed in a single common monitoring map 200 for all drones, and based on this monitoring map 200, action plan information for the plurality of drones 20 is created. Thereby, for example, even if some of the drones 20 deviate from the monitoring action due to tracking or the like, the remaining drones 20 can cover the monitoring of the drones 20 that have deviated from the monitoring action. For example, in the case of monitoring by a general drone, as shown in FIG. 8, a monitoring area is defined for each drone, and monitoring is performed within that range. In this case, for example, when the drone 20-2 discovers an abnormal object during monitoring and starts tracking the object, the monitoring of the left half of the monitoring area 100 is interrupted. In contrast, according to the present embodiment, since the monitoring area is not originally defined for each drone 20, even if the drone 20-2 deviates from the monitoring action, the remaining drone 20-1 performs monitoring based on the monitoring time interval set for each cell. Therefore, continuous monitoring can be continued without creating an area where monitoring is interrupted.

[0051] FIG. 9 is a diagram showing an example of the hardware configuration of the monitoring system according to the embodiment. The computer 900 includes a CPU 901, a main storage device 902, an auxiliary storage device 903, an input / output interface 904, and a communication interface 905. The above-described monitoring plan creation device 10 and control device 21 are implemented in the computer 900. And each of the above-described functions is stored in the auxiliary storage device 903 in the form of a program. The CPU 901 reads the program from the auxiliary storage device 903 and expands it in the main storage device 902, and executes the above processing according to the program. Further, the CPU 901 secures a storage area in the main storage device 902 according to the program. Further, the CPU 901 secures a storage area in the auxiliary storage device 903 for storing data being processed according to the program.

[0052] Note that a program for realizing all or part of the functions of the monitoring plan creation device 10 and the control device 21 may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed to perform processing by each functional unit. Here, the "computer system" is assumed to include hardware such as an OS and peripheral devices. Further, the "computer system" includes a homepage providing environment (or display environment) if it uses a WWW system. Further, the "computer-readable recording medium" refers to a portable medium such as a CD, DVD, USB, or a storage device such as a hard disk built in a computer system. Further, when this program is distributed to the computer 900 via a communication line, the computer 900 that has received the distribution may expand the program in the main storage device 902 and execute the above processing. Further, the above program may be for realizing a part of the above-described functions, and may further be realized in combination with a program already recorded in the computer system for the above-described functions. Further, the functions of the monitoring plan creation device 10 may be distributed and implemented in a plurality of computers 900.

[0053] As described above, although some embodiments according to the present disclosure have been described, all of these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, as well as in the invention described in the claims and the scope equivalent thereto. For example, in the embodiment, it was decided to count down from the set monitoring time interval. However, it may be possible to count up from an initial value of 0 toward the monitoring time interval to count the remaining time with respect to the set monitoring time interval.

[0054] <Appendix> The monitoring plan creation device, monitoring system, monitoring plan creation method, and program described in each embodiment are understood as follows, for example.

[0055] (1) The monitoring plan creation device 10 according to the first aspect divides the monitoring map information (200), which is a virtual area corresponding to the monitoring area indicating the entire area to be monitored by the drone, into small areas (cells), and sets an initial value of the time interval (monitoring time interval) for which each small area should be monitored (Fig. 2). The priority setting unit 14 counts the remaining time until the monitoring deadline based on the initial value of the time interval as time elapses (Fig. 3A, step S5). The action plan creation unit 15 creates action plan information that determines the movement path of the drone so as to preferentially monitor the small areas with less remaining time. The priority setting unit 14 sets the initial value to the remaining time of the small area corresponding to the area when the drone moves based on the action plan information and the area within the monitoring area is monitored by the drone (Fig. 3B, step S10). Thereby, it is possible to continuously execute the monitoring process of preferentially monitoring the areas with high priority while monitoring the entire monitoring area. Thereby, it becomes possible to constantly monitor a wide area.

[0056] (2) The monitoring plan creation device according to the second aspect is the monitoring plan creation device of (1), and in addition to prioritizing the small area with less remaining time, the action plan creation unit prioritizes creating the action plan information such that the movement cost of the drone to the small area (a value quantified by combining one or more of the movement distance, movement time, energy consumption required for movement, ease of movement, etc.) is reduced. Thereby, it is possible to efficiently patrol and monitor areas with high monitoring priority.

[0057] (3) The monitoring plan creation device according to the third aspect is the monitoring plan creation device of (1) to (2), and further includes a monitored area specifying unit 13 that specifies the small area corresponding to the area monitored by the drone as a monitored area. The priority setting unit 14 sets the initial value for the remaining time of the small area corresponding to the area specified by the monitored area specifying unit. Thereby, it is possible to surely manage only the areas that have actually been monitored as monitored areas, preventing situations where an area that was thought to have been monitored was actually not monitored, and improving monitoring accuracy.

[0058] (4) The monitoring plan creation device according to the fourth aspect is the monitoring plan creation device of (3), and the monitored area specifying unit specifies the small area corresponding to the position information as a monitored area based on the time-series position information of the drone. Thereby, it is possible to set the places actually patrolled by the drone as monitored areas.

[0059] (5) The monitoring plan creation device according to the fifth aspect is the monitoring plan creation device of (3) to (4), and the monitored area specifying unit analyzes the image captured by the drone, and when the monitoring target is included in the image, specifies the small area corresponding to the position where the image was captured as a monitored area. Only when the object to be monitored can actually be photographed, the location is regarded as monitored, so that the monitoring accuracy can be further improved.

[0060] (6) The monitoring plan creation device according to the sixth aspect is the monitoring plan creation device of (3) to (5), wherein the monitored area specifying unit calculates the movement route of the drone based on the time-series position information of the drone, and when the difference between the calculated movement route and a predetermined movement route is within an allowable range, specifies the small area corresponding to the calculated movement route as the monitored area. By setting in advance a monitoring route suitable for monitoring (a route that can capture the target well with a camera) and regarding it as monitored only when monitoring is performed while moving along this route, the quality of monitoring can be ensured.

[0061] (7) The monitoring plan creation device according to the seventh aspect is the monitoring plan creation device of (1) to (6), further comprising a monitoring map creation unit that acquires map information of the monitoring area and creates the monitoring map information based on the map information. By automating the creation of the monitoring map information, the monitoring process can be made more efficient.

[0062] (8) The monitoring plan creation device according to the seventh aspect is the monitoring plan creation device of (1) to (7), further comprising a monitoring map creation unit that accepts the designation of the range of the monitoring area and creates the monitoring map information for the accepted range. By simply designating the range of the monitoring target area, the creation of the monitoring map information can be automated, and the monitoring process can be made more efficient.

[0063] (9) The monitoring system according to the ninth aspect includes the monitoring plan creation device of (1) to (8) and one or more drones that act based on the action plan information. Thereby, an appropriate number of drones can be introduced according to the size of the monitoring area, etc., and continuous monitoring can be executed.

[0064] (10) The method for creating a monitoring plan according to the tenth aspect divides the monitoring map information, which is a virtual area corresponding to the monitoring area indicating the entire area to be monitored by the drone, and sets an initial value of the time interval for monitoring each small area thus formed (step S2). As time passes, the remaining time until the monitoring deadline based on the initial value of the time interval is counted (step S5). The method further includes creating action plan information for determining the movement path of the drone so as to preferentially monitor the small areas with less remaining time (step S6), and when the drone moves based on the action plan information and a certain area within the monitoring area is monitored by the drone, setting the initial value to the remaining time of the small area corresponding to the area (step S10).

[0065] (11) The computer 900 according to the eleventh aspect is caused to execute steps of dividing the monitoring map information, which is a virtual area corresponding to the monitoring area indicating the entire area to be monitored by the drone, and setting an initial value of the time interval for monitoring each small area thus formed, counting the remaining time until the monitoring deadline based on the initial value of the time interval as time passes, creating action plan information for determining the movement path of the drone so as to preferentially monitor the small areas with less remaining time, and when a certain area within the monitoring area is monitored by the drone when the drone moves based on the action plan information, setting the initial value to the remaining time of the small area corresponding to the area.

Explanation of Reference Numerals

[0066] 1 ··· Monitoring system 11 ··· Input reception unit 12 ··· Sensor information acquisition unit 13 ··· Monitored area identification unit 14 ··· Priority setting unit 15 ··· Action plan creation unit 16 ··· Instruction unit 17 ··· Monitoring map creation unit 18 ··· Output unit 19 ··· Memory unit 1A ··· Timer 20, 20-1 to 20-N ··· Unmanned aircraft 21, 21-1 to 21-N ··· Control device 22, 22-1 to 22-N ··· Monitoring sensor 23, 23-1 to 23-N ··· Positioning sensor 24, 24-1 to 24-N ··· Communication device 900 ··· Computer 901 ··· CPU 902 ··· Main memory device 903 ··· Auxiliary memory device 904 ··· Input / output interface 905 ··· Communication interface

Claims

1. Regarding surveillance map information, which is a virtual area corresponding to a surveillance area indicating the entire area to be monitored by a drone, for each small area formed by dividing the surveillance map information, an initial value of the time interval is set for repeatedly monitoring the small area at a predetermined time interval, and a priority setting unit that counts the remaining time obtained by subtracting the elapsed time from the initial value of the time interval as time passes; An action plan creation unit that sets a high priority for areas within the surveillance area corresponding to the small areas in ascending order of the remaining time, and creates action plan information that determines the movement route of the drone so as to move to the areas within the surveillance area where the priority is set in descending order of the priority; Comprising; When the drone moves based on the action plan information, the priority setting unit sets the initial value for the remaining time of the small area corresponding to the area monitored by the drone in the surveillance area. Surveillance plan creation device.

2. The action plan creation unit calculates the movement route of the drone from the position of the drone to the goal when the area within the surveillance area with the highest priority is set as the goal, and for each calculated movement route, based on the priority of the small area corresponding to the area within the surveillance area existing on the movement route, a first point is set such that the higher the priority, the larger the value, and a second point is set such that the smaller the movement time or the energy consumption required for movement of the movement route, the larger the value, and the action plan information is created by selecting the movement route with the largest weighted sum of the first point and the second point. The surveillance plan creation device according to Claim 1.

3. A monitored area identification unit that identifies the small area corresponding to the area monitored by the drone as a monitored area; Further comprising; The priority setting unit sets the initial value for the remaining time of the small area corresponding to the area identified by the monitored area identification unit. The surveillance plan creation device according to Claim 1 or Claim 2.

4. The monitored area identification unit identifies the small area corresponding to the position information as a monitored area based on the time-series position information of the drone. The surveillance plan creation device according to Claim 3.

5. The monitored area specifying unit analyzes an image captured by the drone, and when the image includes a monitoring target, specifies the small area corresponding to the position where the image was captured as a monitored area. The monitoring plan creation device according to claim 3 or claim 4.

6. The monitored area specifying unit calculates the movement route of the drone based on the time-series position information of the drone, and when the distance between the calculated movement route and a predetermined movement route is within an allowable range, specifies the small area corresponding to the calculated movement route as a monitored area. The monitoring plan creation device according to any one of claims 3 to 5.

7. The action plan creation unit calculates the movement route of the drone by means of a graph search algorithm. The monitoring plan creation device according to claim 2.

8. A monitoring plan creation device according to any one of claims 1 to 7, One or more drones that act based on the action plan information, A monitoring system comprising the same.

9. A monitoring plan creation method executed by a computer, For monitoring map information, which is a virtual area corresponding to a monitoring area to be monitored by a drone, for each small area formed by dividing the monitoring map information, setting an initial value of the time interval for repeatedly monitoring the small area at a predetermined time interval, and counting the remaining time obtained by subtracting the elapsed time from the initial value of the time interval as time elapses; Setting a high priority for areas within the monitoring area corresponding to the small areas in ascending order of the remaining time, and creating action plan information that defines the movement route of the drone so as to move to the areas within the monitoring area where the priority is set in descending order of the priority; When a certain area within the monitoring area is monitored by the drone when the drone moves based on the action plan information, setting the initial value for the remaining time of the small area corresponding to the area; A monitoring plan creation method having the above steps.

10. On a computer, Regarding surveillance map information, which is a virtual area corresponding to a surveillance area indicating the entire area to be monitored by a drone, for each small area formed by dividing the surveillance map information, an initial value of the time interval is set for repeatedly monitoring the small area at a predetermined time interval, and as time elapses, a step of counting the remaining time obtained by subtracting the elapsed time from the initial value of the time interval; a step of setting a high priority for areas within the surveillance area corresponding to the small areas in ascending order of the remaining time, and creating action plan information that determines the movement path of the drone so as to move to the areas within the surveillance area where the priority is set in descending order of the priority; a step of setting the initial value for the remaining time of the small area corresponding to the area when a certain area within the surveillance area is monitored by the drone when the drone moves based on the action plan information; A program to execute.

Images