Image management device, image management system, and image management method for flying robots

Abstract

To provide an image management device, image management system, and image management method, which enable appropriate determination of a monitoring terminal to be used to output multiple captured images captured by each of multiple flying robots.SOLUTION: An image management device is provided, comprising: a reception unit for receiving multiple captured images captured by each of multiple flying robots; an acquisition unit for acquiring security statuses of monitoring target properties included in flight areas in which the multiple flying robots fly respectively; a determination unit for determining a monitoring terminal to be used to display each of the multiple captured images from among the multiple monitoring terminals on the basis of the security statuses; and an output unit for outputting each of the multiple captured images to the motoring terminal determined by the determination unit.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to an image management device, an image management system, and an image management method for managing a plurality of images taken by a plurality of flying robots. [Background technology]

[0002] In recent years, flying robots have been developed that fly autonomously around monitored properties such as train stations, commercial facilities, and power plants, and transmit images of the monitored properties to a monitoring terminal used by a monitor. The monitoring terminal displays multiple images captured by multiple flying robots so that the monitor can efficiently monitor the monitored properties.

[0003] For example, Patent Document 1 discloses an information display method for displaying information including images provided by multiple flying objects. In this information display method, when remote control is performed from multiple locations using an operation panel, the display is made so that the terminals having the initiative to perform the operation can be recognized by each other. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-195174 Summary of the Invention [Problem to be solved by the invention]

[0005] When multiple captured images taken by multiple flying robots are output to multiple monitoring terminals, it is necessary to appropriately determine the monitoring terminal to which each captured image is output.

[0006] The object of the present invention is to provide an image management device, an image management system, and an image management method that can appropriately determine a monitoring terminal to output multiple captured images taken by each of multiple flying robots. [Means for solving the problem]

[0007] In order to solve this problem, the present invention provides an image management device having a receiving unit that receives multiple captured images taken by each of multiple flying robots, an acquiring unit that acquires the security status of monitored properties included in the flight area in which each of the multiple flying robots flies, a determining unit that determines, based on the security status, which of multiple monitoring terminals will display each of the multiple captured images, and an output unit that outputs each of the multiple captured images to the monitoring terminal determined by the determining unit.

[0008] In this image management device, it is preferable that the decision unit preferentially assigns, to each of the multiple monitoring terminals, images captured by flying robots with the same security status for the monitored property.

[0009] In this image management device, it is preferable that the decision unit assigns to separate monitoring terminals images captured by a flying robot flying in the flight area of ​​a monitored property where an abnormality in the security status has been detected and images captured by a flying robot flying in the flight area of ​​a monitored property where no abnormality in the security status has been detected.

[0010] In this image management device, it is preferable that the determination unit assigns to separate monitoring terminals images captured by a flying robot flying in the flight area of ​​a monitored property whose security status is on alert and images captured by a flying robot flying in the flight area of ​​a monitored property whose security status is off alert.

[0011] In this image management device, the determination unit preferably determines the number of captured images to be displayed or the upper limit of the number to be displayed on the monitoring terminal based on the security status.

[0012] In this image management device, it is preferable that the determination unit determines the monitoring importance of the flying robot using the type of monitored property and the security status of the monitored property, and determines the monitoring terminal on which to display the captured image based on the monitoring importance.

[0013] In this image management device, it is preferable that the decision unit preferentially assigns images captured by a flying robot flying in a flight area where a new abnormality has been detected to a monitoring terminal that is currently displaying a small number of captured images.

[0014] In this image management device, it is preferable that the acquisition unit further acquires attribute information of the monitors who use each of the multiple monitoring terminals, and the determination unit further determines the monitoring terminal that will display each of the multiple captured images based on the attribute information.

[0015] In order to solve such problems, the present invention provides an image management system having an image management device and a plurality of monitoring terminals, wherein the image management device has a receiving unit that receives a plurality of captured images taken by each of a plurality of flying robots, an acquiring unit that acquires the security status of monitored properties included in the flight area in which each of the plurality of flying robots flies, a determining unit that determines, based on the security status, a monitoring terminal from among the plurality of monitoring terminals that will display each of the plurality of captured images, and an output unit that outputs each of the plurality of captured images to the monitoring terminal determined by the determining unit, and the monitoring terminal has a display unit that displays the captured images output from the image management device.

[0016] In order to solve such problems, the present invention provides an image management method that receives multiple captured images taken by each of multiple flying robots, obtains the security status of monitored properties included in the flight area in which each of the multiple flying robots flies, determines which of multiple monitoring terminals will display each of the multiple captured images based on the security status, and outputs each of the multiple captured images to the determined monitoring terminal. [Effects of the Invention]

[0017] The image management device, image management system, and image management method according to the present invention make it possible to appropriately determine the monitoring terminal to which multiple captured images taken by each of multiple flying robots are output. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a diagram showing the overall system configuration of a monitoring system 1. FIG. [Figure 2] 1A is a diagram showing the configuration of the flying robot 10, and FIG. 1B is a diagram showing the configuration of the server 30. FIG. [Figure 3] (A) is a diagram showing an example of the data structure of the status importance table 321, (B) is a diagram showing an example of the data structure of the task importance table 322, and (C) is a diagram showing an example of the data structure of the property importance table 323. [Figure 4] 10A is a diagram showing an example of the data structure of the robot risk table 324, and FIG. 10B is a diagram showing an example of the data structure of the environment risk table 325. FIG. [Figure 5] (A) is a diagram showing an example of the data structure of the flying robot table 326, (B) is a diagram showing an example of the data structure of the environment table 327, and (C) is a diagram showing an example of the data structure of the security device location table 328. [Figure 6] FIG. 10 is a diagram showing an example of the data structure of an attribute table 329. [Figure 7] 10A is a flowchart showing an example of the operation of a decision process, and FIG. 10B is a flowchart showing an example of the operation of an output process. [Figure 8] FIG. 10 is a diagram showing the configuration of another image display device 40 and a plurality of display devices 50. DETAILED DESCRIPTION OF THE INVENTION

[0019] A monitoring system according to an embodiment will be described below with reference to the drawings.

[0020] FIG. 1 is a diagram showing the overall system configuration of a monitoring system 1. As shown in FIG. As shown in FIG. 1, the monitoring system 1 includes multiple flying robots 10, multiple image display devices 20, a server 30, a management device M, and one or more security devices G. The monitoring system 1 is an example of an image management system. The monitoring system 1 monitors and guards one or more monitoring areas S set in facilities such as train stations, commercial facilities, and power plants. Each monitoring area S includes one or more monitored objects, such as station platforms, stores, and power generation facilities. One or more flying robots 10 are assigned to each monitoring area S, and each flying robot 10 flies and monitors and guards a flight area set within one or more monitoring areas S. The monitoring system 1 includes one or more monitoring centers T, each installed within one of the monitoring areas S or outside the monitoring area S. Each image display device 20 is installed on a monitoring desk or the like in one of the monitoring centers T. One or more monitors belong to each monitoring center T, and each monitor monitors the captured images transmitted from each flying robot 10 using one of the image display devices 20 located in the monitoring center T. One monitor is assigned to one image display device 20. The server 30 is located on a monitoring desk or the like in one of the monitoring centers T. The monitoring center T is not limited to one physically defined by a specific building / site, but may also be one virtually configured via a network. For example, monitors and image display devices 20 may exist in multiple remote office environments, and multiple monitors and image display devices 20 may be virtually configured as belonging to a single monitoring center T. Each image display device 20, server 30, management device M, and security device G are connected to a communication network N such as an intranet or the Internet, and each flying robot 10 is connected to the communication network N via a wireless communication network such as a wireless LAN or a mobile phone network. As a result, each flying robot 10, each image display device 20, server 30, management device M, and security device G are connected to each other for communication.

[0021] The management device M is installed on a monitoring desk or the like in a security center operated by a security company. The management device M registers various settings in the flying robot 10 via the server 30 and controls the flying robot 10 according to the operations of a controller at the security center.

[0022] Each security device G is installed at each monitored property and has a predetermined sensor and / or fixed camera. The sensor may be a sensor for detecting intrusion of a person, such as a magnetic sensor for detecting the opening and closing of a door at each property or an infrared sensor for detecting the presence or absence of a person, or a sensor for detecting fire, such as a heat sensor or smoke sensor. When the security device G detects intrusion of a person or the outbreak of a fire by the sensor or fixed camera, it transmits an abnormality notification signal indicating the occurrence of the abnormality to the server 30, and then transmits the signal to the management device M via the server 30.

[0023] The security device G has a plurality of security states, including an alert set state, an alert released state, an abnormal state, and a response state. The alert set state is a state in which an alert mode is set to detect whether or not an abnormality exists and to report if an abnormality occurs, and no abnormality has occurred. The alert release state is a state in which the alert mode is not set and no abnormality has been detected. The abnormal state is a state in which an abnormality has occurred and no action has been taken to deal with the abnormality. The action state is a state in which an abnormality has occurred and action to deal with the abnormality has been taken by a security guard or the like. Each security mode is set from an operation unit provided on the security device G or from the server 30. When the security status changes, the security device G transmits the current security status to the server 30. Each time the server 30 receives a security status from each security device G, it associates the received security status with each security device G, stores it in a second memory unit (described later), and transmits it to the management device M.

[0024] The flying robot 10 is an unmanned small flying object capable of autonomous flight, such as a quad-rotor or single-rotor small unmanned helicopter. The flying robot 10 is also, for example, a drone, a multicopter, or a UAV (Unmanned Aerial Vehicle).

[0025] The image display device 20 is an example of a monitoring terminal. The image display device 20 is an information processing device such as a personal computer, tablet PC, or notebook PC. The image display device 20 has a display unit (not shown). The display unit has an output device such as a liquid crystal display or an organic EL display, and an interface circuit that outputs images to the output device, and displays various information such as images and text. Each image display device 20 receives one or more captured images transmitted from one or more flying robots 10 from the server 30 and displays them on the display unit. That is, the display unit displays the captured images output from the server 30.

[0026] The server 30 is an example of an image management device. The server 30 sets various settings registered from the management device M to the flying robot 10 and the security device G, and collects monitoring results (such as captured images) by the flying robot 10 and the security device G and transmits them to each image display device 20. The server 30 also transmits the security status received from each security device G to the management device M.

[0027] FIG. 2A is a diagram showing the configuration of the flying robot 10. As shown in Figure 2(A), the flying robot 10 has a position and attitude sensor 11, an imaging unit 12, a motor 13, a temperature sensor 14, a battery 15, a first communication unit 16, a first memory unit 17, a first control unit 18, etc.

[0028] The position and orientation sensor 11 acquires the current position and orientation of the flying robot 10. The position and orientation sensor 11 includes, for example, a receiver that receives radio waves (navigation signals) transmitted from navigation satellites (artificial satellites) such as a Global Navigation Satellite System (GNSS), an acceleration sensor that measures acceleration, an electronic compass that measures orientation, and a gyro sensor that measures angular velocity. The receiver receives navigation signals transmitted from multiple navigation satellites and outputs them to the first control unit 18. The acceleration sensor, electronic compass, and gyro sensor output measurement signals indicating the measured acceleration, orientation, and angular velocity to the first control unit 18. The position and orientation sensor 11 may acquire the current position of the flying robot 10 using other known sensors such as a laser scanner and a barometric pressure sensor. The position and orientation sensor 11 may also acquire the orientation of the flying robot 10 using other known sensors.

[0029] The imaging unit 12 includes a visible light camera. The visible light camera includes a photoelectric conversion element sensitive to visible light, such as a CCD element or a C-MOS element, an imaging optical system that forms an image on the photoelectric conversion element, and an A / D converter, and generates and outputs a visible light image based on visible light. The imaging unit 12 sequentially generates visible light images at a predetermined frame rate and outputs them to the first control unit 18 as captured images. The imaging unit 12 may also include a thermal imaging camera that acquires thermal images. The thermal imaging camera includes, for example, two-dimensionally arranged sensors that detect radiant energy of two types of wavelengths of electromagnetic radiation from an object, and an A / D converter that amplifies and analog-to-digital (A / D) converts the electrical signal output from the sensor. The thermal imaging camera generates a thermal image based on a temperature value calculated from the ratio of the two types of radiant energy and outputs it to the first control unit 18 as a captured image at a predetermined frame rate.

[0030] The motor 13 includes one or more (for example, four) motors. A rotor (rotor blade, propeller) is connected to the rotation shaft of each motor. The motor 13 receives a drive signal from the first control unit 18, rotates in accordance with the received drive signal, and generates a drive force to rotate each rotor. The flying robot 10 can generate acceleration in any direction by independently rotating one or more rotors, and can adjust the movement and attitude of the aircraft.

[0031] The temperature sensor 14 detects the temperature inside the flying robot 10 and outputs the detected temperature to the first control unit 18.

[0032] The battery 15 is a storage battery. The battery 15 supplies power to the position and orientation sensor 11, the imaging unit 12, the motor 13, the temperature sensor 14, the first communication unit 16, the first storage unit 17, and the first control unit 18. The battery 15 also outputs the remaining amount of power to the first control unit 18.

[0033] The first communication unit 16 has, for example, an antenna for transmitting and receiving wireless signals and a wireless communication interface circuit for transmitting and receiving signals through a wireless communication line in accordance with a wireless communication protocol such as wireless LAN, and connects to the communication network N via an access point. Alternatively, the first communication unit 16 has, for example, a communication interface circuit compliant with the W-CDMA system or the LTE system, and connects to the communication network N via a network such as a base station and a mobile communication network. The first communication unit 16 outputs data received from the communication network N to the first control unit 18, and transmits data input from the first control unit 18 to the communication network N. The first communication unit 16 also detects the signal strength of the wireless communication and outputs it to the first control unit 18.

[0034] The first storage unit 17 has semiconductor memory such as ROM and RAM, a magnetic disk or an optical disk drive such as a CD-ROM or DVD-ROM, and a recording medium thereof. The first storage unit 17 stores computer programs and various data for controlling the flying robot 10, and inputs and outputs this information to and from the first control unit 18. The computer programs may be installed into the first storage unit 17 from a computer-readable portable recording medium such as a CD-ROM or DVD-ROM using a known setup program. The computer programs may also be stored in a recording medium owned by a specified server and installed via a network. The first storage unit 17 also stores data such as position and attitude information 171 and route information 172. The position and attitude information 171 indicates the current position and current attitude of the flying robot 10 detected based on the signals acquired by the position and attitude sensor 11. The route information 172 is information indicating the planned flight route along which the flying robot 10 will move, and is represented by a coordinate sequence on the flight route. The route information 172 is set by the management device M via the server 30.

[0035] The first control unit 18 has a processor such as a CPU or MPU, memories such as a ROM or RAM, and peripheral circuits, and executes various signal processing for the flying robot 10. The first control unit 18 has a flight control unit 181 and the like implemented as a functional module of a program running on the processor. Note that the first control unit 18 may also be a DSP, LSI, ASIC, FPGA, or the like.

[0036] The flight control unit 181 calculates the current position and current attitude of the flying robot 10 in a three-dimensional movement area (e.g., flight space) from the output of the position / attitude sensor 11 and stores it as position / attitude information 171 in the first memory unit 17. The flight control unit 181 calculates latitude, longitude, and altitude from the navigation signal output from the position / attitude sensor 11, and converts it into a position in the coordinate system of the movement area using pre-stored conversion rules to calculate the current position. The flight control unit 181 also calculates the current attitude in the coordinate system of the movement area from measurement signals of the acceleration sensor and gyro sensor output from the position / attitude sensor 11. The flight control unit 181 may calculate the direction in the coordinate system of the movement area from measurement signals of the electronic compass output from the position / attitude sensor 11, and may also calculate the current attitude using measurement signals from other sensors. Every time the flight control unit 181 calculates the current position and / or current attitude, it transmits the calculated current position and / or current attitude to the server 30 via the first communication unit 16, and transmits them to the management device M and the image display device 20 via the server 30. In addition, every time the imaging unit 12 generates an image, the flight control unit 181 transmits the generated image to the server 30 via the first communication unit 16, and transmits them to the image display device 20 via the server 30.

[0037] The flight control unit 181 receives control signals from the management device M or the server 30 via the first communication unit 16 and drives and controls the motors 13 so that the flying robot 10 performs flight such as ascending, descending, turning (turning), moving forward, and hovering (stationary) in accordance with the received control signals. The flying robot 10 autonomously performs predetermined operations such as calculating its own position, calculating a target position, calculating a route, flying along the route, attitude control, and reporting, without requiring operation by an operator. The flying robot 10 executes one of a plurality of tasks. The control signal specifies the flight area and object to be monitored by the flying robot 10, as well as the task to be executed by the flying robot 10. Tasks are processes executed by the flying robot 10, classified by content (purpose). Tasks are set according to notification signals from the security device G, instructions from a controller at the monitoring center T or security center using the management device M or the server 30, or a task schedule preset in the server 30. The task may be any of a patrol task, a return task, an emergency return task, a tracking task, a detection and movement task, and a designated point movement task, etc. Tasks other than security tasks, such as an inspection task, a rescue task, a delivery task, a surveying task, and a pesticide spraying task, may also be set.

[0038] The patrol task involves flying along a preset flight path (patrol path), taking photographs at checkpoints set on the flight path, and determining whether or not there are any abnormalities at the checkpoints. The flight control unit 181 detects a change area in the captured image generated at the checkpoint. If the size of the detected change area is within a range corresponding to the size of a person, the flight control unit 181 determines that a person has intruded, and if no change area corresponding to the size of a person is detected, it determines that a person has not intruded.

[0039] The return task is a task that, when other tasks are completed or interrupted, searches (calculates) a return route to the takeoff and landing point in accordance with return instructions from the controller, and automatically lands at the takeoff and landing point.

[0040] The emergency return task is a task that, in the event of an emergency such as aircraft trouble, searches (calculates) a return route to the takeoff and landing point in accordance with return instructions from a controller, and automatically lands at the takeoff and landing point.

[0041] The tracking task is a task of flying to track a specified target (person, vehicle, etc.) in accordance with a target designation operation by a controller using the management device M and / or server 30. The flight control unit 181 extracts, as a changed area, an area in each captured image where a group of adjacent high-temperature or high-brightness pixels (e.g., pixels above a threshold) is equal to or larger than a certain size. The flight control unit 181 tracks changed areas extracted within a predetermined distance in consecutive captured images as changed areas containing the same target. The flight control unit 181 controls the motor 13 so that the changed area containing the specified target is included in the captured image at a predetermined position and size.

[0042] The detection movement task is a task in which, when a human intrusion is detected by a sensor or fixed camera installed in the monitored area, the robot flies (or waits to fly) near the sensor or fixed camera that detected the human intrusion and checks the location and cause of the abnormality.

[0043] The designated point movement task is a task of flying to a position designated by a controller using the management device M or the server 30. The designated point movement task may include a movement task and a waiting task as subtasks. The movement task is a task of searching for a route from the current position to a designated position and autonomously moving along the searched route. The waiting task is a task of maintaining a hovering state at a designated position.

[0044] FIG. 2B is a diagram showing the configuration of the server 30. As shown in FIG. 2(B), the server 30 includes a second communication unit 31, a second storage unit 32, a second control unit 33, and the like.

[0045] The second communication unit 31 is an example of an output unit. The second communication unit 31 has a communication interface circuit that complies with, for example, TCP / IP or the like, and is connected to the communication network N. Alternatively, the second communication unit 31 has, for example, an antenna for transmitting and receiving wireless signals and a wireless communication interface circuit for transmitting and receiving signals through a wireless communication line in accordance with a wireless communication protocol such as a wireless LAN, and is connected to the communication network N via an access point. The second communication unit 31 outputs data received from the communication network N to the second control unit 33, and transmits data input from the second control unit 33 to the communication network N.

[0046] The second storage unit 32 has a semiconductor memory such as a ROM or RAM, a magnetic disk or an optical disk drive such as a CD-ROM or DVD-ROM, and a recording medium thereof. The second storage unit 32 stores a computer program and various data for controlling the server 30, and inputs and outputs this information to and from the second control unit 33. The computer program may be installed into the second storage unit 32 from a computer-readable portable recording medium such as a CD-ROM or DVD-ROM using a known setup program or the like. The computer program may also be stored in a recording medium owned by a predetermined server and installed via a network. The second storage unit 32 also stores, as data, a status importance table 321, a task importance table 322, a property importance table 323, a robot risk table 324, an environmental risk table 325, a flying robot table 326, an environment table 327, a security device location table 328, an attribute table 329, and the like. Details of each table will be described later. The second storage unit 32 also stores, as data, map information. The map information indicates, for each divided area within each monitoring area monitored by the monitoring system 1, whether each area is an inhabited area or an uninhabited area and / or a public road area or a private property area. The second storage unit 32 also stores, for each of the multiple flying robots 10, the flight area, the monitored property, and the task specified in the control signal transmitted to each flying robot 10.

[0047] The second control unit 33 has a processor such as a CPU or an MPU, memories such as a ROM or a RAM, and peripheral circuits thereof, and executes various signal processing for the server 30. The second control unit 33 has a receiving unit 331, an acquiring unit 332, a determining unit 333, an output control unit 334, and the like, which are implemented as functional modules of a program running on the processor. Note that the second control unit 33 may also be implemented using a DSP, an LSI, an ASIC, an FPGA, or the like.

[0048] 3(A) is a diagram showing an example of the data structure of the status importance table 321. As shown in FIG. 3(A), the status importance table 321 has a status importance preset for each of a plurality of security statuses. The status importance is an index indicating the importance (necessity) of visually monitoring captured images in each security status when monitoring a monitored property using the flying robot 10. The status importance is set, for example, according to the degree of necessity for visual confirmation by a monitor (or controller), and the higher the necessity for visual confirmation, the higher the value set. When the security status is in the deactivated state, no abnormality has occurred, and visual confirmation is less necessary. When the security status is in the activated state, no abnormality has occurred, but the monitored area is likely to be unmanned outside of business hours, etc., and visual confirmation is more necessary than when the security status is deactivated. When the security status is in the abnormal state, an abnormality has occurred, and no action has yet been taken to address the abnormality. Visual confirmation is most necessary because the monitor must continue to monitor the situation on site until the security guard arrives at the site. When the security status is in the response state, an abnormality has occurred, but the security guard or other person has taken action to address the abnormality. In this case, the monitor must also monitor the situation on site, but visual confirmation is less necessary than when the security status is deactivated. Therefore, the status importance is set to higher values ​​in the order of abnormal state, response state, security status, deactivated state, and deactivated state.

[0049] Fig. 3(B) is a diagram showing an example of the data structure of the task importance table 322. As shown in Fig. 3(B), task importance is set in advance for each of a plurality of tasks in the task importance table 322. The task importance is an index that indicates the importance of each task. The task importance is set, for example, according to the degree of necessity for visual confirmation by an observer (or a controller), and the greater the necessity for visual confirmation, the higher the value set. The observer must visually check the designated target in the tracking task, the person detected by the sensor or fixed camera in the detection and movement task, and the flying robot 10 that has experienced aircraft trouble in the emergency return task. On the other hand, the observer only needs to confirm that no problems have occurred in the patrol task and the designated point movement task, and only needs to confirm that the flying robot 10 returns in the return task. Therefore, the task importance is set to a high value in the tracking task, the detection and movement task, and the emergency return task, a low value in the patrol task and the designated point movement task, and an even lower value in the return task. The task importance may be set by further subdividing the task. For example, among the designated point movement tasks, the task importance of a waiting task may be set to a higher value than the task importance of a movement task. The task importance may also be set based on the position of the flying robot 10. For example, in the designated point movement task, the task importance when the flying robot 10 is at the designated position may be set to a higher value than the task importance when the flying robot 10 is not at the designated position.

[0050] Fig. 3(C) is a diagram showing an example of the data structure of the property importance table 323. As shown in Fig. 3(C), the property importance table 323 has property importance preset for each type of multiple monitored property of the monitoring system 1 (flying robot 10). The property importance is an index that indicates the importance of monitoring each monitored property. The property importance is set, for example, according to the risk of an abnormality occurring or the monitoring service contract content (service guarantee level), and the property importance is set to a higher value as the risk or service guarantee level increases. For properties such as theme parks, tourist destinations, and train stations, the property importance is set to a high value because many people exist within the area and the damage would be great if a security incident such as an incident or accident were to occur. On the other hand, for farmland, forests, rivers, and the like, the property importance is set to a low value because few people exist within the area, security incidents are unlikely to occur, and the damage would be relatively small if a security incident were to occur. Furthermore, for properties such as factories and residential areas, the property importance of properties that have a contract to check and notify even the slightest abnormality is set to a higher value than the property importance of properties that have a contract to notify only if a security incident occurs.

[0051] FIG. 4A is a diagram illustrating an example of the data structure of the robot risk table 324. As shown in FIG. 4A, the robot risk table 324 pre-sets a robot risk for each flying robot state. The flying robot state refers to the body or flight state of the flying robot 10, and includes battery status, flight abnormality, communication status, flight speed, flight altitude, etc. The flying robot state may also include the flight area in which the flying robot 10 flies. The flying robot state may include at least one of battery status, flight abnormality, communication status, flight speed, flight altitude, and flight area. The robot risk is an index indicating the magnitude of risk in the flight of the flying robot 10. The robot risk is set based on the probability of a problem occurring, such as the flying robot 10 being unable to properly perform a task due to factors related to the flying state of the flying robot 10, or the magnitude of damage if a problem occurs. The higher the probability of a problem occurring or the magnitude of damage, the higher the value set. The robot risk is set to a higher value the lower the remaining battery power. The robot risk when an abnormality such as abnormal shaking or abnormal heat occurs is set to a higher value than the robot risk when no abnormality occurs. The robot risk when the communication state is poor is set to a higher value than the robot risk when the communication state is good. The robot risk is set to a higher value the higher the flight speed. The robot risk is set to a higher value the higher the flight altitude.

[0052] FIG. 4B illustrates an example of the data structure of the environmental risk table 325. As illustrated in FIG. 4B, the environmental risk table 325 pre-sets an environmental risk for each environmental condition. The environmental condition may include wind speed, weather, brightness, area attributes, etc. The environmental condition may include at least one of wind speed, weather, brightness, and area attributes. The area attributes may include whether the flight area is a manned or unmanned area, a public road or private property, and / or a small area less than a predetermined area or a large area equal to or greater than a predetermined area. The environmental risk is an index indicating the magnitude of risk for each environmental condition. The environmental risk is set based on, for example, the probability of a problem occurring, such as the flying robot 10 being unable to properly perform a task due to environmental factors in the flight area of ​​the flying robot 10, or the magnitude of damage that would occur if a problem were to occur. The higher the probability of a problem occurring or the magnitude of damage, the higher the environmental risk is set. The stronger (higher) the wind speed, the higher the environmental risk is set. The higher the environmental risk is set when the weather is rainy, the lower the environmental risk is set when the weather is snowy, and the lower the environmental risk is set even more when the weather is sunny. The darker the environment, the higher the environmental risk is set. The environmental risk when the flight area is a manned area is set to a higher value than the environmental risk when the flight area is an unmanned area. The environmental risk when the flight area is a public road area is set to a higher value than the environmental risk when the flight area is a private property area. The environmental risk when the flight area is a small area is set to a higher value than the environmental risk when the flight area is a large area.

[0053] FIG. 5(A) is a diagram showing an example of the data structure of the flying robot table 326. As shown in FIG. 5(A), the flying robot table 326 stores, for each flying robot 10, a task, a monitored object, a flight area, a flying robot status, and the like, in association with each other. A task is a task currently being executed by each flying robot 10. A monitored object is a monitored object currently being monitored by each flying robot 10. A flight area is a flight area in which each flying robot 10 is currently flying. A flying robot status is the current flying robot status of each flying robot 10. The flying robot table 326 is updated by the second control unit 33 in the determination process described below.

[0054] Fig. 5(B) is a diagram showing an example of the data structure of the environment table 327. As shown in Fig. 5(B), the environment table 327 stores the current environmental conditions of each flight area in which the flying robot 10 is currently flying, in association with each other. The environment table 327 is updated by the second control unit 33 in the determination process described below.

[0055] Fig. 5(C) is a diagram showing an example of the data structure of the security device location table 328. As shown in Fig. 5(C), the security device location table 328 stores the installation location of each security device G for one or more security devices G. The security device location table 328 is set in advance by an administrator.

[0056] Fig. 6 is a diagram showing an example of the data structure of attribute table 329. As shown in Fig. 6, attribute table 329 stores attribute information for each of multiple monitors. The attribute information includes device information, qualification information, experience period, affiliation information, age, and monitoring time. Attribute table 329 is set in advance by an administrator and updated periodically. The device information indicates the image display device 20 used by each monitor. The qualification information indicates the rank assigned to the qualifications held by each monitor. The experience period indicates the total period during which each monitor has performed (experienced) monitoring work up to the present. The affiliation information indicates the company or organization to which each monitor belongs, for example, whether it is a security company or not. The age is the age of each monitor. The monitoring time is the continuous monitoring time from the start of monitoring by each monitor to the present, or the cumulative monitoring time for a predetermined period up to the present (for example, one hour or one day).

[0057] Fig. 7(A) is a flowchart showing an example of the operation of the decision processing by the server 30. This flowchart is executed mainly by the second control unit 33 in cooperation with each element of the server 30, based on a program stored in advance in the second storage unit 32. The decision processing shown in Fig. 7(A) is executed periodically.

[0058] First, the acquisition unit 332 acquires the monitoring purpose of each flying robot 10 in the monitoring system 1 (step S101). The monitoring purpose of each flying robot 10 includes the task performed by each flying robot 10 and the monitored object monitored by each flying robot 10. The monitoring purpose of each flying robot 10 is determined by at least one of the content of the task performed by each flying robot 10 and the type of monitored object monitored by each flying robot 10. The acquisition unit 332 acquires the monitoring purpose of each flying robot 10 by reading, for each of the multiple flying robots 10, the latest task and monitored object specified in the control signal sent to each flying robot 10 from the second storage unit 32. The acquisition unit 332 stores the tasks and monitored objects included in the acquired monitoring purpose in the flying robot table 326.

[0059] Next, the acquisition unit 332 acquires flight information of each flying robot 10 possessed by the monitoring system 1 (step S102). The flight information of each flying robot 10 includes the flying robot state of each flying robot 10 and the environmental state of the flight area in which each flying robot 10 flies. The flight information of each flying robot 10 is determined by at least one of the flying robot state of each flying robot 10 and the environmental state of the flight area in which the flying robot 10 flies.

[0060] The acquisition unit 332 transmits a flying robot status request signal requesting acquisition of the flying robot status to each flying robot 10 via the second communication unit 31. The flight control unit 181 of each flying robot 10 acquires the remaining power from the battery 15 to determine the battery status. The flight control unit 181 acquires the current position and attitude of the flying robot 10 from the position / attitude sensor 11 to determine whether an abnormal shaking has occurred. The flight control unit 181 acquires the internal temperature of the flying robot 10 from the temperature sensor 14 to determine whether an abnormal heat generation has occurred. The flight control unit 181 acquires the signal strength of the wireless communication from the first communication unit 16 to determine whether the communication state is good or bad. The flight control unit 181 determines the flight speed from the drive amount of the motor 13. Alternatively, the flight control unit 181 determines the flight speed from changes in the position of the flying robot 10 over time. The flight control unit 181 determines the flight altitude from the current position of the flying robot 10. Then, the flight control unit 181 transmits a flying robot state notification signal indicating the current flying robot state to the server 30 via the first communication unit 16. The acquisition unit 332 acquires the flying robot state of each flying robot 10 by receiving the flying robot state notification signal from each flying robot 10 via the second communication unit 31. Note that each flying robot 10 may periodically transmit a flying robot state notification signal to the server 30. The acquisition unit 332 stores the acquired flying robot state in the flying robot table 326.

[0061] The acquisition unit 332 also acquires the flight area of ​​each flying robot 10 by reading from the second storage unit 32 the latest flight area specified in the control signal sent to each flying robot 10. The acquisition unit 332 stores the acquired flight area in the flying robot table 326.

[0062] The acquisition unit 332 also transmits a weather information request signal requesting weather information for the flight area of ​​each flying robot 10 to the Japan Meteorological Agency's Regional Meteorological Observation System (AMeDAS) or other weather observation system via the second communication unit 31, and receives weather information for each flight area from the weather observation system. The acquisition unit 332 determines the wind speed, weather, and brightness of each flight area from the received weather information. The acquisition unit 332 may have an observer input the wind speed, weather, and brightness of each flight area using an operation unit (not shown). The acquisition unit 332 may also obtain the wind speed, weather, or brightness of each flight area from the flying robot 10. In this case, each flying robot 10 further has a wind speed sensor to measure the wind speed associated with each flying robot 10. Each flying robot 10 also has a humidity sensor to estimate the weather from the humidity and / or temperature of each flying robot 10. Each flying robot 10 also determines the brightness around the flying robot 10 from the grayscale value of each pixel included in the captured image. The acquisition unit 332 transmits a flight area information request signal requesting acquisition of flight area information indicating the wind speed, weather, or brightness of each flight area to each flying robot 10 via the second communication unit 31, and acquires the wind speed, weather, or brightness of each flight area by receiving the flight area information from each flying robot 10. The acquisition unit 332 stores the acquired wind speed, weather, and brightness of each flight area in the environment table 327.

[0063] The acquisition unit 332 also references the map information stored in the second storage unit 32 and identifies whether each flight area is a manned area or an unmanned area, and whether it is a public road area or a private land area. Furthermore, the acquisition unit 332 determines whether each flight area is equal to or larger than a predetermined area. Based on this, the acquisition unit 332 acquires the area attributes of each flight area. The acquisition unit 332 may acquire the area attributes of each flight area by having an observer input the area attributes of each flight area using the operation unit. The acquisition unit 332 stores the acquired area attributes of each flight area in the environment table 327.

[0064] Next, the acquisition unit 332 acquires the security status of the monitored property included in the flight area where each flying robot 10 flies (step S103). The acquisition unit 332 refers to the flying robot table 326 and the security device location table 328 to identify, for each of the multiple flying robots 10, one or more security devices G installed at the monitored property included in the flight area where each flying robot 10 flies. The acquisition unit 332 reads the latest security status received from each identified security device G from the second memory unit 32. The acquisition unit 332 identifies the security status that most requires visual confirmation among the security statuses of the security devices G installed at the monitored property included in the flight area where each flying robot 10 flies as the security status of the monitored property included in the flight area where each flying robot 10 flies. In other words, if the security status of any security device G is abnormal, the acquisition unit 332 identifies the security status of the monitored property included in that flight area as abnormal. If the security status of any security device G is not an abnormal state and the security status of any security device G is an action state, the acquisition unit 332 determines the security status of the monitored property included in the flight area to be an action state. If the security status of any security device G is not an abnormal state or an action state and the security status of any security device G is an alert set state, the acquisition unit 332 determines the security status of the monitored property included in the flight area to be an alert set state. If the security status of all security devices G is an alert disengagement state, the acquisition unit 332 determines the security status of the monitored property included in the flight area to be an alert disengagement state.

[0065] Next, the acquiring unit 332 acquires attribute information of the monitors who use each image display device 20 (step S104). The acquiring unit 332 acquires the attribute information of each monitor by reading it from the attribute table 329.

[0066] Next, the determination unit 333 determines the monitoring importance of each flying robot 10 based on the monitoring purpose, flight information, and / or security status acquired by the acquisition unit 332 (step S105). That is, the determination unit 333 determines the monitoring importance of each flying robot 10 using at least one of the content of the task performed by the flying robot 10, the type of monitored object monitored by the flying robot 10, the flying robot status of the flying robot 10, the environmental status of the area in which the flying robot 10 flies, and the security status of the monitored object included in the flight area in which the flying robot 10 flies. The monitoring importance is an index indicating the importance of monitoring each flying robot 10 or the captured image captured by each flying robot 10. The determination unit 333 refers to the state importance table 321 to determine, for each flying robot 10 in the monitoring system 1, the state importance of the security status of the monitored property included in the flight area in which each flying robot 10 flies. The determination unit 333 refers to the task importance table 322 to determine the task importance of the task executed by each flying robot 10. The determination unit 333 refers to the property importance table 323 to determine the property importance of the monitored property monitored by each flying robot 10. The determination unit 333 refers to the robot risk table 324 to determine the robot risk for each flying robot state of each flying robot 10. The determination unit 333 refers to the environmental risk table 325 to determine the environmental risk for each environmental state in the flight area of ​​each flying robot 10. For example, the determination unit 333 determines the identified state importance or property importance as the monitoring importance. As a result, the determination unit 333 can determine the image display device 20 on which each captured image is to be displayed, taking into consideration the necessity of visual confirmation of the security status of monitored objects included in the flight area in which each flying robot 10 flies, or the importance of monitoring the monitored objects monitored by each flying robot 10. The determination unit 333 may determine the sum, product, weighted sum, etc. of the identified state importance and property importance as the monitoring importance for each flying robot 10. As a result, the determination unit 333 can determine the image display device 20 on which each captured image is to be displayed, taking into consideration both the necessity of visual confirmation of the security status of monitored objects included in the flight area in which each flying robot 10 flies, and the importance of monitoring the monitored objects monitored by each flying robot 10. The determination unit 333 may determine the sum, product, weighted sum, etc. of the identified state importance, property importance, task importance, each robot risk, and / or each environmental risk as the monitoring importance for each flying robot 10. The determination unit 253 may calculate a correction value by correcting the state importance so that the higher the task importance, property importance, robot risk, and / or environmental risk, the higher the correction value, and determine the calculated correction value as the monitoring importance. In this way, the determination unit 333 can determine the image display device 20 that will display each captured image, taking into consideration various factors of each flying robot 10.

[0067] Next, the determination unit 333 determines, from among the multiple image display devices 20 possessed by the monitoring system 1, an image display device 20 that displays the multiple captured images captured by each flying robot 10 possessed by the monitoring system 1 (step S106). Hereinafter, the image display device 20 that displays the captured images captured by the flying robot 10 may be referred to as a target device. The determination unit 333 determines the target device based on the security status, monitoring purpose, flight information, and / or attribute information acquired by the acquisition unit 332.

[0068] For example, the determination unit 333 preferentially assigns to each image display device 20 images captured by flying robots 10 with the same security status for monitored properties included in the flight area. For example, the determination unit 333 assigns only captured images captured by flying robots 10 with the same security status of monitored objects included in the flight area to a single image display device 20. The determination unit 333 assigns captured images captured by flying robots 10 with a specific security status and captured images captured by flying robots 10 with other security statuses to separate image display devices 20. Alternatively, when a captured image that needs to be newly displayed on the image display device 20 is generated, the determination unit 333 identifies the security status of monitored objects included in the flight area of ​​the flying robot 10 that captured the captured image, and determines to preferentially assign the captured image to an image display device 20 that is displaying captured images of monitored objects with the same security status. This allows the monitoring system 1 to display only captured images corresponding to the same security situation on the image display devices 20 monitored by each monitor, allowing each monitor to monitor the captured images from a perspective specific to that security situation. Therefore, the monitoring system 1 can improve the monitoring accuracy of the monitors, and as a result, the safety of the monitored area can be improved. The determination unit 333 may set a corresponding security state in advance for each image display device 20. This allows a monitor using each image display device 20 to recognize in advance points to pay attention to in the displayed captured image, thereby improving monitoring accuracy.

[0069] The determination unit 333 may assign captured images captured by the flying robot 10 flying in the flight area of ​​the monitored property where the security status is in an abnormal state and captured images captured by the flying robot 10 flying in the flight area of ​​the monitored property where the security status is in an abnormal state to separate image display devices 20. That is, the determination unit 333 assigns only captured images captured by the flying robot 10 where the security status is in an abnormal state and / or captured images captured by the flying robot 10 where the security status is in an emergency state to a single image display device 20. Furthermore, the determination unit 333 assigns only captured images captured by the flying robot 10 flying in the flight area of ​​the monitored property where the security status is in an alert set state and / or captured images captured by the flying robot 10 flying in the flight area of ​​the monitored property where the security status is in an alert released state to a single image display device 20. As a result, a monitor monitoring captured images in which an abnormality is detected can monitor multiple captured images collectively from a perspective specific to when an abnormality has occurred, such as determining where a suspicious person is. On the other hand, a monitor monitoring captured images in which no abnormality is detected can monitor multiple captured images collectively from a perspective specific to when no abnormality has occurred, such as determining whether a problem has occurred on the perimeter where no sensors are installed. Therefore, monitoring system 1 can improve the monitoring accuracy of monitors, and as a result, can improve the safety of the monitored area.

[0070] The determination unit 333 may assign captured images captured by the flying robot 10 flying in the flight area of ​​the monitored property whose security status is in the alert set state and captured images captured by the flying robot 10 flying in the flight area of ​​the monitored property whose security status is in the alert disengagement state to separate image display devices 20. That is, the determination unit 333 assigns only captured images captured by the flying robot 10 flying in the flight area of ​​the monitored property whose security status is in the alert set state to a single image display device 20. Also, the determination unit 333 assigns only captured images captured by the flying robot 10 flying in the flight area of ​​the monitored property whose security status is in the alert disengagement state to a single image display device 20. As a result, a monitor monitoring captured images for which the corresponding security status is set can monitor multiple captured images collectively from a perspective unique to the alert state, such as whether there is a suspicious person present. On the other hand, a monitor monitoring captured images for which the corresponding security status is set can monitor multiple captured images collectively from a perspective unique to normal times, such as whether there is a person who has collapsed. Therefore, the monitoring system 1 can improve the monitoring accuracy of monitors, and as a result, the safety of the monitored area can be improved.

[0071] In addition, the determination unit 333 may determine the number of images to be displayed or the maximum number of images to be displayed on a single image display device 20 based on the security status acquired by the acquisition unit 332, and determine the target device so as to satisfy the determined number of images to be displayed or the maximum number of images to be displayed. For example, the determination unit 333 determines the number of captured images to be displayed or the maximum number of images to be displayed on each image display device 20 depending on the necessity of visual confirmation of the security status of a monitored object included in the flight area where the flying robot 10 that captured each captured image flies. The determination unit 333 reduces the number of captured images to be displayed or the maximum number of images to be displayed on each image display device 20 as the necessity of visual confirmation of the security status of a monitored object included in the flight area where the flying robot 10 that captured each captured image flies increases. That is, the determination unit 333 reduces the number of captured images to be displayed or the maximum number of images to be displayed on each image display device 20 in the order of abnormal status, response status, alert set status, and alert released status of the security status of a monitored object included in the flight area where the flying robot 10 that captured each captured image flies. This allows the monitor who monitors the captured images that require a high level of visual confirmation to concentrate on monitoring each captured image, while the monitor who monitors the captured images that require less visual confirmation can efficiently monitor a large number of captured images. Therefore, the monitoring system 1 can improve the monitoring efficiency of the monitor while reducing the occurrence of overlooking abnormalities.

[0072] In addition, the decision unit 333 may preferentially allocate captured images captured by a flying robot 10 flying in the flight area of ​​a monitored property where a new abnormality has been detected to an image display device 20 that is currently displaying a small number of captured images. For example, the determination unit 333 monitors the security status of monitored properties included in each flight area. When the security status of a specific monitored property changes from an alert set state or an alert disengaged state to an abnormal state or a response state, the determination unit 333 determines that a new abnormality has been detected at the monitored property. The determination unit 333 assigns captured images captured by the flying robot 10 flying in the flight area of ​​the monitored property to the image display device 20 currently displaying the fewest captured images. The determination unit 333 may assign captured images captured by the flying robot 10 flying in the flight area of ​​the monitored property to the image display device 20 currently displaying the fewest captured images (e.g., 0 or 1) or less. The determination unit 333 may assign captured images captured by the flying robot 10 flying in the flight area of ​​the monitored property only to the image display device 20 to which captured images corresponding to the abnormal state or the response state have been assigned. This allows the observer monitoring the captured images of the flight area where a new abnormality has been detected to focus on responding to the newly detected abnormality. Therefore, the monitoring system 1 can improve the monitoring accuracy of the observer, and as a result, the safety of the monitored area can be improved.

[0073] The determination unit 333 may also determine the image display device 20 that will display each captured image based on the monitoring importance of each flying robot 10. For example, if the monitoring importance of a newly added flying robot 10 is high, the determination unit 333 may preferentially allocate captured images captured by the flying robot 10 to the image display device 20 with the fewest currently displayed captured images. For example, the determination unit 333 may allocate captured images captured by a flying robot 10 with a monitoring importance equal to or greater than a predetermined importance threshold to the image display device 20 with the fewest currently displayed captured images. The determination unit 333 may allocate captured images captured by a flying robot 10 with a monitoring importance equal to or greater than the importance threshold to the image display device 20 with the number of currently displayed captured images equal to or less than the display number threshold. This allows the observer who monitors the images captured by the flying robot 10, which has a high monitoring importance, to concentrate on monitoring the images. Therefore, the monitoring system 1 can improve the monitoring accuracy of the observer, and as a result, the safety of the monitored area can be improved.

[0074] The determination unit 333 may also determine the target device based on attribute information of each monitor. For example, the determination unit 333 determines the target device based on the proficiency of each monitor. In this case, the determination unit 333 first determines the proficiency of each monitor. The determination unit 333 identifies the experience period included in the attribute information of each monitor and determines the proficiency of each monitor so that the longer the experience period, the higher the proficiency of each monitor. The determination unit 333 also identifies the qualification information included in the attribute information of each monitor and determines the proficiency of each monitor so that the higher the rank indicated in the qualification information, the higher the proficiency of each monitor. In other words, the determination unit 333 determines the proficiency of each monitor so that the proficiency of a monitor who has a predetermined qualification is higher than the proficiency of a monitor who does not have that qualification. The determination unit 333 also determines the proficiency of each monitor so that the proficiency of a monitor who has a qualification of a high rank (level) is higher than the proficiency of a monitor who only has a qualification of a low rank (level). Furthermore, the determination unit 333 identifies the affiliation information included in the attribute information of each monitor, and determines the proficiency level of each monitor so that the proficiency level of a monitor whose affiliation information indicates a security company is higher than the proficiency level of a monitor whose affiliation information indicates a company other than a security company. Note that the proficiency level of each monitor may be set in advance by an administrator or the like. The determination unit 333 assigns captured images captured by the flying robot 10 that are highly necessary for visual confirmation of the security status of the monitored property included in the flight area to the image display device 20 used by a monitor with a higher level of proficiency. For example, the determination unit 333 assigns captured images captured by the flying robot 10 flying in the flight area of ​​the monitored property where an abnormality is detected in the security status to the image display device 20 used by a monitor whose proficiency is equal to or higher than a predetermined proficiency threshold. On the other hand, the determination unit 333 assigns captured images captured by the flying robot 10 flying in the flight area of ​​the monitored property where no abnormality is detected in the security status to the image display device 20 used by a monitor whose proficiency is below the proficiency threshold. Note that the determination unit 333 may assign captured images captured by the flying robot 10 whose security status is on alert to the image display device 20 used by a monitor whose proficiency is equal to or higher than the proficiency threshold. In this case, the decision unit 333 assigns the captured image captured by the flying robot 10 flying in the flight area of ​​the monitored property whose security status is in the alert-off state to the image display device 20 used by the monitor whose proficiency level is below the proficiency threshold. As a result, the monitoring system 1 allows highly skilled monitors to monitor captured images that require a high level of visual confirmation, thereby improving the monitoring accuracy of captured images. On the other hand, the monitoring system 1 allows less skilled monitors to monitor only captured images that require less visual confirmation, thereby preventing less skilled monitors from overlooking abnormalities.

[0075] The determination unit 333 may assign captured images captured by flying robots 10 with higher monitoring importance to image display devices 20 used by monitors with higher proficiency. For example, the determination unit 333 assigns captured images captured by flying robots 10 with monitoring importance levels equal to or higher than a predetermined importance threshold to image display devices 20 used by monitors with proficiency levels equal to or higher than a proficiency threshold. On the other hand, the determination unit 333 assigns captured images captured by flying robots 10 with monitoring importance levels lower than the importance threshold to image display devices 20 used by monitors with proficiency levels lower than the proficiency threshold. As a result, the monitoring system 1 allows highly skilled monitors to monitor captured images that are of high importance, thereby improving the monitoring accuracy of captured images. On the other hand, the monitoring system 1 allows less skilled monitors to monitor only captured images that are of low importance, thereby preventing less skilled monitors from overlooking abnormalities.

[0076] The server 30 may have a normal mode and a training mode for determining a target device to allow less experienced monitors to gain experience. When the training mode is set, the determination unit 33 may assign captured images captured by a flying robot 10 in which the security status of a monitored property included in the flight area is in an abnormal state, a response state, or an alert set state to the image display device 20 used by a monitor whose proficiency level is equal to or lower than the proficiency threshold. Furthermore, when the training mode is set, the determination unit 333 may assign captured images captured by a flying robot 10 in which the monitoring importance level is equal to or higher than the importance threshold to the image display device 20 used by a monitor whose proficiency level is equal to or lower than the proficiency threshold.

[0077] The determination unit 333 may determine the target device based on the fatigue level of each monitor. In this case, the determination unit 333 first determines the fatigue level of each monitor. For example, the determination unit 333 identifies the monitoring time (continuous monitoring time or cumulative monitoring time) included in the attribute information of each monitor, and determines the fatigue level of each monitor so that the longer the monitoring time, the higher the fatigue level. The determination unit 333 also identifies the age included in the attribute information of each monitor, and determines the fatigue level of each monitor so that the fatigue level of a monitor whose age is equal to or greater than a threshold (e.g., 65 years old) is higher than the fatigue level of a monitor whose age is less than the threshold. The determination unit 333 assigns captured images captured by the flying robot 10 that are more likely to require visual confirmation of the security status of the monitored property included in the flight area to the image display device 20 used by a monitor with a lower fatigue level. For example, the determination unit 333 assigns captured images captured by the flying robot 10 flying in the flight area of ​​the monitored property where an abnormality is detected in the security status to the image display device 20 used by a monitor whose fatigue level is equal to or lower than a predetermined fatigue level threshold. On the other hand, the determination unit 333 assigns captured images captured by the flying robot 10 flying in the flight area of ​​the monitored property where no abnormality is detected in the security status to the image display device 20 used by a monitor whose fatigue level is higher than the fatigue level threshold. Note that the determination unit 333 may also assign captured images captured by the flying robot 10 flying in the flight area of ​​the monitored property where the security status is set to alert to the image display device 20 used by a monitor whose fatigue level is equal to or lower than the fatigue level threshold. In this case, the decision unit 333 assigns the captured image captured by the flying robot 10 flying in the flight area of ​​the monitored property whose security status is in the alert-off state to the image display device 20 used by the monitor whose fatigue level is greater than the fatigue level threshold. As a result, the monitoring system 1 allows a less fatigued monitor to monitor captured images that require more visual confirmation, thereby improving the monitoring accuracy of captured images. On the other hand, the monitoring system 1 allows a more fatigued monitor to monitor only captured images that require less visual confirmation, thereby preventing a more fatigued monitor from overlooking an abnormality.

[0078] The determination unit 333 may assign captured images captured by flying robots 10 with higher monitoring importance to image display devices 20 used by monitors with lower fatigue levels. For example, the determination unit 333 assigns captured images captured by flying robots 10 with monitoring importance levels equal to or higher than the importance threshold to image display devices 20 used by monitors with fatigue levels equal to or lower than the fatigue threshold. On the other hand, the determination unit 333 assigns captured images captured by flying robots 10 with monitoring importance levels lower than the importance threshold to image display devices 20 used by monitors with fatigue levels higher than the fatigue threshold. As a result, the monitoring system 1 can improve the monitoring accuracy of captured images by having a less fatigued monitor monitor captured images that are more important to monitor. On the other hand, the monitoring system 1 can prevent a more fatigued monitor from overlooking an abnormality by having a more fatigued monitor monitor only captured images that are less important to monitor. This concludes the explanation of the determination process.

[0079] Fig. 7(B) is a flowchart showing an example of the operation of the output process by the server 30. This flowchart is executed mainly by the second control unit 33 in cooperation with each element of the server 30, based on a program stored in advance in the second storage unit 32. The output process shown in Fig. 7(B) is executed in parallel with the determination process shown in Fig. 7(A).

[0080] First, the receiving unit 331 receives a plurality of captured images captured by each flying robot 10 included in the monitoring system 1 (step S201). The receiving unit 331 acquires each captured image transmitted by each flying robot 10 each time it is generated by receiving the captured image via the second communication unit 31.

[0081] Next, the output control unit 334 outputs each captured image received by the receiving unit 331 by transmitting it to the image display device 20 determined by the determination unit 333 (step S202), and returns the process to step S201. The image display device 20 receives the captured image from the server 30 and displays it. This concludes the description of the output process.

[0082] As described above, the server 30 determines the image display device 20 that displays the images captured by each flying robot 10 based on the security status of the monitored object included in the flight area in which each flying robot 10 flies. Therefore, the server 30 can appropriately determine the image display device 20 that outputs the multiple images captured by each of the multiple flying robots 10. The monitor can monitor the flying robots 10 or the captured images classified based on the security status, and can monitor each flying robot 10 or the captured images safely and efficiently.

[0083] FIG. 8 is a diagram showing the configuration of an image display device 40 and a plurality of display devices 50 according to another embodiment. The image display device 40 and the display device 50 shown in FIG. 8 are used instead of the image display device 20. In this embodiment, the image display device 40 is an example of an image management device, and the display device 50 is an example of a monitoring terminal. Each monitor uses one of the display devices 50 connected to each image display device 40 to monitor the captured images transmitted from each flying robot 10. One monitor is assigned to one display device 50. Each display device 50 has a display unit such as a display for displaying one or more captured images output from the image display device 40. The image display device 40 has a third communication unit 41, an interface unit 42, a third memory unit 43, a third control unit 44, etc.

[0084] The third communication unit 41 has a configuration similar to that of the second communication unit 31, and outputs data received from the communication network N to the third control unit 44, and transmits data input from the third control unit 44 to the communication network N.

[0085] The interface unit 42 is an example of an output unit. The interface unit 42 has an interface circuit that complies with an interface standard such as DisplayPort or a serial bus standard such as USB, and is communicatively connected to the display device 50 to transmit and receive various signals. A plurality of display devices 50 are connected to the interface unit 42.

[0086] The third storage unit 43 has a configuration similar to that of the second storage unit 32, stores a computer program and various data for controlling the image display device 40, and inputs and outputs this information to and from the third control unit 44. The computer program may be installed into the third storage unit 43 from a computer-readable portable recording medium such as a CD-ROM or DVD-ROM using a known setup program or the like. The computer program may also be stored in a recording medium owned by a predetermined server and installed via a network. The third storage unit 43 also stores, as data, a state importance table 431, a task importance table 432, a property importance table 433, a robot risk table 434, an environmental risk table 435, a flying robot table 436, an environment table 437, a security device location table 438, an attribute table 439, etc. Each table contains the same information as the state importance table 321, the task importance table 322, the property importance table 323, the robot risk table 324, the environmental risk table 325, the flying robot table 326, the environment table 327, the security device location table 328, and the attribute table 329. However, in the attribute information contained in the attribute table 439, the device information indicates the display device 50 used by each monitor.

[0087] The third control unit 44 has a configuration similar to that of the second control unit 33, and executes various signal processes for the image display device 40. The third control unit 44 has a receiving unit 441, an acquiring unit 442, a determining unit 443, an output control unit 444, and the like, which are implemented as functional modules of a program that runs on a processor. The receiving unit 441, the acquiring unit 442, the determining unit 443, and the output control unit 444 have the same functions as the receiving unit 331, the acquiring unit 332, the determining unit 333, and the output control unit 334, respectively.

[0088] In this embodiment, the determination process shown in FIG. 7(A) and the output process shown in FIG. 7A, the acquisition unit 442 transmits a monitoring purpose request signal requesting acquisition of the monitoring purpose of each flying robot 10 to the server 30 via the third communication unit 41. The acquisition unit 442 acquires the monitoring purpose of each flying robot 10 by receiving it from the server 30 via the third communication unit 41. In step S102, the acquisition unit 442 transmits a flight area request signal requesting acquisition of the flight area of ​​each flying robot 10 to the server 30 via the third communication unit 41. The acquisition unit 442 acquires the flight area of ​​each flying robot 10 by receiving it from the server 30 via the third communication unit 41. In step S103, the acquisition unit 442 transmits a security status request signal requesting acquisition of the security status of monitored objects included in the flight area in which each flying robot 10 flies to the server 30 via the third communication unit 41. The acquisition unit 442 acquires the security status of monitored objects included in the flight area in which each flying robot 10 flies by receiving it from the server 30 via the third communication unit 41. In step S106, the determination unit 443 determines, from among the multiple display devices 50 connected to the image display device 40, a display device 50 that displays the multiple captured images captured by each flying robot 10 of the monitoring system 1. In this embodiment, the display device 50 that displays the captured images captured by the flying robot 10 is the target device. In step S201 of FIG. 7B, the receiving unit 441 receives a plurality of captured images captured by each flying robot 10 of the monitoring system 1 via the third communication unit 41 and the server 30. In step S202, the output control unit 444 outputs each captured image received by the receiving unit 441 by displaying it on the display device 50 determined by the determining unit 443 via the interface unit .

[0089] As described above, in this embodiment, the image display device 40 determines the display device 50 that will display the images captured by each flying robot 10 based on the security status of the monitored objects included in the flight area in which each flying robot 10 flies. This allows the image display device 40 to appropriately determine the display device 50 that will output the multiple images captured by each of the multiple flying robots 10.

[0090] Although preferred embodiments have been described above, the embodiments are not limited to the above examples. For example, in the monitoring system 1, it is possible to appropriately change whether the components of the server and the image display device are located on the server or on the image display device. Furthermore, in order to provide a decision-making processing service in the form of cloud computing, multiple servers may be distributed and located on a network, and each server may cooperate to share each process.

[0091] Furthermore, in each of the above embodiments, the determination unit 333 calculates the monitoring importance for each of the captured images of the multiple flying robots 10 based on at least one of the monitoring purpose, flight information, and security status acquired by the acquisition unit 332, and determines the image display device 20 to be assigned based on the monitoring importance. However, this is not limited thereto, and the determination unit 333 may calculate the monitoring importance only for the captured images of some of the flying robots 10 and determine the image display device 20 to be assigned based on the monitoring importance. For example, when a large number of flying robots 10 fly and the number of captured images that need to be displayed on the multiple image display devices 20 exceeds a threshold, the determination unit 333 may calculate the monitoring importance only for monitoring images that need to be newly assigned thereafter, and determine the image display device 20 to be assigned based on the monitoring importance.

[0092] Furthermore, the determination unit 333 may determine the image display device 20 to be assigned based on at least one of the information on the monitoring purpose, flight information, and security status acquired by the acquisition unit 332, without calculating the monitoring importance. For example, the server 30 may store in advance a correspondence relationship between a predetermined security status and the identification number of one or more image display devices 20, and the determination unit 333 may determine the image display device 20 to be assigned based on the security status of a monitored object included in the flight area in which the flying robot 10 flies, acquired by the acquisition unit 332, and the correspondence relationship. Furthermore, the determination unit 333 may determine the image display device 20 to be assigned based on the security status without using the monitoring purpose or flight information.

[0093] An image management device, an image management system, and an image display method according to one embodiment of the present invention can contribute to solving social issues such as a declining labor force and long working hours. In addition, the image management device, image management system, and image display method according to one embodiment of the present invention can also contribute to Goal 9 of the Sustainable Development Goals (SDGs) adopted by the United Nations, "Build resilient infrastructure, promote inclusive and sustainable industrialization, and promote inclusive and sustainable industrialization." [Explanation of symbols]

[0094] 1 monitoring system, 10 flying robot, 20, 40 image display device, 30 server, 331, 441 receiving unit, 332, 442 acquiring unit, 333, 443 determining unit, 334, 444 output control unit, 50 display device

Claims

1. a receiving unit that receives a plurality of captured images captured by each of a plurality of flying robots; an acquisition unit that acquires the security status of a monitored object included in a flight area in which each of the plurality of flying robots flies; a determination unit that determines, based on the security status, a monitoring terminal among a plurality of monitoring terminals that is to display each of the plurality of captured images; an output unit that outputs each of the plurality of captured images to the monitoring terminal determined by the determination unit; An image management device comprising:

2. The image management device according to claim 1 , wherein the determination unit preferentially assigns, to each of the plurality of monitoring terminals, images captured by flying robots having the same security status for the monitored property.

3. The image management device described in claim 1, wherein the determination unit assigns to separate monitoring terminals images taken by a flying robot flying in a flight area of ​​a monitored property where the security status is in a state where an abnormality is detected and images taken by a flying robot flying in a flight area of ​​a monitored property where the security status is in a state where no abnormality is detected.

4. The image management device described in claim 1, wherein the determination unit assigns to separate monitoring terminals images captured by a flying robot flying in the flight area of ​​a monitored property whose security status is in an alert set state and images captured by a flying robot flying in the flight area of ​​a monitored property whose security status is in an alert disengaged state.

5. 5. The image management device according to claim 1, wherein the determination unit determines the number of captured images to be displayed or an upper limit of the number of images to be displayed on the monitoring terminal based on the security status.

6. The determination unit determining the importance of monitoring the flying robot using the type of the monitored property and the security status of the monitored property; 5. The image management device according to claim 1, further comprising: determining a monitoring terminal on which the captured image is to be displayed based on the monitoring importance level.

7. The image management device described in any one of claims 1 to 4, wherein the decision unit preferentially assigns images captured by a flying robot flying in a flight area where a new abnormality has been detected to a monitoring terminal that currently displays a small number of the captured images.

8. The acquisition unit further acquires attribute information of a monitor who uses each of the plurality of monitoring terminals, 5. The image management device according to claim 1, wherein the determination unit determines a monitoring terminal on which to display each of the plurality of captured images, further based on the attribute information.

9. An image management system having an image management device and a plurality of monitoring terminals, The image management device a receiving unit that receives a plurality of captured images captured by each of a plurality of flying robots; an acquisition unit that acquires the security status of a monitored object included in a flight area in which each of the plurality of flying robots flies; a determination unit that determines, based on the security status, a monitoring terminal among the plurality of monitoring terminals that is to display each of the plurality of captured images; an output unit that outputs each of the plurality of captured images to the monitoring terminal determined by the determination unit, the monitoring terminal has a display unit that displays the captured image output from the image management device; An image management system characterized by:

10. receiving a plurality of captured images captured by each of a plurality of flying robots; Acquire the security status of the monitored object included in the flight area in which each of the plurality of flying robots flies; determining a monitoring terminal among a plurality of monitoring terminals to display each of the plurality of captured images based on the security status; outputting each of the plurality of captured images to the determined monitoring terminal; An image management method comprising:

Images