Information processing system and information processing method

Abstract

In the event of a disaster, drones equipped with cameras will be used to comprehensively patrol the area, acquiring image data to understand the disaster situation and prevent crime. [Solution] The present invention is an information processing system comprising an information processing device that acquires image data indicating the state of a region from an external source, and a plurality of unmanned aircraft having a shooting means, wherein each of the plurality of unmanned aircraft monitors the state of a region based on image data obtained by aerial photography of the region, wherein the information processing device has a route setting means that divides the area to be monitored into a plurality of sections and provides each of the plurality of unmanned aircraft with route setting information that assigns priority for image acquisition to each of the plurality of sections, and each of the plurality of unmanned aircraft has a flight control means that refers to the route setting information and sets a flight path that prioritizes moving to a section that has not yet been photographed and has a high priority.

Description

Technical Field

[0001] The present invention relates to an information processing system and an information processing method, and can be applied to, for example, a system for monitoring an area to grasp disaster damage conditions and prevent crimes when a disaster occurs.

Background Art

[0002] Non-Patent Document 1 discloses that in the event of a disaster, drones are used to fly around the affected area, photograph the damage status, and collect disaster information from image data and video data.

[0003] Patent Document 1 discloses comprehensively collecting image information obtained by fixed-point surveillance cameras, image information obtained by drone cameras, and image information obtained by residents.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Non-Patent Documents

[0005]

Non-Patent Document 1

Brief Summary of the Invention

Problem to be Solved by the Invention

[0006] For example, when natural disasters such as heavy rains or earthquakes occur, it is necessary to understand the extent of the damage in the affected areas. Furthermore, in disaster-stricken areas after a disaster, there is a possibility of crimes such as theft of houses and shops occurring amidst the chaos, so monitoring is necessary.

[0007] Fixed-point cameras are sometimes installed in specific locations such as city halls, and it is possible to check the extent of damage based on the footage from these cameras. However, because the installation locations are limited, the footage can only cover a limited area.

[0008] Furthermore, from a crime prevention perspective, installing new surveillance cameras can be time-consuming and costly.

[0009] Therefore, in light of the aforementioned challenges, there is a need for an information processing system and method that can acquire image data by comprehensively patrolling the area with drones equipped with photographic devices during a disaster, thereby understanding the disaster situation and deterring crime. [Means for solving the problem]

[0010] To solve these problems, the first part of the present invention is an information processing system comprising an information processing device that acquires image data indicating the state of a region from an external source, and a plurality of unmanned aircraft having a shooting means that can cooperate with the information processing device, wherein each of the plurality of unmanned aircraft monitors the state of a region based on image data obtained by aerial photography of the region, wherein the information processing device has a route setting means that divides the region to be monitored into a plurality of sections and assigns a priority for image acquisition to each of the plurality of sections and provides each of the plurality of unmanned aircraft with route setting information, and each of the plurality of unmanned aircraft has a flight control means that refers to the route setting information and sets a flight path that prioritizes moving to a section that has not yet been photographed and has a high priority.

[0011] The second aspect of the present invention is an information processing method comprising an information processing device that acquires image data indicating the state of a region from an external source, and a plurality of unmanned aircraft having a shooting means that can cooperate with the information processing device, wherein each of the plurality of unmanned aircraft monitors the state of a region based on image data obtained by aerial photography of the region, characterized in that the information processing device divides the region to be monitored into a plurality of sections, provides each of the plurality of unmanned aircraft with route setting information that assigns priority for image acquisition to each of the plurality of sections, and each of the plurality of unmanned aircraft refers to the route setting information to set a flight path that prioritizes movement to the section with the highest priority and photographs the state of the said section. [Effects of the Invention]

[0012] According to the present invention, in the event of a disaster, an unmanned aircraft equipped with a camera can be used to comprehensively patrol the area and acquire image data to capture the condition of the area, thereby enabling an understanding of the disaster situation and deterring crime. [Brief explanation of the drawing]

[0013] [Figure 1] This is an overall configuration diagram showing the overall configuration of the monitoring system according to the embodiment. [Figure 2] This is a configuration diagram showing the configuration of the drone according to the embodiment. [Figure 3] This is a configuration diagram showing the configuration of the information processing device according to the embodiment. [Figure 4] This is an explanatory diagram illustrating the map information to be monitored in this embodiment. [Figure 5] This is a sequence diagram showing the process of setting map information and video acquisition priority information for the drone in an embodiment. [Figure 6] This is a configuration diagram showing the configuration of route setting information according to the embodiment. [Figure 7] This is a sequence diagram showing the processing before the drone is deployed in the monitoring system according to the embodiment. [Figure 8] This is a flowchart illustrating how a drone searches for its own shooting area and determines a flight route to that shooting area. [Figure 9] In an embodiment, it is a diagram showing the flight route of the drone before the start of dispatch. [Figure 10] In the monitoring system according to an embodiment, it is a sequence diagram showing the processing after the start of drone dispatch. [Figure 11] In an embodiment, it is a diagram showing the flight route of the drone before the start of dispatch when a disaster occurs. [Figure 12] In the monitoring system according to an embodiment, it is a sequence diagram showing the flight completion processing for a drone section. MODE FOR CARRYING OUT THE INVENTION

[0014] (A) Main Embodiment Hereinafter, embodiments of the information processing system and information processing method according to the present invention will be described in detail with reference to the drawings.

[0015] In this embodiment, as an example of the information processing system of the present invention, a case where the invention is applied to a monitoring system that comprehensively flies a camera-equipped drone over a disaster-stricken area such as an urban area to quickly grasp the damage situation when a natural disaster such as an earthquake, heavy rain, or heavy snowfall occurs will be exemplified.

[0016] (A-1) Configuration of the Embodiment FIG. 1 is an overall configuration diagram showing the overall configuration of the monitoring system according to the embodiment. In FIG. 1, the monitoring system 1 according to the embodiment includes drones 10-1 to 10-N (N is a positive integer) equipped with a camera 130 serving as an imaging unit, an information processing device 40, an SNS (social network system) server 50, and a local government server 60, all of which are connectable to a network NT. Hereinafter, the drones 10-1 to 10-N basically have the same functions, and when describing common functions, they will be referred to as the drone 10.

[0017] (A-1-1) Drone 10 Drone 10 is an example of an unmanned aircraft, and this example illustrates a case where drone 10 is an unmanned aerial vehicle capable of flight by remote control or autonomous navigation. While it could also be an unmanned aircraft traveling on land, this example uses a drone 10 equipped with a camera to take aerial photographs of road damage.

[0018] Figure 2 is a configuration diagram showing the configuration of the drone 10 according to the embodiment. In Figure 2, the drone 10 according to the embodiment has a communication unit 110, a control unit 120, a camera 130, and a storage unit 150.

[0019] Camera 130 is mounted on drone 10 and is a means of filming the area to capture damage in urban areas and to prevent crime. To accurately recognize the extent of damage, camera 130 may be equipped with features such as a wide-angle lens, a zoom lens, and a high-resolution sensor.

[0020] The communication unit 110 transmits and receives information with the network NT. The communication unit 110 communicates information with other drones 10, the delivery vehicle 30, and the information processing device 40, for example, by broadcast communication. This allows the drone 10 to transmit information such as when it detects damage, information about the flight route of the drone 10, and information about when the drone 10 changes its flight route. The communication method is not limited to broadcast; it may also be multicast, which transmits and receives with nodes belonging to a specific group, or unicast, which transmits and receives with a specific node.

[0021] The control unit 120 is responsible for the various functions of the drone 10. For example, the hardware of the control unit 120 is a device having a CPU, ROM, RAM, EEPROM, etc. For example, processing programs such as monitoring programs are executed by the CPU (computer), thereby causing the CPU to function as a processing means for each functional block in Figure 2.

[0022] In Figure 2, the functions of the control unit 120 are broadly divided into a flight control unit 20 and a disaster detection control unit 21.

[0023] The flight control unit 20 controls the flight based on the flight route. As will be described later, the flight control unit 20 has pre-registered map information of the area to be monitored (i.e., the area to be photographed while flying), and controls the flight of the drone 10 based on the map information and the position information of the drone 10. The flight control unit 20 is capable of acquiring GPS information and can use the GPS information to determine the position information of the drone 10.

[0024] As illustrated in Figure 2, the flight control unit 20 includes a flight route acquisition unit 121, a flight route search unit 122, and a flight route transmission unit 123.

[0025] The flight route acquisition unit 121 acquires map information of the area to be monitored and video acquisition priority information, including the priority for acquiring video for damage status or crime monitoring, from the information processing device 40, creates route setting information, and stores it in the storage unit 150. A detailed explanation of the route setting information will be given later.

[0026] The flight route search unit 122 searches for the flight route of the drone 10 in the map information based on the drone's location information and the priority of video acquisition. Furthermore, when the flight route search unit 122 detects a disaster from the video captured by the drone 10, or when it obtains disaster detection information from the information processing device 40, it searches for a new flight route to prioritize acquiring video of the disaster site. The method for searching the flight route will be explained in detail in the operation section.

[0027] When the flight route of its own drone 10 is determined by the flight route search unit 122, the flight route transmission unit 123 transmits the flight route information of its own drone 10 to other drones 10 via the communication unit 110.

[0028] The disaster detection control unit 21 detects whether a disaster has occurred based on camera image data, including the road conditions in the relevant section, captured by the camera 130. The disaster detection control unit 21 also transmits the detection result indicating the occurrence of a disaster to the information processing device 40.

[0029] Here, the detection result indicating the occurrence of a disaster includes information such as "disaster occurrence area information," which will be described later. The detection result indicating the occurrence of a disaster includes area information where the drone 10 flew (for example, latitude and longitude, information that identifies the area, etc.), video footage taken by the drone 10, information indicating whether or not a disaster occurred based on video analysis, and the time of shooting.

[0030] As illustrated in Figure 2, the disaster detection control unit 21 includes a video acquisition unit 124, a disaster detection processing unit 125, and a detection result notification unit 126.

[0031] The video acquisition unit 124 acquires camera video data from the camera 130. The camera video data includes images and videos showing road conditions, damage to buildings and other structures in urban areas.

[0032] The disaster detection processing unit 125 analyzes camera video data to detect disasters.

[0033] Disaster detection methods can broadly apply a variety of techniques. For example, anomaly detection can be performed by focusing on the difference between the layout of a disaster area during a disaster and the layout of a non-disaster area. During a disaster, the layout of roads and their surrounding areas changes due to landslides, ground fissures, building collapses, floods, etc. Therefore, for example, a reference image dataset can be prepared by collecting only images taken from above in an environment without a disaster in advance and using these as reference images to learn the layout of roads, buildings and their surrounding areas as the "normal state." When camera video data is input from camera 130, the disaster detection processing unit 125 evaluates the degree of deviation between the layout of the surrounding areas of roads in the input camera video data and the "normal state" during training, and if the deviation is large, it may be determined to be an "abnormal state" for disaster detection.

[0034] In this embodiment, the control unit 120 of the drone 10 performs image analysis to detect a disaster, but the camera video data may be transmitted to an information processing device at a disaster center to detect the disaster in real time.

[0035] When the detection result notification unit 126 detects a disaster, it notifies the information processing device 40 via the communication unit 110 of the detection result, including information including the section containing the disaster location (hereinafter also referred to as "disaster location information"). This allows the information processing device 40 to be informed of the disaster location.

[0036] The memory unit 150 is a memory area that stores route information indicating the flight route, information such as sections including disaster locations acquired from other drones 10, and other such information.

[0037] (A-1-2) 60 local government servers, 50 SNS servers The local government server 60 is a management server operated by a local government for disaster countermeasures. Generally, it is assumed that the local government manages fixed-point surveillance cameras 30 (30-1 to 30-M (where M is an integer)) and that the local government server 60 manages the video from the fixed-point surveillance cameras 30.

[0038] In this embodiment, in cooperation with local governments, disaster information acquired by local governments is also utilized to confirm the extent of damage and prevent crime. Specifically, the information processing device 40 connects with the local government server 60 to acquire disaster information (especially video data) acquired by the local government.

[0039] Furthermore, the information processing device 40 may have the same functions as the disaster detection control unit 21 and detect disasters by analyzing video footage captured by the fixed-point surveillance camera 30 from the local government server 60. In this embodiment, the local government server 60 operated by a local government is given as an example, but it may also include government servers managed by government agencies such as the Cabinet Office or the Ministry of Land, Infrastructure, Transport and Tourism (Japan Meteorological Agency).

[0040] The SNS server 50 is accessible via the NT network to retrieve information uploaded to SNS. Immediately after a disaster occurs, disaster information and video are uploaded to SNS in real time, which is extremely useful for identifying high-priority areas that need to be checked. Therefore, the information processing device 40 accesses the SNS server 50 to retrieve disaster-related information (and instructional video data). Although Figure 1 shows only one SNS server 50, it is possible to retrieve disaster information (especially video data) from multiple SNSs.

[0041] (A-1-3) Information processing device 40 The information processing device 40 pre-configures map information for the drone 10 and transmits disaster information to the drone 10. The information processing device 40 also acquires disaster information and video data that are periodically uploaded from the SNS server 50. Furthermore, the information processing device 40 acquires disaster information obtained by the local government from the local government server 60.

[0042] The information processing device 40 can be, for example, a personal computer, a dedicated terminal, a tablet terminal, or a smartphone.

[0043] Figure 3 is a configuration diagram showing the configuration of an information processing device 40 according to the embodiment. In Figure 3, the information processing device 40 includes a control unit 410, a communication unit 420, and a storage unit 430.

[0044] The communication unit 420 transmits and receives information with the network NT.

[0045] The control unit 410 is responsible for the functions of the information processing device 40. For example, the hardware of the control unit 410 can be a device having a CPU, ROM, RAM, EEPROM, etc. For example, a processing program such as a flight route setting program that sets a flight route for flying the drone 10 based on the delivery route is installed, and these processing programs are executed by the CPU (computer), causing the CPU to function as a processing means for various purposes.

[0046] As illustrated in Figure 3, the functions of the control unit 410 include a flight route setting unit 411 that pre-sets map information and video acquisition priority information to be monitored, and a disaster information acquisition unit 412 that acquires video data as disaster information from the SNS server 50 and the local government server 60 to obtain video data for each area to be monitored.

[0047] The memory unit 430 is a memory area that stores information related to setting the flight route of the drone 10, disaster information obtained from the SNS server 50 and the local government server 60, and so on.

[0048] (A-2) Operation of the embodiment Next, the operation of the monitoring method using the drone 10 in the monitoring system 1 according to the embodiment will be described in detail with reference to the drawings.

[0049] (A-2-1) Pre-configuration process Figure 4 is an explanatory diagram illustrating the map information to be monitored in the embodiment. Figure 5 is a sequence diagram showing the process of setting map information and video acquisition priority information to the drone 10 in the embodiment.

[0050] Figure 4 shows an example of a map information structure where the monitored area is arranged in a matrix, for the sake of clarity. In the map information, the address (town name) is displayed as the identification information for each section, but this is not limited to this as long as the section can be identified.

[0051] Regarding the addresses in Figure 4, the town names are numbered "aa", "bb", ..., "gg" from north to south, and the districts are numbered "1-chome", "2-chome", ..., "6-chome" from west to east.

[0052] In Figure 4, the western area faces a mountain and is at risk of landslides, while the eastern area faces a river and is at risk of flooding. Therefore, these areas should be designated for immediate assessment of the extent of damage.

[0053] In Figure 4, fixed-point surveillance cameras 30 managed by the local government are installed in "A City aa3-chome" and "A City aa4-chome". Furthermore, images of the damage in "A City bb2-chome" and "A City cc2-chome" have been uploaded to social media.

[0054] Here, we will illustrate a scenario where four drones, 10-1 to 10-4, are used to divide the monitoring area and photograph the damage situation in each area.

[0055] [Steps S101, S102] First, the information processing device 40 creates map information and video acquisition priority information (step S101), and provides the map information and video acquisition priority information to the drone 10 (step S102).

[0056] [Step S103] When the drone 10 obtains map information and video acquisition priority information from the information processing device 40, it stores route setting information, including the video acquisition priority, in the storage unit 150 (step S103).

[0057] Figure 6 is a configuration diagram showing the configuration of route setting information according to the embodiment.

[0058] In Figure 6, the route setting information to be set for drone 10 includes the following items: "Shooting Area," "Image Acquisition Priority," "Other Drone Flight Schedule," "Distance from Other Drone's Scheduled Route," "Distance from Own Drone," "Shooting Completed," and "Last Date and Time Information." Route setting information is set for each drone 10.

[0059] The "Shooting Area" field contains information about the area in which the aircraft will fly within the monitored area. In this example, each section is treated as an address, so the address is listed in the "Shooting Area" field, but an identification number or similar information could also be used.

[0060] The "Image Acquisition Priority" field contains the image acquisition priority set in advance in the information processing device 40. The image acquisition priority is the priority assigned to the area that the drone 10 should prioritize photographing.

[0061] For example, the priority for acquiring video footage could be assigned to three levels: "High," "Medium," and "Low," from highest to lowest. In this example, areas facing mountains or rivers, areas likely to experience significant damage, or areas where there is already information indicating major damage, would be assigned a "High" priority because rapid situation assessment is necessary. Conversely, areas where drone footage has already been taken and the situation has been assessed, or areas where no damage has been confirmed, would be assigned a "Low" priority. Areas where footage has not yet been taken would be assigned a "Medium" priority. In this way, areas where rapid assessment of damage is necessary should be given a higher priority. Note that the video acquisition priority is not limited to three levels. Furthermore, the method and criteria for assigning video acquisition priority can be defined from various perspectives.

[0062] The "Other Drone Flight Schedule" section contains information indicating whether or not other drones (10) are scheduled to fly in the shooting area. If other drones (10) are scheduled to fly, their identification information is also included. For example, as illustrated in Figure 5, if it says "Other Drone Flight Schedule: Yes (10-2)," it indicates that "Drone 10-2" is scheduled to fly in the shooting area among the four drones 10-1 to 10-4.

[0063] The "Distance from the Planned Flight Path of Other Drones" field displays the distance between the current position of your drone 10 and the planned flight path of other drone 10, when other drone 10 is scheduled to fly. Here, distances less than 1km are denoted as "0(km)", distances between 1km and 2km are denoted as "1(km)", and distances between 2km and 3km are denoted as "2(km)".

[0064] The "Distance from the drone" field displays the distance between the shooting area and the current position of the drone 10. For example, a reference point can be defined in each section of the shooting area, and the distance between that reference point and the current position of the drone 10 can be calculated.

[0065] The "Photographed" field will contain information indicating whether or not the shooting area has already been photographed.

[0066] The "Last Shooting Date and Time" field displays the most recent date and time when drone 10 took the photos.

[0067] In the route setting information in Figure 6, "shooting area," "image acquisition priority," "other drone flight schedule," "photographed," and "last shooting date and time" are shared information among all drones 10. Therefore, drones 10-1 to 10-4 update their information when they acquire a flight route from another drone 10 or when they acquire the latest information from the information processing device 40.

[0068] On the other hand, the "distance from the planned path of other drones" and the "distance from the own drone" are updated by each of the drones 10-1 to 10-4 in accordance with the change in the current position of the own drone 10.

[0069] (A-2-2) Pre-deployment procedures for Drone 10 Figure 7 is a sequence diagram showing the processing before the drone 10 is deployed in the monitoring system 1 according to the embodiment.

[0070] [Steps S104, S105] The disaster information acquisition unit 412 of the information processing device 40 connects with the SNS server 50 and the local government server 60 to check whether there is video data within the monitored area, and if there is video data within the area (shooting area), it acquires that video data (step S104).

[0071] If video data is available within the monitored area, the disaster information acquisition unit 412 can understand the extent of the damage in that area. Therefore, it lowers the video acquisition priority for that area (shooting area) and then transmits the video priority information to each drone 10 (step S105). If there is no video data within the monitored area, the video priority is not updated.

[0072] For example, an operator accesses social media to check whether video data has been uploaded. If video data of the disaster situation in the monitored area has been uploaded, the disaster information acquisition unit 412 acquires that video data and the location information attached to it. Various methods can be widely applied to search for video data, and a method in which an operator searches using the functions of social media can be applied.

[0073] In the example shown in Figure 4, it is assumed that a user has uploaded video data of the damage situation in "A City bb2-chome" and "A City cc2-chome" to social media. In this case, the disaster information acquisition unit 412 identifies the area where the video was taken based on the location information attached to the video data. The disaster information acquisition unit 412 checks the video acquisition priority for the area and changes it to a lower priority if it is set high. In other words, video data for these areas has been acquired and the extent of the damage has been grasped. Therefore, the video acquisition priority for the drones 10 can be lowered, so for example, if it is set to "high", it is changed to "low" or "medium", and the changed video acquisition priority information is transmitted to all drones 10.

[0074] For example, in the event of a disaster, the fixed-point surveillance camera 30 may be able to capture video footage of the event. Here, we assume that the local government server 60 stores the video data from the fixed-point surveillance camera 30. In that case, if there is video data of the damage situation within the monitored area, the disaster information acquisition unit 412 acquires that video data and the location information of the fixed-point surveillance camera 30.

[0075] In the example in Figure 4, fixed-point surveillance cameras 30 are installed in "A City aa3-chome" and "A City aa4-chome," and the disaster information acquisition unit 412 is able to acquire video data of "A City aa3-chome" and "A City aa4-chome" when an incident occurs. In that case, since video of "A City aa3-chome" and "A City aa4-chome" has been acquired, the disaster information acquisition unit 412 checks the video acquisition priority for the area, changes it to a lower priority if it is high, and transmits the changed video acquisition priority information to all drones 10.

[0076] Although this explanation describes the process before deployment, access to the SNS server 50 and the local government server 60 is performed periodically even after deployment has begun. Therefore, each time the latest video data of the damage situation is acquired, the video acquisition priority is updated and communicated to all drones 10.

[0077] [Step S106] In each drone 10, when the flight route acquisition unit 121 receives video acquisition priority information from the information processing device 40, the flight route search unit 122 updates the route setting information based on the video acquisition priority (step S106).

[0078] This allows the latest video acquisition priority, based on footage posted on social media and images captured by fixed-point surveillance cameras 30, to be shared with all drones 10.

[0079] [Step S107] In drone 10-1, the flight route search unit 122 searches for and determines the flight route of its own drone 10 based on the route setting information (step S107).

[0080] Figure 8 is a flowchart showing how, in this embodiment, the drone 10 searches for its own shooting area and determines the flight route to that shooting area.

[0081] Figure 8 shows an example of the procedure for selecting a shooting area. The shooting area is narrowed down in the following steps S21 to S25, and once a shooting area is found, the flight route for that searched shooting area is determined.

[0082] First, the flight route search unit 122 refers to the "Photographed" item in the route setting information and selects an unphotographed shooting area (step S21). Here, if photography has been completed for all shooting areas, it may select a shooting area whose last shooting date and time has been set after a predetermined time.

[0083] Next, the flight route search unit 122 refers to the "Other Drone Flight Schedule" item in the route setting information and selects a shooting area that is not scheduled to be flown by the other drone 10 (step S22).

[0084] The flight route search unit 122 falsifies the "imaging acquisition priority" item in the route setting information and preferentially selects an imaging area with a "high" priority (step S23). If there is no area with a "high" priority at this time, the process may proceed to step S24.

[0085] Alternatively, if there is no "high" priority area, you may select a shooting area with a "medium" priority. In other words, if the priority is "medium," select that area; if the priority is "low," proceed to step S24.

[0086] The flight route search unit 122 refers to the "distance from the drone" item in the route setting information and selects a shooting area close to the current location information of the drone 10 (step S24).

[0087] Furthermore, the flight route search unit 122 refers to the route setting information item "distance from the planned flight path of another drone" and selects an area that is far from the planned flight path of the other drone 10 (step S25).

[0088] [Steps S108, S109, S110, S111] In drone 10-1, once the flight route search unit 122 determines the flight route of drone 10-1, the flight route transmission unit 123 transmits the flight route of drone 10-1 to the other drones 10-2 to 10-4 (step S108).

[0089] In the other drone 10-2, the flight route acquisition unit 121 receives the flight route of drone 10-1 (step S109), and the flight route acquisition unit 121 updates the route setting information item "Other drone flight schedule" based on the flight route of drone 10-1 (step S110).

[0090] Furthermore, the processing in steps S106 to S110 is performed for each of the drones 10-1 to 10-4. That is, based on the flight routes of other drones 10, the drone 10 updates the "Flight Schedule of Other Drones" item in its own route setting information, and then searches its own drone 10's shooting area to determine its flight route. This kind of communication is repeated sequentially among the drones 10.

[0091] Figure 9 shows the flight routes of drones 10-1 to 10-4 before deployment in one embodiment.

[0092] In the example shown in Figure 9, drone 10-1 flies in a straight line to capture images in an area facing a mountain, where video capture is a high priority. The flight route of drone 10-1 is "A City aa 1-chome" → "A City bb 1-chome" → "A City cc 1-chome" → "A City dd 1-chome" → "B City ee 1-chome" → "B City ff 1-chome" → "B City gg 1-chome".

[0093] Drone 10-2 will fly in a straight line to capture images in areas facing the river where video capture is a high priority. The flight route for Drone 10-2 will be "A City aa 6-chome" → "A City bb 6-chome" → "A City cc 6-chome" → "A City dd 6-chome" → "B City ee 6-chome" → "B City ff 6-chome" → "B City gg 6-chome".

[0094] Drone 10-3 will fly and take photos in the area from "A City bb3-chome" to "A City cc3-chome".

[0095] Drone 10-4 will fly and photograph the area from "B City gg 3-chome" to "B City ff 3-chome" to "B City ff 4-chome" to "B City gg 4-chome".

[0096] (A-2-3) After the deployment of drone 10 Figure 10 is a sequence diagram showing the processing after the drone 10 is deployed in the monitoring system 1 according to the embodiment.

[0097] Here, we will explain the case where drone 10-1 detects a disaster, representing drones 10-1 to 10-4, but drones 10-2 to 10-4 will perform the same processing as drone 10-1.

[0098] [Step S111] The drone 10 begins flying based on the flight route set before deployment, and while flying, the drone 10 takes pictures of the situation in the shooting area with the camera 130 (step S111).

[0099] [Steps S112, S113] In the deployed drone 10, the video acquisition unit 124 acquires camera image data captured by the camera 130, the disaster detection processing unit 125 analyzes the camera image data (step S112), and the disaster detection processing unit 125 confirms whether or not a disaster has occurred (step S113).

[0100] When the disaster detection processing unit 125 analyzes the camera image data and detects the occurrence of a disaster (step S113 / YES), the process proceeds to step S113.

[0101] Furthermore, when the information processing device 40 acquires disaster occurrence information through the SNS server 50, the local government server 60, and the deployed drones 10, it transmits that disaster occurrence information to all drones 10. At this time, the disaster detection processing unit 125 monitors the reception of disaster occurrence information, and when disaster occurrence information is received (step S113 / YES), it proceeds to step S113.

[0102] On the other hand, if no disaster is detected and no information about the disaster is received (step S113 / NO), the process returns to S110, and the drone 10 continues to analyze the camera image data while flying.

[0103] Here, we will illustrate a case where drone 10-1 detects a disaster and, acting as a representative of the other drones 10, communicates with drone 10-2 to convey a change in the flight path.

[0104] [Steps S114, S115] Steps S114 and S115 are processes performed by the drone 10-1 after it has detected the occurrence of a disaster.

[0105] When the drone 10-1 detects the occurrence of a disaster, the detection result notification unit 126 transmits the disaster occurrence section information to the information processing device 40 (step S114). In the information processing device 40, the disaster information acquisition unit 412 acquires the disaster occurrence section information from the drone 10-1 and stores it in the storage unit 430 (step S115).

[0106] Here, disaster occurrence area information includes at least the disaster location, indicated by, for example, latitude and longitude, address, etc., and area identification information for the disaster location. In addition to this information, disaster occurrence area information may also include one or more camera images of the damage, the time of shooting, and other information.

[0107] [Steps S116, S117] Since a disaster has been detected, it is necessary to quickly assess the extent of the damage in the affected area.

[0108] Therefore, in order to prioritize flying around the disaster site, the flight route acquisition unit 121 of the drone 10-1 refers to the route setting information and changes the "image acquisition priority" item for the shooting area of ​​the disaster site to a higher value. Subsequently, the flight route search unit 122 uses the modified route setting information to change the flight route of the drone 10-1 (step S116).

[0109] At this time, using the route setting information with the changed video acquisition priority, the flight route of the drone 10-1 is modified according to the flowchart in Figure 8.

[0110] Depending on the current location of the drone, the priority for acquiring video footage from the area where the disaster occurred is higher, so it is possible to prioritize selecting the shooting area with the highest priority.

[0111] To prevent the flight route from overlapping with that of other drones 10-2, the flight route transmission unit 123 of drone 10-1 transmits flight route change information to other drones 10-2 (step S117).

[0112] [Steps S118, S119] In Drone 10-2, the priority for acquiring video footage from the disaster site has also been increased. In addition, Drone 10-2 obtains flight route change information from Drone 10-1 and changes the "Other Drone Flight Schedule" item in the route setting information.

[0113] Subsequently, if the flight route of drone 10-1 overlaps with that of drone 10-2, the flight route search unit 122 of drone 10-2 changes the flight route of drone 10-2 using the modified route setting information (step S118).

[0114] If drone 10-2 changes its flight path, the flight path transmission unit 123 of drone 10-2 transmits the changed flight path to the other drones 10-1, 10-3 to 10-4 (step S119).

[0115] [Step S120] When drone 10-1 receives a modified flight route from drone 10-2, the flight route acquisition unit 121 updates the route setting information item "Other drone flight schedule" based on the flight route of drone 10-2 (step S120).

[0116] Here, an example of the process in steps S111 to S120 will be explained using Figure 11.

[0117] Figure 11 shows the flight routes of drones 10-1 to 10-4 before deployment in the event of a disaster, in an embodiment.

[0118] Assume that disasters have occurred in "City A, cc6-chome" and "City B, ff2-chome". In each drone 10-1 to 10-4, the flight route acquisition unit 121 changes the priority of acquiring video footage of the disaster site to "high".

[0119] In this situation, since the disaster may have occurred not only in the area where the disaster occurred but also in the surrounding areas, it is necessary to quickly grasp the situation there as well, so the priority for acquiring video footage of the surrounding areas may also be changed to "high".

[0120] For example, if the disaster occurred in "City A, cc6-chome," the surrounding area Y1 would be set to "High" for acquiring video footage of "City A, bb5-chome," "City A, bb6-chome," "City A, cc5-chome," "City A, dd5-chome," and "City A, dd6-chome."

[0121] Similarly, as the surrounding area Y2 of "B City ff2-chome" where the disaster occurred, the priority for acquiring video footage of "B City gg2-chome," "B City ee2-chome," and "A City ee3-chome" will be set to "High."

[0122] In Figure 11, after the deployment begins, drones 10-1 and 10-2 are already flying in areas with a "high" priority for video acquisition, making it difficult to change their flight routes.

[0123] On the other hand, drones 10-3 and 10-4 can each change their flight routes. After changing the route setting information, drones 10-3 and 10-4 each change their flight routes as shown in Figure 11, according to the flowchart in Figure 8.

[0124] For example, the flight route of drone 10-3 before deployment was "A City bb3-chome" → "A City cc3-chome", but it changes to "A City bb3-chome" → "A City cc3-chome" → "A City cc4-chome" → "A City cc5-chome" → "A City dd6-chome" → "A City cc5-chome" → "A City bb5-chome".

[0125] Similarly, for example, the flight route of drone 10-4 before deployment was "B City gg 3-chome" → "B City ff 3-chome" → "B City ff 4-chome" → "B City gg 4-chome", but it becomes "B City gg 3-chome" → "B City ff 3-chome" → "B City gg 2-chome" → "B City ff 2-chome" → "B City ee 2-chome" → "B City ee 3-chome".

[0126] In this way, the area where the disaster occurred can be photographed preferentially, allowing for a rapid assessment of the situation. Furthermore, by communicating their flight routes to each other, the other drones can change their flight routes, thus avoiding overlapping flight paths.

[0127] (A-2-4) Flight completion process for each section Figure 12 is a sequence diagram showing the flight completion process for the drone 10 in the monitoring system 1 according to the embodiment.

[0128] Drone 10 continues to take photographs while flying (step S121), and when the flight of the relevant section is completed (step S122), the flight route transmission unit 123 transmits a flight completion notification, including the area where the flight has been completed and the time of flight completion, to the other drone 10-2 (step S123).

[0129] When the other drone 10-2 receives a flight completion notification from drone 10-1, it receives and stores the flight completion notification (step S124).

[0130] Furthermore, not only drone 10-1, but also drones 10-2 through 10-4 will undergo the same processing as drone 10-1.

[0131] In the drone 10-1, the flight route search unit 122 checks whether there are any uncaptured shooting areas by referring to the route setting information (step S125). If there are uncaptured shooting areas (step S125 / there are), the process returns to step S106 and is repeated. If the last shooting date and time has exceeded a predetermined time, the process may be returned to step S106 in order to capture images of that shooting area.

[0132] On the other hand, if there are no areas that have not yet been photographed (step S125 / none), it can be said that the photography of the damage situation in all monitored areas has been completed. In this case, the flight route search unit 122 switches to a crime prevention flight route (step S126), and the flight route transmission unit 123 transmits the crime prevention flight route to the other drones 10-2 (step S127).

[0133] Other drones 10-2 receive and memorize crime prevention flight routes from drone 10-1 (step S128).

[0134] The flight route for crime prevention can be a flight route that patrols a pre-defined area. In particular, the flight route can be designed to comprehensively patrol areas where fixed surveillance cameras 30 are not installed, with the drone 10 taking photographs while flying. In this way, crime can be prevented by having the drone 10 patrol.

[0135] (A-3) Effects of the embodiment As described above, according to this embodiment, when natural disasters such as heavy rain or earthquakes occur, multiple camera-equipped drones that have been on standby fly over urban areas and take photographs. By determining the flight route according to "the degree of risk of natural disaster," "whether footage has already been taken," and "whether there are areas where time has passed since the last shooting," it becomes possible to grasp the latest information on the disaster situation and deter criminal activity.

[0136] (B) Other embodiments Although various modified embodiments were mentioned in the embodiments described above, the following modified embodiments can also be applied.

[0137] (B-1) In the embodiment described above, when setting and updating the flight route of the drone by referring to the route setting information, an example was given in which the area is determined in the order illustrated in Figure 8. However, the order is not limited to Figure 8, as long as it takes into account that the area has not been photographed, that a disaster has occurred, that it is an area close to the drone, and that it avoids overlapping photography with other drones. Furthermore, while the process illustrated in Figure 8 is conceivable, the order is not limited to the order in Figure 8.

[0138] (B-2) The drone 10, as an unmanned aircraft, may analyze the captured images to determine the extent of the damage. For example, a method for evaluating the extent of damage using machine learning or the like can be applied. [Explanation of Symbols]

[0139] 1: Surveillance system, NT: Network, 30: Fixed-point surveillance camera, 50: SNS server, 60: Municipal server, 10(10-1~10-N): Drone, 20: Flight control unit, 21: Disaster detection control unit, 110: Communication unit, 120: Control unit, 121: Flight route acquisition unit, 122: Flight route search unit, 123: Flight route transmission unit, 124: Video acquisition unit, 125: Disaster detection processing unit, 126: Detection result notification unit, 130: Camera, 150: Memory unit, 40: Information processing unit, 410: Control unit, 411: Flight route setting unit, 412: Disaster information acquisition unit, 420: Communication unit, 430: Memory unit.

Claims

1. An information processing system comprising an information processing device that acquires image data indicating the state of a region from an external source, and a plurality of unmanned aircraft having a shooting means that can cooperate with the information processing device, wherein each of the plurality of unmanned aircraft monitors the state of the region based on image data obtained by aerial photography of the region, The aforementioned information processing device The system has a routing means that divides the area to be monitored into multiple sections and provides each of the multiple unmanned aircraft with routing information that assigns priority to image acquisition to each of the multiple sections. Each of the aforementioned multiple unmanned aircraft, The flight control means has a system that, by referring to the aforementioned route setting information, sets a flight path that prioritizes movement to areas that have not yet been photographed and have a high priority. An information processing system characterized by the following:

2. Each of the plurality of unmanned aircraft is equipped with disaster detection control means that detects whether or not a disaster has occurred based on image data including the state of the area captured by the imaging means, and notifies the information processing device of the disaster detection result. Each of the aforementioned multiple unmanned aircraft is When the disaster detection control means itself detects the occurrence of a disaster in the area, or when it receives information indicating the occurrence of a disaster from the information processing device, The flight control means updates the priority of the disaster-affected area in the route setting information and updates the flight path by referring to the updated route setting information. The information processing system according to feature 1.

3. Each of the disaster detection and control means of the plurality of unmanned aircraft, A disaster detection processing unit analyzes image data including the state of the section captured by the aforementioned photographic means to detect whether or not a disaster has occurred based on the damage status of the section, A detection result transmission unit that transmits the detection results from the disaster detection processing unit to the information processing unit. The information processing system according to claim 2, characterized by having the following features.

4. Each of the aforementioned flight control means for the plurality of unmanned aircraft, A flight route search unit that, upon detecting a disaster, updates the route setting information to prioritize the affected area and its surrounding areas, and then modifies its own flight path to prioritize movement to the higher-priority area by referring to the updated route setting information. A flight route transmission unit that transmits flight route change information, including its own modified flight path, to other unmanned aircraft. The information processing system according to claim 2, characterized by comprising:

5. Of the plurality of unmanned aircraft, the flight control means of the first unmanned aircraft transmits the flight route change information to the second unmanned aircraft, which is another unmanned aircraft. The flight control means of the second unmanned aircraft changes the flight path of the second unmanned aircraft if the modified flight path of the first unmanned aircraft, which is included in the received flight path change information, overlaps with the flight path of the second unmanned aircraft. The information processing system according to feature 2.

6. Each of the aforementioned flight control means for the plurality of unmanned aircraft, The information processing system according to claim 1, characterized in that, by referring to the route setting information, if the latest date and time of photographing the state of a certain section has elapsed a predetermined time, the flight path including movement to that section is set.

7. An information processing method comprising an information processing device that acquires image data indicating the state of a region from an external source, and a plurality of unmanned aircraft having a shooting means that can cooperate with the information processing device, wherein each of the plurality of unmanned aircraft monitors the state of a region based on image data obtained by aerial photography of the region, The aforementioned information processing device The area to be monitored is divided into multiple sections, and route setting information assigning priority for image acquisition to each of the multiple unmanned aircraft is provided to each of the multiple sections. Each of the aforementioned multiple unmanned aircraft, Referencing the aforementioned route setting information, a flight path prioritizing movement to the high-priority section is set, and the state of that section is photographed. An information processing method characterized by the following:

Images