Information processing system and information processing method

Abstract

In the event of a disaster, the system will use onboard cameras to photograph and analyze routes, allowing for the exploration of supply delivery routes and contributing to a rapid initial response. [Solution] The present invention is a system for which a plurality of unmanned aircraft derive a delivery route that allows for the delivery of goods based on image data obtained by aerial photography of the road conditions of a route connecting a first city and a second city, comprising: flight control means for controlling the flight of a plurality of unmanned aircraft based on a flight route that includes sections divided into a plurality of parts, each of a plurality of routes between the first city and the second city; disaster detection control means for detecting whether or not a disaster has occurred based on image data including the road conditions of the photographed sections; and delivery route derivation means for collecting detection results indicating whether or not a disaster has occurred for each section obtained from the disaster detection control means of each of the plurality of unmanned aircraft, and deriving a delivery route that bypasses sections including roads that are impassable.

Description

[Technical Field]

[0001] The present invention relates to an information processing system and an information processing method, and is applicable to, for example, a monitoring device, a monitoring system, and a monitoring method for monitoring road damage conditions when a disaster occurs. [Background Art]

[0002] Non-Patent Document 1 discloses collecting road damage conditions by utilizing drones in the event of a disaster.

[0003] In addition, Non-Patent Document 2 discloses that when transporting supplies by drone during a disaster, a flight route is set to manage navigation.

[0004] Furthermore, Patent Document 1 discloses that a drone flies along a flight route that avoids dangerous areas. [Prior Art Documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2019-202682 [Non-Patent Documents]

[0006] [Non-Patent Document 1] New Energy and Industrial Technology Development Organization (NEDO), Guidelines for Utilization of Drones in Disasters "Project for Realization of Energy-Saving Society where Robots and Drones Play an Active Role / Development of Unmanned Aerial Vehicle Operation Management System and Collision Avoidance Technology / Demonstration Project of Operation Management System Considering Regional Characteristics and Scalability", February 24, 2022 (Reiwa 4) [Non-Patent Document 2] NEXT DELIVERY Co., Ltd., "Demonstration experiment of drone logistics in preparation for disasters conducted in Nisshin City", December 6, 2023, PR TIMES, https: / / prtimes.jp / main / html / rd / p / 000000024.000093605.html [Overview of the project] [Problems that the invention aims to solve]

[0007] For example, when natural disasters such as heavy rains or earthquakes occur, it is necessary to deliver relief supplies to the affected areas, but delivery routes may be blocked by landslides, road closures, etc.

[0008] In this case, it is necessary to restore the delivery route using heavy machinery, but because there is no information on the condition of the delivery route immediately after a natural disaster, requests for the deployment of heavy machinery can only be made after they have actually arrived on site and assessed the situation, which presents a challenge as it can lead to delays in the initial response.

[0009] Therefore, there is a need for an information processing system and method that can contribute to a rapid initial response by using photographic equipment mounted on unmanned aerial vehicles to photograph and analyze routes in the event of a disaster, thereby searching for supply delivery routes. [Means for solving the problem]

[0010] To solve these problems, the first aspect of the present invention is an information processing system in which each of a plurality of unmanned aircraft having a shooting means derives a delivery route in which goods can be delivered based on image data obtained by aerial photography of the road conditions of a route connecting a first city and a second city, characterized in that (1) each of the plurality of unmanned aircraft has (1-1) flight control means that controls the flight based on a flight route that includes sections obtained by dividing each of the plurality of routes connecting the first city and the second city into a plurality of sections, and (1-2) disaster detection control means that detects whether or not a disaster has occurred based on image data including the road conditions of the section photographed by the shooting means, and (2) delivery route derivation means that collects detection results indicating whether or not a disaster has occurred for each section obtained from the disaster detection control means of each of the plurality of unmanned aircraft, and derives a delivery route that bypasses sections including roads that are impassable.

[0011] The second aspect of the present invention is an information processing method for which a plurality of unmanned aircraft, each having a photography means, derive a delivery route that allows for the delivery of goods based on image data obtained by aerial photography of the road conditions of a route connecting a first city and a second city, characterized in that (1) each of the plurality of unmanned aircraft (1-1) performs flight control based on a flight route that includes sections obtained by dividing each of the plurality of routes connecting the first city and the second city into a plurality of sections, (1-2) detects whether or not a disaster has occurred based on image data including the road conditions of the section photographed by the photography means, and (2) a delivery route derivation means collects detection results indicating whether or not a disaster has occurred for each section obtained from each of the plurality of unmanned aircraft, and derives a delivery route that bypasses sections including roads that are impassable. [Effects of the Invention]

[0012] According to the present invention, in the event of a disaster, a photographic device mounted on an unmanned aerial vehicle can be used to photograph and analyze routes, thereby searching for supply delivery routes and contributing to a rapid initial response. [Brief explanation of the drawing]

[0013] [Figure 1] This is an overall configuration diagram showing the overall configuration of the monitoring system according to the embodiment. [Figure 2]It is a configuration diagram showing the configuration of a drone according to an embodiment. [Figure 3] It is a configuration diagram showing the configuration of an information acquisition device mounted on a delivery vehicle according to an embodiment. [Figure 4] It is a configuration diagram showing the configuration of an information processing apparatus according to an embodiment. [Figure 5] It is an explanatory diagram for explaining a delivery route in an embodiment. [Figure 6] It is a sequence diagram showing a process for setting route information to a drone in a monitoring system according to an embodiment. [Figure 7] It is a configuration showing the configuration of information indicating a flight route according to an embodiment. [Figure 8] It is a sequence diagram showing a process at the start of drone dispatch in the monitoring system according to an embodiment. [Figure 9] It is a sequence diagram showing a process after the start of drone dispatch in the monitoring system according to an embodiment. [Figure 10] It is a flowchart showing a flight route change process of a drone that has detected a disaster occurrence in an embodiment. [Figure 11] It is a flowchart showing a flight route change process of another drone that has received flight route change information in an embodiment. [Figure 12] It is a sequence diagram showing a flight completion process for each section of a drone in the monitoring system according to an embodiment. [Figure 13] It is a sequence diagram showing a delivery route determination process in the monitoring system according to an embodiment. [Figure 14] It is an explanatory diagram for explaining the processes from step S105 to step S109. [Figure 15] It is an explanatory diagram for explaining the processes from step S113 to step S119. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0014] (A) Main Embodiment Hereinafter, embodiments of the information processing system and information processing method according to the present invention will be described in detail with reference to the drawings.

[0015] In this embodiment, as an example of the information processing system of the present invention, a monitoring system is described that, when a natural disaster such as an earthquake, heavy rain, or heavy snow occurs, photographs the road by flying a camera-equipped drone along a main road or other road from the source to the destination of relief supplies, thereby quickly grasping the road conditions. In particular, a monitoring system that contributes to initial response is given as an example.

[0016] (A-1) Configuration of the embodiment Figure 1 is an overall configuration diagram showing the overall configuration of the monitoring system according to the embodiment. In Figure 1, the monitoring system 1 according to the embodiment includes drones 10-1 to 10-N (where N is a positive integer) equipped with cameras 130 as imaging units, delivery vehicles 30-1 to 30-M (where M is a positive integer) for delivering support supplies, and an information processing device 40.

[0017] In the following, drones 10-1 to 10-N essentially have the same functions, and when describing common functions, they will be referred to as drone 10. The same applies to delivery vehicles 30-1 to 30-M.

[0018] In the monitoring system 1, the drones 10-1 to 10-N, the delivery vehicles 30-1 to 30-M, and the information processing device 40 all have communication capabilities that allow them to connect to the network NT.

[0019] (A-1-1) Drone 10 Drone 10 is an example of an unmanned aircraft, and this example illustrates a case where drone 10 is an unmanned aerial vehicle capable of flight by remote control or autonomous navigation. While it could also be an unmanned aircraft traveling on land, this example uses a drone 10 equipped with a camera to take aerial photographs of road damage.

[0020] Figure 2 is a configuration diagram showing the configuration of the drone 10 according to the embodiment. In Figure 2, the drone 10 according to the embodiment has a communication unit 110, a control unit 120, a camera 130, and a storage unit 150.

[0021] Camera 130 is mounted on drone 10 and is a means of capturing images of road conditions. To accurately recognize the extent of damage to the road, camera 130 may be equipped with, for example, a wide-angle lens, a zoom lens, and a high-resolution sensor.

[0022] The communication unit 110 transmits and receives information with the network NT. The communication unit 110 communicates information with other drones 10, the delivery vehicle 30, and the information processing device 40, for example, by broadcast communication. This allows the drone 10 to transmit information such as when it detects damage, information about the flight route of the drone 10, and information about when the drone 10 changes its flight route. The communication method is not limited to broadcast; it may also be multicast, which transmits and receives with nodes belonging to a specific group, or unicast, which transmits and receives with a specific node.

[0023] The control unit 120 is responsible for the various functions of the drone 10. For example, the hardware of the control unit 120 is a device having a CPU, ROM, RAM, EEPROM, etc. For example, processing programs such as monitoring programs are executed by the CPU (computer), thereby causing the CPU to function as a processing means for each functional block in Figure 2.

[0024] In Figure 2, the functions of the control unit 120 are broadly divided into a flight control unit 20 and a disaster detection control unit 21.

[0025] The flight control unit 20 controls the flight based on the flight route. As illustrated in Figure 2, the flight control unit 20 includes a flight route acquisition unit 121, a flight route search unit 122, and a flight route transmission unit 123.

[0026] The flight route acquisition unit 121 acquires information indicating the flight route from the delivery source to the delivery destination from the information processing device 40 and stores it in the storage unit 150. This allows the flight route information to be pre-set.

[0027] When the disaster detection processing unit 125 detects the occurrence of a disaster, the flight route search unit 122 searches for a flight route that bypasses the disaster site. Furthermore, if another drone 10 searches for a flight route, the flight route search unit 122 changes its own flight route to avoid overlapping flight photography sections. The method for searching for a flight route will be explained in detail in the operation section.

[0028] When the flight route transmission unit 123 changes its own flight route, it transmits flight route change information indicating the flight route of its own drone 10 to other drones 10 via the communication unit 110.

[0029] The disaster detection control unit 21 detects whether a disaster has occurred based on camera image data, including the road conditions in the section captured by the camera 130. The disaster detection control unit 21 also transmits the detection result indicating whether a disaster has occurred to the other drones 10 and delivery vehicles 30. As illustrated in Figure 2, the disaster detection control unit 21 includes a video acquisition unit 124, a disaster detection processing unit 125, and a detection result notification unit 126.

[0030] The video acquisition unit 124 acquires camera video data from the camera 130. The camera video data includes images and videos of road conditions.

[0031] The disaster detection processing unit 125 analyzes camera video data to detect disasters.

[0032] Disaster detection methods can broadly apply a variety of techniques. For example, anomaly detection can be performed by focusing on the difference between the layout of a road during a disaster and during a non-disaster state. During a disaster, the layout of roads and their surrounding areas changes due to landslides, ground fissures, building collapses, floods, etc. Therefore, for example, a reference image dataset can be prepared by collecting only images taken from above in an environment without a disaster in advance and using these as reference images to learn the layout of roads and their surrounding areas as the "normal state." When camera video data is input from camera 130, the disaster detection processing unit 125 evaluates the degree of deviation between the layout of the road surrounding areas in the input camera video data and the "normal state" during learning, and if the deviation is large, it may be determined to be an "abnormal state" for disaster detection.

[0033] In this embodiment, the control unit 120 of the drone 10 performs image analysis to detect a disaster, but the camera video data may be transmitted to an information processing device at a disaster center to detect the disaster in real time.

[0034] When the detection result notification unit 126 detects a disaster, it notifies the communication unit 110 of the detection result, including information including the section containing the disaster location (hereinafter also referred to as "disaster location information"). This allows other drones 10 and delivery vehicles 30 to be notified of the disaster location.

[0035] The memory unit 150 is a memory area that stores route information indicating the flight route, information such as sections including disaster locations acquired from other drones 10, and other such information.

[0036] (A-1-2) Delivery vehicle 30 The delivery vehicle 30 is a mobile vehicle used to deliver goods, and can be a truck or other vehicle. The delivery vehicle 30 has an information acquisition device 31 that acquires information related to the occurrence of a disaster using the monitoring system 1.

[0037] Figure 3 is a configuration diagram showing the configuration of an information acquisition device 31 mounted on a delivery vehicle 30 according to the embodiment. In Figure 3, the information acquisition device 31 includes a control unit 310, a communication unit 320, and a storage unit 330.

[0038] The communication unit 320 transmits and receives information with the network NT.

[0039] The control unit 310 is responsible for the function of the information acquisition device 31. For example, the hardware of the control unit 310 can be a device having a CPU, ROM, RAM, EEPROM, etc. For example, processing programs such as an information acquisition program that acquires information about the disaster-stricken area and a detour route search program are installed, and these processing programs are executed by the CPU (computer), causing the CPU to function as a means of various processing.

[0040] The control unit 310 includes a detection result acquisition unit 311 that acquires information about the section including the disaster site from the drone 10 via the communication unit 320 and stores it in the storage unit 330, and a delivery route derivation unit 312 that collects camera image data for each section taken by each of the drones 10 and derives the delivery route from the delivery source to the delivery destination. This makes it possible to recognize the section including the disaster site detected by the camera-equipped drone 10.

[0041] The memory unit 330 is a memory area that stores information such as the section containing the disaster location, which is acquired from the drone 10.

[0042] (A-1-3) Information processing device 40 The information processing device 40 pre-sets route information for the drone 10, indicating the flight route from the delivery source to the delivery destination. The information processing device 40 can be, for example, a personal computer, a dedicated terminal, a tablet terminal, or a smartphone.

[0043] Figure 4 is a configuration diagram showing the configuration of an information processing device 40 according to an embodiment. In Figure 4, the information processing device 40 includes a control unit 410, a communication unit 420, and a storage unit 430.

[0044] The communication unit 420 transmits and receives information with the network NT.

[0045] The control unit 410 is responsible for the functions of the information processing device 40. For example, the hardware of the control unit 410 can be a device having a CPU, ROM, RAM, EEPROM, etc. For example, a processing program such as a flight route setting program that sets a flight route for flying the drone 10 based on the delivery route is installed, and these processing programs are executed by the CPU (computer), causing the CPU to function as a processing means for various purposes.

[0046] As illustrated in Figure 4, the functions of the control unit 410 include a flight route setting unit 411 that sets a flight route for the drone 10 based on the delivery route, and a flight route transmission unit 412 that transmits route information indicating the set flight route to the drone 10.

[0047] The memory unit 430 is a memory area that stores information related to setting the flight route of the drone 10.

[0048] (A-2) Operation of the embodiment Next, the operation of the monitoring method using the drone 10 in the monitoring system 1 according to the embodiment will be described in detail with reference to the drawings.

[0049] (A-2-1) Flight route setting process Figure 5 is an explanatory diagram illustrating the delivery route in an embodiment. Figure 6 is a sequence diagram showing the process of setting route information to the drone 10 in the monitoring system 1 according to the embodiment.

[0050] In Figure 5, "A" is the base from which supplies are distributed (also called "City 1"), and "B" is the city to which supplies are distributed (also called "City 2"). "C" is a city located between "A" and "B", and is the city where the disaster occurred.

[0051] Let's assume that the drones 10 are pre-positioned. For example, one drone 10-1 is positioned at delivery source A, two drones 10-2 and 10-3 are positioned at delivery destination B, and one drone 10-4 is positioned at city C.

[0052] For the sake of explanation, we will focus on drones 10-1 and 10-2 among drones 10-1 to 10-4 to explain the processing operation of drone 10, but the other drones perform the same processing operation.

[0053] The delivery route from origin "A" to destination "B" shall consist of important roads connecting key locations, such as main roads. Of course, there may also be detours, roads with little traffic, or narrow roads, and these may also be included as roads to be used in the delivery route. However, from the perspective of roads that can be used for goods delivery, this section will focus on important roads such as main roads and wide roads.

[0054] "R1" through "R12" are sections that divide the route from the delivery source to the delivery destination. Drone 10 flies along one of the roads in one of the sections "R1" through "R12" as its flight route, and photographs the road conditions in that section. In other words, "R1" through "R12" can also be said to be sections that divide the flight route of drone 10.

[0055] The route can be divided (created) into sections based on factors such as whether or not there are branches and the length of the sections. For example, since multiple drones 10 may end up filming the same road (section) repeatedly, dividing the sections at branches is effective. Also, since one section becomes the flight route of the drone 10, if the section length is long, the flight distance will also be long, which may affect the selection of an efficient flight route, so dividing the sections by distance is effective. Sections can be divided from these perspectives, but they may also be divided from other perspectives.

[0056] [Steps S101, S102] First, the information processing device 40 creates information indicating the flight route of the drone 10 in order to create a route from the delivery source A to the delivery destination B (step S101), and provides the information indicating the flight route to the drone 10 (step S102).

[0057] Figure 7 shows the configuration of information indicating a flight route according to the embodiment. As illustrated in Figure 7, the route information indicating the flight route of the drone 10 has "flight route identification information" and "flight route" as items, and the "flight route identification information" and "flight route" are associated. Note that the items are not limited to these, and the latitude routes or addresses of the starting and ending points of each section may also be associated.

[0058] Here's a brief explanation of one method for creating a flight route. First, since the flight route is the section that drone 10 will fly, when creating information indicating the flight route, we identify the path from delivery source A to delivery destination B. The path should follow roads such as main roads.

[0059] Furthermore, various methods can be applied to determine the route; for example, existing route search functions using map information and GPS data can be used.

[0060] Next, considering branching points and segment lengths for each route, multiple segments are determined for each route. Furthermore, each segment and multiple segments within each route are combined to create a flight route.

[0061] For example, as illustrated in Figure 7, a flight route is defined as a single flight pattern formed by combining multiple segments, such as the flight route identification information "xx1" (in this case, a flight pattern combining "R1" and "R2"). In this case, there is no particular limit to the number of segments to be combined; it could be two, three, or four segments, or it could combine all segments of the delivery route from source A to destination B. Note that a flight route may also consist of a single segment, such as "R1," without combining segments.

[0062] When creating a flight pattern, the starting point can be the placement point of the drone 10. For example, since destination B is reached by routes R12 and R11, these can be used as the starting points. Similarly, since city C is reached by routes R2, R7, and R9, these can also be used as the starting points. Furthermore, the drone 10 may be placed between a certain section R4 and another adjacent section R6, in which case R4 or R6 can be used as the starting points.

[0063] Here, as illustrated in Figure 5, it is assumed that the following flight routes are pre-set for drone 10-1: "R1→R2", for drone 10-2: "R12→R8→R5", for drone 10-3: "R11→R10", and for drone 10-4: "R9→(R10)→R7".

[0064] Note that in Figure 5, there are some sections that are not assigned, but this can be resolved by assigning them to the flight route during the second flight. Also, regarding "R10", drones 10-3 and 10-4 will fly, but one drone 10 may perform aerial photography while the other drone 10 flies but does not perform photography. Alternatively, although it will result in overlapping photography, both drones 10 may perform aerial photography.

[0065] [Steps S103, S104] When the drone 10 obtains information indicating the flight route from the information processing device 40, it stores the information indicating the flight route, as illustrated in Figure 7, in the storage unit 150 (step S103), and sets the flight route of its own drone based on the information indicating the flight route (step S104).

[0066] All drones 10 (10-1 to 10-4) share common information indicating the same flight route. As will be described later, after the start of flight, the drones 10 can autonomously determine their flight route while communicating with other drones 10. Therefore, it is desirable for all drones 10 to have common information indicating the flight route in advance. Accordingly, for example, the flight route information shown in Figure 7 is set for all drones 10.

[0067] It is desirable to set the flight routes for drone 10 in advance before deploying drone 10. For example, when creating flight routes, the initial flight route to be assigned to drone 10 should be set in advance. This allows you to predetermine the initial flight route for each drone 10. Also, by deciding in advance which drone 10 will fly which flight route before the start of flight, efficient assignment can be achieved without missing any shots.

[0068] Furthermore, information indicating flight routes can be changed or added. In such cases, if there are changes to the information indicating flight routes, it may be possible to set (change) the flight route after deployment.

[0069] (A-2-2) Processing at the start of deployment of drone 10 Figure 8 is a sequence diagram showing the process at the start of deployment of the drone 10 in the monitoring system 1 according to the embodiment.

[0070] [Steps S105, S106] Before deployment, drone 10-1 transmits deployment availability information to other drones 10-2 indicating whether or not it is ready to deploy (step S105), and also confirms deployment availability information from other drones 10-2 (step S106).

[0071] For example, the drone 10 can broadcast a confirmation signal to inquire whether it is available for deployment and receive a response signal indicating whether it is available or not. If no response signal is received, the drone 10 may be considered unavailable for deployment.

[0072] [Steps S107, S108, S109] For each drone 10, if its own drone 10 is available for deployment (step S107 / available for deployment) and there are other drones 10 that are unavailable for deployment (step S108 / YES), the flight route of the other unavailable drone 10 is added to the next flight route of its own drone 10 (step S109).

[0073] In other words, after completing the flight route assigned to its own drone 10, the flight route of another drone 10 that is unavailable is added as a candidate for the next flight route. This prevents any missed shots by having the own drone 10 fly and take pictures in place of the other drones 10 that are unavailable.

[0074] Figure 14 is an explanatory diagram illustrating the process from steps S105 to S109.

[0075] For example, as illustrated in Figure 14, suppose that significant damage occurs in city C, making it impossible to deploy drone 10-4.

[0076] In that case, for the flight route assigned to the unavailable drone 10-4 ("R9→(R10)→R7"), for example, drone 10-3 would add "R7→R9" as the next flight route after its own flight route ("R11→R10"). Alternatively, the order of the flight routes could be changed, such as changing "R9→(R10)→R7" to "R7→R9".

[0077] If, in step S107, the system's own drone 10 is unavailable for deployment (step S107 / unavailable for deployment), the process returns to S105 and is repeated. Also, if, in step S108, there are no other drones 10 unavailable for deployment (step S108 / NO), the process proceeds to the step shown in Figure 9.

[0078] (A-2-3) Processing after the deployment of drone 10 Figure 9 is a sequence diagram showing the processing after the drone 10 is deployed in the monitoring system 1 according to this embodiment.

[0079] Here, we will explain the case where drone 10-1 detects a disaster, representing drones 10-1 to 10-4, but drones 10-2 to 10-4 will perform the same processing as drone 10-1.

[0080] [Step S110] The deployable drone 10 begins flying based on the pre-configured flight route, and while flying, the drone 10 takes pictures of the road conditions with the camera 130 (step S110).

[0081] [Steps S112, S113] In the deployed drone 10, the video acquisition unit 124 acquires camera image data captured by the camera 130, the disaster detection processing unit 125 analyzes the camera image data (step S112), and the disaster detection processing unit 125 confirms whether or not a disaster has occurred (step S113).

[0082] If the disaster detection processing unit 125 analyzes the camera image data and detects the occurrence of a disaster (step S113 / YES), the process proceeds to step S113. On the other hand, if no disaster is detected (step S113 / NO), the process returns to S110, and the drone 10 continues analyzing the camera image data while flying.

[0083] Here, we will illustrate a case where drone 10-1 detects a disaster and, acting as a representative of the other drones 10, communicates with drone 10-2 to convey a change in the flight path.

[0084] [Steps S114, S115] In step S113, when drone 10-1 detects the occurrence of a disaster, the detection result notification unit 126 transmits disaster occurrence area information to delivery vehicles 30-1 and 3-2 (step S114). This allows delivery vehicles 30-1 and 30-2 to be notified of the location of the disaster.

[0085] In delivery vehicles 30-1 and 30-2, the detection result acquisition unit 311 acquires disaster occurrence section information from the drone 10-1 and stores it in the storage unit 330 (step S115).

[0086] Here, disaster occurrence area information includes at least the disaster occurrence point, indicated by, for example, latitude and longitude, address, etc., and the section within that disaster occurrence point. In addition to this information, disaster occurrence area information may also include one or more camera images of the damage, the time of the images taken, and other information.

[0087] In addition, delivery vehicle 30 can utilize disaster-affected area information to detour delivery routes, but this will not be explained here.

[0088] [Steps S116, S117] Since it has detected that a disaster has occurred on the main road in the relevant section, drone 10-1 changes its flight route to search for a route that bypasses the disaster site (step S116), and transmits the flight route change information to another drone 10-2 (step S117).

[0089] Here, the process of changing the flight route of drone 10-1 after detecting a disaster intends to change the following section of the flight route assigned to drone 10-1 itself to a different section. From the perspective of initial response measures, since it has become clear through disaster detection that the section cannot be used as a delivery route, the aim is to identify roads that can be used as a delivery route by flying and photographing a different section.

[0090] Figure 10 is a flowchart showing the process of changing the flight route of drone 10-1 when it detects the occurrence of a disaster, in an embodiment.

[0091] When the disaster detection processing unit 125 detects a disaster (step S21), the flight route search unit 122 searches for the closest section from the current location (for example, the disaster site) (step S22).

[0092] Since the disaster site has been detected, the drone will photograph nearby roads to explore alternative routes for delivering supplies and determine if detours are feasible. Therefore, the next section of the flight path for the disaster-detecting drone 10-1 is crucial, and efficiency is required to minimize flight time and complete road photography in a short amount of time.

[0093] In this example, from the perspective described above, we will illustrate a case where a section close to the drone 10-1's current location is searched for and that section is designated as the next flight section (filming section). However, the drone 10-1 may also fly to the next pre-set section following the current section. However, since it is difficult to use a disaster-stricken section as a delivery route for supplies, we will explain a case where a different section is designated as the next section.

[0094] Furthermore, in step S22, it is also possible to determine whether another drone 10 is currently flying in that section, or whether another drone 10 has already flown in that section. In other words, if the section has been flown and photographed by another drone 10, either currently or in the past, it will result in duplicate photography. Therefore, the flight route search unit 122 may exclude sections that have been flown and photographed by another drone 10, either currently or in the past.

[0095] Next, the flight route search unit 122 determines whether the section searched in step S22 matches the next pre-set section (step S23). Here, it checks whether the searched section is a pre-set section.

[0096] If the searched section matches the next section (step S23 / YES), the objective can be achieved by having drone 10-1 fly along the pre-set flight route, so the process is terminated without changing the flight route or transmitting information about the flight route change.

[0097] On the other hand, if the searched section and the next section do not match (step S23 / NO), the searched section is changed to become the next section for drone 10-1 (step S24), and the flight route transmission unit 123 transmits the flight route change information to the other drone 10-2 (step S25).

[0098] Here, the flight route change information includes at least the identification information of the drone 10 and the modified flight route indicating which section has been changed. Other drones 10 that receive this flight route change information will be able to see which drone 10 changed which section. In addition to this information, the flight route change information may also include the time of the change, the section before the change, etc.

[0099] As described above, when drone 10-1 detects a disaster, it transmits information about the affected area to the delivery vehicle 30 and, if necessary, transmits information about changing the flight route to other drones 10. The information about the affected area may also be transmitted to other drones 10.

[0100] [Step S118] When drone 10-2 receives flight route change information from drone 10-1, the flight route search unit 122 checks the flight route of drone 10-1 included in the received flight route change information with its own flight route. If it flies the same section as drone 10-1, the flight route search unit 122 of drone 10-2 changes its own flight route (step S118).

[0101] Upon receiving the flight path change information, drone 10-2 changes its own flight path so as not to fly over the same section as drone 10-1, which detected the disaster.

[0102] Figure 11 is a flowchart showing the flight route change process of another drone 10-2 that has received flight route change information in an embodiment.

[0103] When drone 10-2 acquires flight route change information (step S31), the flight route search unit 122 analyzes the received flight route change information and confirms the flight route changed by drone 10-1 and the flight route of drone 10-2 itself (step S32).

[0104] Then, the flight route search unit 122 checks whether the modified section of drone 10-1 is included in the flight route of drone 10-2 (step S33). If it is included (step S33 / YES), the process proceeds to step S34. If it is not included (step S33 / NO), the process ends because there is no overlapping section between drone 10-2 and drone 10-1.

[0105] In step S34, the flight route search unit 122 modifies the section of the drone 10-2's flight path that overlaps with the section modified by drone 10-1 (step S34).

[0106] Here, the section of the flight path of drone 10-2 that overlaps with the flight path of drone 10-1 is modified. Step S34 in Figure 11 illustrates the case where the section is changed to a different section, but the overlapping section may also be removed from the flight path of drone 10-2.

[0107] In this example, it is assumed that the next section after the section in which drone 10-2 is currently flying is an overlapping section. In that case, the flight route search unit 122 identifies the position of the endpoint of the currently flying section and searches for another section close to that endpoint (step S34).

[0108] For example, if the endpoint is branched, and any of the branched sections overlap, the flight route search unit 122 will search for other sections. Also, for example, if it is not possible to find another section close to the endpoint, the overlapping section may be removed from the flight route of the drone 10-2.

[0109] If another section close to the endpoint is searched for, the flight route search unit 122 checks whether the searched section is included in the flight route change information of drone 10-1 (step S35). If it is included (step S35 / YES), the unit returns to step S34 to search for another section. If it is not included (step S35 / NO), the searched section is swapped with the overlapping section, and the flight route of drone 10-2 is changed (step S36).

[0110] [Step S119] When drone 10-2 receives flight route change information and changes its own flight route, the flight route transmission unit 123 of drone 10-2 transmits the changed flight route to drone 10-1 (step S119).

[0111] Figure 15 is an explanatory diagram illustrating the process from steps S113 to S119.

[0112] For example, drone 10-1, which was taking aerial photographs of R2, detects that R2 is impassable.

[0113] In this case, drone 10-1's flight path was "R1→R2" and it was planned to film up to the end of R2, but since it detected a disaster, it searched for another section, R5, and changed its flight path to "R5→R6→R7". It then transmitted information about this changed flight path, including the modified flight path, to the other drones 10.

[0114] Upon receiving the flight route change information, drone 10-2 confirms that R5 is a section that overlaps with the shooting area of ​​drone 10-1. It then searches for a section that does not overlap, for example, searching for R4 instead of R5, and changes the modified flight route to "R12→R8→R4".

[0115] [Step S120] When drone 10-1 receives a modified flight route from drone 10-2, it stores the modified flight route of drone 10-2 in the memory unit 150 and updates it (step S120).

[0116] For example, drone 10-2 broadcasts its changed flight route, and this information is transmitted to other drones 10, including drone 10-1, which detected the disaster. At this time, since drone 10-1 has already changed its flight route, it does not change its flight route even if it receives a notification of the change from drone 10-2. On the other hand, other drones 10, upon receiving the notification of the change in drone 10-2's flight route, can check whether there are any overlapping sections with drone 10-2's route and, if necessary, perform a flight route change process similar to that shown in Figure 11.

[0117] (A-2-4) Flight completion processing for each section Figure 12 is a sequence diagram showing the flight completion process for each section of the drone 10 in the monitoring system 1 according to the embodiment.

[0118] Drone 10-1 continues aerial photography, and when it completes the flight of the section without detecting a disaster (step S121 / YES), the detection result notification unit 126 transmits a section flight completion notification to the other drones 10-2 and the delivery vehicle 30 (step S122).

[0119] If the flight in the relevant section has not been completed (step S121 / NO), the process proceeds to step S110, and drone 10-1 continues flight photography.

[0120] When the other drone 10-2 and the delivery vehicles 30-1 and 30-2 receive a section flight completion notification from drone 10-1, they each store the section flight completion notification (step S123).

[0121] Furthermore, not only drone 10-1, but also the other drones 10-2 will undergo the same processing as drone 10-1.

[0122] (A-2-5) Process for determining the delivery route Figure 13 is a sequence diagram showing the process of determining the delivery route in the monitoring system 1 according to the embodiment.

[0123] As described above, each drone 10 takes photographs while flying along its own flight route. The information obtained from the flight route and road conditions at each section by each drone 10 is provided to delivery vehicles 30-1 and 30-2. In other words, it is assumed that road conditions at all sections of the route from delivery source A to delivery destination B are photographed, and that the information obtained from all of these sections is transmitted to delivery vehicles 30-1 and 30-2.

[0124] In each of the delivery vehicles 30-1 and 30-2, the information captured from photographs taken along all sections is aggregated to determine the delivery route from the delivery source A to the delivery destination B (step S124).

[0125] Here, we illustrate the case where each of the delivery vehicles 30-1 and 30-2 determines the delivery route. However, for example, the information processing device 40 may aggregate information from the drone 10 to determine the delivery route and transmit the result to the delivery vehicles 30-1 and 30-2.

[0126] (A-3) Effects of the Embodiment As described above, this embodiment makes it possible to photograph and analyze delivery routes using a camera mounted on a drone when natural disasters such as heavy rain or earthquakes occur, thereby securing delivery routes and contributing to a rapid initial response by delivering relief supplies.

[0127] (B) Other embodiments Although various modified embodiments were mentioned in the embodiments described above, the following modified embodiments can also be applied.

[0128] (B-1) In the embodiment described above, an example was given in which multiple delivery vehicles 30 each have a delivery route extraction unit, but it is also possible for only one of the delivery vehicles 30 to have this unit and transmit the resulting delivery route to the other delivery vehicles 30.

[0129] (B-2) The drone 10, as an unmanned aircraft, may analyze images of the road to determine the extent of the damage. For example, a method for evaluating the extent of the damage can be applied using machine learning or the like. [Explanation of Symbols]

[0130] 1: Surveillance system, NT: Network, 10 (10-1 to 10-4): Drone, 30 (30-1 to 30-2): Delivery vehicle, 31: Information acquisition device, 40: Information processing device, 110: Communication unit, 120: Control unit, 20: Flight control unit, 21: Disaster detection control unit, 121: Flight route acquisition unit, 122: Flight route search unit, 123: Flight route transmission unit, 124: Video acquisition unit, 125: Disaster detection processing unit, 126: Detection result notification unit, 130: Camera, 150: Memory unit, 310: Control unit, 311: Detection result acquisition unit, 312: Delivery route derivation unit, 320: Communication unit, 330: Storage unit, 410: Control unit, 411: Flight route setting unit, 412: Flight route transmission unit, 420: Communication unit, 430: Memory unit.

Claims

1. An information processing system that derives a delivery route for goods based on image data obtained by each of several unmanned aircraft equipped with a camera, which takes aerial photographs of the road conditions along a route connecting a first city and a second city, Each of the aforementioned multiple unmanned aircraft, Flight control means that controls flight based on a flight route that includes sections obtained by dividing each of the multiple routes connecting the first city and the second city into multiple segments, Disaster detection control means detects whether or not a disaster has occurred based on image data including the road conditions of the section captured by the aforementioned shooting means. It has, A delivery route derivation means collects detection results indicating whether or not a disaster has occurred in each section, obtained from the disaster detection control means of each of the multiple unmanned aircraft, and derives a delivery route that bypasses sections including roads that are impassable. An information processing system characterized by comprising the following features.

2. Each of the disaster detection and control means of the plurality of unmanned aircraft, A disaster detection processing unit analyzes image data including the road condition of the section captured by the aforementioned photographing means to detect whether or not a disaster has occurred based on the extent of damage to the road in that section. A detection result transmission unit that transmits the detection results from the disaster detection processing unit to other wireless devices and the delivery route guidance means. The information processing system according to claim 1, characterized by having the following features.

3. Each of the aforementioned flight control means for the plurality of unmanned aircraft, A flight route search unit that, upon detecting a disaster, searches for a different section from the disaster-affected section and changes the flight route accordingly. A flight route transmission unit that transmits flight route change information, including the modified flight route, to other unmanned aircraft. The information processing system according to claim 1, characterized by comprising:

4. Of the aforementioned multiple unmanned aircraft, the flight control means of the first unmanned aircraft that detected the occurrence of the disaster transmits the flight route change information to the second unmanned aircraft, which is another unmanned aircraft. The flight control means of the second unmanned aircraft changes the flight route of the second unmanned aircraft if there is an overlapping section, based on the modified flight route of the first unmanned aircraft included in the received flight route change information and the flight route of the second unmanned aircraft. The information processing system according to feature 1.

5. The information processing system according to claim 1, characterized in that each of the flight control means of the plurality of unmanned aircraft transmits information on whether or not they can be deployed to each other at the start of flight, and if there is an unmanned aircraft that cannot be deployed, sets the flight route of that unmanned aircraft as its own next flight route.

6. The information processing system according to claim 1, characterized in that the delivery route derivation means is provided on one or more mobile bodies that deliver goods.

7. An information processing method for deriving a delivery route for goods based on image data obtained by each of several unmanned aircraft equipped with a camera to photograph the road conditions of a route connecting a first city and a second city, Each of the aforementioned multiple unmanned aircraft, Flight control is performed based on a flight route that includes segments obtained by dividing each of the multiple routes connecting the first city and the second city into multiple sections. Based on image data including the road conditions of the section captured by the aforementioned photographic means, the presence or absence of a disaster is detected. The delivery route derivation means collects detection results indicating whether or not a disaster has occurred in each section, obtained from each of the multiple unmanned aircraft, and derives a delivery route that bypasses sections including roads that are impassable. An information processing method characterized by the following:

Images