Computer Programs

Abstract

To provide an assistive technology that enables detailed inspection of an object using a drone while suppressing cost increases.SOLUTION: A computer program implements: a function that acquires information for setting, to a terminal that controls an unmanned flight device equipped with an imaging function, a first operation of an unmanned flight device so as to image an object; a function that acquires an image obtained as a result of the unmanned flight device performing the first operation from the unmanned flight device; a function that accepts designation of a part of the target from a user using the image; and a function that sets a second operation of the unmanned flight device so as to acquire the designated part image of the target that is more detailed than the designated part image of the target obtained in the first operation.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a computer program for controlling an unmanned aerial vehicle equipped with an imaging function. [Background technology]

[0002] In recent years, the miniaturization and weight reduction of batteries and the sophistication of sensors such as gyro sensors and acceleration sensors have improved the operational stability of unmanned small flying devices called drones, and such drones have become available at low cost. Japan's Ministry of Economy, Trade and Industry has advocated an "industrial revolution in the sky" and is promoting technological development and environmental improvements for the safe use of drones (see, for example, Non-Patent Document 4).

[0003] Patent Document 1 discloses a technology for taking aerial photographs using a drone and generating a three-dimensional model of a target. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] US Patent Application Publication No. 2018 / 0218533 [Non-patent literature]

[0005] [Non-Patent Document 1] https: / / www.dji.com / jp / mavic, searched on August 21, 2020 [Non-Patent Document 2] https: / / dronebank.jp / dronedeploy / index.html, searched on August 21, 2020 [Non-Patent Document 3] https: / / www.softbank.jp / corp / news / press / sbkk / 2018 / 20181107_01 / , retrieved on August 22, 2020 [Non-Patent Document 4] https: / / www.meti.go.jp / policy / mono_info_service / mono / robot / drone.html, searched on August 22, 2020 Summary of the Invention [Problem to be solved by the invention]

[0006] When generating a 3D model from images captured by a drone's camera, the communication speed between the drone and its control terminal can become a bottleneck. In particular, to generate a detailed 3D model of the target, it is necessary to acquire many high-resolution, i.e. large-sized, images and transmit them from the drone to the processing terminal. However, communication between the drone and the processing terminal is wireless, which is not suitable for transmitting and receiving large amounts of data in a short period of time. Alternatively, if a high-speed, large-capacity wireless communication module that would enable such transmission and reception is installed, the cost of the drone will increase.

[0007] The present invention has been made in consideration of these problems, and its purpose is to provide support technology that enables detailed inspection of targets using drones while suppressing increases in costs. [Means for solving the problem]

[0008] One aspect of the present invention relates to a computer program, which causes a terminal controlling an unmanned flying device having an imaging function to perform the following functions: acquire information for setting a first operation of the unmanned flying device to capture an image of an object, acquire an image obtained as a result of the unmanned flying device performing the first operation from the unmanned flying device, accept a designation of a part of the object from a user using the image, set a second operation of the unmanned flying device to capture an image of the designated part of the object that is more detailed than the image of the designated part of the object captured in the first operation, register in a holding means specific information for identifying the designated part of the object and a flight path determined for capturing the image of the part, refer to the holding means and re-set the flight path stored in association with the part when a request is received to capture the designated part of the object again, and display on a display an image of the designated part of the object captured in the second operation at a different time point or a three-dimensional model generated from the image in a comparable manner.

[0009] In addition, any combination of the above components, or mutual substitution of the components or expressions of the present invention between devices, methods, systems, recording media storing computer programs, etc. are also valid aspects of the present invention. Effect of the Invention

[0010] According to the present invention, it is possible to provide an assistance technology that enables detailed inspection of an object using a drone while suppressing an increase in costs. [Brief description of the drawings]

[0011] [Figure 1] 1 is a schematic diagram for explaining an examination support system according to an embodiment; [Diagram 2] 1 is a schematic diagram showing a drone flying along a general flight path set corresponding to a base station device and capturing an image of the base station device. [Diagram 3]13 is a schematic diagram showing a state in which designation of a part to be checked in detail from a user is accepted in the generated outline 3D model. FIG. [Figure 4] 1 is a schematic diagram showing a drone flying along a detailed flight path set corresponding to a specified part of a base station device and capturing an image of the specified part. [Diagram 5] FIG. 13 is a schematic diagram showing a user checking a generated detailed three-dimensional model. [Figure 6] FIG. 2 is a hardware configuration diagram of the mobile terminal of FIG. [Figure 7] 2 is a block diagram showing the functions and configuration of the mobile terminal of FIG. 1. [Figure 8] 8 is a data structure diagram showing an example of the overview image information storage unit of FIG. 7. [Figure 9] 8 is a data structure diagram showing an example of a detailed image information storage unit in FIG. 7. [Figure 10] FIG. 8 is a data structure diagram showing an example of the detailed flight history storage unit of FIG. 7. [Figure 11] 2 is a flowchart showing the flow of a series of processes in the mobile terminal of FIG. 1. [Figure 12] 13 is a representative screen diagram of a detailed imaging history selection screen displayed on the display of the mobile terminal. FIG. [Figure 13] FIG. 13 is a representative screen diagram of a designated part confirmation screen displayed on the display of the mobile terminal. [Figure 14] FIG. 13 is a representative screen diagram of a designated part transition screen displayed on the display of a mobile terminal. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the same or equivalent components, members, and processes shown in each drawing will be denoted by the same reference numerals, and duplicated explanations will be omitted as appropriate. In addition, some of the members that are not important for the explanation will be omitted in each drawing.

[0013] FIG. 1 is a schematic diagram for explaining an inspection support system 2 according to an embodiment. The inspection support system 2 supports an inspection of an inspection target performed by an operator (user 4) using an unmanned flying device such as a drone 8. In the present embodiment, a base station device 6 of a mobile phone network is assumed as an inspection target, but the inspection target is not limited to the base station device 6, and in other embodiments, for example, power-related infrastructure facilities such as power transmission lines, and buildings such as buildings, bridges, and dams may be inspection targets.

[0014] The inspection support system 2 includes a mobile terminal 10 of the user 4 and a drone 8. The mobile terminal 10 and the drone 8 are configured to be communicable, and this communication may be realized by direct wireless communication means such as Bluetooth (registered trademark) or WiFi (registered trademark), or may be realized via a network such as a mobile phone network or the Internet. The mobile terminal 10 is a terminal that controls the drone 8.

[0015] The mobile terminal 10 is a mobile terminal such as a smartphone, a tablet terminal, a notebook PC, or a dedicated controller. The user 4 downloads and installs an inspection support application program (hereinafter referred to as an inspection support app) on the mobile terminal 10 via a network from a download site. Alternatively, the inspection support app may be pre-installed on the mobile terminal 10. Furthermore, the inspection support app may be configured in an ASP type or a SaaS type. When the inspection support app is executed by the mobile terminal 10, the mobile terminal 10 communicates with the drone 8 to realize various functions. Hereinafter, functions realized by the mobile terminal 10 (processing units such as the CPU (Central Processing Unit) of the mobile terminal 10) executing the inspection support app may be described as functions of the mobile terminal 10, but these functions are actually functions realized by the inspection support app on the mobile terminal 10.

[0016] The drone 8 is a relatively small device that flies unmanned, and may fly by remote control via wireless communication or may fly autonomously. In this embodiment, the drone 8 is assumed to be a general-purpose drone equipped with an imaging function such as a camera, a positioning function such as a GPS (Global Positioning System), and a communication function with the mobile terminal 10, such as the DJI Mavic Pro (see Non-Patent Document 1).

[0017] Referring to FIG. 1, the user 4 first specifies an area (hereinafter referred to as an imaging area 12) to be imaged by the drone 8 on the display 102 of the mobile terminal 10. In particular, the user 4 specifies the base station device 6 to be inspected on the electronic map displayed on the display 102, and then sets the imaging area 12 so that the base station device 6 is included in the imaging area 12. The inspection object such as the base station device 6 may be displayed as an object that can be specified on the electronic map. The imaging area 12 may be specified by a polygon that connects the points 14 that the user 4 tapped on the display 102. Alternatively, the imaging area 12 may be automatically set when an object corresponding to the base station device 6 is selected on the electronic map. For example, the imaging area 12 may be set to a circular area of ​​a predetermined radius centered on the position of the base station device 6.

[0018] 2 is a schematic diagram showing how the drone 8 flies along a general flight path 16 set corresponding to the base station device 6 and captures an image of the base station device 6. When the base station device 6 and its associated imaging area 12 are designated by the user 4, the mobile terminal 10 sets the general flight path 16 of the drone 8 so as to acquire images required to generate a general or coarse three-dimensional model (hereinafter referred to as a general three-dimensional model) of the designated base station device 6.

[0019] 2, the drone 8 captures an image of the base station device 6 while flying along a general flight path 16, and transmits the obtained image to the mobile terminal 10. The mobile terminal 10 generates a general three-dimensional model of the base station device 6 using the acquired image. Setting the general flight path 16 of the drone 8 and generating the general three-dimensional model using the image acquired from the drone 8 may be realized using a known Drone 3D Mapping technology (for example, see Non-Patent Document 2).

[0020] FIG. 3 is a schematic diagram showing a state in which a designation of a part to be checked in detail is received from a user 4 in the generated general three-dimensional model 18. The mobile terminal 10 displays the generated general three-dimensional model 18 of the base station device 6 on the display 102. The user 4 designates a part of the base station device 6 to be checked in detail by drawing a rectangle 20 on the general three-dimensional model 18 displayed on the display 102. The mobile terminal 10 specifies the part within the rectangle 20 as the designated part. The mobile terminal 10 sets a detailed flight path 22 of the drone 8 so as to acquire images required for generating a detailed or fine three-dimensional model (hereinafter referred to as a detailed three-dimensional model) of the designated part based on positioning information when the designated part is imaged on the general flight path 16. The detailed three-dimensional model has a smaller granularity and / or a higher resolution and / or a larger amount of data per object than the general three-dimensional model.

[0021] Note that multiple types of rectangles 20 with different colors, line types, and shapes may be available, and in this case, different types of rectangles may correspond to different instruction contents. For example, when a rectangle with a red frame is specified, the mobile terminal 10 may interpret it as an instruction to capture only the surface of the rectangle in detail. When a rectangle with a blue frame is specified, the mobile terminal 10 may interpret it as an instruction to capture the area within the rectangle in detail in all directions. Also, not only rectangles but also shapes such as circles and triangles may be used as long as an area can be specified. Instead of an area, the user may specify only a point, and the area around that point may be the target area.

[0022] 4 is a schematic diagram showing a drone 8 flying along a detailed flight path 22 set corresponding to a specified part of the base station device 6 and capturing an image of the specified part. While flying along the detailed flight path 22, the drone 8 captures an image of the specified part of the base station device 6 and transmits the obtained image to the mobile terminal 10. The mobile terminal 10 generates a detailed three-dimensional model 24 of the specified part using the acquired image.

[0023] 5 is a schematic diagram showing a state in which a user 4 is checking a generated detailed three-dimensional model 24. The mobile terminal 10 displays the generated detailed three-dimensional model 24 of the specified part on the display 102. The user 4 checks the condition of the specified part (cracks, splits, discoloration, deterioration, falling off, attachment of foreign matter, etc.) from the detailed image of the specified part being displayed.

[0024] In the inspection of infrastructure or buildings, it is usually required to find abnormalities such as cracks on the order of 1 mm to several mm. In order to find abnormalities of this level, it is necessary to obtain relatively detailed images, but obtaining images of that level when generating a three-dimensional model of the entire inspection target is not realistic in terms of time and processing load. Therefore, in the support system 2 according to this embodiment, first, a general three-dimensional model 18 of the inspection target is generated, and the general three-dimensional model 18 is used to accept the designation of the part to be inspected from the user 4. Next, the drone 8 is flown again so as to capture the part designated by the user 4 in detail, and a detailed three-dimensional model 24 is generated. This allows the user 4 to grasp the overall inspection target in an overview during the inspection, and then check the condition of the required part in detail. In addition, since it is not necessary to generate a detailed three-dimensional model of the entire inspection target, the time required for the inspection can be shortened. Furthermore, since high communication performance is not required for the drone 8, costs can be suppressed.

[0025] Fig. 6 is a hardware configuration diagram of the mobile terminal 10 in Fig. 1. The mobile terminal 10 includes a memory 104, a processor 106, a communication interface 108, a display 102, and an input interface 110. Each of these elements is connected to a bus 112 and communicates with each other via the bus 112.

[0026] The memory 104 is a storage area for storing data and programs. The data and programs may be permanently or temporarily stored in the memory 104. In particular, the memory 104 stores an examination support application. The processor 106 executes the programs stored in the memory 104, particularly the examination support application, to realize various functions in the mobile terminal 10. The communication interface 108 is an interface for transmitting and receiving data to and from the outside of the mobile terminal 10. For example, the communication interface 108 includes an interface for accessing a network and an interface for direct wireless communication with the drone 8. The display 102 is a device for displaying various information, and is, for example, a liquid crystal display or an organic EL (Electroluminescence) display. The input interface 110 is a device for receiving input from a user. The input interface 110 includes, for example, a touch panel provided on the display 102, various input keys, and the like.

[0027] Fig. 7 is a block diagram showing the functions and configuration of the mobile terminal 10 of Fig. 1. Each block shown here can be realized in hardware by elements and mechanical devices such as a computer CPU, and in software by a computer program, etc., but here, functional blocks realized by the cooperation of these are depicted. Therefore, those skilled in the art who have read this specification will understand that these functional blocks can be realized in various ways by combining hardware and software.

[0028] The mobile terminal 10 includes a general processing unit 114, a detail processing unit 116, a model utilization unit 118, a general image information holding unit 120, a three-dimensional model holding unit 122, a detailed image information holding unit 124, and a detailed flight history holding unit 126. The general processing unit 114 performs processing related to the generation of the general three-dimensional model 18. The detail processing unit 116 performs processing related to the generation of the detailed three-dimensional model 24. The model utilization unit 118 performs processing related to the use of the generated detailed three-dimensional model 24.

[0029] FIG. 8 is a data structure diagram showing an example of the summary image information storage unit 120 of FIG. 7. The summary image information storage unit 120 stores a summary or rough image (hereinafter referred to as a summary image) acquired by the drone 8 while flying along the summary flight path 16 and a situation when the summary image was acquired. The summary image information storage unit 120 stores a summary flight ID that identifies the flight of the drone 8 following the summary flight path 16, a time when the drone 8 acquired the summary image, a position of the drone 8 when the drone 8 acquired the summary image, and a file of the summary image in association with each other. The position of the drone 8 when the drone 8 acquired the summary image may be provided by a positioning function of the drone 8. The size of the file of the summary image may be relatively small.

[0030] FIG. 9 is a data structure diagram showing an example of the detailed image information storage unit 124 of FIG. 7. The detailed image information storage unit 124 stores a detailed or fine image (hereinafter referred to as a detailed image) acquired by the drone 8 while flying along the detailed flight path 22 and a situation when the detailed image was acquired. The detailed image information storage unit 124 stores a detailed flight ID that identifies the flight of the drone 8 following the detailed flight path 22 (hereinafter referred to as a detailed flight), a designated part ID that identifies a part designated by the user 4, a time when the drone 8 acquired the detailed image, a position of the drone 8 when the drone 8 acquired the detailed image, and a file of the detailed image in association with each other. The size of the file of the detailed image may be relatively large, and may be larger than the size of the file of the overview image, for example.

[0031] FIG. 10 is a data structure diagram showing an example of the detailed flight history storage unit 126 of FIG. 7. The detailed flight history storage unit 126 stores a history of detailed flights. The detailed flight history storage unit 126 stores a detailed flight ID, an object ID for identifying an inspection object, a designated part ID, a designated rectangular image file, a time when the detailed flight was performed, information on a detailed flight path 22 in the detailed flight, and a detailed image file of the designated part in association with each other. The designated rectangular image file is an image file including a rectangle 20 drawn by the user 4 on the general three-dimensional model 18 when accepting the designation of the designated part from the user 4. For example, the designated rectangular image file may be a screen capture of the display 102 shown in FIG. 3. The file of the detailed image of the designated part may be a file of a designated detailed image described later.

[0032] 7, the overview processing unit 114 includes an imaging area acquisition unit 128, an overview flight path setting unit 130, an overview image acquisition unit 132, and an overview model generation unit 134. The imaging area acquisition unit 128 acquires information for setting the operation of the drone 8 to capture an image of the base station device 6. The imaging area acquisition unit 128 displays an electronic map on the display 102 of the mobile terminal 10, and accepts designation of the base station device 6 as an inspection target and designation of the imaging area 12 from the user 4 via the electronic map.

[0033] The general flight path setting unit 130 generates the operation of the drone 8 to capture an image of the base station device 6, i.e., the general flight path 16, based on the information acquired by the imaging area acquisition unit 128. The general flight path setting unit 130 sets the general flight path 16 for the drone 8 by transmitting the generated general flight path 16 to the drone 8.

[0034] The overview image acquisition unit 132 acquires overview images from the drone 8 obtained as a result of the drone 8 flying along the overview flight path 16 (hereinafter referred to as overview flight). The overview image acquisition unit 132 accepts overview images successively transmitted from the drone 8 during the overview flight of the drone 8, as well as the acquisition time and acquisition position of the overview image, and stores them in the overview image information holding unit 120.

[0035] The overview model generation unit 134 reads out the overview image stored in the overview image information storage unit 120, and generates an overview 3D model 18 of the base station device 6 based on the read out overview image. The overview model generation unit 134 registers the generated overview 3D model 18 in the 3D model storage unit 122.

[0036] The detail processing unit 116 includes a designation receiving unit 136, a detailed flight path setting unit 138, a position and attitude control unit 140, a detailed image acquisition unit 142, and a detailed model generation unit 144. The designation receiving unit 136 receives a designation of a part of the base station device 6 from the user 4 using the outline three-dimensional model 18 generated from the outline image. When the designation receiving unit 136 receives a detailed confirmation request of the base station device 6 from the user 4, it reads out the outline three-dimensional model 18 of the base station device 6 held in the three-dimensional model holding unit 122 and displays it on the display 102. The designation receiving unit 136 identifies the part of the base station device 6 displayed within the rectangle 20 drawn by the user 4 on the displayed outline three-dimensional model 18 as the designated part. The function of the designation receiving unit 136 when re-designating will be described later.

[0037] The detailed flight path setting unit 138 generates the operation of the drone 8, i.e., the detailed flight path 22, so as to acquire a detailed image of the designated part that is more detailed than the overview image of the designated part obtained in the overview flight. The detailed flight path setting unit 138 sets the detailed flight path 22 for the drone 8 by transmitting the generated detailed flight path 22 to the drone 8.

[0038] When generating the detailed flight path 22, the detailed flight path setting unit 138 uses the positioning information obtained during the overview flight. Specifically, the detailed flight path setting unit 138 first identifies an overview image in which the specified part is shown. This may be realized, for example, by identifying an overview image that is the basis for constituting the specified part in the overview three-dimensional model 18, or by identifying an overview image corresponding to the rectangle 20 drawn by the user 4 when specifying the part from among the overview images stored in the overview image information storage unit 120. The detailed flight path setting unit 138 refers to the overview image information storage unit 120 and acquires a position associated with the identified overview image. The detailed flight path setting unit 138 sets the position of the first destination of the detailed flight path 22 based on the acquired position. The detailed flight path setting unit 138 sets the flight path of the drone 8 after arriving at the first destination so that the specified part is imaged from a plurality of different viewpoints. The shape of the detailed flight path 22 from the start point to the first destination may be set to be the shortest route to the specified part (or the position of the first destination) that avoids any obstacles on the way.

[0039] The position and attitude control unit 140 controls the drone 8 to adjust the position or attitude for capturing an image of the designated part by comparing the outline image of the designated part identified by the detailed flight path setting unit 138 with the current image acquired by the imaging function of the drone 8. The position and attitude control unit 140 transmits the outline image of the designated part identified by the detailed flight path setting unit 138 to the drone 8. When the drone 8 flies along the detailed flight path 22 and arrives near the first destination, it compares the received outline image with the current image of the designated part acquired by the imaging function of the drone 8. The drone 8 adjusts the position and attitude of the drone 8 and / or the line of sight and focal length of the camera of the drone 8 so that the difference between the two images becomes small. The detailed image of the designated part captured by the drone 8 whose position and attitude have been adjusted in this way is called a designated detailed image.

[0040] It is also possible to adjust the position and attitude of the drone 8 by other methods. For example, the position and attitude control unit 140 may adjust the detailed flight path 22 generated by the detailed flight path setting unit 138 using the display orientation of the general three-dimensional model 18 displayed on the display 102 when the user 4 specifies the part and the position corresponding to the drawn rectangle 20. Alternatively, when relatively detailed positioning information is assigned to each object of the general three-dimensional model 18 (for example, see Non-Patent Document 3), the position and attitude control unit 140 may generate or adjust the detailed flight path 22 based on the positioning information of the object corresponding to the part specified by the rectangle 20.

[0041] A specific example of the detailed flight path 22 is shown below. (1) Fly in a straight line from the starting point to the first destination → Adjust position and attitude by comparing images → Capture specified detailed images → Capture images from multiple angles while flying around the specified area → Return (2) Fly in an arc from the starting point to the first destination (to avoid obstacles) → Control the PTZ (pan-tilt-zoom) while hovering at the first destination to set the camera to the specified direction and focal length → Capture a specified detailed image → Fly around the specified area and capture images from multiple angles → Return

[0042] The detailed image acquisition unit 142 acquires detailed images obtained as a result of the drone 8 performing a detailed flight from the drone 8. The detailed image acquisition unit 142 accepts detailed images sequentially transmitted from the drone 8 during the detailed flight of the drone 8, as well as the acquisition time and acquisition position of the detailed images, and stores them in the detailed image information storage unit 124.

[0043] The detailed model generation unit 144 reads out the detailed image stored in the detailed image information storage unit 124, and generates a detailed three-dimensional model 24 of the specified part based on the read out detailed image. The detailed model generation unit 144 registers the generated detailed three-dimensional model 24 in the three-dimensional model storage unit 122.

[0044] When one detailed flight is completed and the corresponding detailed 3D model 24 is stored in the 3D model storage unit 122 , the detailed processing unit 116 registers information about the detailed flight in the detailed flight history storage unit 126 .

[0045] The model utilization unit 118 includes a detailed image providing unit 146 and a contrast image providing unit 148. The detailed image providing unit 146 reads out the detailed 3D model 24 of the specified part stored in the 3D model storage unit 122, and displays it on the display 102. Alternatively, the detailed image providing unit 146 may display the specified detailed image on the display 102 instead of the detailed 3D model 24. The function of the contrast image providing unit 148 will be described later.

[0046] The operation of the mobile terminal 10 configured as above will now be described. FIG. 11 is a flowchart showing a series of processing steps in the mobile terminal 10 of FIG. 1. The mobile terminal 10 accepts the designation of the imaging area 12 by the user 4 (S202). The mobile terminal 10 sets the outline flight path 16 based on the accepted imaging area 12 (S204). The mobile terminal 10 acquires an outline image from the drone 8 (S206). The mobile terminal 10 generates an outline three-dimensional model 18 using the acquired outline image (S208). The mobile terminal 10 accepts designation of a part to be a target of detailed inspection from the user 4 via the display of the outline three-dimensional model 18 (S210). The mobile terminal 10 sets the detailed flight path 22 so as to acquire a detailed image of the designated part (S212). The mobile terminal 10 acquires a detailed image from the drone 8 (S214). The mobile terminal 10 generates a detailed three-dimensional model 24 using the acquired detailed image (S216). The mobile terminal 10 presents the detailed 3D model 24 of the specified part or the specified detailed image to the user 4 (S218). The mobile terminal 10 registers the specified part ID, the detailed flight path 22, and the specified detailed image in the detailed flight history storage unit 126 (S220).

[0047] FIG. 12 is a representative screen diagram of a detailed imaging history selection screen 150 displayed on the display 102 of the mobile terminal 10. The detailed imaging history selection screen 150 displays information 152 for identifying a past detailed flight and a confirmation button 154 with an image for each past detailed flight. The information 152 for identifying a past detailed flight includes the date on which the detailed flight was performed, the name of the inspection target, and the name of the designated part. When the designation reception unit 136 receives a request to display the imaging history from the user 4, the designation reception unit 136 generates the detailed imaging history selection screen 150 by referring to the detailed flight history storage unit 126, and displays the detailed imaging history selection screen 150 on the display 102. The date on the detailed imaging history selection screen 150 corresponds to the time when the detailed flight was performed in the detailed flight history storage unit 126, the name of the inspection target in the detailed imaging history selection screen 150 corresponds to the target ID in the detailed flight history storage unit 126, and the name of the designated part in the detailed imaging history selection screen 150 corresponds to the designated part ID in the detailed flight history storage unit 126.

[0048] 13 is a representative screen diagram of a designated part confirmation screen 156 displayed on the display 102 of the mobile terminal 10. When the user 4 taps a confirmation button 154 of a desired detailed flight on the detailed image capture history selection screen 150, the designation receiving unit 136 reads out a designated rectangle image file corresponding to the detailed flight of the tapped confirmation button 154 from the detailed flight history storage unit 126, and generates a designated part confirmation screen 156. The designated part confirmation screen 156 has an image including a rectangle 20 drawn by the user 4 on the general three-dimensional model 18 when designation of a designated part is accepted from the user 4 regarding the detailed flight of the tapped confirmation button 154, and a re-designation button 158. For example, when user 4 taps the confirmation button for a past detailed flight identified as "8 / 15, BS#1000, bottom end of antenna No. 1" from the detailed flight history shown on the detailed imaging history selection screen 150 of Figure 12, the screen transitions to display a specified part confirmation screen 156, in which a rectangle that user 4 drew on 8 / 15 (a past time) on display 102, on which a general three-dimensional model of BS#1000 (the name of base station device 6) is displayed, to specify the bottom end of antenna No. 1 (specified part) is displayed together with the general three-dimensional model.

[0049] The user 4 checks the designated part on the designated part confirmation screen 156, and if there is no problem, taps the re-designation button 158. The designation reception unit 136 then accepts the tap as a request to image the designated part again. The detailed flight path setting unit 138 reads out from the detailed flight history storage unit 126 a detailed flight path corresponding to the designated rectangular image file read out when the designated part confirmation screen 156 was generated, and transmits the detailed flight path to the drone 8, thereby re-setting the drone 8 to the same detailed flight path as the past detailed flight path.

[0050] In another embodiment, the designation receiving unit 136 may display on the display 102 a screen displaying a previously acquired designated detailed image or detailed three-dimensional model instead of the designated part confirmation screen 156. In this case, when the user 4 requests re-designation, the designation receiving unit 136 reads out a detailed flight path corresponding to the displayed designated detailed image or detailed three-dimensional model from the detailed flight history storage unit 126 and sets it again on the drone 8.

[0051] In the inspection of infrastructure facilities such as base station equipment and buildings, it is known from experience which parts should be inspected with priority and which parts should be inspected every time. In the example of a base station equipment, deterioration starts from the tip of the antenna, so the tip is a part that must be checked every time. In this embodiment, if the designated parts that need to be inspected repeatedly are identified from the outline three-dimensional model in the first inspection, the detailed flight path setting for the drone 8 can be easily completed by simply selecting from the detailed flight history in the next and subsequent inspections. This improves user convenience.

[0052] FIG. 14 is a representative screen diagram of a designated site transition screen 160 displayed on the display 102 of the mobile terminal 10. The comparison image providing unit 148 causes the display 102 to display a designated site transition screen 160 that displays, in a comparable manner, detailed images or detailed three-dimensional models of the same designated site obtained in the detailed flight routes 22 at different times. When the comparison image providing unit 148 receives a comparison request accompanied by the designation of an inspection target and a designated site from the user 4, it acquires from the detailed flight history holding unit 126 a file of a designated detailed image corresponding to the designated inspection target (target ID) and the designated designated site (designated site ID). The comparison image providing unit 148 generates the designated site transition screen 160 by arranging the acquired designated detailed images in chronological order. Note that, instead of or in addition to the designated detailed images, detailed three-dimensional models that can be acquired from the three-dimensional model holding unit 122 may be displayed on the designated site transition screen 160.

[0053] In the designated detailed image displayed on the designated site transition screen 160, the difference from the immediately previous designated detailed image displayed on the same screen is emphasized and displayed. For example, in the designated detailed image corresponding to "8 / 17" of the designated site transition screen 160 in FIG. 14, a new crack 162, which is the difference from the immediately previous designated detailed image of "8 / 15", is displayed in a different manner from other cracks, for example, in an emphasized manner such as being displayed thickly.

[0054] In this way, by displaying the designated detailed images of the designated site in chronological order, the user 4 can more accurately and at a glance confirm the deteriorated areas and the progress of deterioration.

[0055] In the above-described embodiment, examples of the holding unit are a hard disk and a semiconductor memory. Also, based on the description in this specification, it will be understood by those skilled in the art touched upon in this specification that each part can be realized by a CPU (not shown), modules of installed application programs, modules of system programs, semiconductor memories that temporarily store the contents of data read from a hard disk, and the like.

[0056] According to the inspection support system 2 of the present embodiment, the generation of a three-dimensional model by aerial photography of the drone 8 is divided into two stages, namely, an outline and a detailed stage, so that it is not necessary to generate a detailed three-dimensional model of the inspection object other than the desired / necessary portion. This makes it possible to reduce the processing load and processing time while realizing detailed confirmation of the desired / necessary portion. In addition, since the communication performance required for the drone 8 is not so high, it is possible to suppress an increase in costs by using a relatively inexpensive drone.

[0057] The configuration and operation of the inspection support system 2 according to the embodiment have been described above. This embodiment is merely an example, and it will be understood by those skilled in the art that various modifications are possible in the combination of each component and each process, and that such modifications are also within the scope of the present invention.

[0058] In the embodiment, the case where the user 4 goes near the base station device 6 and operates the mobile terminal 10 to perform the inspection has been described, but the present invention is not limited to this. For example, the designation of the base station device 6 as the inspection target, the designation of the imaging area 12, the designation of the designated portion using the obtained outline three-dimensional model 18, and the confirmation of the designated detailed image may be performed by an operator at a center facility located away from the base station device 6. The worker only needs to transport the drone 8 to the vicinity of the base station device 6. The processing in this case can be understood by replacing the mobile terminal 10 in the embodiment with a desktop terminal of the operator.

[0059] Alternatively, a three-dimensional model of each part of the base station device 6 may be stored in advance on the operator's desktop terminal, and the operator may select a specified part from the three-dimensional model. [Explanation of symbols]

[0060] 2 Inspection support system, 4 User, 6 Base station equipment, 8 Drone, 10 Mobile terminal.

Claims

1. A terminal that controls an unmanned flying device equipped with imaging capabilities, Obtaining information for setting a first operation of the unmanned aerial vehicle to capture an image of a target; A function of acquiring from the unmanned aerial device an image obtained as a result of the unmanned aerial device performing the first operation; a function of receiving designation of a part of the target from a user using the image; A function of setting a second operation of the unmanned aerial vehicle to obtain an image of a specified portion of the target that is more detailed than the image of the specified portion of the target obtained in the first operation; A function of registering, in a holding means, specific information that identifies a specified portion of the target and a flight path determined for capturing an image of the specified portion; a function of, when receiving a request to re-image a specified portion of the target, referring to the storage means and re-setting a flight path stored in association with the portion; and displaying, on a display, in a comparable manner, images of a specified portion of the object obtained in the second operation at a different time point or a three-dimensional model generated from the images.

2. The computer program described in claim 1, further enabling the terminal to implement a function of controlling the unmanned aerial device to adjust its position or attitude for imaging the specified portion of the target by comparing an image used when accepting the designation of the portion of the target with a current image acquired by the imaging function of the unmanned aerial device.

3. The computer program described in claim 1 or 2, further enabling the terminal to implement a function of retaining the position of the unmanned aerial device when a first image was acquired, which is an image of a specified portion of the target that is more detailed than the image of the specified portion of the target acquired in the first operation, in association with the file of the first image.

4. A computer program as described in any one of claims 1 to 3, further enabling the terminal to implement a function of accepting and retaining a second image, which is an image of a specified portion of the target, acquired during the first operation and transmitted sequentially from the unmanned aerial device, the time when the second image was acquired, and the location where the second image was acquired.

Images