Inspection support system and inspection support method

Abstract

To provide an inspection support system and an inspection support method for facilitating inspection of a power distribution facility.SOLUTION: The check support system W1 includes the processor 20 that causes the display 40 to display the photographed image obtained by photographing the ground with the camera 51 of the unmanned flying object 50, and when the distribution facility to be checked is shown in the photographed image, the processor 20 superimposes and displays at least one of the mark indicating the position of the distribution facility, the facility name of the distribution facility, and the facility number of the distribution facility on the photographed image in accordance with the display position of the distribution facility in the photographed image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an inspection support system and an inspection support method. [Background technology]

[0002] Regarding technology for capturing images of the ground using a camera on an unmanned aerial vehicle such as a drone, for example, the technology described in Patent Document 1 is known. That is, Patent Document 1 describes that "an AR (Augumented Reality) image is generated and displayed in which the movement path of the mobile device is superimposed as a virtual object on a real object, which is an image captured by a camera." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2021-193538 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Patent Document 1 allows a user to check the movement path of a mobile device on a captured image, but does not particularly consider making it easier for the user to distinguish specific inspection targets shown in the captured image. For example, when an inspector inspects multiple power distribution facilities sequentially while looking at images captured by an unmanned aerial vehicle, it is often difficult to distinguish between the power distribution facilities in the captured image because the appearances of the facilities are similar or the facilities are surrounded by trees. It is desirable to reduce the burden on users when inspecting power distribution facilities, but Patent Document 1 does not describe such technology.

[0005] Therefore, an object of the present disclosure is to provide an inspection support system and an inspection support method that facilitate the inspection of power distribution equipment. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, the inspection support system of the present disclosure includes a processing unit that displays on a display device images obtained by photographing the ground with a camera of an unmanned aerial vehicle, and when the distribution equipment to be inspected appears in the photographed image, the processing unit superimposes at least one of a mark indicating the position of the distribution equipment, the equipment name of the distribution equipment, and the equipment number of the distribution equipment on the photographed image, in accordance with the display position of the distribution equipment in the photographed image. [Effects of the Invention]

[0007] According to the present disclosure, an inspection support system and an inspection support method that facilitate inspection of power distribution equipment can be provided. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a functional block diagram of an inspection support system according to an embodiment. [Figure 2] FIG. 2 is an explanatory diagram showing the state of an unmanned aerial vehicle in flight in the inspection support system according to the embodiment. [Figure 3] FIG. 2 is an explanatory diagram of facility information in the inspection support system according to the embodiment. [Figure 4] 1 is a flowchart relating to setting of inspection targets and inspection sequences in the inspection support system according to the embodiment. [Figure 5] 10 is a flowchart of a process relating to the superimposed display of marks and the like of power distribution facilities in the inspection support system according to the embodiment. [Figure 6] 10 is an example of a screen display including a mark indicating the location of a power distribution facility in the inspection support system according to the embodiment. [Figure 7] 10 is an example of a screen display including a direction mark indicating the direction to the next power distribution facility in the inspection support system according to the embodiment. [Figure 8] 10 is a flowchart of a process related to a pass determination of a distribution facility to be inspected in the inspection support system according to the embodiment. [Figure 9]10 is a flowchart of a process related to assigning a file name to still image information in the inspection support system according to the embodiment. [Figure 10] 10 is an example of a screen display relating to the assignment of a file name to still image information in the inspection support system according to the embodiment.

[0009] <Embodiment> <Configuration of inspection support system> FIG. 1 is a functional block diagram of an inspection support system W1 according to an embodiment. The inspection support system W1 shown in Fig. 1 is a system for supporting the inspection of power distribution facilities, and includes a data processing device 100, an input device 30, a display device 40, and an unmanned aerial vehicle 50. Examples of power distribution facilities to be inspected include electric wires and utility poles, as well as steel towers and pole-mounted transformers.

[0010] In this embodiment, as an example, an unmanned aerial vehicle 50 such as a drone is made to fly autonomously along a predetermined flight route, and photographs of ground-based power distribution facilities are taken by a camera 51 of the unmanned aerial vehicle 50. Note that "autonomous flight" means that the unmanned aerial vehicle 50 flies autonomously along a predetermined flight route that has been set in advance.

[0011] The data processing device 100 has a function of displaying, in a predetermined manner, captured images of the ground obtained by the camera 51 of the unmanned aerial vehicle 50 on the display device 40. The inspector inspects the power distribution equipment while looking at the captured images displayed on the display device 40. The data processing device 100 also has a function of acquiring still image information 13 from the captured images (video) at a timing specified by the inspector (the timing when the capture button is pressed), and storing this still image information 13 in the memory unit 10 after assigning a predetermined file name to it.

[0012] 1 is used by a user (such as an inspector) to perform a predetermined input operation, and is connected to the data processing device 100. For example, a keyboard or a mouse is used as the input device 30. The input device 30 is used when the user specifies the power distribution equipment to be inspected and the inspection sequence, and when setting the flight route of the unmanned aerial vehicle 50.

[0013] The display device 40 displays the processing results of the data processing device 100 in a predetermined manner. For example, a display is used as the display device 40. Note that a touch panel terminal that combines the functions of the data processing device 100, the input device 30, and the display device 40, such as a smartphone or a tablet, may also be used.

[0014] The unmanned aerial vehicle 50 is an unmanned aircraft that flies by remote control or predetermined automatic control. A drone, for example, is used as such an unmanned aerial vehicle 50. The unmanned aerial vehicle 50 is capable of ascending and descending vertically and diagonally, as well as hovering (stopping in mid-air), moving forward and backward, moving left and right, and rotating counterclockwise and clockwise.

[0015] As shown in FIG. 1, the unmanned aerial vehicle 50 is equipped with a camera 51. The unmanned aerial vehicle 50 is caused to fly along a predetermined flight route, and the camera 51 captures images of the power distribution equipment to be inspected. Although not shown, the camera 51 has a large number of image pickup elements arranged vertically and horizontally in a matrix. The light incident on each image pickup element is photoelectrically converted to generate data for the captured image. The images (video) captured by the camera 51 from moment to moment are transmitted to the data processing device 100 via wireless communication and displayed in real time on the display device 40.

[0016] 1, the data processing device 100 includes a storage unit 10 and a processing unit 20. Although not shown, the hardware configuration of the storage unit 10 includes non-volatile memory such as a read-only memory (ROM) or a hard disk drive (HDD), and volatile memory such as a random access memory (RAM) or a register. The storage unit 10 pre-stores a predetermined program (not shown) to be executed by the processing unit 20, as well as facility information 11 and altitude information 12. The storage unit 10 also stores still image information 13 acquired by a still image information acquisition unit 24 (described later).

[0017] The facility information 11 is a database relating to the power distribution facility to be inspected. Such facility information 11 includes the facility name and facility number of the power distribution facility, as well as latitude and longitude information indicating the location of the power distribution facility. The elevation information 12 is information for specifying the elevation (i.e., topography) of each point specified by latitude and longitude. For example, a digital elevation model (DEM) is used as this elevation information 12. A digital elevation model is a model in which geographic space is divided into meshes and an elevation value is associated with the center (latitude and longitude) of each mesh. Publicly available elevation information models include the geographic information of the Geospatial Information Authority of Japan and the global high-precision digital 3D map (ALOS World 3D) of the Japan Aerospace Exploration Agency (JAXA).

[0018] The still image information 13 is information about a still image acquired from the unmanned aerial vehicle 50 by the still image information acquisition unit 24. For example, when a user (inspector) viewing an image captured on the display device 40 presses a capture button (not shown) at a predetermined timing, the image captured at that timing is stored as still image information 13 in a storage medium (e.g., an SD card) of the unmanned aerial vehicle 50.

[0019] When the still image information acquisition unit 24 transmits an acquisition signal for the still image information 13 to the unmanned aerial vehicle 50 via wireless communication, the still image information 13 is transmitted from the unmanned aerial vehicle 50. The still image information 13 acquired by the still image information acquisition unit 24 is assigned a predetermined file name and stored in the memory unit 10.

[0020] The processing unit 20 has a function of displaying on the display device 40 an image of the ground captured by the camera 51 of the unmanned aerial vehicle 50. The processing unit 20 includes a processor such as a CPU (Central Processing Unit) as its hardware configuration, which is not shown in the figure. The CPU reads out a program stored in the ROM or hard disk drive and expands it into RAM, thereby executing predetermined processing.

[0021] 1, the processing unit 20 includes an equipment information acquisition unit 21, an altitude information acquisition unit 22, an aircraft information acquisition unit 23, and a still image information acquisition unit 24. In addition to the components described above, the processing unit 20 also includes an equipment information display unit 25, an AR display content generation unit 26, an equipment passing determination unit 27, an equipment naming unit 28, and a terrain following processing unit 29.

[0022] The facility information acquisition unit 21 appropriately acquires facility information 11 from the storage unit 10. Such facility information 11 is used, for example, when a user selects a distribution facility to be inspected or when setting the inspection sequence of the distribution facility. The facility information 11 is also appropriately used when superimposing a mark indicating the location of the distribution facility on a captured image or when assigning a predetermined file name to the still image information 13. The altitude information acquisition unit 22 appropriately acquires the altitude information 12 from the storage unit 10. Such altitude information 12 is used when the terrain following processing unit 29, which will be described later, creates a flight route for the unmanned aerial vehicle 50.

[0023] The aircraft information acquisition unit 23 acquires aircraft information from the unmanned aerial vehicle 50 on a moment-by-moment basis. The aircraft information includes the latitude, longitude, and altitude of the unmanned aerial vehicle 50 during flight, the speed of the unmanned aerial vehicle 50, and the direction of travel of the unmanned aerial vehicle 50. Furthermore, the aircraft information includes not only images captured by the camera 51 of the unmanned aerial vehicle 50 on a moment-by-moment basis, but also the zoom ratio of the camera 51 and the direction of the optical axis of the camera 51 (or the rotation angle of the camera 51 relative to the body of the unmanned aerial vehicle 50). Note that if the camera 51 is fixed and does not rotate relative to the body of the unmanned aerial vehicle 50, there is no particular need for the direction of the optical axis of the camera 51 to be included in the aircraft information described above.

[0024] The still image information acquisition unit 24 acquires still image information 13 at a timing specified by the user in the captured image (video) from the unmanned aerial vehicle 50. As described above, the still image information 13 acquired by the still image information acquisition unit 24 is given a predetermined file name and then stored in the memory unit 10.

[0025] When the user selects the distribution equipment to be inspected or specifies the inspection order of the distribution equipment, the equipment information display unit 25 displays the equipment number, etc. in a predetermined manner on the display device 40 in accordance with the position of the distribution equipment on the map.

[0026] When a power distribution facility appears in an image captured by the camera 51 of the unmanned aerial vehicle 50, the AR display content generation unit 26 superimposes a mark indicating the location of the power distribution facility and a facility number on the captured image (i.e., AR display). Note that "AR" is an abbreviation for "Augmented Reality." The equipment passing determination unit 27 determines whether the unmanned aerial vehicle 50 has approached the next distribution equipment that it is scheduled to arrive at in a planar view as the unmanned aerial vehicle 50 passes sequentially over the distribution equipment to be inspected.

[0027] If the distribution equipment to be inspected is captured in still image information 13 at a timing specified by the user in the captured image (video), the equipment name assignment unit 28 causes the file name of this still image information 13 to include the equipment name and equipment number of the distribution equipment. The terrain following processing unit 29 creates a flight route for the unmanned aerial vehicle 50 so that the altitude above the ground when the unmanned aerial vehicle 50 flies falls within a predetermined range.

[0028] FIG. 2 is an explanatory diagram showing the unmanned aerial vehicle 50 in flight. Note that Figure 2 shows a case in which an unmanned aerial vehicle 50 is flown autonomously and photographs a power distribution facility E1 in a mountainous area sequentially with a camera 51. Incidentally, although each utility pole in Figure 2 is labeled as a power distribution facility E1, the electric wires (overhead wires) strung across the utility poles are also included in the "power distribution facility." The controller 60 shown in Figure 2 is a device that transmits and receives predetermined signals to and from the unmanned aerial vehicle 50. Note that the data processing device 100 (see Figure 1) described above may be built into the controller 60 or may be provided separately; either is acceptable.

[0029] Incidentally, although it is possible for inspector P1 to change the direction of travel of unmanned aerial vehicle 50 during flight by operating controller 60, as described above, in this embodiment, unmanned aerial vehicle 50 is made to fly autonomously. In this case, the operation of controller 60 performed by inspector P1 is limited to making it possible to send and receive data to and from unmanned aerial vehicle 50 and starting the flight of unmanned aerial vehicle 50.

[0030] The controller 60 is electrically connected to a display device 40, such as a tablet terminal, via wiring. Images captured by the camera 51 of the unmanned aerial vehicle 50 are displayed in real time on the display device 40. The inspector P1 inspects the power distribution equipment by viewing the captured images displayed on the display device 40. It is also possible to record the captured images of the power distribution equipment so that the inspector P1 can check the captured images after the flight of the unmanned aerial vehicle 50 has finished.

[0031] In recent years, the number of inspectors has been declining, and the inspectors are also aging. However, by using an unmanned aerial vehicle 50 to inspect the power distribution equipment E1, inspections can be performed even in situations where there is a shortage of personnel. Furthermore, even in cases where natural disasters cause fallen trees or landslides, making it difficult to access areas on foot or by vehicle, a rapid response can be achieved by using the unmanned aerial vehicle 50 to photograph the power distribution equipment E1 from the sky. Furthermore, inspecting each piece of power distribution equipment E1 by hand requires a great deal of time and effort, and in some cases, there may be areas with poor footing. In contrast, the use of an unmanned aerial vehicle 50 in this embodiment can improve the efficiency and reliability of inspections and reduce costs.

[0032] FIG. 3 is an explanatory diagram of the facility information 11. As described above, the equipment information 11 shown in FIG. 3 is a database related to the power distribution equipment to be inspected, and is created in advance and stored in the storage unit 10 (see FIG. 1). In the example of FIG. 3, the equipment information 11 associates an "equipment name," an "equipment number," a "latitude," and a "longitude." The "equipment name" is the name of the power distribution equipment to be inspected. The "equipment number" is an identification number assigned to the power distribution equipment to be inspected. The "latitude" and "longitude" are the latitude and longitude of the location of the power distribution equipment. In addition to the "latitude" and "longitude" described above, the equipment information 11 may also include a value indicating the "altitude" (e.g., altitude above ground) of the power distribution equipment, if necessary. Specific methods for using the equipment information 11 will be described later.

[0033] <Pre-flight processing> Figure 4 is a flowchart for setting inspection targets and inspection sequences (see also Figure 1 as appropriate). 4 is performed in advance, prior to the start of flight of the unmanned aerial vehicle 50. In step S101, the processing unit 20 determines whether or not a power distribution facility that is a candidate for inspection has been designated. In step S101, if a power distribution facility that is a candidate for inspection has not been designated (S101: No), the processing unit 20 repeats the processing of step S101. In addition, if a power distribution facility that is a candidate for inspection has been designated (S101: Yes) in step S101, the processing of the processing unit 20 proceeds to step S102.

[0034] For example, suppose that the user presses an "area search" button (not shown) on the display screen and specifies a predetermined inspection target area (region name). In this case, the processing unit 20 determines that the power distribution equipment included in this inspection target area has been specified as a candidate for inspection (S101: Yes). Also, for example, suppose that the user presses a "Facility Search" button (not shown) on the display screen through an input operation to specify a predetermined facility name. In this case, the processing unit 20 determines that the power distribution facility corresponding to the facility name has been specified as a candidate for inspection (S101: Yes). Alternatively, the user can specify a predetermined business establishment name and specify the power distribution facilities under the jurisdiction of that business establishment as candidates for inspection.

[0035] In step S102, the processing unit 20 causes the equipment information display unit 25 to display on the map a mark (not shown) indicating the location of the power distribution equipment that is a candidate for inspection. That is, when the user specifies an inspection target area or equipment name through an operation via the input device 30 (S101: Yes), the processing unit 20 causes the processing unit 25 to display on the map a mark indicating the location of the power distribution equipment that corresponds to the inspection target area or the equipment name as a candidate for inspection (S102).

[0036] More specifically, the processing unit 20 first refers to the equipment information 11 (see FIG. 3 ) and identifies the latitude and longitude of the power distribution equipment that is a candidate for inspection and that was selected in step S101. The processing unit 20 then displays a predetermined mark indicating the location of the power distribution equipment at the latitude and longitude position of the power distribution equipment on the map. Note that instead of (or together with) the mark indicating the location of the power distribution equipment, letters, symbols, or numbers indicating the equipment name or equipment number of the power distribution equipment may be displayed on the map.

[0037] In step S103, the processing unit 20 determines whether or not the power distribution equipment to be inspected has been selected and the inspection order has been specified. The power distribution equipment to be inspected is appropriately selected from the power distribution equipment candidates to be inspected (those specified in step S101). Furthermore, the inspection order for the power distribution equipment to be inspected is appropriately set by the user's operation via the input device 30. For example, if the user sequentially selects marks (not shown) of multiple power distribution equipment to be inspected on the display screen, the inspection order for the power distribution equipment may be set in the order of selection.

[0038] In step S103, if the inspection object is not selected and the inspection order is not specified (S103: No), the processing unit 20 repeats the process of step S103. Also, in step S103, if the inspection object is selected and the inspection order is specified (S103: Yes), the processing unit 20 proceeds to step S104.

[0039] In step S104, the processing unit 20 causes the equipment information display unit 25 to display marks (not shown) of the power distribution equipment to be inspected on the map, as well as the inspection sequence (for example, a predetermined number), thereby allowing the user to check the power distribution equipment to be inspected and the inspection sequence on the map.

[0040] Next, in step S105, the processing unit 20 creates a flight route for the unmanned aerial vehicle 50. That is, when the user operates the input device 30 to select a power distribution facility to be inspected from among candidates for inspection, and further specify an inspection sequence (S103: Yes), the processing unit 20 creates a flight route for the unmanned aerial vehicle 50 based on the inspection facility and the inspection sequence (S105).

[0041] Specifically, the processing unit 20 sets a plurality of waypoints (WP) so that the unmanned aerial vehicle 50 flies sequentially over a plurality of power distribution facilities to be inspected in an inspection order specified by the user. Note that a "waypoint" is location information indicating a pass point of the unmanned aerial vehicle 50, and includes information indicating the pass order of the unmanned aerial vehicle 50 in addition to latitude, longitude, and altitude. For example, waypoints may be set one by one above the power distribution facilities to be inspected. When creating a flight route, the processing unit 20 sets waypoints using the terrain following processing unit 29 so that the ground altitude of the unmanned aerial vehicle 50 during flight falls within a predetermined range. After performing the processing of step S105, the processing unit 20 ends the series of processes related to creating a flight route (END).

[0042] In conventional technology, inspectors set waypoints one by one according to the location of the power distribution equipment to be inspected while comparing equipment information stored on paper or in a specified file format with a specified map. Therefore, setting waypoints required a great deal of time. In contrast, in this embodiment, a flight route (multiple waypoints) is automatically created when the inspection target and inspection sequence are specified. This significantly reduces the effort and time required to create a flight route, thereby achieving high efficiency. Furthermore, the processing unit 20 can appropriately set each waypoint based on the latitude and longitude of the power distribution equipment to be inspected.

[0043] <In-flight processing> FIG. 5 is a flowchart of a process relating to the superimposed display of marks and the like of power distribution facilities (also see FIG. 1 as appropriate). It is assumed that at the time of "START" in FIG. 5, the unmanned aerial vehicle 50 is flying along a predetermined flight route. In step S201, the processing unit 20 calculates the orientation (angle) of the unmanned aerial vehicle 50 relative to the power distribution facility and the distance between the unmanned aerial vehicle 50 and the power distribution facility based on the above-mentioned facility information 11 and aircraft information. Information indicating the moment-by-moment orientation of the unmanned aerial vehicle 50 is included in the aircraft information and is acquired from the unmanned aerial vehicle 50 by the aircraft information acquisition unit 23. The distance between the unmanned aerial vehicle 50 and the power distribution facility is calculated based on the moment-by-moment latitude and longitude of the unmanned aerial vehicle 50 included in the aircraft information and the latitude and longitude of the power distribution facility included in the facility information 11.

[0044] In step S202, the processing unit 20 calculates the display position of the power distribution facility on the display device 40. That is, based on the calculation result of step S201 as well as the number of vertical and horizontal pixels of the display device 40, the processing unit 20 determines at what position (i.e., in what pixel) the power distribution facility photographed by the camera 51 of the unmanned aerial vehicle 50 will be displayed on the display device 40.

[0045] In step S203, processing unit 20 determines whether or not the power distribution facility is displayed within the display screen of display device 40. That is, processing unit 20 determines whether or not the display position (pixel) of the power distribution facility is within the range of the display screen of display device 40, in which a large number of pixels are arranged vertically and horizontally in a matrix. In step S203, if the power distribution facility is displayed within the display screen (captured image) (S203: Yes), processing by processing unit 20 proceeds to step S204.

[0046] In step S204, the processing unit 20 displays a mark or the like indicating the location of the power distribution facility using AR. Here, "AR display" means that a predetermined mark or the like indicating the location of the power distribution facility is superimposed on the captured image as if it were located near the power distribution facility.

[0047] To explain step S204 in more detail, the processing unit 20 superimposes (in AR display) on the captured image a mark or the like indicating the location of the power distribution facility based on the latitude and longitude indicating the location of the power distribution facility, the position, orientation, and zoom ratio of the camera 51 at the time of capturing the image, and the number of vertical and horizontal pixels of the display device 40. This allows the user to grasp at a glance where in the captured image the power distribution facility is located and what its facility number is.

[0048] After performing the process of step S204, the process of the processing unit 20 returns to "START" (RETURN). When the display position of the power distribution facility on the display device 40 moves on the screen, the mark or the like indicating the position of the power distribution facility also moves on the screen accordingly. Display examples of such marks or the like (see FIG. 6) will be described later.

[0049] Also, in step S203, if the power distribution facility is not displayed on the display screen (photographed image) (S203: No), the processing by the processing unit 20 proceeds to step S205. In step S205, the processing unit 20 displays, in AR, a mark or the like indicating the direction to the next power distribution facility. This allows the user to easily understand in which direction the unmanned aerial vehicle 50 is scheduled to reach next. After performing the processing of step S205, the processing by the processing unit 20 returns to "START" (RETURN).

[0050] FIG. 6 is an example of a screen display including marks M11 and M12 indicating the locations of the power distribution facilities. In the example of Figure 6, two utility poles E11 and E12 included in the power distribution equipment to be inspected are shown in the image captured by the unmanned aerial vehicle 50 (see Figure 2). In addition, a circular mark M11 indicating the position of utility pole E11 is displayed above the screen of utility pole E11. When this image was captured, utility pole E11 was the next target position for the unmanned aerial vehicle 50, so mark M11 is displayed in a predetermined color (e.g., red, shown by hatching in Figure 6) indicating that it is the target position.

[0051] The position of the power distribution facility that the unmanned aerial vehicle 50 is scheduled to reach next is set as the "target position." Therefore, the power distribution facility whose position is the target position will be switched sequentially as the unmanned aerial vehicle 50 moves. As shown in Figure 6, the number "11," which is the facility number of the utility pole E11, is displayed above the mark M11 on the screen. This facility number is identified from the facility information 11 (see Figure 3) described above.

[0052] The other utility pole E12 shown in Figure 6 is located farther away than utility pole E11 in the direction of travel of unmanned aerial vehicle 50 (see Figure 2). When this image was captured, utility pole E12 was not the target position of unmanned aerial vehicle 50, so mark M12 is displayed in a predetermined color (e.g., white, shown as a white outline in Figure 6) that simply indicates that it is one of the inspection targets. Furthermore, the number "12," which is the equipment number of utility pole E12, is displayed above mark M12 on the screen.

[0053] The dashed line L1 in FIG. 6 is a straight line passing through the marks M11 and M12. This line L1 may be, for example, a straight line parallel to the flight route of the unmanned aerial vehicle 50 (see FIG. 2) (a straight line whose display position on the captured image is slightly away from the flight route), or may be a straight line that approximately coincides with the flight route. For example, the processing unit 20 connects the multiple marks M11, M12,... that are adjacent in the inspection order of the power distribution equipment (utility poles E11, E12,...) corresponding to the marks M11, M12,... with a line segment (for example, line L1 in FIG. 6) and displays the line segment superimposed on the captured image. This allows the user to confirm in the captured image the direction in which the unmanned aerial vehicle 50 is heading. Note that the flight route of the unmanned aerial vehicle 50 is often a polygonal line overall, so depending on the range captured in the captured image, the line L1 may not be a straight line but a polygonal line.

[0054] 6 shows an example in which marks M11 and M12 indicating the positions of utility poles E11 and E12 and an equipment number are displayed, but this is not limiting. For example, the equipment names of utility poles E11 and E12 may be superimposed on the captured image. In this way, when the distribution equipment to be inspected appears in the captured image (S203: Yes in FIG. 5), the processing unit 20 superimposes at least one of the mark indicating the position of the distribution equipment, the equipment name of the distribution equipment, and the equipment number of the distribution equipment on the captured image, in accordance with the display position of the distribution equipment in the captured image (S204).

[0055] This allows the user to grasp the location of the power distribution equipment on the captured image, as well as the equipment number of the power distribution equipment. For example, in mountainous areas, power distribution equipment is often surrounded by trees, etc., making it difficult for the user to identify the power distribution equipment (identify its equipment number) based on the captured image alone. Furthermore, if the unmanned aerial vehicle 50 rotates during an inspection while looking at the captured image and then returns to its original orientation, it is difficult to determine based on the captured image alone whether the power distribution equipment visible at that time is the same as the one before the rotation, or whether it is a different power distribution equipment.

[0056] In particular, since power distribution facilities such as utility poles have very similar appearances, it can be difficult to distinguish between them. In contrast, in this embodiment, marks M11 and M12 indicating the position of the utility poles, etc. are displayed, so that the user can grasp the location of the inspection target at a glance.

[0057] Although not shown in Fig. 6, a panel indicating which power distribution facility the unmanned aerial vehicle 50 has currently passed (at the time of photographing) and which power distribution facility it is scheduled to reach next may be displayed on the photographing screen, making it easier for the user to check the progress of the inspection.

[0058] Furthermore, the display of real-time captured images and the display of a map may be switched by a user's input operation. When the map is displayed, the location of the unmanned aerial vehicle 50, the location of the home point (takeoff point), as well as the location of the power distribution equipment to be inspected and the flight route are displayed on the map. By displaying such a map, the user can easily understand how far the unmanned aerial vehicle 50 has traveled from a specified home point.

[0059] FIG. 7 is an example of a screen display including a direction mark M20 indicating the direction to the next power distribution facility. As described above, if the power distribution facility to be inspected is not visible in the captured image (S203: No in FIG. 5), the processing unit 20 superimposes on the captured image a direction mark M20 indicating the direction to the power distribution facility that the unmanned aerial vehicle 50 is scheduled to reach next (S205). In the example of FIG. 7, the direction mark M20 is displayed near a line L1 indicating the general flight route of the unmanned aerial vehicle 50. It is assumed that power distribution facilities to be inspected are located in the lower right of the screen in FIG. 7 (an area not visible on the screen) and in the upper left of the screen (an area not visible on the screen), and that a line segment connecting these positions is displayed on the captured image as line L1.

[0060] This allows the user to understand that the photographed image does not show the power distribution facility to be inspected. Also, the user can easily understand the direction of the power distribution facility that the unmanned aerial vehicle 50 is scheduled to arrive at next (the direction from the unmanned aerial vehicle 50 to the power distribution facility in a planar view).

[0061] FIG. 8 is a flowchart of a process related to the pass determination of the power distribution facility to be inspected (see also FIG. 1 as appropriate). The series of processes shown in Figure 8 are performed in parallel with the series of processes in the flowchart shown in Figure 5. Also, at the time of "START" in Figure 8, it is assumed that the unmanned aerial vehicle 50 is flying along a predetermined flight route. In step S301, the processing unit 20 acquires aircraft information from the unmanned aerial vehicle 50. The aircraft information includes latitude and longitude information indicating the position of the unmanned aerial vehicle 50 from moment to moment.

[0062] In step S302, the processing unit 20 determines whether the unmanned aerial vehicle 50 has reached a distance from the power distribution facility that is less than a predetermined value. That is, the processing unit 20 determines whether the unmanned aerial vehicle 50 has entered a circle (not shown) with a predetermined radius in a planar view, centered on the position of the power distribution facility at the target location (the power distribution facility that the unmanned aerial vehicle 50 is scheduled to reach next). The "predetermined value" mentioned above is a threshold value that serves as a criterion for determining whether the unmanned aerial vehicle 50 has approached the power distribution facility in a planar view, and is set in advance. The position (latitude and longitude) of the power distribution facility is identified based on the facility information 11 (see FIG. 3). The position (latitude and longitude) of the unmanned aerial vehicle 50 is also identified based on moment-to-moment aircraft information.

[0063] In step S302, if the body of the unmanned aerial vehicle 50 has not reached a distance from the power distribution facility that is equal to or less than the predetermined value (S302: No), the processing of the processing unit 20 proceeds to step S303. In step S303, the processing unit 20 displays a mark indicating the target position of the unmanned aerial vehicle 50 in a first manner. That is, when the captured image shows a power distribution facility that the unmanned aerial vehicle 50 is scheduled to reach next among the multiple power distribution facilities to be inspected, the processing unit 20 superimposes a mark indicating the position of that power distribution facility on the captured image in a first manner. In the example of FIG. 6, the mark M11 is displayed in a predetermined color (e.g., red, shown by hatching in FIG. 6) indicating that it is the target position. This allows the user to easily grasp the position of the power distribution facility E11 that the unmanned aerial vehicle 50 is scheduled to reach next.

[0064] The display mode of the mark M11 is not particularly limited as long as it can be easily recognized by the user. For example, various display modes are possible, such as blinking the mark M11, pulsating the mark, or changing the size of the mark M11.

[0065] Furthermore, in step S302, if the body of the unmanned aerial vehicle 50 reaches a distance from the power distribution facility that is less than or equal to a predetermined value (S302: Yes), the processing of the processing unit 20 proceeds to step S304. In step S304, the processing unit 20 determines, using the facility passage determination unit 27, that the unmanned aerial vehicle 50 has passed the power distribution facility. In other words, the processing unit 20 considers that the unmanned aerial vehicle 50 has passed above the power distribution facility that was the target position and was to be inspected.

[0066] Next, in step S305, the processing unit 20 switches the power distribution facility at the target position. For example, when the unmanned aerial vehicle 50 passes the utility pole E11 on the front side of the screen shown in Figure 6, the processing unit 20 sets the position of another utility pole E12 on the other side of this utility pole E11 as the next target position of the unmanned aerial vehicle 50.

[0067] In step S306, the processing unit 20 displays the mark of the new target position in a first manner, and displays the mark indicating the position of the power distribution facility that has already been passed (including the case where the power distribution facility is actually approaching) in a second manner. That is, when the distance in a plan view between the power distribution facility that is next to be reached and the unmanned aerial vehicle 50 becomes equal to or less than a predetermined value (S302: Yes), the processing unit 20 switches the display manner of the mark indicating the position of the power distribution facility from the first manner to the second manner (S306).

[0068] As a specific example, the processing unit 20 changes the display mode of the mark M11 indicating the position of the utility pole E11 in Fig. 6 from a first mode (hatched in Fig. 6) to a second mode (white in Fig. 6). This allows the user to know in real time that the unmanned aerial vehicle 50 is approaching the utility pole E11 in a planar view and is about to pass over this utility pole E11. In addition, the processing unit 20 changes the display mode of the mark M12 indicating the position of another utility pole E12 in Figure 6 from the second mode (white in Figure 6) to the first mode (hatched in Figure 6). This allows the user to understand that the position of the utility pole E12 has been set as the target position that the unmanned aerial vehicle 50 is scheduled to reach next. After performing the processing of step S303 or S306, the processing of the processing unit 20 returns to "START" (RETURN).

[0069] <Processing during or after flight> FIG. 9 is a flowchart of a process for assigning a file name to still image information (also see FIG. 1 as appropriate). 9 may be performed while the unmanned aerial vehicle 50 is flying, or after the flight has ended. At the time of "START" in FIG. 9, still image information 13 at the timing specified by the user (the timing when the capture button is pressed) in the captured image (video) is assumed to be stored in the storage medium (e.g., an SD card) of the unmanned aerial vehicle 50.

[0070] In step S401, the processing unit 20 acquires still image information 13 from the unmanned aerial vehicle 50 using the still image information acquisition unit 24. The still image information 13 is linked to information indicating the date and time of the image capture, as well as the position (latitude, longitude, and altitude) of the unmanned aerial vehicle 50 and the orientation of the camera 51.

[0071] In step S402, the processing unit 20 identifies the position of the unmanned aerial vehicle 50 and the orientation of the camera 51 based on the still image information 13. That is, the processing unit 20 reads information linked to the still image information 13 to identify the position of the unmanned aerial vehicle 50 and the orientation of the camera 51 at the time the still image was captured.

[0072] In step S403, the processing unit 20 calculates the value of the center coordinates of the screen of the display device 40 using the equipment naming unit 28. That is, the processing unit 20 calculates the latitude and longitude of the position shown at the center coordinates of the screen of the display device 40, which has pixels arranged in a matrix in the vertical and horizontal directions. Note that the processing of step S403 uses the position of the unmanned aerial vehicle 50 and the orientation of the camera 51 (the information identified in step S402).

[0073] In step S404, the processing unit 20 determines whether or not a power distribution facility exists within a range of a predetermined distance from the center coordinates of the screen using the facility naming unit 28. That is, the processing unit 20 determines whether or not a power distribution facility exists within a range of a distance (distance in a planar view) from the latitude and longitude position corresponding to the center coordinates of the display device 40 as a reference.

[0074] In step S404, if there is a power distribution facility within a range of a predetermined distance or less from the center coordinates (S404: Yes), the processing unit 20 proceeds to step S405. In step S405, the processing unit 20 causes the equipment name assigning unit 28 to assign the equipment name of the power distribution equipment that is closest to the center coordinates of the screen as the file name of the still image information 13. Note that the equipment number of the power distribution equipment may be included in the file name of the still image information 13 together with (or instead of) the equipment name.

[0075] In this way, when still image information is acquired as a captured image and the still image information shows the distribution equipment to be inspected, the processing unit 20 includes one or both of the equipment name and equipment number of the distribution equipment in the file name of the still image information 13. This allows the user to easily determine which distribution equipment is shown in the still image information 13 from the file name.

[0076] In step S404, if there is no power distribution facility within a range that is equal to or less than the predetermined distance from the center coordinates of the screen (S404: No), the processing unit 20 proceeds to step S406. In step S406, the processing unit 20 assigns a predetermined file name to the still image information 13, indicating that the still image does not show any power distribution equipment. For example, a file name such as "No Power Distribution Equipment" may be included in the still image information 13. Alternatively, a file name including information on the date and time of shooting and the shooting location may be assigned to the still image information 13 based on a predetermined naming rule. This allows the user to understand from the file name that the still image information 13 does not show any power distribution equipment. After performing the processing of step S405 or S406, the processing unit 20 ends the series of processes (END).

[0077] FIG. 10 shows an example of a screen display relating to the assignment of file names to still image information. It is assumed that the file name of the still image is displayed in bar B1 shown in Fig. 10. In the example of Fig. 10, the equipment name of utility pole E13 shown in the still image is "equipment N" and its equipment number is "number 13", so the file name "equipment N: number 13" is assigned to still image information 13 (see Fig. 1). These equipment names and equipment numbers are identified based on the above-mentioned equipment information 11 (see Fig. 3).

[0078] In this way, the equipment name and equipment number of the power distribution equipment shown in the still image are assigned as file names. Therefore, the user can easily identify the main power distribution equipment shown in the still image (or the fact that no power distribution equipment is shown in the still image) without having to check each still image in each file of still image information 13 (see FIG. 1). Note that still image information 13 is used appropriately when the user considers whether or not maintenance of the power distribution equipment is necessary and the specific maintenance method, etc.

[0079] Furthermore, the user can understand what is mainly shown in the still image from the file name of the still image without having to open the still image file as shown in FIG. 10 and check the contents, which improves convenience.

[0080] <Effects> According to this embodiment, when the ground is photographed by the camera 51 of the unmanned aerial vehicle 50 and the power distribution equipment to be inspected is captured in the captured image, the location and name of the power distribution equipment are superimposed on the captured image (see FIG. 6). This allows the user to easily identify the specific power distribution equipment that is captured in the captured image (i.e., the power distribution equipment that the user is looking at). This significantly reduces the burden on the user when inspecting power distribution equipment. Furthermore, the user can quickly and appropriately grasp the status of the power distribution equipment.

[0081] Furthermore, in this embodiment, when the distribution equipment to be inspected is shown in the still image information 13, the equipment name and equipment number of the distribution equipment are included in the file name (see FIG. 10). This allows the user to ascertain the equipment name and equipment number of the distribution equipment shown in the still image from the file name without having to open each still image file and check it, thereby reducing the effort required to manage the still image information 13. In this way, this embodiment can provide an inspection support system W1 or the like that makes it easy to inspect distribution equipment.

[0082] <<Variations>> The inspection support system W1 and the inspection support method according to the present disclosure have been described above in the embodiments, but the present disclosure is not limited to these descriptions and various modifications can be made. For example, in the embodiment, the case where the inspection target is a power distribution facility has been described, but the present invention is not limited to this. That is, the embodiment can be applied to the inspection of power transmission facilities, substation facilities, plants, railways, bridges, roads, solar panels, wind turbines for wind power generation, and the like.

[0083] In addition, in the embodiment, the case where an inspector inspects the power distribution equipment has been described, but this is not limiting. For example, a predetermined AI (Artificial Intelligence) may perform the inspection based on image recognition together with (or instead of) the inspector.

[0084] In addition, in the embodiment, the case where the unmanned aerial vehicle 50 performs autonomous flight has been described, but this is not limited to this. That is, the direction, etc. of the unmanned aerial vehicle 50 may be changed during flight by the pilot operating the controller 60.

[0085] Furthermore, the processes (processes such as the inspection support method) executed by the inspection support system W1 may be executed as a predetermined computer program. The program may be provided via a communication line or may be written to a predetermined storage medium and distributed.

[0086] Furthermore, the present disclosure is not limited to the embodiments and includes various modifications. For example, the embodiments have been described in detail to clearly explain the present disclosure, and the present disclosure is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to add, delete, or replace part of the configuration of the embodiments with other configurations.

[0087] Furthermore, the above-mentioned configurations, functions, processing units, processing means, etc. may be partly or entirely implemented in hardware, for example, by designing them as integrated circuits. Furthermore, the above-mentioned configurations, functions, etc. may be implemented in software, with a processor interpreting and executing a program that implements each function. Information such as the programs, tables, and files that implement each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0088] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0089] 10 Storage section 11 Equipment information 12 Elevation information 13 Still image information 20 Processing section 21 Equipment information acquisition department 22 Altitude information acquisition unit 23 Aircraft information acquisition unit 24 Still image information acquisition section 25 Equipment information display section 26 AR display content generation section 27 Equipment passage determination section 28 Equipment Naming Section 29 Terrain following processing unit 30 Input Devices 40 Display device 50 Unmanned Aerial Vehicles 51 Camera 60 Controller 100 Data processing device B1 Bar E1 Power distribution equipment E11, E12, E13 utility poles (power distribution equipment) L1 Line (line segment) M11, M12, M13 marks (marks indicating the location of power distribution equipment) M20 Orientation Mark P1 Inspector W1 Inspection Support System

Claims

1. a processing unit that displays on a display device an image of the ground captured by the camera of the unmanned aerial vehicle; When the distribution equipment to be inspected appears in the captured image, the processing unit superimposes at least one of a mark indicating the position of the distribution equipment, the equipment name of the distribution equipment, and the equipment number of the distribution equipment on the captured image, in accordance with the display position of the distribution equipment in the captured image.

2. When the power distribution facility to be inspected is not shown in the photographed image, the processing unit superimposes a direction mark indicating the direction to the power distribution facility that the unmanned aerial vehicle is scheduled to reach next on the photographed image.

2. The inspection support system according to claim 1, wherein:

3. The processing unit connects the plurality of marks that correspond to adjacent inspection orders of the power distribution facilities with a line segment and displays the line segment superimposed on the captured image.

2. The inspection support system according to claim 1, wherein:

4. The processing unit When the photographed image shows a distribution facility that the unmanned aerial vehicle is scheduled to arrive at next among a plurality of distribution facilities to be inspected, the mark indicating the position of the distribution facility is superimposed on the photographed image in a first manner; When the distance between the power distribution facility and the unmanned aerial vehicle in a plan view becomes equal to or less than a predetermined value, the display mode of the mark indicating the position of the power distribution facility is switched from the first mode to a second mode.

2. The inspection support system according to claim 1, wherein:

5. When the processing unit acquires still image information as the photographed image and the still image information shows the power distribution equipment to be inspected, the processing unit includes one or both of the equipment name and equipment number of the power distribution equipment in the file name of the still image information.

2. The inspection support system according to claim 1, wherein:

6. The processing unit When an inspection target area or facility name is designated by a user through an input device, a mark indicating the location of the power distribution facility corresponding to the inspection target area or facility name is displayed on a map as a candidate for inspection; When a user operates the input device to select a power distribution facility to be inspected from the candidates and further specify an inspection sequence, a flight route for the unmanned aerial vehicle is created based on the inspection target and the inspection sequence.

2. The inspection support system according to claim 1, wherein:

7. A process of displaying on a display device an image of the ground captured by a camera of the unmanned aerial vehicle, In the above process, if the distribution equipment to be inspected appears in the captured image, at least one of a mark indicating the position of the distribution equipment, the equipment name of the distribution equipment, and the equipment number of the distribution equipment is superimposed on the captured image in accordance with the display position of the distribution equipment in the captured image.

Images