Repair support system and repair support method

Abstract

To provide a repair support system that provides appropriate and efficient repair support for substation equipment. [Solution] The repair support system R1 is equipped with a processing unit 30 that acquires images obtained by photographing substation equipment with a camera D11 of an unmanned aerial vehicle D1. The unmanned aerial vehicle D1 moves along a predetermined inspection route and takes images sequentially with the camera D11 at multiple shooting positions on the inspection route. The processing unit 30 compares the photographed images with a predetermined reference image for each shooting position and extracts damaged areas in the substation equipment from the photographed images. If repair of the damaged area is required, the processing unit 30 identifies a damage image similar to the photographed image from multiple damage images prepared in advance, and links the type of damage, repair method, and repair work procedure corresponding to the damage image to the photographed image showing the damaged area and displays them on the display unit 40.

Description

[Technical Field]

[0001] The present disclosure relates to a repair support system and the like. [Background technology]

[0002] As a technology for inspecting substation facilities using unmanned aerial vehicles such as drones, for example, the system described in Patent Document 1 is known. That is, Patent Document 1 describes that "based on the appearance characteristics of a specific part of the substation facility in a normal state where an abnormality accompanied by the occurrence of a foreign object may occur, the specific part is extracted from an image captured by the unmanned vehicle, regardless of whether the abnormality actually occurs." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-177995 Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology described in Patent Document 1, a "specific part" where a predetermined abnormality may occur is set in advance, and maintenance personnel visually check images of this "specific part." As a result, if an abnormality occurs in a location other than the "specific part" in the substation equipment, the abnormality may not be discovered or may be discovered late. Furthermore, since maintenance personnel visually check the captured images to determine whether or not an abnormality exists, there is a possibility that an abnormality may be overlooked if the maintenance personnel is inexperienced. As such, with the technology described in Patent Document 1, there is a possibility that an abnormality may be overlooked, or that the results of an abnormality determination may vary depending on the maintenance personnel's level of experience.

[0005] Therefore, an object of the present disclosure is to provide a repair support system or the like that provides appropriate and efficient repair support for substation equipment. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, the repair support system disclosed herein includes a processing unit that acquires images obtained by photographing substation equipment with a camera of an unmanned aerial vehicle, and the unmanned aerial vehicle moves along a predetermined inspection route and takes images sequentially with the camera at multiple shooting positions on the inspection route.The processing unit compares the captured images with a predetermined reference image for each shooting position and extracts damaged areas in the substation equipment from the captured images.If repair of the damaged area is required, the processing unit identifies a damage image similar to the captured image from multiple damage images prepared in advance, and links the type of damage, repair method, and repair work procedure corresponding to the damage image to the captured image containing the damaged area and displays them on a display device. [Effects of the Invention]

[0007] According to the present disclosure, it is possible to provide a repair support system or the like that provides appropriate and efficient repair support for substation equipment. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a configuration diagram of a repair support system according to an embodiment. [Figure 2] FIG. 1 is an explanatory diagram of a repair support system according to an embodiment. [Figure 3] FIG. 2 is an explanatory diagram of a reference image database of the repair support system according to the embodiment. [Figure 4] FIG. 2 is an explanatory diagram of a photographed image database of the repair support system according to the embodiment. [Figure 5] FIG. 2 is an explanatory diagram of a repair method database of the repair support system according to the embodiment. [Figure 6] An explanatory diagram regarding setting the flight destination position of an unmanned aerial vehicle in the repair support system of the embodiment. [Figure 7] 4 is a flowchart showing processing by a processing unit of the repair support system according to the embodiment. [Figure 8]1A and 1B are explanatory diagrams showing examples of a reference image and a captured image in a repair support system according to an embodiment; [Figure 9] 10 is a display example of a screen including repair method information of the repair support system according to the embodiment. [Figure 10] 10 is an example of a display screen relating to a comparison between a reference image and a photographed image for each photographing position in the repair support system according to the embodiment. [Figure 11] 10 is an example of a screen display when a captured image including an oil leakage location is displayed as a pop-up in the repair support system according to the embodiment. [Figure 12] 10 is a display example of a screen including repair method information of the repair support system according to the embodiment. [Figure 13] FIG. 10 is a configuration diagram of a repair support system according to a modified example. [Figure 14] FIG. 10 is an explanatory diagram of a repair support system according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0009] <Embodiment> <Configuration of repair support system> FIG. 1 is a configuration diagram of a repair support system R1 according to an embodiment. The repair support system R1 shown in Fig. 1 is a system for supporting the repair of substation equipment. Although not shown, components of the substation equipment to be repaired include transformers and busbars, as well as switchgear, circuit breakers, measuring instruments, and specified auxiliary equipment. As shown in Fig. 1, the repair support system R1 is configured to include a repair support device 100 and an unmanned aerial vehicle D1.

[0010] The repair support device 100 is a device that executes predetermined processing based on images captured by the camera D11 of the unmanned aerial vehicle D1. Such a repair support device 100 may be configured with one computer, or may be configured with multiple computers connected in a predetermined manner. In this embodiment, as an example, a case where the repair support device 100 is used in a so-called stand-alone form will be described. Note that a case where the repair support device is connected to a server via a network will be described in a modified example below.

[0011] The unmanned aerial vehicle D1 is an unmanned aircraft that flies by remote control or predetermined automatic control. For example, a drone is used as such an unmanned aerial vehicle D1. The unmanned aerial vehicle D1 is capable of ascending and descending vertically and diagonally, as well as hovering (stopping in mid-air), moving forward and backward, moving left and right, and rotating left and right. The unmanned aerial vehicle D1 is configured to measure (locate) its own position by receiving radio waves from a positioning satellite (not shown) in the Global Navigation Satellite System, for example.

[0012] As shown in FIG. 1, the unmanned aerial vehicle D1 is equipped with a camera D11. The unmanned aerial vehicle D1 moves (e.g., autonomously flies) along a predetermined inspection route, and sequentially photographs the substation equipment with the camera D11 at multiple photographing positions along the inspection route. Here, "autonomous flight" means that the unmanned aerial vehicle D1 flies autonomously along a predetermined inspection route that has been set in advance. In addition to the inspection route, the photographing position and photographing direction of the camera D11 are also set in advance and stored in a memory (not shown) of the unmanned aerial vehicle D1.

[0013] After receiving a flight start command from a controller (not shown), the unmanned aerial vehicle D1 flies around and inside the substation along a predetermined inspection route, taking photographs at each shooting position in sequence. Note that the images taken by the camera D11 may be still images or may be videos (multiple still images).

[0014] As shown in Fig. 1, the repair support device 100 includes an input unit 10, a storage unit 20, a processing unit 30, and a display unit 40 (display device). The input unit 10 accepts input of images (photographed images and reference images described below) captured by a camera D11 of the unmanned aerial vehicle D1. For example, the photographed images input via the input unit 10 are converted from color images to grayscale, and then linked to predetermined facility information before being stored in a photographed image database 22.

[0015] The storage unit 20 stores a reference image database 21, a photographed image database 22, and a repair method database 23. Although not shown, the hardware configuration of the storage unit 20 includes a hard disk drive (HDD), a random access memory (RAM), and a read-only memory (ROM).

[0016] The reference image database 21 is a database in which multiple reference images are each associated with a specific photographing position. A "reference image" is an image of the substation equipment in a state where damage is relatively minimal, and is associated with each photographing position. Such reference images are acquired by photographing the substation equipment shortly after it starts operating or immediately after repairs. As will be described in more detail later, the reference images are used for comparison with images photographed during inspection.

[0017] The photographed image database 22 is a database in which the degree of damage and the damage level are associated with the photographed image (images taken by the camera D11 of the unmanned aerial vehicle D1) at each photographing position. The degree of damage and the damage level will be described later. The repair method database 23 is a database that includes repair methods and repair work procedures for repairing substation equipment. The repair methods and repair work procedures may be ones that have already been verified, or may be ones that have been newly developed.

[0018] The processing unit 30 acquires a photographed image of the substation facility taken by the camera D11 of the unmanned aerial vehicle D1, and executes a predetermined process based on the photographed image. Although not shown, the hardware configuration of the processing unit 30 includes a processor such as a CPU (Central Processing Unit). The processor reads a predetermined program stored in the ROM (not shown) or hard disk drive (not shown) and loads it into RAM (not shown), thereby executing the predetermined process.

[0019] As shown in FIG. 1, the processing unit 30 includes an image conversion unit 31, a damage degree calculation unit 32, a similar image search unit 33, a repair method information generation unit , and a display control unit . The image conversion unit 31 converts the R, G, and B color images input from the unmanned aerial vehicle D1 via the input unit 10 into grayscale. For example, when a predetermined reference image (color image) is input, the image conversion unit 31 converts this reference image into grayscale and stores the converted reference image in the reference image database 21. Furthermore, when a predetermined captured image (color image) is input from the unmanned aerial vehicle D1 after it has completed flight, the image conversion unit 31 converts this captured image into grayscale and stores the converted captured image in the captured image database 22.

[0020] The damage degree calculation unit 32 refers to the equipment information (substation equipment installation location, equipment control number, and part name) linked to the photographed image, and compares the photographed image with a reference image whose equipment information matches. Then, based on the comparison result between the reference image and the photographed image, the damage degree calculation unit 32 calculates the damage degree and damage level of the part shown in the photographed image. The damage degree and damage level will be described later.

[0021] The similar image search unit 33 searches for the most similar damage image based on the part determined to require repair based on the photographed image and its damage level. Here, a "damage image" is an image showing the state of damage in a specific part of the substation equipment, and is prepared in advance in association with the name of the part and the damage level. For example, an image of a specific damaged part photographed during past repairs is used as such a damage image.

[0022] The repair method information generation unit 34 extracts information on the type of damage, repair method, and repair work procedure linked to the damage image that is most similar to the captured image from the repair method database 23, and generates specified repair method information.

[0023] The display control unit 35 causes the display unit 40 to display data including the repair method information in a predetermined manner. The display unit 40 is a device that displays the processing results of the processing unit 30. For example, a liquid crystal display is used as the display unit 40. Note that a touch panel terminal such as a smartphone or tablet may also be used as the display unit 40.

[0024] FIG. 2 is an explanatory diagram of the repair support system R1. 2, the repair support device 100 (see FIG. 1) is configured with a client terminal 100a and a management terminal 100b. The client terminal 100a and the management terminal 100b can communicate with each other via wired or wireless communication.

[0025] After the unmanned aerial vehicle D1 has finished taking photographs, the data of the captured images is input from the unmanned aerial vehicle D1 to the client terminal 100a. For example, an SD card (not shown) storing the captured image data may be removed from the unmanned aerial vehicle D1, and the SD card may be inserted into a slot (not shown) in the client terminal 100a. Alternatively, the captured image data may be transmitted from the unmanned aerial vehicle D1 in response to a request signal from the client terminal 100a. The captured image data is stored in the captured image database 22 of the client terminal 100a.

[0026] In addition to the photographed image database 22, the above-mentioned reference image database 21 and repair method database 23 are also stored in the client terminal 100a. The management terminal 100b refers to the repair method database 23, extracts the damage image that is most similar to the photographed image, and provides data including the type of damage, repair method, and repair work procedure linked to this damage image to the client terminal 100a. As a result, the data including the type of damage, repair method, and repair work procedure is displayed in a predetermined manner on the client terminal 100a.

[0027] <Inspection of substation equipment> In a substation facility, a gas-insulated switchgear (not shown), for example, is provided in the section from an incoming or outgoing transmission line (not shown) to a transformer (not shown). The gas-insulated switchgear is configured such that a busbar, a switchgear, and auxiliary equipment are housed in a metal container filled with a predetermined insulating gas. Such a gas-insulated switchgear has many narrow spaces. Therefore, even if a maintenance technician attempts to perform an external inspection of the gas-insulated switchgear, it is difficult for the technician to enter such narrow spaces. Furthermore, if ancillary equipment such as a snow-proof roof is provided, the roof frame is also inspected, but it takes time and effort for the maintenance technician to climb onto the roof frame.

[0028] Therefore, in the first embodiment, the unmanned aerial vehicle D1 flies along a predetermined inspection route, and the camera D11 sequentially photographs each part of the substation equipment at the photographing position. This allows the unmanned aerial vehicle D1 to enter a narrow space, for example, in the gas-insulated switchgear, and take photographs. Furthermore, it is also possible to easily inspect ancillary equipment such as a snow-proof roof. For example, the unmanned aerial vehicle D1 may be small, measuring several tens of centimeters in plan view.

[0029] When flying an unmanned aerial vehicle D1 inside a substation building, if it is difficult for the unmanned aerial vehicle D1 to receive a GPS (Global Positioning System) signal, the photographing position should be set as follows: That is, a predetermined reference point that can communicate with the unmanned aerial vehicle D1 should be set inside the building, and the photographing position should be set based on the relative position from that reference point. The image captured by the camera D11 of the unmanned aerial vehicle D1 is compared with a predetermined reference image, which will be explained next.

[0030] FIG. 3 is an explanatory diagram of the reference image database 21. As shown in FIG. As described above, the reference image database 21 shown in Fig. 3 is a database in which each of a plurality of reference images is associated with a predetermined photographing position. As shown in Fig. 3, the reference image database 21 associates a "reference image ID," "photographing date and time," "installation location," "equipment management number," "part," and "photographing position."

[0031] The "reference image ID" is identification information assigned to each of a plurality of reference images. Because the reference image is identified by this reference image ID, the reference image database 21 also includes the reference image itself. The "photographed date and time" shown in FIG. 3 is the date and time when the reference image was photographed. For example, an image photographed when the substation equipment is in a relatively minor state of damage (shortly after it has started operation or immediately after repairs) is used as the reference image.

[0032] The "Installation location" shown in Figure 3 is information indicating the location where the substation equipment being photographed is installed. The "Equipment control number" is an identification number assigned to the substation equipment being photographed. The "Location" is the name of the part of the substation equipment that appears in the reference image. Note that data including the "Installation location," "Equipment control number," and "Location" described above is referred to as "equipment information" of the substation equipment. The "Photographing position" shown in Figure 3 is information indicating the three-dimensional position of the substation equipment when it was photographed.

[0033] FIG. 4 is an explanatory diagram of the photographed image database 22. As shown in FIG. As described above, the captured image database 22 shown in Fig. 4 is a database that includes captured images for each capture position obtained by the camera D11 (see Fig. 1) of the unmanned aerial vehicle D1 (see Fig. 1). As shown in Fig. 4, the captured image database 22 associates a "captured image ID," a "capture date and time," an "installation location," an "equipment management number," a "part," a "capture position," a "damage degree," and a "damage level."

[0034] The "photographed image ID" is identification information assigned to each of a plurality of photographed images. Since the photographed image is identified by this photographed image ID, the photographed image database 22 also includes the photographed image itself. The "photographed date and time" and "installation location" shown in FIG. 4, as well as the "equipment management number," "part," and "photographed position" are the same as those in the reference image database 21 (see FIG. 3), and therefore their explanation will be omitted.

[0035] The "damage level" shown in FIG. 4 is a numerical value indicating the degree of damage to a part shown in a photographed image. In the first embodiment, the "damage level" is calculated as the ratio of the number of pixels in the damaged part (pixels whose brightness values ​​differ from those in the reference image) to the number of pixels in a predetermined area of ​​the photographed image. The "damage level" shown in FIG. 4 indicates the level of damage to a part shown in a photographed image, and is associated with a predetermined numerical range of the damage level. For example, the damage level may be set in five stages, from level 1 to level 5, in order of decreasing degree of damage.

[0036] When the processing unit 30 (see Figure 1) acquires a photographed image from the unmanned aerial vehicle D1 (see Figure 1), it associates the photographed image with the degree of damage and damage level, as well as the photographing location and specified equipment information, thereby generating a photographed image database 22 as shown in Figure 4. As described above, the "equipment information" includes the installation location of the substation equipment, the equipment management number of the substation equipment, and the name of the part of the substation equipment photographed at the specified photographing location.

[0037] FIG. 5 is an explanatory diagram of the repair method database 23. As described above, the repair method database 23 shown in Fig. 5 is a database that includes repair methods and repair work procedures for substation equipment. As shown in Fig. 5, the repair method database 23 associates a "damage image ID," "photograph date and time," "type of damage," "location," "damage level," "repair method," and multiple "work procedures 1, 2, ..." (i.e., repair work procedures).

[0038] The "damage image ID" is identification information assigned to each of a plurality of damage images. Since the damage image is identified by this damage image ID, the repair method database 23 also includes the damage image itself. As mentioned above, the "damage image" is an image obtained by photographing a specific damaged area during past repair work, and is prepared in advance.

[0039] The "photograph date and time" shown in Figure 5 is the date and time when the damage image was photographed. The "type of damage" is the type of damage in a specific part of the substation equipment. This "type of damage" is set, for example, by an inspector's input operation when the substation equipment was previously repaired. In the example of Figure 5, "type of damage" is set to "oil leakage," "foundation defect," or "corrosion." The "part" is the name of the part shown in the damage image. The "damage level" indicates the level of damage in the part shown in the damage image.

[0040] "Repair method" is the name of the method for repairing a specified part. In the example of Figure 5, "repair methods" such as "oil leak sealing," "repair of foundation," and "repair welding after cutting corroded parts" are set. "Work procedure 1," "work procedure 2," etc. shown in Figure 5 indicate a series of repair work procedures associated with a specified repair method. The number immediately following the "work procedure" indicates the order of the work.

[0041] For example, if an "oil leak" has occurred in "XX part" of a substation and the damage level is "Level 3," the first repair method, "oil leak sealing," is performed in the following repair procedure. That is, a maintenance worker prepares a specified "sealing material" and performs "oil removal" (procedure 1) as a material cleaning process, and then performs "scraping work" (procedure 2), "cleaning work" (procedure 3), "oil leak sealing work" (procedure 4), and "post-repair painting" (procedure 5), in this order, and so on, with multiple tasks proceeding in sequence.

[0042] A name of one repair method (for example, "oil leak sealing treatment") is given to each of these series of repair work procedures. That is, in the repair method database 23, each of the multiple damage images is associated with the name and damage level of the part in the substation equipment, as well as the type of damage, the repair method, and the repair work procedure.

[0043] FIG. 6 is an explanatory diagram regarding setting of the flight destination position of the unmanned aerial vehicle D1. The example in Figure 6 shows how the space W1 inside and around transformer V1 is divided into multiple cubes Q1. These cubes Q1 are adjacent to each other in the three-dimensional X, Y, and Z directions. In each cube Q1, the square faces marked with black dots indicate the range of photography per shot by camera D11. In the example in Figure 6, camera D11 takes photos from the front of the squares marked with black dots. Note that the direction of the optical axis of camera D11 does not necessarily have to be in any of the X, Y, or Z directions, and may be oblique.

[0044] As shown in Figure 6, the shooting positions (i.e., the flight destination position of the unmanned aerial vehicle D1) are set so that the shooting range of the camera D11 (the faces of the cube Q1) is adjacent in each of the X, Y, and Z directions. Then, the inspection route is set in advance so that it passes through each shooting position in sequence. This allows the substation equipment to be photographed without omission, preventing oversights in inspection.

[0045] FIG. 7 is a flowchart showing the processing of the processing unit (also see FIG. 1 as appropriate). At the time of "START" in Figure 7, it is assumed that the reference image database 21 and repair method database 23 are already stored in the memory unit 20. The remaining captured image database 22 is generated when the captured images are acquired (S101, S102). Also, at the time of "START" in Figure 7, it is assumed that the unmanned aerial vehicle D1 has flown along a predetermined inspection route and has already completed the process of sequentially capturing images at each capturing position.

[0046] In step S101, the processing unit 30 acquires a captured image from the unmanned aerial vehicle D1 via the input unit 10 (captured image acquisition process). For example, an SD card (not shown) storing the captured image data is removed from the unmanned aerial vehicle D1 and then inserted into a slot (not shown) of the repair support device 100, thereby acquiring the captured image data.

[0047] In step S102, the processing unit 30 associates the captured image with facility information using the input unit 10. That is, when the processing unit 30 acquires the captured image captured by the camera D11, facility information including the installation location of the substation facility being the subject of the image, the facility management number of the substation facility, and the name of a specific part of the substation facility is associated with the captured image. Note that the name of a part of the substation facility may be determined by identifying information of that part.

[0048] In step S103, the processing unit 30 converts the captured color image into a grayscale image using the image conversion unit 31. A predetermined luminance value (e.g., one of 256 luminance values) indicating the shade of grayscale is associated with each pixel of the grayscaled captured image. By performing grayscaling in this manner, the processing unit 30 can speed up the process of extracting damaged areas from the captured image (S104) and extracting damage images similar to the captured image (S108). As described above, the grayscaled captured image is linked to facility information and stored in the storage unit 20 as the captured image database 22.

[0049] In step S104, the processing unit 30 compares the reference image with the captured image and extracts damaged areas from the captured image. First, the processing unit 30 references the equipment information linked to the captured image and reads out from the reference image database 21 a reference image in which the installation location, equipment control number, and part name of the substation equipment match. Then, the processing unit 30 compares the brightness values ​​of each pixel between the reference image and the captured image. That is, the processing unit 30 compares the brightness values ​​of pixels with the same vertical and horizontal coordinate values ​​in the area in which the equipment or component to be inspected is captured between the reference image and the captured image, and extracts pixels with a brightness value difference of a predetermined value or more from the captured image as damaged areas.

[0050] In step S105, the processing unit 30 calculates the damage level of the part shown in the photographed image using the damage level calculation unit 32. For example, the processing unit 30 calculates the damage level as the proportion of the number of pixels of the damaged part to the total number of pixels of the photographed image. Alternatively, the processing unit 30 may extract an area in the photographed image in which the device or component to be inspected is shown, and calculate the damage level as the proportion of the number of pixels of the damaged part in this area. In other words, the processing unit 30 calculates the damage level as the proportion of the number of pixels of the damaged part to the number of pixels in a predetermined area of ​​the photographed image. The "predetermined area" mentioned above may be, for example, the entire photographed image, or may be the area in the photographed image in which the device or component to be inspected is shown.

[0051] In step S106, the processing unit 30 identifies the damage level of the part shown in the photographed image. To give a specific example, if the damage level is less than 20%, the damage level is classified as level 1. If the damage level is 20% or more but less than 40%, the damage level is classified as level 2, and if the damage level is 40% or more but less than 60%, the damage level is classified as level 3. If the damage level is 60% or more but less than 80%, the damage level is classified as level 4, and if the damage level is 80% or more, the damage level is classified as level 5. Data on the damage level and damage level are associated with the photographed image and stored in the photographed image database 22 (see FIG. 4).

[0052] FIG. 8 is an explanatory diagram showing an example of the reference image and the captured image. The image on the left side of Figure 8 is a reference image including the flange connection F1 of the pipe. The reference image shows the flange connection F1 in a state with almost no damage. The image on the right side of Figure 8 is a photograph of the flange connection F1 taken by camera D11 of unmanned aerial vehicle D1 during inspection. The photographing position, photographing range, and resolution of the reference image and the photographed image are assumed to be approximately the same. Furthermore, both the reference image and the photographed image are assumed to have been converted to grayscale.

[0053] In the example of FIG. 8, an oil leak has occurred at the flange connection F1 in the captured image, and pixels in some areas A1 and A2 (near the seam of the flange connection F1) of the captured image are closer to black, and their brightness values ​​are lower than those in the reference image. These areas A1 and A2 are "damaged areas" in the captured image. The processing unit 30 counts the total number of pixels in areas A1 and A2 in the captured image that have brightness values ​​different from those in the reference image, and calculates the percentage of the number of pixels in areas A1 and A2 to the total number of pixels in a predetermined area of ​​the captured image (e.g., an area in the captured image showing the device or component to be inspected) as the damage level (S105 in FIG. 7). Furthermore, the processing unit 30 identifies the damage level based on this damage level (S106 in FIG. 7). This process is performed for each captured image from each shooting position.

[0054] Returning to FIG. 7 again, the explanation will be continued. In step S107, the processing unit 30 determines whether or not the damaged portion of the photographed image at each photographing position needs to be repaired. That is, the processing unit 30 determines whether or not the damaged portion shown in the photographed image needs to be repaired based on the damage level described above. Note that the damage level (e.g., level 3 or higher) that serves as the criterion for determining whether or not repair is needed is set in advance.

[0055] In this way, the processing unit 30 converts all of the captured color images acquired from the unmanned aerial vehicle D1 into grayscale (S103), compares the grayscale captured images with the reference image for each capture position, and extracts, as damaged areas, pixel areas in the captured images that have different brightness values ​​from those in the reference image from areas in which the equipment or component to be inspected is captured (S104).The processing unit 30 then calculates the proportion of pixels in the damaged area to the number of pixels in a predetermined area of ​​the captured image as the damage level (S105), and determines that repair of the damaged area is required if the damage level associated with a predetermined numerical range including the value of the damage level is equal to or greater than the predetermined level (S106, S107).

[0056] In step S108, the processing unit 30 uses the similar image search unit 33 to identify damage images that are similar to the photographed image requiring repair (i.e., a photographed image showing a part requiring repair) (damage image identification process). Explaining this in more detail, the processing unit 30 first identifies the name and damage level of the part of the substation equipment shown in the photographed image requiring repair based on the photographed image database 22 (see FIG. 4). Next, the processing unit 30 uses the name and damage level of this part as search conditions to identify the damage image from the repair method database 23 (see FIG. 5). That is, the processing unit 30 extracts damage images from the repair method database 23 whose part names and damage levels match those in the photographed image.

[0057] Although the search criteria described above may narrow down the damage image to one, in some cases, there may be multiple damage images that match the search criteria. Taking this into consideration, the following processing may be performed. Specifically, when a damaged portion captured in a photographed image needs to be repaired, the processing unit 30 extracts, from the multiple damage images, those that meet the search criteria, using the name of the portion containing the damaged portion and the damage level of the damaged portion as search criteria. Then, the processing unit 30 identifies, from the damage images that meet the search criteria, the one that is most similar to the photographed image based on pattern matching or the like (S108). This allows the identification of a damage image that meets the predetermined search criteria and is most similar to the photographed image.

[0058] In step S109, the processing unit 30 reads the repair method and the like linked to the damage image from the repair method database 23 (see FIG. 5) by the repair method information generation unit 34. As described above, in the repair method database 23, data on the type of damage, repair method, and repair work procedure (work procedures 1, 2, ... in FIG. 5) are associated with a predetermined damage image. In step S109, the processing unit 30 reads the type of damage, repair method, and repair work procedure as data corresponding to the damage image identified in step S108.

[0059] In step S110, the processing unit 30 generates repair method information using the repair method information generation unit 34. Specifically, the processing unit 30 generates repair method information by applying the type of damage, repair method, repair work procedure, photographed image of the area requiring repair, name of the area, damage level, and photographed position to a predetermined format (template). Of the above-mentioned data, the photographed image of the area requiring repair and the photographed position are identified from the photographed image database 22 (see FIG. 4).

[0060] In step S111, the processing unit 30 causes the display control unit 35 to display the repair method information in a predetermined manner on the display unit 40 (display processing). After performing the processing of step S111, the processing unit 30 ends the series of processing (END).

[0061] In this way, the processing unit 30 acquires a photographed image from the unmanned aerial vehicle D1 (photographed image acquisition process: S101), compares a predetermined reference image with the photographed image for each photographed position, and extracts a damaged portion of the substation equipment from the photographed image (damaged portion extraction process: S104). If the damaged portion needs to be repaired, the processing unit 30 identifies a damage image similar to the photographed image from among multiple damage images prepared in advance (damage image identification process: S108), and displays the type of damage, repair method, and repair work procedure corresponding to the damage image on the display unit 40 (display device) in association with the photographed image having the damaged portion (display process: S111).

[0062] FIG. 9 is an example of a screen display including repair method information. In the example of Figure 9, a photographed image E1 of a part requiring repair is displayed in the upper left of the screen. The photographed image ID, the name of the part, the photographed location, the type of damage, and the repair method are displayed below the photographed image E1, in that order. In the example of Figure 9, the type of damage is "oil leakage," and the repair method is "oil leakage sealing." The repair work procedure for this "oil leakage sealing" is displayed on the right side of the screen. Specifically, after "oil removal," a series of repair work procedures, such as "scraping work," "cleaning work," "oil leakage sealing work," and "painting after repair" are performed in that order, are displayed in the form of a flowchart. By viewing this screen, maintenance personnel can easily understand the type of damage to the part shown in the photographed image E1, as well as the repair method and repair work procedure.

[0063] When the maintenance worker selects one of the repair work procedures (for example, "cleaning work") by inputting an input operation, a predetermined explanatory diagram or video showing the repair work procedure may be displayed. Also, when there are multiple damaged areas, for example, data for each area may be displayed in list form, and when the maintenance worker selects one of them, a screen like that shown in Figure 9 may be displayed.

[0064] FIG. 10 shows an example of a display screen for comparing a reference image and a photographed image for each photographing position. In the example of Figure 10, a reference image (e.g., an image of a brand new state) of a primary bushing 51, which is a component of a substation, is displayed on the left side of the screen. To explain in more detail, an unmanned aerial vehicle D1 (see Figure 1) is moved sequentially from the bottom to the top of the primary bushing 51, and the images taken at each of the photographing positions A, B, C, and D are stitched together vertically and displayed as the "reference image."

[0065] The same is true for the "photographed image" (image taken during inspection) on the right side of the screen. In this way, the processing unit 30 displays a plurality of reference images with adjacent photographing ranges side by side on one screen, and also displays a plurality of photographed images with adjacent photographing ranges side by side on one screen. Furthermore, one device or one component of the substation equipment (primary bushing 51 in the example of FIG. 10) is displayed across a plurality of photographed images lined up on one screen. A legend G1 (color bar) showing five levels of damage is displayed below the photographed image.

[0066] As indicated by the horizontal white arrows, the processing unit 30 (see FIG. 1) compares the reference image with the captured image at each of the image capturing positions A, B, C, and D. In the example of FIG. 10, oil leakage has occurred at the bottom of the primary bushing 51, as shown as oil leakage point 51a in the image captured at image capturing position A.

[0067] For example, suppose the damage level of the oil leakage point 51a in the image captured at photographing position A is level 3. In this case, the processing unit 30 (see FIG. 1) displays the background of the primary bushing 51 in the image captured at photographing position A in a predetermined color or pattern (dots in FIG. 10) corresponding to level 3. At the remaining photographing positions B, C, and D, the damage level is level 1, so the background of the primary bushing 51 in the photographed images is white.

[0068] In this way, the processing unit 30 displays each captured image in a predetermined display mode corresponding to the level of damage. This allows the maintenance personnel to visually grasp the damage level of the primary bushing 51 at the image capture positions A, B, C, and D. Furthermore, by displaying the entire primary bushing 51, the maintenance personnel can easily grasp the extent of damage in each part. Note that the display mode indicating the damage level is not limited to the background color or pattern of the primary bushing 51. For example, display modes such as surrounding the edge of the captured image with a frame line of a predetermined color or displaying a predetermined mark next to the captured image may also be used.

[0069] In addition, when the threshold for the damage level requiring repair is level 3, the background of the primary bushing 51 may be displayed in a predetermined color or pattern (e.g., white) for images taken at levels 1 and 2, which are subject to follow-up observation.

[0070] On the other hand, for images requiring repair with a damage level of Level 3 or higher, the background of the primary bushing 51 may be displayed in a different color or pattern. That is, among the multiple images displayed side by side on the screen, the processing unit 30 (see FIG. 1) displays, in different display modes, images with a damage level of a predetermined level or higher (in the example of FIG. 10, the image taken at photographing position A) and images with a damage level below the predetermined level (in the example of FIG. 10, the images taken at photographing positions B, C, and D). The aforementioned "predetermined level" is a damage level threshold (e.g., Level 3) that serves as a criterion for determining whether repair is required, and is set in advance. By changing the display mode according to whether repair is required, the maintenance technician can at a glance determine which image contains the area requiring repair.

[0071] As shown by icon 1a in FIG. 10, when the image captured at photographing position A is selected by an input operation by the maintenance worker, an enlarged image such as that shown in FIG. 11 is displayed as a pop-up on the display unit 40 (see FIG. 1).

[0072] FIG. 11 shows an example of a screen display in which a photographed image including an oil leakage location 51a is displayed as a pop-up. When one of the multiple captured images (images captured at capture positions A, B, C, and D in Figure 10) displayed side by side on one screen is selected by a user (maintenance worker, etc.) input operation, the processing unit 30 displays the captured image in a pop-up display and also displays the degree of damage and damage level of the part shown in the captured image in association with the captured image.

[0073] In the example of Figure 11, the damage degree and damage level values ​​for the oil leakage point 51a (i.e., the damaged area) shown in the captured image are displayed in the upper left of the screen. In addition, a link to "Display repair method information" is displayed in the lower right of the screen. This allows the maintenance personnel to grasp at a glance the damage degree and damage level of the area shown in the captured image. As shown in icon 2a, when the maintenance personnel selects the link to "Display repair method information," a screen containing repair method information such as that shown in Figure 12 below is displayed.

[0074] FIG. 12 is an example of a screen display including repair method information. In the example of FIG. 12, a captured image E2 of a part requiring repair is displayed in the upper left of the screen. The captured image ID, the name of the part, the photographed position, the type of damage, and the repair method are displayed in that order below the captured image E2. The repair procedure is displayed in the form of a flowchart on the right side of the screen. In this way, the processing unit 30 displays the captured image, the name of the part shown in the captured image, the type of damage in the part, the repair method, and the repair procedure on a single screen. This allows the maintenance worker to easily understand what type of damage exists in which part and what procedure should be used to perform the repair.

[0075] <Effects> According to this embodiment, an unmanned aerial vehicle D1 (see FIG. 1) flies along a predetermined inspection route, taking photographs sequentially at multiple preset photographing positions. This makes it easy to photograph narrow spaces inside substation equipment that are difficult for maintenance personnel to enter. This also makes it possible to improve efficiency and reduce the number of personnel required when inspecting substation equipment, as well as significantly reduce the workload of maintenance personnel.

[0076] Furthermore, the degree of damage is calculated based on a comparison between a predetermined reference image and the photographed image, and the necessity of repair is determined based on the damage level corresponding to the degree of damage. Because this process is performed for each photographed position, it is possible to prevent omissions (oversights) when determining the necessity of repair.

[0077] In this embodiment, the processing unit 30 converts the color reference image and the photographed image into grayscale, and then compares the reference image with the damaged image. This reduces the amount of calculation required for the processing unit 30 to identify the damaged area, thereby speeding up the processing.

[0078] In conventional technology, the maintenance personnel select and determine the repair method and repair work procedure. In this case, an inexperienced maintenance personnel and an experienced maintenance personnel may select different repair methods, etc., even for the same damage. In contrast, in this embodiment, the processing unit 30 identifies the repair method and repair work procedure based on the location and level of the damage. Therefore, an appropriate repair method, etc. can be presented regardless of the maintenance personnel's level of proficiency (i.e., repair work can be leveled out). In this way, this embodiment can provide a repair support system R1, etc., that provides appropriate and efficient repair support for substation equipment.

[0079] <<Variations>> The repair support system R1 and the repair support method according to the present disclosure have been described above in the embodiments, but the present disclosure is not limited to these descriptions and various modifications can be made. For example, in the embodiment, the repair support device 100 (see FIG. 1) is used in a stand-alone form, but the present invention is not limited to this. That is, as shown in FIG. 13, the repair support device 100A may be connected to a server 70 via a network N1.

[0080] FIG. 13 is a configuration diagram of a repair support system RA1 according to a modified example. The repair support system RA1 shown in Figure 13 includes a repair support device 100A, a server 70, and an unmanned aerial vehicle D1. The repair support device 100A includes an input unit 10, a memory unit 20A, a processing unit 30, a display unit 40, and a communication unit 60. The memory unit 20A stores a reference image database 21 and a captured image database 22. The communication unit 60 performs predetermined data communication with the server 70 via a network N1.

[0081] The server 70 is configured to include a repair method database 71 and a processor 72. The repair method database 71 includes data similar to that of the repair method database 23 (see FIG. 5) described in the embodiment. When the processor 72 receives a request signal for repair method information from the repair support device 100A, the processor 72 reads predetermined repair method information from the repair method database 23 in response to the request signal, and transmits the repair method information to the repair support device 100A via the network N1.

[0082] 13 have the same configuration as the repair support device 100A. For example, tablets carried by maintenance personnel of a predetermined substation facility may be used as these client terminals PC1, PC2, and PC3. Data in the repair method database 71 is shared between the repair support device 100A and the client terminals PC1, PC2, and PC3.

[0083] FIG. 14 is an explanatory diagram of a repair support system RA1 according to a modified example. The repair support system RA1 shown in FIG. 14 corresponds to the configuration shown in FIG. A management terminal 80 shown in Fig. 14 (not shown in Fig. 13) is a terminal that provides predetermined data to the server 70, and is connected to the server 70 via a network (not shown). The management terminal 80 provides the server 70 in advance with data such as predetermined damage images and locations (names of damaged locations), as well as damage levels, repair methods, and repair work procedures. These data are stored in the server 70 as a repair method database 71.

[0084] As shown in Figure 14, data on captured images is input from the unmanned aerial vehicle D1 to the repair support device 100A. The repair support device 100A transmits data including the captured images as well as the names of the parts and the damage levels to the server 70. The server 70, having received this data, extracts multiple damage images that match the conditions of the damage part and damage level, and identifies the one from these damage images that is most similar to the captured image. The server 70 then transmits data on the type of damage, repair method, and repair work procedure corresponding to this damage image to the repair support device 100A.

[0085] As a result, the display unit 40 of the repair support device 100A (see FIG. 13) displays predetermined data including the type of damage, the damaged location, the repair method, and the repair work procedure in addition to the captured image. This allows a maintenance worker looking at the screen of the display unit 40 to easily understand the procedure for repairing the substation equipment. Furthermore, because data updates for the repair method and repair work procedure are performed collectively by the server 70, there is no need to update the data for the repair method and repair work procedure in the repair support device 100A or the client terminals PC1, PC2, and PC3 (see FIG. 13).

[0086] In the embodiment, when a part shown in a photographed image needs to be repaired, the processing unit 30 extracts from a plurality of damage images those that meet the search conditions, using the part and damage level as search conditions, and identifies the damage image that is most similar to the photographed image from among the damage images. However, the present invention is not limited to this. For example, the processing unit 30 may identify the damage image that is most similar to the photographed image from among the plurality of damage images without setting specific search conditions for the part and damage level.

[0087] In addition, in the embodiments, the case where the object of inspection and repair is a substation facility has been described, but the present invention is not limited to this. For example, the embodiments can be applied to the inspection and repair of power transmission facilities, power distribution facilities, solar panels, wind turbines for wind power generation, communication facilities, air conditioning facilities, refrigeration facilities, bridges, power generation plants, manufacturing plants, water treatment plants, and chemical plants.

[0088] Furthermore, the processes (such as processes in the repair support method) executed by the repair support device 100, 100A may be executed as a predetermined program by a computer. The program may be provided via a communication line, or may be written to a recording medium such as a CD-ROM and distributed.

[0089] Furthermore, the present disclosure is not limited to the embodiments and includes various modifications. For example, the embodiments have been described in detail to clearly explain the present disclosure, and the present disclosure is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to add, delete, or replace part of the configuration of the embodiments with other configurations.

[0090] Furthermore, the above-mentioned configurations, functions, processing units, processing means, etc. may be partly or entirely implemented in hardware, for example, by designing them as integrated circuits. Furthermore, the above-mentioned configurations, functions, etc. may be implemented in software, with a processor interpreting and executing a program that implements each function. Information such as the programs, tables, and files that implement each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0091] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0092] 10 Input section 20,20A storage section 21 Reference Image Database 22 Photo database 23,71 Repair Method Database 30 Processing section 31 Image conversion unit 32 Damage degree calculation section 33 Similar Image Search Unit 34 Repair method information generation unit 35 Display control unit 40 Display section (display device) 60 Communications Department 70 servers 80 Management terminal 100,100A repair support device D1 Unmanned Aerial Vehicle D11 Camera Repair support system R1, RA1 S101 Step (Image Acquisition Processing) Step S104 (Damaged part extraction process) Step S108 (damage image identification processing) S111 step (display processing)

Claims

1. a processing unit for acquiring a photographed image of the substation facility by a camera of the unmanned aerial vehicle; The unmanned aerial vehicle moves along a predetermined inspection route, and photographs are taken sequentially by the camera at a plurality of photographing positions on the inspection route; The processing unit comparing a predetermined reference image with the photographed image for each of the photographing positions, and extracting a damaged portion in a portion of the substation equipment from the photographed image; If the damaged area needs to be repaired, the repair support system identifies a damage image similar to the photographed image from among multiple damage images prepared in advance, and links the type of damage, repair method, and repair work procedure corresponding to the damage image to the photographed image containing the damaged area and displays them on a display device.

2. The processing unit Converting all of the captured images into grayscale; comparing the grayscale photographed image with the reference image for each photographing position, and extracting, as the damaged portion, a region of pixels having a brightness value different from that of the reference image from a region in which the device or component to be inspected is photographed in the photographed image; calculating a ratio of the number of pixels occupied by the damaged portion to the number of pixels in a predetermined region of the photographed image as a degree of damage; determining that the damaged portion needs to be repaired when a damage level associated with a predetermined numerical range including the value of the damage degree is equal to or greater than a predetermined level; The repair support system according to claim 1 ,

3. The processing unit Calculating the percentage of pixels occupied by the damaged part in the area of ​​the photographed image in which the device or component to be inspected is captured as the degree of damage. The repair support system according to claim 2,

4. When the processing unit acquires the photographed image from the unmanned aerial vehicle, the processing unit associates the degree of damage and the damage level with the photographed image, and also associates the photographed position with predetermined facility information, thereby generating a photographed image database; The facility information includes the installation location of the substation facility, the facility management number of the substation facility, and the name of the part of the substation facility to be photographed at the photography position. The repair support system according to claim 2,

5. a reference image database in which each of the plurality of reference images is associated with a predetermined one of the photographing positions; a repair method database in which each of the plurality of damage images is associated with a name and a damage level of a portion of the substation equipment, and is associated with the type of damage, the repair method, and the repair work procedure. The repair support system according to claim 1 ,

6. When the damaged portion needs to be repaired, the processing unit extracts images of the damage that meet the search criteria, using the name of the portion including the damaged portion and the damage level of the damaged portion, from among the plurality of damage images, and further identifies the image that is most similar to the photographed image from among the damage images that meet the search criteria. The repair support system according to claim 1 ,

7. the processing unit displays the plurality of captured images having adjacent shooting ranges side by side on one screen, One device or one component in the substation facility is displayed across a plurality of the photographed images arranged on one screen, The processing unit displays each of the captured images in a predetermined display mode corresponding to the level of the damage level. The repair support system according to claim 2,

8. the processing unit displays the plurality of captured images having adjacent shooting ranges side by side on one screen, One device or one component in the substation facility is displayed across a plurality of the photographed images arranged on one screen, The processing unit displays, among the plurality of photographed images, photographed images whose damage level is equal to or greater than the predetermined level and photographed images whose damage level is less than the predetermined level in different display modes. The repair support system according to claim 2,

9. the processing unit displays the plurality of captured images having adjacent shooting ranges side by side on one screen, One device or one component in the substation facility is displayed across a plurality of the photographed images arranged on one screen, When one of the plurality of photographed images is selected by a user's input operation, the processing unit displays the photographed image in a pop-up display, and also displays the damage degree and the damage level in association with the photographed image. The repair support system according to claim 2,

10. The processing unit displays the photographed image, the name of the part shown in the photographed image, the type of damage, the repair method, and the repair work procedure on one screen. The repair support system according to claim 1 ,

11. an image acquisition process for acquiring images from an unmanned aerial vehicle that moves along a predetermined inspection route and sequentially photographs the substation equipment with a camera at a plurality of photographing positions on the inspection route; a damaged portion extraction process for comparing a predetermined reference image with the photographed image for each of the photographing positions and extracting a damaged portion in a portion of the substation equipment from the photographed image; a damage image identification process for identifying a damage image similar to the photographed image from among a plurality of damage images prepared in advance when the damaged portion needs to be repaired; A repair support method in which a processing unit sequentially executes the following steps: a display process in which the type of damage, repair method, and repair work procedure corresponding to the damage image are linked to the captured image having the damaged area and displayed on a display device.

Images