Diagnostic system, diagnostic method and diagnostic program

Abstract

To enhance accuracy of a diagnosis for determining presence / absence of damage using an image obtained by photographing a building.SOLUTION: A diagnostic device (1) comprises: a division unit (104) which divides a photographic image (111) of a wind power generation facility to be diagnosed to generate a plurality of divided images; and a damage detection unit (107) which detects damage of the wind power generation facility to be diagnosed, based upon reconstruction errors of when the respective divided images are reconstituted by using a reconstitution model (113).SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a diagnostic device for diagnosing whether or not a structure is damaged. [Background technology]

[0002] It is preferable to periodically diagnose various structures for damage after construction. One example of a technology for such diagnosis is the wind power generation facility monitoring system described in Patent Document 1 below. In this monitoring system, a drone is controlled to avoid collision with the blades of the wind power generation facility while the drone photographs the blades. The monitoring system then compares the image information obtained by photographing the blades with normal image information prepared in advance to diagnose the presence or absence of abnormalities in the wind power generation facility. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-176821 Summary of the Invention [Problem to be solved by the invention]

[0004] When conducting inspections using drones, the cost of introducing and maintaining the drones is high. In addition, because wind power generation facilities are often installed in windy locations, there is a risk that the drone will collide with the blades due to the influence of wind, even if the drone is properly controlled.

[0005] For this reason, it would be desirable to be able to diagnose the presence or absence of damage using images of wind power generation facilities taken from, for example, the ground, without using a drone. However, when taking images from the ground, the shooting distance is significantly greater than when using a drone, and the method of comparing image information obtained by taking images with normal image information in Patent Document 1 makes it difficult to perform a diagnosis with sufficient accuracy. If the accuracy of diagnosis using images could be improved, diagnosis using images taken from the ground would also become possible, which would be desirable, but Patent Document 1 does not describe a method for improving the accuracy of diagnosis. This problem is not limited to wind power generation facilities, but is a problem that occurs commonly in various types of buildings.

[0006] An object of one aspect of the present invention is to provide a diagnostic device or the like that can highly accurately diagnose the presence or absence of damage using images of a building. [Means for solving the problem]

[0007] In order to solve the above problems, a diagnostic device according to one aspect of the present invention includes a division unit that divides an image of a structure to be diagnosed for damage by a predetermined division unit to generate a plurality of divided images, and a damage detection unit that detects damage to the structure to be diagnosed based on the reconstruction error when each of the divided images is reconstructed using a reconstruction model that has learned the characteristics of images of undamaged structures.

[0008] In addition, in order to solve the above-mentioned problems, a diagnostic device according to another aspect of the present invention includes a target area detection unit that detects a target area for diagnosis in a captured image of a building that is the target of diagnosis for the presence or absence of damage, using a detection model generated by machine learning to detect a target area for diagnosis in the building from an image of the building, and a damage detection unit that detects damage to the building that is the target of diagnosis based on a reconstruction error when a portion of the target area in the captured image is reconstructed using a reconstruction model that has learned the features of an image of an undamaged building.

[0009] In addition, in order to solve the above-mentioned problems, a diagnostic method according to one aspect of the present invention is a diagnostic method executed by at least one information processing device, and includes a division step of dividing a captured image of a building to be diagnosed for the presence or absence of damage into a predetermined division unit to generate a plurality of divided images, and a damage detection step of detecting damage to the building to be diagnosed based on a reconstruction error when each of the divided images is reconstructed using a reconstruction model that has learned the characteristics of images of undamaged buildings.

[0010] In addition, in order to solve the above-mentioned problems, a diagnostic method according to another aspect of the present invention is a diagnostic method executed by at least one information processing device, and includes a target area detection step of detecting a target area for diagnosis in a captured image of a building to be diagnosed for the presence or absence of damage, using a detection model generated by machine learning to detect a target area for diagnosis in the building from an image of the building, and a damage detection step of detecting damage to the building to be diagnosed based on a reconstruction error when a portion of the target area in the captured image is reconstructed using a reconstruction model that has learned the features of an image of an undamaged building. [Effects of the Invention]

[0011] According to one aspect of the present invention, it is possible to improve the accuracy of diagnosing the presence or absence of damage using images of a building. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing an example of a configuration of a main part of a diagnostic device according to an embodiment of the present invention; [Figure 2] 4 is a flowchart illustrating an example of a diagnostic method executed by the diagnostic device. [Figure 3] FIG. 10 is a diagram showing an example of a diagnostic target area detected from a captured image. [Figure 4] 4 is a diagram showing an example of divided images generated from the photographed image shown in FIG. 3. FIG. [Figure 5] FIG. 5 is a diagram showing an example of an error image generated from the divided images shown in FIG. [Figure 6] FIG. 6 is a diagram showing an example of a binarized image generated from the error image shown in FIG. 5. [Figure 7] FIG. 10 is a diagram showing an example of a display of damage detection results. DETAILED DESCRIPTION OF THE INVENTION

[0013] [Configuration of diagnostic device] The configuration of the diagnostic device 1 will be described with reference to FIG. 1. FIG. 1 is a block diagram showing an example of the configuration of the main components of the diagnostic device 1. The diagnostic device 1 is a device for detecting damage to a structure. An example of detecting damage to a wind power generation facility using the diagnostic device 1 will be described below. A wind power generation facility generates electricity using wind power and includes blades that rotate when exposed to the wind, supports that support the blades, and a generator. In a wind power generation facility, power is generated by converting the rotational motion of the blades into electricity using a generator. The diagnostic device 1 detects damage that appears on the exterior of such a wind power generation facility. Note that damage detection targets are not limited to wind power generation facilities, and the diagnostic device 1 can be used to detect damage to any structure. Therefore, the term "wind power generation facility" in the following description can be replaced with any "structure." For example, the diagnostic device 1 can also be used to detect damage to a bridge. The diagnostic device 1 can also be used to detect damage to a part of a structure, such as a chimney.

[0014] 1, diagnostic device 1 includes a control unit 10 that controls all the components of diagnostic device 1, and a storage unit 11 that stores various data used by diagnostic device 1. Diagnostic device 1 also includes a communication unit 12 that enables diagnostic device 1 to communicate with other devices, an input unit 13 that accepts input to diagnostic device 1, and an output unit 14 that enables diagnostic device 1 to output various data.

[0015] The control unit 10 includes a data acquisition unit 101, a defective area detection unit 102, a target area detection unit 103, a division unit 104, a reconstructed image generation unit 105, an error image generation unit 106, a damage detection unit 107, and a display control unit 108. The memory unit 11 stores a captured image 111, a detection model 112, and a reconstruction model 113.

[0016] The data acquisition unit 101 acquires captured images 111 of the wind power generation facility to be diagnosed for damage. While Fig. 1 shows an example in which the data acquisition unit 101 acquires the captured images 111 stored in the storage unit 11, the method and source of acquiring the captured images 111 are not particularly limited. For example, the data acquisition unit 101 may acquire the captured images 111 from a camera that captures the captured images 111 via the communication unit 12 or the input unit 13.

[0017] The defective area detection unit 102 detects a defective area where the wind power generation facility is not captured normally in the captured image 111. When the defective area detection unit 102 detects a defective area, the damage detection unit 107, which will be described later, detects damage in the remaining areas excluding the defective area.

[0018] Here, the wind power generation facility is an outdoor facility. Therefore, lens flare may occur depending on the way external light hits the facility during shooting. Furthermore, due to the influence of water droplets or the like on the surface of the wind power generation facility, an image may be captured in which it is difficult to determine whether or not there is damage. For this reason, it is preferable to provide the diagnostic device 1 with a defective area detection unit 102, which detects defective areas where the wind power generation facility is not photographed normally, and then detects damage in areas other than the defective areas. This makes it possible to prevent erroneous detection of damage in defective areas.

[0019] The target area detection unit 103 detects the target area for diagnosis in the captured image 111 using a detection model 112 generated by machine learning to detect the target area for diagnosis from an image of the wind power generation facility. When the target area detection unit 103 detects the target area, the damage detection unit 107, which will be described later, detects damage in the target area. This reduces the possibility that areas other than the target area for diagnosis will affect the diagnosis result. For example, the above configuration also reduces the possibility that an area with a large reconstruction error in the background area of ​​the wind power generation facility will be erroneously detected as a damaged area.

[0020] It is arbitrary to determine the target area as an area in which a component of the wind power generation facility is captured. For example, the target area detection unit 103 may detect, as the target area, an area in the captured image 111 in which the blades of the wind power generation facility are captured. In this case, a model generated by machine learning to detect the area in which the blades are captured may be used as the detection model 112.

[0021] The detection model 112 may be a semantic segmentation (area division) model that classifies each pixel of the captured image 111 according to the object captured in that pixel. An example of such a model is U-net. By using such a model as the detection model 112, the image area of ​​the captured image 111 can be divided into three types: an area capturing a blade, an area capturing a pole, and a background area of ​​the wind power generation facility. In this case, the target area detection unit 103 can detect the area capturing a blade, or both the area capturing a blade and the area capturing a pole, as the target area for diagnosis.

[0022] Furthermore, the target area detection unit 103 can also detect an outline area in which the outline of the wind power generation facility is captured from the captured image 111 based on the detection result of the detection model 112. The target area detection unit 103 that performs such detection can be said to function as an outline detection unit. When the target area detection unit 103 detects an outline area, the damage detection unit 107 (described later) detects damage in areas other than the outline area.

[0023] Experiments conducted by the inventors of the present application have confirmed that the reconstruction error, which will be described later, tends to be large in the contour portions of wind power generation facilities, even if there is no damage in those portions. Although detecting the contour region is not essential, detecting the contour region and then detecting damage in the regions other than the detected contour region makes it possible to prevent erroneous diagnosis of damage in undamaged contour portions.

[0024] Furthermore, when diagnosing the presence or absence of damage to the blades, the target area detection unit 103 may detect a pillar area in which the pillar of the wind power generation facility is captured from the captured image 111. The target area detection unit 103 that performs such detection can be said to function as a pillar detection unit. When the target area detection unit 103 detects a pillar area, the damage detection unit 107 (described later) detects damage in areas other than the area detected by the target area detection unit 103. Note that the wind power generation facility includes, in addition to the blades that rotate in response to wind, a nacelle that houses a generator and the like, and pillars that support the nacelle and the blades. The target area detection unit 103 only needs to detect an area that includes at least the pillar among these components of the wind power generation facility. The area may include only the area in which the pillar is captured, or may include an area in which the nacelle is captured in addition to the area in which the pillar is captured.

[0025] When a blade is photographed from the ground, a support pillar or nacelle is usually visible in the background. The above configuration allows the presence or absence of damage to be diagnosed while eliminating the influence of the support pillar or other object visible in the image, making it possible to diagnose the presence or absence of damage to the blade with high accuracy. As mentioned above, the support pillar area may also be included in the target area for diagnosis.

[0026] The dividing unit 104 divides the captured image 111 of the wind power generation facility to be diagnosed for damage into a predetermined division unit to generate a plurality of divided images. The division unit may be determined in advance depending on the required accuracy of diagnosis, etc.

[0027] The reconstructed image generation unit 105 generates a reconstructed image by reconstructing the segmented image generated by the segmentation unit 104 using the reconstruction model 113. The reconstruction model 113 is a model that has learned the characteristics of images of undamaged wind power generation equipment. When a segmented image showing a damaged portion is reconstructed using the reconstruction model 113, the reconstruction error of the damaged portion in the generated reconstructed image will be larger than that of other image regions. This is because the reconstruction model 113 has not learned about the damaged portion and therefore cannot properly reconstruct the damaged portion.

[0028] The reconstruction model 113 may be any model that receives a segmented image as input data and outputs a reconstructed image that reproduces the features of the segmented image. For example, an autoencoder (AE), a convolutional autoencoder (CAE), or a variational autoencoder (VAE) may be applied as the reconstruction model 113.

[0029] Here, a reconstruction model 113 may be generated for each component that makes up the wind power generation facility. In this case, the reconstruction image generation unit 105 generates a reconstructed image using one of the multiple reconstruction models 113, a reconstruction model 113 that corresponds to the component that appears in the segmented image. As a result, the damage detection unit 107, which will be described later, detects damage to each component based on the reconstruction error when reconstructing a segmented image that shows the component using the reconstruction model 113 generated for each component that makes up the wind power generation facility. With this configuration, it is possible to calculate the reconstruction error for each component with high accuracy and diagnose the presence or absence of damage to each component with high accuracy.

[0030] For example, a reconstruction model 113 for reconstructing a segmented image in which a blade appears, and a reconstruction model 113 for reconstructing a segmented image in which a support pole appears may be stored in advance in the storage unit 11, etc. In this case, the reconstructed image generation unit 105 reconstructs the segmented image in which a blade appears using the reconstruction model 113 for reconstructing the segmented image in which a blade appears. On the other hand, the reconstructed image generation unit 105 reconstructs the segmented image in which a support pole appears using the reconstruction model 113 for reconstructing the segmented image in which a support pole appears. This makes it possible to diagnose the presence or absence of damage to each of the blade and the support pole with high accuracy.

[0031] The error image generation unit 106 generates a plurality of partial error images that respectively indicate the reconstruction errors of the plurality of segmented images. The error image generation unit 106 also concatenates the generated partial error images to generate an error image that indicates the distribution of the reconstruction errors in the captured image 111. When the error image generation unit 106 generates an error image, the damage detection unit 107 (described later) uses the error image to detect damage to the wind power generation facility to be diagnosed. This makes it possible to correctly detect damage that spans multiple segmented images as a single piece of damage.

[0032] Damage detection unit 107 detects damage to the wind power generation equipment to be diagnosed based on the reconstruction error when each of the segmented images is reconstructed using reconstruction model 113 that has learned the features of images of undamaged wind power generation equipment. More specifically, damage detection unit 107 detects, based on the distribution of reconstruction errors shown in the error image generated by error image generation unit 106, an area consisting of pixels where the reconstruction error is equal to or greater than a threshold as a damaged area.

[0033] The display control unit 108 causes a display device to display the damage detection results obtained by the damage detection unit 107. The display device may be provided in the diagnostic device 1 or may be a device external to the diagnostic device 1. The manner in which the diagnostic results are displayed is not particularly limited. For example, the display control unit 108 may display the diagnostic results by superimposing information indicating the damaged area detected by the damage detection unit 107 on the captured image 111.

[0034] As described above, the diagnostic device 1 includes a division unit 104 that divides a captured image 111 of the wind power generation equipment to be diagnosed for damage in predetermined division units to generate a plurality of divided images, and a damage detection unit 107 that detects damage to the wind power generation equipment to be diagnosed based on the reconstruction error when each of the divided images is reconstructed using a reconstruction model 113 that has learned the features of images of undamaged wind power generation equipment.

[0035] According to the above configuration, damage is detected based on the reconstruction error of the reconstruction model 113. This reconstruction model 113 is one that has learned the features of images of undamaged wind power generation facilities. Therefore, for images that show damaged parts that have not been learned, the reconstruction error of the damaged parts will be large, making it possible to detect damage based on the reconstruction error.

[0036] Furthermore, with the above configuration, the captured image 111 is divided into predetermined division units to generate a plurality of divided images, and each of the generated divided images is reconstructed using the reconstruction model 113, so damage detection can be performed with higher accuracy than when the entire captured image 111 is reconstructed. Therefore, with the above configuration, it is possible to diagnose the presence or absence of damage using captured images of wind power generation equipment with higher accuracy.

[0037] As described above, the diagnostic device 1 is equipped with a target area detection unit 103 that detects a target area for diagnosis in a captured image 111 of the wind power generation equipment to be diagnosed for damage using a detection model 112 generated by machine learning to detect a target area for diagnosis in the wind power generation equipment from an image of the wind power generation equipment, and a damage detection unit 107 that detects damage to the wind power generation equipment to be diagnosed based on a reconstruction error when a portion of the target area in the captured image 111 is reconstructed using a reconstruction model 113 that has learned the features of an image of undamaged wind power generation equipment.

[0038] The above configuration reduces the possibility that areas other than the target area for diagnosis will affect the diagnosis results, thereby enabling more accurate diagnosis of the presence or absence of damage using images of the wind power generation facility.

[0039] [Diagnostic method flow] The flow of the diagnostic method executed by the diagnostic device 1 will be described with reference to FIG. 2. FIG. 2 is a flowchart showing an example of the diagnostic method executed by the diagnostic device 1. FIG. 2 also shows processing after an image obtained by photographing the wind power generation facility to be diagnosed is input to the diagnostic device 1 and the image is stored in the storage unit 11 as a photographed image 111. In the following, specific examples will be described with reference to FIGS. 3 to 7 as appropriate. FIG. 3 is a diagram showing an example in which a target area for diagnosis is detected from the photographed image 111. FIG. 4 is a diagram showing an example in which a segmented image is generated from the photographed image 111 shown in FIG. 3. FIG. 5 is a diagram showing an example in which an error image 114 is generated from the segmented image shown in FIG. 4. FIG. 6 is a diagram showing an example in which a binarized image 115 is generated from the error image 114 shown in FIG. 5. FIG. 7 is a diagram showing an example in which a damage detection result is displayed.

[0040] In S1, the data acquisition unit 101 acquires captured images 111 obtained by photographing the wind power generation facility to be diagnosed. Note that for one wind power generation facility, the data acquisition unit 101 may acquire multiple captured images 111 obtained by photographing the wind power generation facility from multiple directions, or multiple captured images 111 obtained by photographing the wind power generation facility at different times. In this case, the processing from S2 onwards is performed for each captured image 111.

[0041] In S2, the defective area detection unit 102 detects a defective area in the photographed image 111 acquired in S1, where the wind power generation facility is not normally captured. There are no particular limitations on what kind of area is determined to be a defective area, and on the method for detecting the defective area. For example, the defective area detection unit 102 may detect, as a defective area, a continuous area made up of pixels whose brightness value is equal to or greater than a threshold value.

[0042] In S3 (target area detection step), the target area detection unit 103 detects a target area for diagnosis in the captured image 111 acquired in S1. For example, the target area detection unit 103 may classify the image area of ​​the captured image 111 from the output value obtained by inputting the captured image 111 to the detection model 112, and detect an area classified as a wind power generation facility or its components (e.g., blades) as a target area. An area not detected as a target area in S3 (e.g., an area classified as a background area of ​​the wind power generation facility) is not a target for diagnosis. For this reason, it can be said that the target area detection unit 103 detects an area that is not a target for diagnosis.

[0043] For example, assume that a captured image 111 shown in FIG. 3 is acquired in S1. This captured image 111 shows the support poles and blades of a wind power generation facility. Since lens flare or the like does not occur in this captured image 111, no defective area is detected in S2, and processing in S3 is performed. In S3, the target area detection unit 103 detects the target area for diagnosis. At this time, the target area detection unit 103 may generate a captured image 111' shown in FIG. 3. In the captured image 111', the pixel values ​​of pixels in the area not targeted for diagnosis in the captured image 111 are uniformly set to 256. In this way, the target area detection unit 103 may change the pixel values ​​of pixels in the area not targeted for diagnosis to values ​​that do not affect the diagnosis (in other words, to values ​​that prevent damage from being detected in that area). The processing in S3 can also be said to be processing for cutting out or extracting the target area for diagnosis from the captured image 111.

[0044] In S4 (division step), the division unit 104 divides the photographed image 111 acquired in S1 into a predetermined division unit to generate a plurality of divided images. For example, as shown in FIG. 4, the division unit 104 may divide the photographed image 111' (an image obtained by extracting a target area from the photographed image 111) into 6 equal parts in the height direction and 8 equal parts in the width direction to generate 48 divided images. In FIG. 4, each divided image is represented by a combination of numbers 1 to 6 indicating its position in the height direction and letters a to h indicating its position in the width direction. For example, in FIG. 4, the divided image in the lower left corner of the photographed image 111' is represented as "6a" by combining the number "6" indicating its position in the height direction and the letter "a" indicating its position in the width direction. Note that the division unit 104 only needs to generate divided images for the target area. For example, the division unit 104 does not need to generate divided images 1a to 1e, 2a to 2d, 2h, etc.

[0045] In S5, the reconstructed image generation unit 105 reconstructs the segmented images generated in S4 to generate a reconstructed image. The reconstruction is performed using the reconstruction model 113. Then, in S6, the error image generation unit 106 generates a partial error image using the reconstructed image generated in S5. Then, in S7, the error image generation unit 106 connects the partial error images generated in S6 to generate an error image.

[0046] For example, suppose that divided images 6a, 5b, 4c, etc. shown in Fig. 4 are generated in S4. In this case, in S5, the reconstructed image generation unit 105 reconstructs the divided image 6a using the reconstruction model 113 to generate a reconstructed image 6a', as shown in Fig. 5. Similarly, the reconstructed image generation unit 105 also reconstructs the divided images 5b, 4c, etc. to generate reconstructed images 5b', 4c', etc. In S6, the error image generation unit 106 generates a partial error image 6a" using the reconstructed image 6a'. The partial error image of the reconstructed image 6a' is a difference image between the divided image 6a and the reconstructed image 6a'. In other words, the pixel value of each pixel in the partial error image of the reconstructed image 6a' is the difference between the pixel value of each pixel of the divided image 6a and the pixel value of the reconstructed image 6a' (i.e., the reconstruction error at each pixel). Similarly, the error image generation unit 106 calculates the differences between the divided images 5b, 4c, etc. and the reconstructed images 5b', 4c', etc., to generate partial error images 5b", 4c", etc. Then, in S7, the error image generation unit 106 connects the partial error images 6a", 5b", 4c", etc. to generate an error image 114. At this time, the error image generating unit 106 places the partial error image 6a'' at the position of the divided image 6a in the photographed image 111, and similarly places the partial error images 5b'', 4c'', etc. at the positions of the divided images 5b, 4c, etc. in the photographed image 111.

[0047] In S8, the target area detection unit 103 detects a contour area in which the contour of the wind power generation facility is captured in the captured image 111 acquired in S1. For example, the target area detection unit 103 may detect the boundary line between the target area in which the wind power generation facility is captured and other areas as the contour line of the wind power generation facility, and detect the contour area based on the contour line. For example, the target area detection unit 103 may detect an area of ​​a predetermined width centered on the contour line as the contour area.

[0048] In S9, the target area detection unit 103 excludes the contour area detected in S8 and the pillar area detected when detecting the target area in S3 from the diagnosis target. The area in which the nacelle is captured may also be excluded from the diagnosis target. In this case, the area in which the nacelle is captured may also be detected in S3. If the target area detection unit 103 does not detect the pillar area in S3, it may detect the pillar area by using, for example, a pillar detection model generated by machine learning using an image of the pillar as training data. The area in which the nacelle is captured may also be detected in a similar manner.

[0049] In S10, the damage detection unit 107 binarizes the error image generated in S7 using a predetermined threshold. Through binarization, it is possible to classify each pixel included in the error image into pixels whose reconstruction error exceeds the threshold and pixels whose reconstruction error is equal to or less than the threshold. A pixel whose reconstruction error exceeds the threshold has an abnormal reconstruction error and is likely to correspond to a damaged location. On the other hand, a pixel whose reconstruction error is equal to or less than the threshold has a reconstruction error within a normal range and is likely to correspond to a location without damage. In the binarization process, the damage detection unit 107 considers that the reconstruction error of each pixel included in the region excluded from the diagnosis target in S9 is equal to or less than the threshold. This makes it possible to prevent damage from being detected in the region excluded from the diagnosis target.

[0050] In S11, the damage detection unit 107 performs a closing process on the binarized error image. The closing process is a process of forming a continuous region based on pixels classified as abnormal values ​​by the binarization process. For example, the damage detection unit 107 may form a continuous region made up of abnormal value pixels by performing an extension process in which pixels surrounding a pixel classified as an abnormal value are also considered to be abnormal values. Alternatively, for example, the damage detection unit 107 may form a continuous region made up of abnormal value pixels by performing an interpolation process in which each pixel in a region sandwiched between pixels classified as abnormal values ​​is considered to be an abnormal value.

[0051] In S12 (damage detection step), the damage detection unit 107 detects damage to the wind power generation facility to be diagnosed. Damage detection is performed based on the reconstruction error when each of the segmented images generated in S4 is reconstructed by the reconstruction model 113. Specifically, the damage detection unit 107 detects, in the error image that has been subjected to the closing process in S11, a continuous region consisting of a predetermined number of pixels or more, in other words, a region whose area exceeds a predetermined value, as a damaged area.

[0052] For example, if the error image 114 shown in FIG. 5 is generated in S7, the damage detection unit 107 may generate the binarized image 115 shown in FIG. 6 from this error image 114 in S10. The binarized image 115 shown in FIG. 6 is specifically generated as follows. That is, the damage detection unit 107 sets the pixel values ​​of pixels in the image region of the error image 114 that were excluded from the diagnosis target in S9 and pixels whose pixel values ​​are equal to or less than a threshold to a predetermined value (e.g., 0). On the other hand, the damage detection unit 107 sets the pixel values ​​of pixels in the image region of the error image 114 whose pixel values ​​exceed the threshold to another predetermined value (e.g., 255). The binarized image generated in this manner usually contains scattered pixels whose pixel values ​​are the other predetermined value. For this reason, the damage detection unit 107 performs the closing process described above on these pixels to create continuous regions having a certain width, and detects these continuous regions as damaged areas. As a result, the binarized image 115 shown in FIG. 6 is generated.

[0053] In S13, the display control unit 108 causes the display device to display the detection result of S12. If no damage is detected in S12, the display control unit 108 may display a message to that effect. When the process of S13 ends, the diagnostic method of FIG. 2 ends.

[0054] For example, in S13, the display control unit 108 may display a result image 116 as shown in FIG. 7, in which information indicating the detection result of S12 is superimposed on the photographed image 111 acquired in S1. In the result image 116, for the area containing damage detected in S12, a line indicating the outer edge of the area is superimposed on the photographed image 111. By displaying this line, the display control unit 108 allows the user of the diagnostic device 1 to easily recognize the position and range of the detected damage. For example, FIG. 7 shows a partial enlarged view of a blade. From this partial enlarged view, it is easy to recognize that damage exists in the area surrounded by line E.

[0055] 2, the pillar region is excluded from the diagnosis target in S9, but the pillar region may also be excluded from the diagnosis target in S3. In this case, the processing from S4 onwards is performed on the region in which the blade is captured. Alternatively, in S4, a segmented image may be generated for the entire captured image 111, and in S9, the background region as well as the outline region and pillar region may be excluded from the diagnosis target.

[0056] 2 includes a division step (S4) of dividing a captured image 111 of a wind power generation facility to be diagnosed for the presence or absence of damage into a predetermined division unit to generate a plurality of divided images, and a damage detection step (S12) of detecting damage to the wind power generation facility to be diagnosed based on a reconstruction error when each of the divided images is reconstructed using a reconstruction model 113 that has learned the features of images of undamaged wind power generation facility. This makes it possible to highly accurately diagnose the presence or absence of damage using images of the wind power generation facility.

[0057] 2 includes a target area detection step (S3) for detecting a target area for diagnosis in a captured image 111 of the wind power generation equipment to be diagnosed for the presence or absence of damage, using a detection model 112 generated by machine learning to detect a target area for diagnosis in the wind power generation equipment from an image of the wind power generation equipment, and a damage detection step (S12) for detecting damage to the wind power generation equipment to be diagnosed, based on a reconstruction error when a portion of the target area in the captured image 111 is reconstructed using a reconstruction model 113 that has learned the features of an image of undamaged wind power generation equipment. This makes it possible to highly accurately diagnose the presence or absence of damage using an image of the wind power generation equipment.

[0058] [Modification] The execution entity of each process described in the above embodiment is arbitrary and is not limited to the above example. In other words, the same functions as the diagnostic device 1 can be realized by a plurality of information processing devices (which can also be called processors) that can communicate with each other. For example, each process shown in FIG. 2 may be shared and executed by a plurality of information devices. In other words, the execution entity of the diagnostic method according to the above embodiment may be one information processing device (e.g., the diagnostic device 1) or multiple information processing devices.

[0059] [Reference example] A diagnostic device 1 according to one reference example includes a target area detection unit 103 that detects a target area for diagnosis in a captured image 111 of the wind power generation equipment to be diagnosed for the presence or absence of damage, using a detection model 112 generated by machine learning to detect a target area for diagnosis in the wind power generation equipment from an image of the wind power generation equipment, and a damage detection unit 107 that detects damage to the wind power generation equipment to be diagnosed, based on a reconstruction error when a portion of the target area in the captured image 111 is reconstructed using a reconstruction model 113 that has learned the features of an image of undamaged wind power generation equipment. This diagnostic device 1 does not necessarily have to include a division unit 104.

[0060] That is, the damage detection unit 107 of the diagnostic device 1 according to one reference example may detect damage based on a reconstruction error when a target region in an unsegmented captured image 111 is reconstructed using the reconstruction model 113. The diagnostic device 1 according to this reference example reduces the possibility that regions other than the target region for diagnosis will affect the diagnosis result, thereby making it possible to highly accurately diagnose the presence or absence of damage using an image.

[0061] Moreover, a diagnostic method according to one reference example includes a target area detection step of detecting a target area for diagnosis in a captured image 111 of the wind power generation equipment to be diagnosed for the presence or absence of damage, using a detection model 112 generated by machine learning to detect a target area for diagnosis in the wind power generation equipment from an image of the wind power generation equipment, and a damage detection step of detecting damage to the wind power generation equipment to be diagnosed, based on a reconstruction error when a portion of the target area in the captured image 111 is reconstructed using a reconstruction model 113 that has learned the features of an image of undamaged wind power generation equipment. In this diagnostic method, the image to be reconstructed may be a divided image obtained by dividing the captured image 111, or may be an undivided captured image 111.

[0062] [Software implementation example] The functions of the diagnostic device 1 can be realized by a program (diagnostic program) for causing a computer to function as the diagnostic device 1, and a program for causing a computer to function as each control block of the diagnostic device 1 (particularly each part included in the control unit 10).

[0063] In this case, the diagnostic device 1 includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The functions described in the above embodiment are realized by executing the program using the control device and storage device.

[0064] The program may be stored non-transitory on one or more computer-readable storage media. The storage media may or may not be included in the diagnostic device 1. In the latter case, the program may be supplied to the diagnostic device 1 via any wired or wireless transmission medium.

[0065] Furthermore, some or all of the functions of the control blocks can be realized by logic circuits. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of the present invention. In addition, the functions of the control blocks can also be realized by, for example, a quantum computer.

[0066] 〔summary〕 A diagnostic device according to a first aspect of the present invention includes a division unit that divides an image of a wind power generation facility to be diagnosed for damage by a predetermined division unit to generate a plurality of divided images, and a damage detection unit that detects damage to the wind power generation facility to be diagnosed based on a reconstruction error when each of the divided images is reconstructed using a reconstruction model that has learned the characteristics of images of undamaged wind power generation facility.

[0067] The diagnostic device according to aspect 2 of the present invention is the same as in aspect 1, but includes a target area detection unit that detects a target area for diagnosis in the captured image using a detection model generated by machine learning to detect a target area for diagnosis from an image of wind power generation equipment, and the division unit divides the target area detected by the target area detection unit to generate the divided image.

[0068] A diagnostic device according to aspect 3 of the present invention is in accordance with aspect 1 or 2, and includes an error image generation unit that concatenates a plurality of partial error images that respectively indicate the reconstruction errors of a plurality of the segmented images to generate an error image that indicates the distribution of reconstruction errors in the captured image, and the damage detection unit uses the error images to detect damage to the wind power generation equipment that is the target of diagnosis.

[0069] A diagnostic device according to aspect 4 of the present invention is any of aspects 1 to 3, and includes a contour detection unit that detects the contour of the wind power generation facility from the captured image, and the damage detection unit detects damage in areas other than the area where the contour is detected.

[0070] A diagnostic device according to aspect 5 of the present invention is any of aspects 1 to 4, wherein the wind power generation equipment to be diagnosed includes blades that rotate in the wind and poles that support the blades, and is equipped with a pole detection unit that detects the poles of the wind power generation equipment from the captured image, and the damage detection unit detects damage in areas other than the area where the poles are detected.

[0071] A diagnostic device according to aspect 6 of the present invention is any of aspects 1 to 5, and includes a faulty area detection unit that detects faulty areas in the captured image where the wind power generation equipment is not shown normally, and the damage detection unit detects damage in areas other than the faulty areas.

[0072] A diagnostic device according to aspect 7 of the present invention is any of aspects 1 to 6, wherein the damage detection unit detects damage to each component of the wind power generation facility based on a reconstruction error when the segmented image showing the component is reconstructed using a reconstruction model generated for each component of the wind power generation facility.

[0073] A diagnostic device according to an eighth aspect of the present invention includes a target area detection unit that detects a target area for diagnosis in a captured image of wind power generation equipment that is the target for diagnosis for the presence or absence of damage, using a detection model generated by machine learning to detect a target area for diagnosis in the wind power generation equipment from an image of the wind power generation equipment, and a damage detection unit that detects damage to the wind power generation equipment that is the target for diagnosis, based on a reconstruction error when a portion of the target area in the captured image is reconstructed using a reconstruction model that has learned the features of an image of undamaged wind power generation equipment.

[0074] A diagnostic method according to aspect 9 of the present invention is a diagnostic method executed by at least one information processing device, and includes a division step of dividing a captured image of a wind power generation facility to be diagnosed for damage by a predetermined division unit to generate a plurality of divided images, and a damage detection step of detecting damage to the wind power generation facility to be diagnosed based on a reconstruction error when each of the divided images is reconstructed using a reconstruction model that has learned the characteristics of images of undamaged wind power generation facility.

[0075] A diagnostic method according to aspect 10 of the present invention is a diagnostic method executed by at least one information processing device, and includes a target area detection step of detecting a target area for diagnosis in a captured image of the wind power generation equipment to be diagnosed for damage using a detection model generated by machine learning to detect a target area for diagnosis in the wind power generation equipment from an image of the wind power generation equipment, and a damage detection step of detecting damage to the wind power generation equipment to be diagnosed based on a reconstruction error when a portion of the target area in the captured image is reconstructed using a reconstruction model that has learned the features of an image of undamaged wind power generation equipment.

[0076] A diagnostic program according to an eleventh aspect of the present invention is a diagnostic program for causing a computer to function as the diagnostic device according to the first aspect, and causes the computer to function as the dividing unit and the damage detection unit.

[0077] A diagnostic program according to a twelfth aspect of the present invention is a diagnostic program for causing a computer to function as the diagnostic device according to the eighth aspect, and causes the computer to function as the target area detection unit and the damage detection unit.

[0078] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0079] 1 Diagnostic equipment 102 Defective area detection unit 103 Object area detection unit (contour detection unit, support detection unit) 104 Division 106 Error image generation unit 107 Damage detection unit 111 images 112 Detection Model 113 Reconstruction Model

Claims

1. a division unit that divides a photographed image of a building that is to be diagnosed for the presence or absence of damage into a predetermined division unit to generate a plurality of divided images; A diagnostic device comprising: a damage detection unit that detects damage to the structure being diagnosed based on the reconstruction error when each of the divided images is reconstructed using a reconstruction model that has learned the characteristics of images of undamaged buildings.

2. a target area detection unit that detects a target area for diagnosis in the captured image using a detection model generated by machine learning to detect a target area for diagnosis from an image of a building; The diagnostic device according to claim 1 , wherein the dividing unit divides the target region detected by the target region detection unit to generate the divided image.

3. an error image generating unit that generates an error image that indicates a distribution of reconstruction errors in the captured image by connecting a plurality of partial error images that respectively indicate reconstruction errors of the plurality of segmented images; The diagnostic device according to claim 1 , wherein the damage detection unit detects damage to the structure to be diagnosed using the error image.

4. a contour detection unit that detects a contour of the building from the captured image; The diagnostic device according to claim 1 , wherein the damage detection unit detects damage in an area other than the area where the contour is detected.

5. The structure to be diagnosed is a wind power generation facility, and includes blades that rotate when exposed to wind, a nacelle that houses a generator, and a support that supports the nacelle and the blades, a pillar detection unit that detects an area including at least a pillar of the wind power generation facility from the captured image, The diagnostic device according to claim 1 , wherein the damage detection unit detects damage in an area other than the area detected by the support column detection unit.

6. a defective area detection unit that detects a defective area in the captured image where the structure is not normally captured, The diagnostic device according to claim 1 , wherein the damage detection unit detects damage in areas other than the defective area.

7. 3. The diagnostic device according to claim 1, wherein the damage detection unit detects damage to each component of the building structure based on a reconstruction error when the segmented image containing the component is reconstructed using a reconstruction model generated for that component.

8. a target area detection unit that detects a target area for diagnosis in a photographed image of a building that is to be diagnosed for the presence or absence of damage, using a detection model generated by machine learning to detect a target area for diagnosis in the building from an image of the building; a damage detection unit that detects damage to the structure being diagnosed based on a reconstruction error when the target area in the captured image is reconstructed using a reconstruction model that has learned the characteristics of images of undamaged buildings.

9. A diagnostic method executed by at least one information processing device, comprising: a division step of dividing a photographed image of a building to be diagnosed for the presence or absence of damage by a predetermined division unit to generate a plurality of divided images; A diagnostic method including a damage detection step of detecting damage to the structure to be diagnosed based on the reconstruction error when each of the divided images is reconstructed using a reconstruction model that has learned the features of images of undamaged buildings.

10. A diagnostic method executed by at least one information processing device, comprising: a target area detection step of detecting a target area for diagnosis in a photographed image of a building to be diagnosed for the presence or absence of damage, using a detection model generated by machine learning to detect a target area for diagnosis in the building from an image of the building; and a damage detection step of detecting damage to the structure to be diagnosed based on a reconstruction error when a portion of the target area in the captured image is reconstructed using a reconstruction model that has learned the features of images of undamaged buildings.

11. A diagnostic program for causing a computer to function as the diagnostic device according to claim 1, the diagnostic program causing a computer to function as the dividing unit and the damage detecting unit.

12. A diagnostic program for causing a computer to function as the diagnostic device according to claim 8, the diagnostic program causing a computer to function as the target region detection unit and the damage detection unit.

Images