Damage detection device and damage detection method for structure

Abstract

To provide technique for accurately detecting a damaged part of a large-sized structure.SOLUTION: A damage detection device that detects damage to a structure without contacting has: a visible camera 3 which images the structure to acquire a captured image; a high-resolution image acquisition unit 5A which performs super-resolution processing so as to acquire an image of the structure having higher resolution than the captured image from the captured image that the visible camera 3 acquires; and an evaluation unit 5B which analyzes the image that the super-resolution image acquisition unit 5A acquires to evaluate the damage to the structure.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technology for non-contact damage detection of large structures. The present invention is particularly suitable for structures that are constantly undergoing slight movements. Furthermore, the present invention is a technology particularly suitable for easily detecting damage to the entire structure. [Background technology]

[0002] Steelworks operate multiple large pieces of equipment. These include unloaders that unload raw materials from bulk carriers, and stackers and reclaimers that load and unload raw materials in raw material yards. Large pieces of equipment such as unloaders and stackers and reclaimers are installed in berths where raw materials are unloaded and in raw material yards where raw materials are stored. Because these equipment are installed in outdoor environments near the coast, they are prone to corrosion and thinning. In recent years, the equipment has tended to age, increasing the risk of serious breakdowns due to deterioration of the equipment.

[0003] Therefore, it is necessary to periodically diagnose and manage the deterioration of the structures that make up large-scale facilities, and to repair and renew the facilities as necessary.

[0004] Normally, the primary diagnosis of deterioration of large facilities (structures) such as those described above is performed by inspectors visually checking for any abnormalities in the structure's appearance. However, because steelworks have numerous pieces of equipment scattered across a vast site, comprehensive visual inspection of all of them requires a lot of manpower, time, and money.

[0005] To solve this problem, technologies have been developed in recent years to detect damage to structures using images. For example, Patent Document 1 discloses a deterioration state diagnosis method in which a structure is imaged with an ultra-high pixel count and the image is used to determine damage to the structure. Furthermore, Patent Document 2 discloses a deterioration state detection method in which an image of the structure is acquired and the number of pixels in the structure area is compared with the number of pixels in the deteriorated area to diagnose the deterioration of the structure. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent Publication No. 2021-32042 [Patent Document 2] Patent Publication No. 2021-32650 Summary of the Invention

[0007] It is difficult for people to approach and photograph areas where damage is to be inspected for large structures. For this reason, in the damage detection methods described in Patent Documents 1 and 2, a drone equipped with a camera is brought close to the structure to capture images of the structure, thereby obtaining high-resolution images of the inspection areas of the structure. However, drone flight is dependent on the weather. In addition, work and equipment operation around the drone's flight area are restricted. For this reason, it is difficult to perform imaging using a drone frequently and easily. Furthermore, while close-up imaging can obtain high-resolution images, the imaging area of ​​each image is narrow. For this reason, to evaluate the deterioration of a large structure, it is necessary to individually capture images of multiple areas.

[0008] On the other hand, when trying to capture images of a structure without using a drone, it is necessary to install a camera in a fixed position away from the structure (equipment). This makes it impossible to capture high-resolution images of the structure. As a result, damage to the structure must be detected from images that do not have high resolution, making it difficult to accurately detect damage to large structures. This is particularly problematic when capturing images of a wide area.

[0009] The present invention has been made in view of the above points, and one of the objects of the present invention is to provide a technique for easily and accurately detecting damaged areas in large structures. [Means for solving the problem]

[0010] In order to solve the problem, one aspect of the present invention is a damage detection device that detects damage to a structure in a non-contact manner, and includes an image acquisition unit that captures an image of the structure to acquire the captured image, a high-resolution image acquisition unit that performs super-resolution processing to acquire an image of the structure with higher resolution than the captured image from the captured image acquired by the image acquisition unit, and an evaluation unit that analyzes the image acquired by the high-resolution image acquisition unit and evaluates damage to the structure. [Effects of the Invention]

[0011] According to an aspect of the present invention, by performing super-resolution processing to increase the resolution of captured images, for example, it is possible to safely and stably acquire high-resolution images of large facilities from a distant location, and by using these high-resolution images, it is possible to accurately detect damaged areas in the structures. Furthermore, when an image of the entire structure is acquired from a distant location, it becomes possible to evaluate damage detection for the entire large structure from a single image, enabling damage to the large structure to be detected easily and in a short time. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a schematic diagram showing an example of the configuration of a structure damage detection device according to an embodiment of the present invention. [Figure 2] 1 shows an example of an image of a reclaimer captured by a visible light camera, where (a) is an image of the entire image, and (b) is an image of a cut-out area. [Figure 3] 1 shows an example of images obtained by cutting out the same area from two consecutively captured image data, where (a) is the image cut out from the Nth frame, and (b) is the image cut out from the N+1th frame. [Figure 4]FIG. 10 is a diagram showing an image after super-resolution processing based on continuously captured images. [Figure 5] FIG. 10 is a diagram showing an example of a detection result of a damaged portion. DETAILED DESCRIPTION OF THE INVENTION

[0013] Next, an embodiment of the present invention will be described with reference to the drawings. (About the test subjects) FIG. 1 is a schematic diagram showing an example of the configuration of a structure damage detection device according to an embodiment of the present invention. This embodiment is a device for detecting damage to a structure that constitutes a large facility. In this embodiment, a reclaimer 2 installed in a raw material yard of a steelworks is used as an example of a structure that constitutes a large facility. In the raw material yard of the steelworks, raw materials for steelmaking stored as raw material piles 1 are removed by the reclaimer 2 and sent to the next process.

[0014] The damage detection device of this embodiment is particularly effective when applied to large-scale facilities, for example, with a vertical or horizontal dimension of 10 m or more. The large-scale facilities may be composed of multiple structures. Furthermore, when the large-scale facilities are composed of a single structure, the damage detection device is particularly effective when applied to a structure with a vertical or horizontal dimension of 3 m or more.

[0015] (composition) As shown in FIG. 1, the damage detection device of this embodiment includes a visible camera 3, an image storage device 4, and an image processing device 5.

[0016] <Visible Camera 3> The visible camera 3 constitutes an image acquisition unit that captures an image of a structure and acquires the captured image. The visible light camera 3 captures an image of the reclaimer 2, which is a structure. In this embodiment, an example is given in which the entire reclaimer 2 is captured as a single image (captured area). The reclaimer 2 may be divided into multiple areas, and each area may be individually imaged and its damage assessed.

[0017] Here, the reclaimer 2 moves along the raw material pile 1 by remote control. For this reason, people cannot approach or enter the reclaimer 2 during operation. For this reason, it is preferable that the visible light camera 3 is installed outside the operating range of the reclaimer 2 and in a fixed position where it can capture an image of the entire reclaimer 2 facility. Also, from the viewpoint of imaging the entire facility as a single imaging region, it is preferable to image the entire facility at a position distant from the facility. Also, from the viewpoint of easily diagnosing damage to the entire facility in a short time, it is preferable to image the entire facility as a single imaging region. Note that "distant" refers to imaging from a distance of, for example, 50 m or more, preferably 100 m or more.

[0018] As will be described in detail below, the visible camera 3 is configured to continuously capture multiple images of the reclaimer 2 at preset intervals.

[0019] <Image Storage Device 4> The image storage device 4 collects and stores data (image data) of a plurality of images continuously captured by the visible camera 3.

[0020] <Image processing device 5> The image processing device 5 includes a high-resolution image acquisition unit 5A and an evaluation unit 5B.

[0021] <High-resolution image acquisition unit 5A> The high-resolution image acquisition unit 5A performs processing to acquire a high-resolution image, which is an image of the reclaimer 2 having a higher resolution than the image data (captured image) captured by the visible camera 3. The high-resolution image acquisition unit 5A performs super-resolution processing as processing to acquire a high-resolution image.

[0022] At least one captured image may be used for the super-resolution processing, but it is preferable to use a plurality of captured images captured consecutively at a predetermined interval for the super-resolution processing.

[0023] Large equipment such as the reclaimer 2 is constantly vibrating slightly (continuously moving) due to natural vibration sources such as wind power and artificial vibration sources such as mechanical vibration. Therefore, in the image data continuously captured by the visible light camera 3, the position of the pixels capturing the contour of the object being imaged will shift, and the contour captured in each image will have different degrees of change in brightness or color for adjacent pixels. The high-resolution image acquisition unit 5A of this embodiment acquires a high-resolution image by image processing using a plurality of image data with different pixel positions and different degrees of change in brightness or color.

[0024] In this embodiment, a plurality of images acquired by successively capturing images are aligned and reconstructed by super-resolution processing, thereby generating a high-resolution image.

[0025] For the super-resolution processing, a conventionally known processing method may be employed. For example, a known learning model created in advance using an artificial intelligence algorithm such as a generative adversarial network may be used for the super-resolution processing. Note that the super-resolution processing does not necessarily have to be performed using multiple image data. For example, by using a learning model created in advance using a generative adversarial network, it is also possible to perform the super-resolution processing using only one image data.

[0026] [About imaging intervals] As mentioned above, the reclaimer 2 is constantly moving slightly. For this reason, the pixel positions and the degree of change in brightness or color differ between the multiple images captured continuously by the visible camera 3. However, when aligning and reconstructing multiple images using super-resolution processing, if there is a large misalignment between two consecutively captured images, alignment becomes difficult. For this reason, in order to generate a high-resolution image with high accuracy, it is desirable that the misalignment between two consecutively captured images be one pixel or less. In other words, overlapping pixels at the same position makes alignment and reconstruction easier.

[0027] Since microtremors occur at the natural frequency of the structure, it is desirable to set the imaging interval based on the natural frequency and vibration amplitude of the reclaimer 2 so that the positional deviation between the two images is one pixel or less.

[0028] Specifically, if the natural frequency of the structure (reclaimer 2) is f [Hz] and the vibration amplitude is A [mm], the vibration velocity V of the structure is expressed as 2πfA. If the pixel resolution of the image data is R [mm / pixel] and the imaging interval is I [s], then in order to keep the positional deviation between two consecutively captured images to one pixel or less, the relationship R≧V×I=2πfA×I must hold. Even if the natural frequency is the same, in the case of a large structure, the vibration amplitude is greater in the distal portion than in the center. Therefore, it is preferable to use the vibration amplitude A [mm] at the point with the largest amplitude in the target structure.

[0029] By setting the imaging interval so as to satisfy "I≦R / 2πfA", the positional deviation between two consecutively captured images can be reduced to one pixel or less.

[0030] As an example, large equipment such as the reclaimer 2 has a small natural frequency (f) of approximately 1 Hz or less due to its heavy weight. On the other hand, a small natural frequency generally results in a large vibration amplitude (A), with a maximum amplitude of approximately 20 mm (several mm to several tens of mm). In this case, the vibration velocity (V) (= 2πfA) of constant microtremors of large equipment such as the reclaimer 2 can be as high as 126 mm / s. Therefore, for the reclaimer 2, the imaging interval (I) [s] should be R / 126 or less. For example, if a 4K camera with 3840 pixels horizontally and 2160 pixels vertically captures an area 19.2 m horizontally and 10.8 m vertically, the pixel resolution (R) is 5 mm / pixel. Therefore, by setting the imaging interval to 0.039 s or less, the positional deviation between the two images can be reduced to less than one pixel.

[0031] <Evaluation Section 5B> The evaluation unit 5B analyzes the images acquired by the high-resolution image acquisition unit 5A and performs processing to evaluate damage to the structure. As part of the evaluation, damaged areas of the reclaimer 2 are detected by image processing.

[0032] The evaluation unit 5B of this embodiment is configured to detect damaged areas in a structure by analyzing images using a machine-learned learning model. Machine learning can utilize image segmentation techniques such as semantic segmentation. Semantic segmentation is a technique for classifying each pixel in an image into a pre-trained class. A learning model can be obtained by pre-training a convolutional neural network using images of the damage to be detected (wall thinning or holes) as training data. Damaged areas can be extracted by inputting high-resolution images into the learning model.

[0033] The detected damaged areas may be highlighted, for example, by overlaying color on a high-resolution image. In this case, the facility manager can visually confirm the damaged areas and the extent of the damage. The results of the detection of damaged areas may be notified to the structure manager by any method, such as displaying them on a monitor screen. Damage evaluation may also be performed, such as displaying the number of damages exceeding a predetermined number of pixels (area) per unit area.

[0034] The evaluation unit 5B may be configured to evaluate and output the degree of damage to the entire target reclaimer 2 or each part based on the determined damaged location and degree of damage. The degree of damage may be, for example, the degree of urgency of maintenance. The damage detection according to this embodiment corresponds to the first stage of deterioration diagnosis of large-scale equipment.

[0035] (Operation etc.) According to this embodiment, by performing super-resolution processing to increase the resolution of the image, high-resolution images can be obtained safely and stably from a distant position, and by using these images, damaged areas of the structure can be detected with high accuracy. Furthermore, by acquiring an image of the entire structure from a distance, it becomes possible to evaluate damage detection for the entire large structure from a single image. As a result, in this embodiment, damage to a large structure can be detected easily and in a short time.

[0036] (Variation) In the above embodiment, a visible camera 3 is used as the imaging device. However, an imaging unit other than a general visible camera 3 may be used as the imaging unit of the image acquisition unit. For example, a hyperspectral camera that disperses light into multiple wavelengths or a thermal camera that records infrared light may be used depending on the type of damage to be detected and the accuracy required. Also, a configuration may be adopted in which a video is captured instead of a still image, multiple images (still images) are extracted from the captured video, and the super-resolution processing is performed using the captured images.

[0037] Furthermore, although the reclaimer 2 has been used as an example of a structure, the target of damage detection may be a large facility other than the reclaimer 2 or a large structure such as a high-rise building. "Damage" also includes various defects such as corrosion, rust, thinning, holes, etc.

[0038] (others) The present disclosure may also have the following configuration. (1) Disclosure 1 is a damage detection device for detecting damage to a structure in a non-contact manner, an image acquisition unit that captures an image of the structure; a high-resolution image acquisition unit that performs super-resolution processing to acquire an image of the structure having a higher resolution than the captured image from the captured image acquired by the image acquisition unit; an evaluation unit that analyzes the image acquired by the high-resolution image acquisition unit and evaluates damage to the structure; A damage detection device for a structure having the above structure. (2) Disclosure 2 states that the high-resolution image acquisition unit performs super-resolution processing using a plurality of captured images captured by the image acquisition unit. (3) In Disclosure 3, the high-resolution image acquisition unit sets the imaging interval between the multiple captured images to be used based on the natural frequency and vibration amplitude of the structure. (4) Disclosure 4 sets the imaging interval I so as to satisfy I≦R / 2πfA, where f [Hz] is the natural frequency of the structure, A [mm] is the vibration amplitude, R [mm / pixel] is the pixel resolution of the image acquired by the image acquisition unit, and I [s] is the imaging interval. (5) Disclosure 5 discloses that the evaluation unit detects damaged areas of the structure by analyzing the image using a machine-learned learning model. (6) Disclosure 6 provides a method for obtaining an image of a structure having a higher resolution than the captured image from the captured image of the structure, and evaluating damage to the structure from the obtained image. Methods for detecting damage in structures. [Example]

[0039] Next, an example of super-resolution processing and an example of detection of damaged portions in the processing of this embodiment will be described. Figure 2(a) is an example of an image of the entire reclaimer 2 captured by the visible camera 3. Specifically, Figure 2(a) is image data of the entire facility, captured by capturing the entire reclaimer 2 as a single imaging area. Figure 2(a) is shown as a schematic diagram for ease of understanding. Also, Figure 2(b) is an example of a screen where a portion of the image is cut out from the image of the entire facility to make the processing easier to understand. Note that the super-resolution processing was performed on an image capturing the entire reclaimer 2. In other words, the super-resolution processing was not performed on the cut-out image. However, the super-resolution processing may be performed on each cut-out image, but considering the micro-movement, it is preferable to perform the super-resolution processing on the original captured image.

[0040] Here, the visible light camera 3 records the intensity of light from the subject using an image sensor with a fixed number of pixels, so there is a limit to the number of pixels in the image data that can be captured. Therefore, when the image data is enlarged and displayed as shown in Figure 2(b), the subject is displayed in a pixel-by-pixel grid, making it difficult to grasp the details.

[0041] Figure 3 shows an example of the same area cut out from two consecutive images of the entire facility. As mentioned above, due to the constant slight movement of Reclaimer 2, the position of the pixels capturing the outline of the imaged object shifts, causing the color of each pixel to change.

[0042] Figure 4 shows the results of super-resolution processing. Figure 4 is an image of the area shown in Figure 3, extracted from an image of the entire facility that has been converted into a high-resolution image using super-resolution processing. Note that Figure 4 is a high-resolution image generated with a resolution increased 16 times, four times vertically and four times horizontally. In this example, as shown in Figure 4, the holes that were difficult to identify in the image of the entire facility before super-resolution processing were clearly visible. For the super-resolution processing, a publicly known learning model created in advance using an artificial intelligence algorithm such as a generative adversarial network was used.

[0043] Figure 5 shows an example of the results of detecting damaged areas. Figure 5 shows an example of detecting holes as damaged areas from a high-resolution image using a learning model generated by image segmentation. Figure 5 also shows an image in which the damaged area S is highlighted by coloring it red and overlaying it. Specifically, Figure 5 shows an example in which the outline of the damaged area S is displayed with a thick line, and the inside of the damaged area S is left blank. By automatically identifying and displaying damaged areas through image processing, facility managers can easily check the presence and extent of damage. In addition, by calculating and recording the number of pixels (area) of the damaged area, the degree of deterioration can be quantitatively grasped and managed.

[0044] As described above, by using this embodiment, damaged areas of the entire facility can be easily and early detected without being overlooked. Furthermore, even when a large number of facilities are installed on a large site, it is possible to inspect them frequently while reducing time and costs. As a result, repairs can be made before a serious breakdown occurs. Furthermore, by properly understanding the deterioration status of the facilities, it is possible to correctly prioritize facilities renewal and to formulate rational investment plans. [Explanation of symbols]

[0045] 1 raw material pile 2 Reclaimer (structure) 3. Visible light camera 4. Image storage device 5. Image processing device 5A High-resolution image acquisition unit 5B Evaluation Section S Damaged area

Claims

1. A damage detection device for detecting damage to a structure in a non-contact manner, an image acquisition unit that captures an image of the structure; a high-resolution image acquisition unit that performs super-resolution processing to acquire an image of the structure having a higher resolution than the captured image from the captured image acquired by the image acquisition unit; an evaluation unit that analyzes the image acquired by the high-resolution image acquisition unit and evaluates damage to the structure; A damage detection device for a structure having the above structure.

2. the high-resolution image acquisition unit performs super-resolution processing using a plurality of captured images captured by the image acquisition unit. The structure damage detection device according to claim 1 .

3. the high-resolution image acquisition unit sets an imaging interval between the plurality of captured images to be used based on a natural frequency and a vibration amplitude of the structure; The structure damage detection device according to claim 2 .

4. When the natural frequency of the structure is f [Hz], the vibration amplitude is A [mm], the pixel resolution of the captured image acquired by the image acquisition unit is R [mm / pixel], and the capturing interval is I [s], the capturing interval I is set so as to satisfy I≦R / 2πfA. The damage detection device for a structure according to claim 3.

5. The evaluation unit detects a damaged portion of the structure by analyzing the image using a machine-learned learning model. The damage detection device for a structure according to any one of claims 1 to 4.

6. acquiring an image of the structure having a higher resolution than the captured image from the captured image of the structure, and evaluating damage to the structure from the acquired image; Methods for detecting damage in structures.

Images