Image analysis apparatus, image analysis method, and recording medium

Abstract

Provided is an image analysis device, an image analysis method, and a program capable of reducing false detection of defects. The image analysis device includes a processor that acquires an infrared thermal image of a structure in which a test object is present, acquires a visible image of the structure in which the test object is present, determines a temperature change from the infrared thermal image, and infers a cause of the temperature change based on at least temperature change information obtained from the infrared thermal image and surface change information obtained from the visible image.

Description

Technical Field

[0001] This invention relates to an image analysis apparatus, an image analysis method, and a program. Background Technology

[0002] The technique of using infrared thermal images acquired by an infrared camera to photograph structures such as concrete to identify defects such as bulges and cracks, as well as intact parts within a structure, is well-known. When there are defects in a structure, the surface temperature of the defective part is higher than that of the intact part during a temperature rise, such as during the day, and conversely, the surface temperature of the defective part is lower than that of the intact part during a temperature drop, such as at night. Therefore, if a part with a surface temperature different from the surrounding area is present in the infrared thermal image, it can be determined that there is a defect inside that part.

[0003] On the other hand, in infrared surveys, even if there are no defects such as bulges inside the structure, there are still areas where the surface temperature differs from the surrounding area. For example, during structural repair, sometimes the thermal conductivity of the repair material differs from that of the surrounding concrete, resulting in a different surface temperature of the repair mark compared to the surrounding concrete. Additionally, sometimes the infrared emissivity at the repair mark differs from that of the surrounding concrete, causing the surface temperature of the repair mark in the infrared thermal image to differ from the surrounding concrete. Sometimes, when foreign matter such as free lime adheres to the surface, the actual temperature and / or surface temperature of that area also differs from the surrounding concrete. Furthermore, uneven color (mold, moss, stripping agents, water effects, etc.), joints, steps, scum, reinforcing steel, rust, rust, leaks, surface irregularities, honeycomb panels, etc., can all cause areas where the actual and / or surface temperature differs from the surrounding area. Thus, in infrared surveys, there are problems such as areas where the surface temperature differs from the surrounding area even when there are no defects such as bulges inside the structure, leading to a high rate of false detections.

[0004] Regarding this issue, Patent Document 1 discloses the following method. It involves a multivariate analysis formula relating a specific factor that affects the thermal image of a structure to the probability that an abnormality extracted from the thermal image of the structure using information from this factor contains a defect. Then, an infrared camera is used to capture a thermal image of the structure. An abnormality with a temperature different from the surrounding area is extracted from the thermal image. Information about the factors within this abnormality is identified. The identified factor information is then quantified, and this value is applied to the multivariate analysis formula to calculate the probability that an abnormality contains a defect.

[0005] Previous technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2013-096741 Summary of the Invention

[0008] The technical problem to be solved by the invention

[0009] Patent Document 1 discloses a method that involves extracting anomalies with surface temperatures different from the surrounding area from an infrared thermal image, then determining the presence or absence of cracks and the surface condition using visual images or human observation. This information is then applied to a multivariate analysis formula to calculate the probability that the anomaly contains a defect. Here, "surface condition" refers to the presence or absence of variations on the surface of a concrete structure, such as uneven color, uneven surfaces, or loose lime. Since false detections (anomalies where the surface temperature differs from the surrounding area even though there are no internal defects in the structure) are often caused by variations on the structure's surface, determining the presence or absence of surface variations is considered effective in reducing false detections.

[0010] However, in the invention of Patent Document 1, the probability of an adverse condition is calculated statistically based solely on the information of the presence or absence of changes (including cracks) on the surface of the structure. This method is insufficient to reduce false detections.

[0011] The present invention was made in view of the following circumstances, and its object is to provide an image analysis apparatus, image analysis method and program that can reduce false detection of defects.

[0012] means for solving technical problems

[0013] The image analysis device involved in the first method is an image analysis device equipped with a processor, wherein the processor performs the following processing: acquiring an infrared thermal image of the structure of the object to be inspected; acquiring a visual image of the structure of the object to be inspected; determining temperature changes based on the infrared thermal image; and, for the temperature changes, inferring the cause of the temperature changes based at least on the temperature change information obtained from the infrared thermal image and the surface change information obtained from the visual image.

[0014] In the image analysis apparatus involved in the second approach, the temperature change information includes the temperature distribution of the infrared thermal image and / or information obtained from the temperature distribution.

[0015] In the image analysis apparatus involved in the third approach, the temperature change information includes information on the shape and / or magnitude of the temperature change.

[0016] In the image analysis apparatus involved in the fourth approach, the surface change information includes the brightness distribution of the visible image and / or information obtained from the brightness distribution.

[0017] In the image analysis apparatus involved in the fifth method, the surface change information includes at least one of the following: the type, shape, and location of the surface change.

[0018] In the image analysis device involved in the sixth method, the processor infers the cause of the temperature change based on the similarity between temperature change information and surface change information.

[0019] In the image analysis apparatus involved in the seventh method, when the processor infers that the cause of the temperature change is a surface change, it infers the temperature distribution based on the surface change and reduces it from the infrared thermal image.

[0020] In the image analysis apparatus involved in the eighth method, similarity includes partial similarity.

[0021] In the image analysis apparatus of the ninth method, the processor performs the following processing: determining the surface change corresponding to the temperature change as a crack or peeling; evaluating the similarity between the temperature change information and the surface change information; and when at least some of them are similar, inferring that the cause of the temperature change is the lifting that accompanies the crack or peeling.

[0022] In the image analysis apparatus of the tenth method, the processor performs the following processing: determines the surface change corresponding to the temperature change as a crack or peeling; evaluates whether there is a surface change in the magnitude of the temperature change and / or near the boundary of the temperature change; and if there is a surface change, infers that the cause of the temperature change is the lifting accompanying the crack or peeling.

[0023] In the image analysis apparatus involved in the eleventh method, the processor infers the cause of the temperature change based on the temperature slope of the boundary of the temperature change.

[0024] In the image analysis apparatus involved in the twelfth method, surface changes include at least one of repair marks, free lime, seams, steps, cracks, and peeling.

[0025] In the image analysis apparatus of the thirteenth method, the visual image is an image obtained by imagerizing the reflection intensity distribution of two or more different wavelength ranges in the wavelength range of visible light.

[0026] The image analysis device involved in the fourteenth method also includes a display device, on which the processor displays the inference results of the cause of the temperature change.

[0027] The image analysis method involved in the fifteenth method includes: the step of acquiring an infrared thermal image of the structure of the object to be inspected; the step of acquiring a visual image of the structure of the object to be inspected; the step of determining temperature changes based on the infrared thermal image; and the step of inferring the cause of temperature changes based at least on temperature change information obtained from the infrared thermal image and surface change information obtained from the visual image.

[0028] The sixteenth method involves a program for executing a computer that performs the following steps: acquiring an infrared thermal image of the structure of the object to be inspected; acquiring a visual image of the structure of the object to be inspected; determining a temperature change based on the infrared thermal image; and inferring the cause of the temperature change based at least on temperature change information obtained from the infrared thermal image and surface change information obtained from the visual image.

[0029] Invention Effects

[0030] The image analysis apparatus, image analysis method, and program according to the present invention can reduce false detections of defects. Attached Figure Description

[0031] Figure 1 This is a block diagram illustrating an example of the hardware structure of an image analysis device.

[0032] Figure 2 It is a block diagram representing the processing functions implemented by the CPU.

[0033] Figure 3 It is a diagram representing the information stored in the storage unit.

[0034] Figure 4 This is a flowchart illustrating an analysis method using an image analysis device.

[0035] Figure 5 It refers to infrared thermal images and visual images obtained by photographing structures covered with rust liquid.

[0036] Figure 6 Using binary representation relative to Figure 5 The image of the changing shape of the image.

[0037] Figure 7 It refers to infrared thermal images and visual images of structures, including the repaired parts.

[0038] Figure 8 Using binary representation relative to Figure 7 The image of the changing shape of the image.

[0039] Figure 9 It refers to infrared thermal images and visual images obtained by capturing structures, including the stripped-off parts.

[0040] Figure 10 It consists of infrared thermal images and visual images of a structure including two detached parts.

[0041] Figure 11 Using binary representation relative to Figure 9 The image of the changing shape of the image.

[0042] Figure 12 Using binary representation relative to Figure 10 The image of the changing shape of the image.

[0043] Figure 13 It refers to infrared thermal images and visual images obtained by taking pictures of structures, including those with cracks.

[0044] Figure 14 It is an infrared thermal image and visual image obtained by taking pictures of a structure, including other cracked parts.

[0045] Figure 15 Using binary representation relative to Figure 13 The image of the changing shape of the image.

[0046] Figure 16 Using binary representation relative to Figure 14 The image of the changing shape of the image.

[0047] Figure 17 This is a diagram illustrating an example of displaying the inference result on a display device. Detailed Implementation

[0048] The image analysis apparatus involved in the implementation is an image analysis apparatus equipped with a processor. The processor acquires an infrared thermal image of the structure of the object to be inspected, acquires a visual image of the structure of the object to be inspected, determines temperature changes based on the infrared thermal image, derives temperature change information, derives surface change information from the surface changes of the visual image of the part corresponding to the temperature change, and infers the cause of the temperature change based on the temperature change information and the surface change information.

[0049] Regarding the invention involved in the implementation of the above-described structure, the inventors conducted in-depth research on reducing false detections of defective parts, and as a result, discovered the following insights, leading to the present invention.

[0050] To reduce false detections, it is necessary to determine the impact of identified surface changes (hereinafter also referred to as surface variations) on infrared thermal images (the cause of anomalies). Therefore, analyzing the relationship between anomalies extracted from infrared thermal images and the location or shape of surface changes identified from visual images is indispensable. The inventors conducted a comparative investigation of infrared thermal images and visual images of concrete structures and found that the relationship between surface changes (uniform color, joints, steps, rust, etc.) identified in visual images and temperature changes identified in infrared thermal images (hereinafter, the portion of the surface temperature in the infrared thermal image that differs from the surrounding area will be referred to as temperature change) varies depending on the thermal environment, such as the location within the structure or the time of photography. In other words, the impact of surface changes on surface temperature varies depending on the thermal environment, such as the location within the structure or the time of photography.

[0051] For example, when photographing the railings of a bridge on a clear day, the location and shape of color unevenness or rust as identified by the visual image and the temperature change as identified by the infrared thermal image are clearly consistent. That is, the color unevenness or rust significantly affects the surface temperature. On the other hand, when photographing the underside of the bridge deck of the same bridge at the same time, the same color unevenness or rust as the railing is visible in the visual image, but sometimes the corresponding temperature change is not visible in the infrared thermal image. The inventors believe that the reason for this difference is that on a clear day, there is solar radiation on the railing, and the amount absorbed differs between the parts with color unevenness or rust and other parts. Thus, the inventors have found that since the effect of surface changes on surface temperature varies depending on the thermal environment, it is necessary to determine its effect (determine the cause of the temperature change). Therefore, analyzing the relationship between the location or shape of surface changes and temperature changes is indispensable.

[0052] Hereinafter, preferred embodiments of the image analysis apparatus, image analysis method and program involved in the present invention will be described with reference to the accompanying drawings.

[0053] [Hardware Structure of the Image Analysis Device]

[0054] Figure 1 This is a block diagram illustrating an example of the hardware structure of an image analysis device according to an embodiment.

[0055] As Figure 1The image analysis device 10 shown can be a computer or workstation. In this example, the image analysis device 10 mainly consists of an input / output interface 12, a storage unit 16, an operation unit 18, a CPU (Central Processing Unit) 20, RAM (Random Access Memory) 22, ROM (Read Only Memory) 24, and a display control unit 26. A display device 30 is connected to the image analysis device 10, and under the instructions of the CPU 20 and the control of the display control unit 26, the image is displayed on the display device 30. The display device 30 is, for example, a monitor.

[0056] The input / output interface 12 (I / F in the figure) can input various data (information) to the image analysis device 10. For example, data stored in the storage unit 16 can be input via the input / output interface 12.

[0057] The CPU (processor) 20 reads various programs stored in the storage unit 16 or ROM 24, expands them in the RAM 22, and performs calculations, thereby providing unified control over all parts. Additionally, the CPU 20 reads programs stored in the storage unit 16 or ROM 24, uses the RAM 22 to perform calculations, and performs various processing operations on the image analysis device 10.

[0058] Figure 1 The infrared camera 32 shown captures an infrared thermal image of the structure 36, which is the object of inspection. The visual camera 34 captures a visual image of the structure 36, which is the object of inspection.

[0059] The image analysis device 10 can acquire infrared thermal images from the infrared camera device 32 via the input / output interface 12. Additionally, the image analysis device 10 can acquire visual images from the visual camera device 34 via the input / output interface 12. The acquired infrared thermal images and visual images can be stored, for example, in the storage unit 16.

[0060] Figure 2 This is a block diagram representing the processing functions implemented by CPU20.

[0061] CPU 20 includes an infrared thermal image acquisition unit 51, a visual image acquisition unit 53, a temperature change information export unit 55, a surface change information export unit 57, a cause inference unit 59, and an information display unit 61. The specific processing functions of each unit will be explained later. Since the infrared thermal image acquisition unit 51, visual image acquisition unit 53, temperature change information export unit 55, surface change information export unit 57, cause inference unit 59, and information display unit 61 are part of CPU 20, it can also be said that CPU 20 executes the processing of each unit.

[0062] Back Figure 1 The storage unit 16 stores data and programs that enable the image analysis device 10 to operate, such as the operating system and programs that execute image analysis methods. Additionally, the storage unit 16 stores information used in the embodiments described below.

[0063] Figure 3 This diagram represents information stored in the storage unit 16. The storage unit 16 is a memory composed of various semiconductor memories such as CD (Compact Disk), DVD (Digital Versatile Disk), Hard Disk, and Flash Memory.

[0064] The storage unit 16 mainly stores infrared thermal images 101, temperature change information 102, visual images 103, and surface change information 104.

[0065] Infrared thermal image 101 is an image captured by infrared camera device 32. It is an image that detects the infrared radiation energy emitted from structure 36 and converts the infrared radiation energy into temperature to represent the temperature distribution on the surface of the structure.

[0066] Temperature change information 102 is information about temperature changes, which is the temperature distribution of the infrared thermal image 101 and / or information obtained from the temperature distribution.

[0067] Visual image 103 is an image captured by visual imaging device 34, representing the distribution of visible light reflection intensity from the surface of structure 36. Typically, a visual image is composed of an RGB image obtained by imagerizing the reflection intensity distribution in three different wavelength ranges within the visible light wavelength range; that is, each pixel has color information (RGB signal value). In this example, the structure is also assumed to have color information. Furthermore, the brightness of visual image 103, described later, represents the signal value of visual image 103, and the brightness of each pixel in visual image 103 reflects the visible light reflection intensity at the corresponding position on the surface of structure 36.

[0068] In addition, the surface change information 104 is information on the surface changes at the location corresponding to the temperature change in the visible image 103, and is the brightness distribution of the visible image 103 and / or information obtained from the brightness distribution.

[0069] Figure 1 The operation unit 18 shown includes a keyboard and mouse, which allow the user to perform necessary processing on the image analysis device 10. The display device 30 can function as an operation unit by using a touch panel type device.

[0070] The display device 30 is, for example, a liquid crystal display or other device, capable of displaying the results obtained by the image analysis device 10.

[0071] Figure 4 This is a flowchart illustrating an image analysis method using the image analysis device 10. For example... Figure 4 As shown, the image analysis method includes an infrared thermal image acquisition step (step S1), a visual image acquisition step (step S2), a temperature change information export step (step S3), a surface change information export step (step S4), a cause inference step (step S5), and an inference result display step (step S6).

[0072] <Steps for acquiring infrared thermal images>

[0073] The infrared thermal image acquisition unit 51 acquires an infrared thermal image of the structure 36 of the object being inspected (infrared thermal image acquisition step: step S1). The infrared thermal image is an infrared thermal image 101 stored in the storage unit 16. The infrared thermal image 101 is acquired by the infrared thermal image acquisition unit 51 from the storage unit 16. Furthermore, if the storage unit 16 does not store the infrared thermal image 101, the infrared thermal image acquisition unit 51 acquires the infrared thermal image 101 from an external source. For example, the infrared thermal image acquisition unit 51 can acquire the infrared thermal image 101 via a network through the input / output interface 12; alternatively, the infrared thermal image acquisition unit 51 can acquire the infrared thermal image 101 from the infrared camera device 32 via the input / output interface 12.

[0074] <Steps for obtaining a visual image>

[0075] The visual image acquisition unit 53 acquires a visual image of the structure 36 of the object being inspected (visual image acquisition step: step S2). The visual image is a visual image 103 stored in the storage unit 16. The visual image 103 is acquired by the visual image acquisition unit 53 from the storage unit 16. Furthermore, if the storage unit 16 does not store the visual image 103, the visual image acquisition unit 53 acquires the visual image 103 from an external source. For example, the visual image acquisition unit 53 can acquire the visual image 103 via a network through the input / output interface 12, or it can acquire the visual image 103 from the visual camera device 34 through the input / output interface 12.

[0076] <Steps for exporting temperature change information>

[0077] Next, the temperature change information export unit 55 determines the temperature change based on the infrared thermal image 101 and exports the temperature change information 102 (temperature change information export step: step S3).

[0078] The temperature change information exporting unit 55, based on the infrared thermal image 101, identifies and extracts the parts of the surface temperature that are different from the surrounding area as temperature changes, and exports the temperature change information 102.

[0079] For example, a portion of the surface of a structure in the infrared thermal image 101 that exceeds a temperature difference specified by the average temperature (a portion where the temperature is higher than the surroundings when the temperature rises, like during the day, or a portion where the temperature is lower than the surroundings when the temperature falls, like at night) can be identified as a temperature change and extracted. A portion of the surface temperature that is different from the surroundings and is spatially uniformly connected, or even if not connected, that is distributed at a distance closer than specified, can be identified as a temperature change. Furthermore, a temperature change is not necessarily a portion where the actual surface temperature differs from the surroundings. That is, even if the actual surface temperature is the same as the surroundings, the infrared emissivity may differ, thus sometimes a surface temperature that differs from the surroundings in the infrared thermal image 101 can also be identified as a temperature change.

[0080] Here, the infrared thermal image 101 can also be the infrared thermal image of the concrete structure 36 of the object captured by the infrared camera device 32. Alternatively, to facilitate the determination of temperature changes and / or the extraction of temperature change information, it can be a processed version of the original infrared thermal image 101. For example, a temperature gradient often occurs on the surface of the structure 36 because the amount of heat received from or dissipated from the surface of the structure 36 is partially different. Therefore, it is also possible to process the original infrared thermal image 101 to reduce the temperature gradient, determine and extract temperature changes based on the processed image, and extract temperature change information 102.

[0081] Temperature change information 102 is information representing the temperature distribution (spatial distribution of temperature) of the temperature change, and it represents the temperature distribution over a range that includes at least the entire temperature change (including the range up to the end if the temperature change extends to the end of the infrared thermal image 101). In the cause inference step (step S5) described later, it is preferable to represent the temperature distribution over the largest possible range in order to infer the cause.

[0082] The temperature change information 102 can be the temperature distribution itself in the original infrared thermal image 101, or it can be a distribution obtained by coarsely quantizing the original temperature distribution, such as a binarized, ternary, or quaternary distribution. The temperature change information 102 can also be information representing the shape of the temperature change. For example, a binarized temperature distribution can be said to represent the shape of the temperature change. The temperature change information can also represent the magnitude of the temperature change when approximated by rectangles or ellipses.

[0083] Therefore, the temperature change information 102 regarding temperature change is the temperature distribution of the infrared thermal image 101 and / or information obtained from that temperature distribution.

[0084] The temperature change information export unit 55 can determine multiple temperature changes (i = 1, 2, 3, ... N) and export temperature change information 102.

[0085] <Steps for exporting surface change information>

[0086] The surface change information exporting unit 57 exports surface change information 104 based on the visual image 103 (surface change information exporting step: step S4).

[0087] The surface change information export unit 57 exports surface change information 104 corresponding to the temperature changes (i = 1, 2, 3, ... N) from the visible image 103. A change can mean "a state that has changed from its initial state" or "a state different from the ordinary state," but in this embodiment, the state of a surface that affects the temperature of the concrete surface in the infrared thermal image and can be a cause of temperature change, such as repair marks, attachment of foreign matter like loose lime, uneven color (mold, moss, stripping agent, water effects, etc.), joints, steps, scum, sand reinforcement, rust liquid or rust, leakage, surface unevenness, and honeycomb panels, is specifically referred to as a "surface change." Additionally, cracks or peeling are also referred to as "surface changes."

[0088] Surface change information 104 indicates the presence, type, shape, and location of surface changes, and is obtained from the brightness distribution (spatial distribution of brightness) of the visual image 103. Furthermore, the brightness distribution in the visual image 103, representing surface changes such as color inhomogeneity, rust, or rust liquid, is effective for inferring the cause of temperature changes.

[0089] The brightness of each pixel in the visual image 103 reflects the intensity of visible light reflected at the corresponding position on the surface of the structure 36. Furthermore, the difference in brightness reflects the difference in the intensity of visible light reflected relative to the surface of the structure 36, i.e., the difference in reflectivity. That is, it reflects the difference in the absorption rate of visible light. Therefore, the brightness distribution of the surface of the structure 36 in the visual image 103 reflects the distribution of the absorption rate of visible light, such as sunlight, illuminating the surface of the structure 36. Similarly, the brightness distribution of each surface variation on the surface of the structure 36 reflects the distribution of the absorption of visible light during that surface variation, thus making the inference of the cause of temperature changes valid. Therefore, the surface change information 104 can be information representing the brightness distribution of the surface variation, or it can be the brightness distribution of the visual image 103 itself, or it can be a distribution obtained by coarsely quantizing that brightness distribution, such as a binarized, ternary, or quaternary distribution.

[0090] Therefore, the surface change information 104 is information about the surface changes at the location corresponding to the temperature change in the visible image 103, and is the brightness distribution of the visible image 103 and / or information obtained from the brightness distribution.

[0091] The brightness distribution of the part corresponding to the temperature change (the brightness distribution of the visual image 103 itself, or the distribution obtained by coarse quantization of the brightness distribution) is information (included information) indicating the presence, type, shape, location and brightness distribution of the surface change of that part. The brightness distribution can also be used as surface change information 104.

[0092] When at least one of the following surface change information 104—presence, type, shape, location, and brightness distribution—is explicitly derived from the brightness distribution of the area corresponding to the temperature change, cracks and spalling are distinguished from other surface changes (the state of the surface that affects the temperature of the concrete surface in the infrared thermal image 101 and may be a cause of false detection of internal defects in the structure 36) and extracted. From the visible image 103, portions with a brightness difference exceeding a specified value compared to the surrounding area and forming a line of length exceeding a specified value are identified as cracks and extracted. Various methods using machine learning and methods focusing on line shape and extracting based on its features have been proposed for crack extraction; any method can be used.

[0093] The peeled area has a clearly different brightness compared to the original concrete surface, and the texture, contrast, and spectrum of the brightness distribution are also different. Therefore, areas that exceed the specified differences in brightness, brightness distribution texture, contrast, and spectrum compared to the surrounding area are uniformly identified as peeled and extracted. Since there are steps between the peeled area and the original concrete surface, and the stepped parts are darker (lower brightness), the darkness (degree of low brightness) of the boundary area can also be a valid feature for determining peeling. In addition, as other surface changes, areas with different brightness and / or different brightness distribution texture and / or different brightness distribution spectrum and / or different brightness distribution contrast compared to the surrounding area are locally and uniformly identified and extracted based on the visible image 103. For example, areas with brightness exceeding the specified brightness difference from the average are identified as other surface changes and extracted. Areas that are different from the surrounding area in any one or more of brightness, brightness distribution texture, spectrum, and contrast, and are spatially uniformly connected, or even if not connected, are distributed at a distance closer than specified, are identified as a surface change and extracted.

[0094] Through the above processing, for each of the cracks, peeling, and other surface changes, at least one of the following is obtained: presence, shape, location, and brightness distribution. Here, for other surface changes, in the cause deduction step (step S5) described later, it is not necessarily necessary to carefully determine the type of surface change (repair marks, loose lime, uneven color, joints, steps, scum, sand veins, rust liquid or rust, leakage, surface unevenness, honeycomb panels, etc.). However, the party that has determined the type of surface change can more appropriately perform cause deduction in the cause deduction step (step S5).

[0095] Specifically, for surface changes such as uneven color, rust liquid or rust, etc., which are different from the surrounding concrete in terms of the amount of visible light absorbed, the surface temperature in the infrared thermal image 101 is mainly due to the difference between the surface temperature and the surrounding environment. As mentioned above, the brightness distribution reflects the distribution of the amount of visible light absorbed. Therefore, the brightness distribution, as surface change information 104, is valid for cause inference in the cause inference step (step S5).

[0096] On the other hand, the following applies to surface changes where the brightness distribution is ineffective for cause inference, i.e., surface changes where the brightness distribution is more like noise. For surface changes where the difference in thermal conductivity or infrared emissivity between the surface and the surrounding concrete, such as repair marks or free lime, is the main cause of the difference in surface temperature between the surface and the surroundings in the infrared thermal image 101, or surface changes where the difference in surface temperature between the surface and the surroundings in the infrared thermal image 101 is caused by structural reasons such as joints or steps, the shape, as surface change information 104, is valid for cause inference in the cause inference step (step S5). Therefore, for other surface changes, it is preferable to carefully determine the type. Based on the extracted features of the surface change, such as average brightness, contrast, brightness dispersion, texture, spectrum, and shape, its type can be carefully determined.

[0097] Here, the visible image 103 is typically composed of RGB images obtained by imagerizing the reflection intensity distributions of three different wavelength ranges within the visible light wavelength range. Cracks, peeling, and other surface changes can also be determined and extracted based on any of the RGB brightness distributions. However, since the brightness difference (difference in reflection intensity) between surface changes such as rust or rust and concrete differs significantly according to RGB, it is preferable to determine and extract based on the brightness distribution of the channel with the largest difference. For example, in the case of rust or rust, the brightness of B is particularly low compared to concrete, meaning the difference in reflection intensity across the wavelength range of B is large, thus it is preferable to determine based on the brightness distribution of B. However, the channel with the largest brightness difference from concrete is considered to vary depending on the type of surface change. For example, it is also considered that, unlike rust or rust, the brightness difference of the R channel becomes particularly large due to different types of color inhomogeneity. Therefore, it is preferable to evaluate the magnitude of the brightness deviation in the brightness distributions of each RGB channel and determine and extract cracks, peeling, and other surface changes based on the channel with the largest deviation. For example, the standard deviation of the brightness at the location corresponding to the temperature change in the brightness distribution of each RGB channel can be divided by the average brightness of the concrete, i.e., the variation coefficient. Cracks, peeling, and other surface changes can be identified and extracted based on the channel with the largest variation coefficient. The average brightness of the concrete can be the average brightness at the location corresponding to the temperature change, or it can be the average brightness over a larger range that includes the location corresponding to the temperature change. For other surface changes, when carefully identifying their type, they can be determined based on features such as the average brightness, contrast, brightness dispersion, texture, and spectrum of each RGB channel of the extracted surface changes. When the visible image 103 has two or more types, cracks, peeling, and other surface changes are also identified and extracted using the channel with the largest deviation. In addition, the type of other surface changes can be carefully identified based on features such as the average brightness and contrast of each of the two or more channels.

[0098] Here, the area corresponding to the temperature change is the area larger than the spatial range corresponding to the temperature change, as determined by the infrared thermal image 101 in the temperature change information derivation step (step S3). As described later, in the cause inference step (step S5), the surface change information of the area larger than the spatial range corresponding to the temperature change is required in the analysis of the cause inference of the temperature change. In particular, for peeling, it is necessary to analyze the relationship between the temperature change and peeling at different locations.

[0099] Furthermore, in the above description, the steps of infrared thermal image acquisition (step S1), visual image acquisition (step S2), temperature change information derivation (step S3), and surface change information derivation (step S4) have been described in that order, but their order may be appropriately changed within the scope of the implementation.

[0100] <Steps for Deducing the Cause>

[0101] The cause inference unit 59 infers the cause of the temperature change based on the temperature change information 102 and the surface change information 104 (cause inference step: step S5).

[0102] Furthermore, since the processing for inferring the cause differs slightly depending on the type of surface change information 104 exported in the surface change information export step (step S4), it will be described as a first method, a second method, and a third method respectively.

[0103] <First Method>

[0104] As a first approach, the case where the brightness distribution of the region corresponding to the temperature change is used as surface change information in the surface change information derivation step (step S4) will be explained. For each of cracks and peeling, and other surface changes, a method for inferring the cause will be explained.

[0105] (Other surface changes)

[0106] Extract the range of temperature changes that matches or includes a slightly larger range of temperature changes from the brightness distribution derived in the temperature change information derived in the temperature change information derived in the surface change information extraction step (step S3). Evaluate the similarity between the brightness distribution and the temperature distribution within this range. If they are similar, the cause of the temperature change is inferred to be the surface change. On the other hand, if they are dissimilar, the cause is inferred to be something other than the surface change, such as an internal defect like a float.

[0107] There are many methods to evaluate the similarity between two distributions. For example, the distributions can be standardized so that the minimum and maximum values ​​of the brightness and temperature distributions are the same. Then, the similarity can be evaluated by calculating the Euclidean distance given by the following equation (1). For example, equation (1) can also be calculated, and when the distance is below a specified value (the closer it is to 0, the more similar it is), it is considered similar.

[0108] Furthermore, since the relationship between brightness and temperature can sometimes be the same and sometimes opposite, depending on the type of surface change or the time of shooting, it is necessary to calculate both scenarios. That is, the Euclidean distance is calculated for both the case where the brightness distribution remains the same and the distribution where the brightness levels are reversed (e.g., the distribution obtained by subtracting the original brightness value from 255). The distributions are considered similar when the distance falls below a specified value.

[0109] sqrt((v(1,1)-t(1,1)) 2 +(v(2,1)-t(2,1)) 2 +……) Equation (1)

[0110] Here, v(x, y) represents the pixel value at coordinate (x, y) in the normalized brightness distribution, and t(x, y) represents the pixel value at coordinate (x, y) in the normalized temperature distribution.

[0111] Alternatively, as another method for evaluating similarity, one could calculate the Pearson product-moment correlation coefficient given by equation (2). If the absolute value of the correlation coefficient is above a specified value (the closer to 1, the more similar the similarity), then similarity is considered. Here, by calculating the absolute value of the correlation coefficient, similarity can be evaluated whether the relationship between brightness and temperature is the same or opposite.

[0112] ∑(v(x,y)-v_ave)*(t(x,y)-t_ave) / sqrt(∑(v(x,y)-v_ave) 2 ) / sqrt(∑(t(x,y)-t_ave) 2 Equation (2)

[0113] Here, v(x, y) represents the pixel value at coordinate (x, y) in the brightness distribution, v_ave represents the average value of the brightness distribution, t(x, y) represents the pixel value at coordinate (x, y) in the temperature distribution, and t_ave represents the average value of the temperature distribution.

[0114] Figure 5 These are images of concrete structures taken during the day. Figure 5 (A) is a visual image. Figure 5 (B) is an infrared thermal image. Furthermore, in Figure 5 In the photograph, rust liquid adhered to the surface of the concrete structure of the object being inspected.

[0115] In this example, in the temperature change information extraction step (step S3), based on the infrared thermal image ( Figure 5 (B) was determined to be a temperature change where the surface temperature was higher than the surrounding temperature and was extracted. A visual image of the rust fluid reflected in the area corresponding to the temperature change is shown. Figure 5 In (A), since its brightness distribution and temperature distribution of temperature change are similar (the relationship between brightness and temperature is opposite), it is inferred that the cause of temperature change is the surface change (rust liquid).

[0116] Figure 6 Use binary representation relative to Figure 5 The image shows the changing shape, with 1 (white) representing pixels that have changed and 0 (black) representing pixels that have not changed. Figure 6 (A) indicates that the image is from a visual image ( Figure 5 (A)) The shape of the surface changes (rust solution) derived from the surface changes. Figure 6 (B) indicates that the infrared thermal image ( Figure 5 (B) The shape of the temperature change derived. Compare. Figure 6 (A) and Figure 6 (B) is judged to be similar.

[0117] Figure 7 These are images of concrete structures taken during the day. Figure 7 (A) is a visual image. Figure 7 (B) is an infrared thermal image. The surface of the concrete structure of the inspected object contains repaired areas.

[0118] In this example, in the temperature change information extraction step (step S3), based on the infrared thermal image ( Figure 7 (B) is used to determine and extract temperature changes where the surface temperature is higher than the surrounding temperature. Repair marks are reflected in visual images of the areas corresponding to these temperature changes. Figure 7 In (A), since its brightness distribution and temperature distribution of temperature change are similar (the relationship between brightness and temperature is the same), it is inferred that the cause of temperature change is the surface change (repair marks).

[0119] Figure 8 Use binary representation relative to Figure 7 The changing shape of the image is represented by 1 (white) for pixels that have changed and 0 (black) for pixels that have not changed, similar to the above. Figure 8 (A) is from a visual image ( Figure 7 (A) The shape of the surface changes (repair marks) derived from the data. Figure 8 (B) is from infrared thermal images ( Figure 7 (B) The shape of the temperature change derived. Compare. Figure 8 (A) and Figure 8 (B) is judged to be similar.

[0120] exist Figure 7 In the infrared thermographic image of the repair trace in (B), it can be seen that the temperature slope at the boundary of the temperature change is relatively steep. Thus, the characteristic of temperature changes caused by surface changes is that the temperature slope is relatively steep at its boundary (however, this depends on the type of surface change). In the case of internal defects such as buoyancy, because heat diffuses between the internal defect and the surface, the temperature slope at the boundary of the temperature change caused by the internal defect is gentler. The deeper the internal defect, i.e., the wider the gap between the internal defect and the surface, the more heat diffuses, and therefore the gentler the temperature slope becomes.

[0121] However, in the case of surface changes, because the surface temperature differs from the surrounding environment due to differences in the heat absorption rate, thermal conductivity, total infrared emissivity, etc., of the surface change, the slope of this surface temperature difference is steeper than in the case of internal defects. Therefore, the cause of the temperature change can also be inferred based on this characteristic (the steeper temperature slope at the boundary of the temperature change).

[0122] Specifically, if the temperature distribution and brightness distribution are similar within the range of temperature changes derived in the temperature change information derivation step (step S3), and / or the temperature slope at the boundary of the temperature change is above a specified value (pre-set threshold), the cause of the temperature change can also be inferred to be surface change.

[0123] The slope of temperature at the boundary of temperature change can be used as a gradient vector of f. For example, if the temperature distribution is f(x, y), the maximum slope at each point on the boundary of temperature change can be calculated. Size The average of the maximum slope at each point on the boundary can be obtained. For example, equation (3) can be calculated at each point (x, y) on the boundary, and its average value can be obtained.

[0124] sqrt((f(x+1,y)-f(x,y)) 2 +(f(x,y+1)-f(x,y)) 2 Equation (3)

[0125] Furthermore, the slope of temperature at the boundary of temperature change varies significantly depending on thermal environments such as solar radiation or external air temperature. Therefore, it is also possible to calculate the absolute value of the difference between the temperature change and the surrounding temperature, or the absolute value of the difference between the temperature change and the average temperature of the structure's surface within a large specified range that includes the temperature change, and then standardize the slope of temperature at the boundary by using the maximum or average value of this absolute value (e.g., by division). If the standardized slope of temperature is above a specified value, the cause of the temperature change is inferred to be surface change.

[0126] When evaluating the similarity between temperature distribution and brightness distribution under temperature changes, the feature of a steeper temperature slope at the boundary of temperature changes caused by surface variations is also included in the evaluation. However, this feature can also be evaluated as described above. Furthermore, the method described above is an example of a method for evaluating the slope of temperature, and the method is not limited to the method described above. For example, the slope of temperature can also be indirectly evaluated using the spatial second derivative of temperature. The slope of temperature can also be evaluated using any method.

[0127] This feature is particularly prominent in "repair marks", "seams", and "steps", and is effective for inferring causes.

[0128] (stripping)

[0129] Figure 9 and Figure 10 These are images of concrete structures taken during the day. Figure 9 (A) is a visual image. Figure 9 (B) is an infrared thermal image. Additionally, Figure 10 (A) is a visual image. Figure 10 (B) is an infrared thermal image. Figure 9 and Figure 10 In the photograph, the concrete structure of the object being inspected contains areas of peeling.

[0130] like Figure 9 As shown in the visual image (A), it can be understood that peeling occurred in the upper right region, exhibiting a step-like appearance. Additionally, as... Figure 10 As shown in the visual image (A), it can be understood that there are peeling areas on the right side and peeling areas on the left side, which are small but also have steps.

[0131] from Figure 9 (B) and Figure 10 (B) The infrared thermal image shows a region adjacent to the detached area with a high surface temperature (a lighter color in the infrared thermal image). This region can be considered to have entered the air from the detached area and become floating. Thus, at the detached area, many regions were observed to have entered the air and floated up.

[0132] Based on the temperature distribution derived in the temperature change information deriving step (step S3) and the brightness distribution derived in the surface change information deriving step (step S4), it can be determined that there is a lift accompanied by peeling (or peeling accompanied by lift). Specifically, the similarity of the temperature distribution and brightness distribution is evaluated in both the overall and partial temperature change. If the overall distribution is dissimilar but the partial distribution is similar, it is determined to be a lift accompanied by peeling.

[0133] The overall similarity of temperature changes is evaluated by extracting a range from the brightness distribution that corresponds to the temperature change and assessing the similarity between the brightness distribution and the temperature distribution within that range. The similarity of partial temperature changes is evaluated by extracting a defined range centered on each point on the boundary of the temperature change from the brightness distribution based on the shape of the temperature change and assessing the similarity between the brightness distribution and the temperature distribution within each range. If any part is similar, it is determined to be a lift accompanied by peeling. Since the boundaries (steps) of peeling areas in the brightness distribution and the boundaries of temperature changes in the temperature distribution have the same shape, this evaluation is possible. Furthermore, as already explained, the similarity evaluation covers both cases where the relationship between the brightness level of the brightness distribution and the temperature level of the temperature distribution is the same and cases where it is opposite.

[0134] from Figure 9 and Figure 10 Infrared thermal images show that the surface temperature at the peeled-off area is lower than the surrounding area. Conversely, when the same area is photographed at night, the surface temperature at the peeled-off area is higher than the surrounding area. The reason for this difference in surface temperature at the peeled-off area is that it is located further inward. Thus, because the surface temperature at the peeled-off area is different from the surrounding area, it is determined to be a temperature change and extracted. Regarding temperature changes caused by peeling, they can be distinguished from those caused by internal defects such as lifting or other surface changes, as their temperature relationship with the surrounding area differs. Furthermore, because the brightness distribution of the peeled-off area coincides with the area that differs from the surrounding area in the temperature distribution, the overall temperature distribution and brightness distribution of the temperature change are clearly similar. Therefore, peeling and other surface changes can be distinguished based on the characteristic brightness distribution of the dark areas (low-brightness areas) caused by the steps, making it easy to determine that the cause of the temperature change is peeling. It should be noted that in the case of lifting accompanied by peeling, the temperature change and peeling are adjacent but in different locations.

[0135] As from Figure 9 and Figure 10 As understood, the phenomenon of buoyancy accompanying peeling, like other surface changes, is characterized by a steeper temperature slope at the boundary of the temperature change, a feature that can be used to infer the cause. However, in the case of buoyancy accompanying peeling, within the boundary of the temperature change, only the portion adjacent to the peeling has a steeper temperature slope, while the other portions exhibit a gentler temperature slope, similar to typical buoyancy. Therefore, in the case of buoyancy accompanying peeling, if the overall temperature slope of the boundary of the temperature change is gentle, but a portion of the boundary has a steeper temperature slope, it can be determined as buoyancy accompanying peeling.

[0136] Specifically, it can also be done by evaluating the similarity of the temperature distribution derived in the temperature change information deriving step (step S3) and the brightness distribution derived in the surface change information deriving step (step S4) at both the overall and partial temperature change levels. If the overall distributions are dissimilar but the partial distributions are similar, the slope of the temperature distribution at the boundary of the temperature change is evaluated at both the overall and partial levels. If the overall slope is below a specified value while the partial slope is above a specified value, the cause of the temperature change is inferred to be buoyancy accompanying peeling. The slope of the overall temperature at the boundary of the temperature change is determined similarly to other surface changes. At each point on the boundary of the temperature change, the maximum slope, i.e., the magnitude of the gradient vector of the temperature distribution, is calculated, and its average value (the average of the maximum slopes at all points on the boundary of the temperature change) is calculated. For the slope of the temperature at the boundary of the temperature change, a specified range centered on each point on the boundary is extracted, and the average of the maximum slopes at all points on the boundary within the extracted specified range is calculated.

[0137] Here, in the temperature distribution of temperature changes, the portions similar to the brightness distribution and the portions with steep temperature slopes at the boundaries are the same. That is, in the temperature distribution of temperature changes, in the portions adjacent to the peeling, the temperature distribution is similar to the brightness distribution, and the temperature slope at the boundaries is steep. Therefore, the preferred method is to simultaneously evaluate the similarity to the brightness distribution and the temperature slope for each portion of the temperature change, and in cases where there are portions similar to the brightness distribution and with steep temperature slopes, infer that the cause of the temperature change is buoyancy accompanying the peeling.

[0138] Furthermore, the lifting associated with peeling, like other surface changes, can be normalized by the absolute difference between the temperature change and the surrounding temperature, or the absolute difference between the temperature change and the average temperature of the larger structure surface. Similarly, in evaluating the similarity to brightness distribution, the steep temperature slope at the boundary is also included in the evaluation of the lifting associated with peeling, but the temperature slope can also be evaluated as described above.

[0139] Figure 11 Use binary representation relative to Figure 9 The changing shape of the image Figure 12 Use binary representation relative to Figure 10 The changing shape of the image. Pixels that have changed are represented by 1 (white), and pixels that have not changed are represented by 0 (black). Figure 11 (A) indicates that the image is from a visual image ( Figure 9 (A)) The shape of the surface change (peeling) derived from it. Figure 11 (B) indicates that the infrared thermal image ( Figure 9(B) The shape of the temperature change derived from it.

[0140] like Figure 11 As shown, comparison Figure 11 (A) and Figure 11 (B) is not similar overall, but in the upper right part of the temperature change, it is adjacent to the peeling. In the part adjacent to the peeling, the shape of the temperature change and the peeling boundary is similar, so it can be inferred that it is a buoyancy accompanied by peeling.

[0141] Figure 12 (A) indicates that the image is from a visual image ( Figure 10 (A)) The shape of the surface change (peeling) derived from it. Figure 12 (B) indicates that the infrared thermal image ( Figure 10 (B) The shape of the temperature change derived from it.

[0142] like Figure 12 As shown, comparison Figure 12 (A) and Figure 12 (B) In addition to the peeling on the right, there is also small peeling on the left, all of which are adjacent to temperature changes. In the adjacent parts, due to the similarity of temperature changes and shapes, the temperature changes in both places can be inferred to be floating accompanied by peeling.

[0143] (crack)

[0144] Figure 13 and Figure 14 These are images of concrete structures taken during the day. Figure 13 (A) is a visual image. Figure 13 (B) is an infrared thermal image. Additionally, Figure 14 (A) is a visual image. Figure 14 (B) is an infrared thermal image. Figure 13 and Figure 14 The concrete structure being inspected contained areas with cracks on its surface.

[0145] from Figure 13 The visual image in (A) can be understood as the crack entering laterally. Additionally, from... Figure 13 The infrared thermal image in (B) can be interpreted as showing a higher surface temperature (a lighter-colored area in the infrared thermal image) on the upper side of the crack. This higher-temperature area is in a floating state. Figure 13 As shown, cracks are often observed accompanying the floating, and in most cases, cracks are present along the boundary of the floating structure. Additionally, as... Figure 14 As shown, cracks were also observed within the area of ​​the floating region.

[0146] Based on the temperature distribution derived in the temperature change information deriving step (step S3) and the brightness distribution derived in the surface change information deriving step (step S4), the lifting of cracks (or cracks accompanying lifting) can be determined. First, known edge detection is performed on the temperature distribution to extract the boundaries of temperature changes. Furthermore, various known methods such as the Sobel method, Laplace method, and Carney method are used for edge detection. Next, based on the shape of the temperature change, the brightness distribution within a specified range centered on each point on the boundary of the temperature change and the distribution after the boundary extraction of the temperature change are extracted, and their similarity is evaluated. If similarity is found in any part of the boundary, it is determined that there is a crack along the boundary in that part, i.e., a lifting of cracks.

[0147] like Figure 14 As shown, the situation where cracks exist within the raised area can also be determined as follows. For example, from... Figure 14 As is understandable, within the area of ​​the rise, in the cracked section, the temperature changes drastically. Therefore, in the distribution where edge detection of the temperature distribution has been performed, not only at the boundaries of the temperature changes, but also within the temperature changes, there are linear portions extracted as edges. If the shape of these linear portions matches the shape of the cracks in the brightness distribution, it can be determined that the rise is accompanied by cracks.

[0148] Specifically, based on the shape of the temperature change, the similarity is evaluated by extracting the brightness distribution within a defined range centered on each point within the temperature change and the distribution after edge extraction of the temperature change. If the similarity is found in at least a portion of the interior of the temperature change, it is determined that a crack exists within the interior of the temperature change, i.e., a crack-related lifting. Furthermore, the similarity evaluation is performed in the same manner as described above, where the values ​​of the two distributions evaluated by coverage are the same or opposite. Alternatively, the similarity can also be evaluated based on the fact that the brightness of the crack is lower than that of the surrounding area in the brightness distribution.

[0149] Similar to peeling, the lifting associated with cracks exhibits a steeper temperature gradient at certain boundaries compared to temperature changes caused by internal defects such as lifting. That is, the temperature gradient is steep in the portion of the boundary along the temperature change. Furthermore, the temperature gradient is also steep in the portion of the temperature change within the crack. Therefore, this feature can be used to determine lifting associated with cracks. For example, edge detection of the temperature distribution can be performed, extracting the boundaries of temperature changes and the portions where the temperature steepens within them, and then extracting portions within those portions where the edge size is greater than a specified value, i.e., the temperature gradient is steeper than a specified value. If there are no portions with a temperature gradient greater than a specified value at the boundaries of temperature changes or within the portions within the temperature changes, the cause of the temperature change is inferred to be something other than lifting associated with cracks, such as internal defects like lifting. Alternatively, in cases where the temperature slope exceeds a specified value, the similarity between each part and the brightness distribution is evaluated. In cases where any part is similar (such as a linear part with a sharp temperature slope, and the shape of the linear part is consistent with the shape of the crack in the brightness distribution), the cause of the temperature change is inferred to be the lifting of the crack.

[0150] The lifting associated with cracks, like other surface changes or lifting associated with peeling, can also have its temperature tilt normalized by the absolute difference between the temperature change and the surrounding temperature, or the absolute difference between the temperature change and the average temperature of a larger area of ​​the structure's surface. In evaluating the similarity between the temperature distribution and the brightness distribution after edge detection, the steeper temperature tilt of the temperature distribution in the cracked area is also included in the evaluation, but the temperature tilt can also be evaluated as described above.

[0151] As mentioned above, visual images are typically composed of RGB images. While causal inference can be made based on any RGB brightness distribution, it is preferable to evaluate the magnitude of brightness deviation across the brightness distributions of each RGB channel and infer the cause based on the channel with the largest deviation. The same applies when there are two or more types of visual images.

[0152] Figure 15 Use binary representation relative to Figure 13 The changing shape of the image Figure 16 Use binary representation relative to Figure 14 The changing shape of the image. Pixels that have changed are represented by 1 (white), and pixels that have not changed are represented by 0 (black).

[0153] Figure 15 (A) indicates that the image is from a visual image ( Figure 13 (A)) The shape of the surface variation (crack) derived from it. Figure 15 (B) indicates that the infrared thermal image ( Figure 13(B) The shape of the boundary of the temperature change derived from it. For example... Figure 15 As shown, due to Figure 15 (A) Surface variations (cracks) and shape Figure 15 (B) The shape of the boundary of the temperature change is at least partially similar, so it can be inferred that it is accompanied by the lifting of the crack.

[0154] Figure 16 (A) indicates that the image is from a visual image ( Figure 14 (A)) The shape of the surface variation (crack) derived from it. Figure 16 (B) indicates that the infrared thermal image ( Figure 14 (B) The shape of the inner edge (the part where the temperature changes drastically) derived from the temperature change. Furthermore, in Figure 16 In this process, the boundaries of temperature changes are also extracted as edges, but the boundaries are omitted. Because... Figure 16 (A) Surface variations (cracks) and shape Figure 16 (B) The shapes of the internal edges of the temperature changes are at least partially similar, so it can be inferred that they are associated with the lifting of cracks.

[0155] For cracks, peeling, and other surface changes, the process for causal deduction is explained. Regardless of the order of causal deduction, the process for causal deduction when proceeding in the order of other surface changes, peeling, and cracks is illustrated as an example.

[0156] First, the range of temperature changes that matches the temperature changes derived in the temperature change information extraction step (step S3) is extracted from the brightness distribution derived in the surface change information extraction step (step S4), and the similarity between the brightness distribution and the temperature distribution within this range is evaluated. At this point, the temperature slope of the temperature change boundary can also be evaluated.

[0157] In similar situations (and / or when the temperature slope at the boundary is steep), the cause of the temperature change can be inferred to be surface change. Alternatively, the relationship between the temperature change and the surrounding temperature difference can be used to determine whether the surface change is peeling or other types of surface change.

[0158] In cases of dissimilarity (and / or where the temperature slope of the boundary is gentle), proceed to the next step.

[0159] Next, in the brightness distribution exported in the surface change information export step (step S4), a defined range centered on each point on the boundary of the temperature change exported in the temperature change information export step (step S3) is extracted, and the similarity between the brightness distribution and temperature distribution of each range is evaluated. At this time, the temperature slope of the boundary of the temperature change in each range can also be evaluated.

[0160] In cases where similarities exist in any part (and the temperature slope at the boundary is steep), the cause of the temperature change is inferred to be buoyancy accompanying the peeling. In cases where no parts are similar (or the temperature slope at the boundary is gentle), proceed to the next step.

[0161] Next, edge detection is performed on the temperature distribution of the temperature change exported in the temperature change information export step (step S3). In the brightness distribution exported in the surface change information export step (step S4), a defined range centered on each point on and within the boundary of the temperature change exported in the temperature change information export step (step S3) is extracted, and the similarity between the brightness distribution of each range and the edge-detected temperature distribution is evaluated. Alternatively, only the portion with an edge size greater than a defined value can be extracted from the edge-detected temperature distribution.

[0162] In cases where parts are similar, the cause of the temperature change is inferred to be the lifting of the crack. In cases where parts are dissimilar, the cause is inferred to be lifting.

[0163] It is also possible to deduce the presence, type, shape, location, and brightness distribution of cracks, peeling, and other surface changes from the brightness distribution of the area corresponding to the temperature change, and then infer the cause. However, since the method is the same as the method of inferring the cause after explicitly deriving the presence, type, etc. of surface changes in the surface change information derivation step (step S4), the explanation is omitted.

[0164] <Second Method>

[0165] As a second approach, in the surface change information extraction step (step S4), the presence, type, shape, location, and brightness distribution of surface changes are explicitly extracted from the brightness distribution of the area corresponding to the temperature change. For other surface changes, the case where no type determination is performed is explained. For each of cracks, peeling, and other surface changes, the method for inferring the cause is explained.

[0166] (Other surface changes)

[0167] In the absence of other surface changes, the cause of the temperature change is inferred to be something other than the surface change, such as an internal defect like buoyancy. In the presence of other surface changes, the similarity between the brightness distribution and temperature distribution of the surface change is evaluated using the same method described in the first approach. If similar, the cause of the temperature change is inferred to be the surface change; otherwise, it is inferred to be something other than the surface change, such as an internal defect like buoyancy. However, since the shape of other surface changes has already been derived, similarity can also be evaluated within the range including the surface change and the range conforming to the temperature change derived in the temperature change information derivation step (step S3). Alternatively, the similarity can be evaluated based on shape, not on temperature and brightness distributions, for example, by binarizing the distribution after setting pixels with temperature changes to 1 and pixels without temperature changes to 0, and binarizing the distribution after setting pixels with other surface changes to 1 and pixels without other surface changes to 0.

[0168] As described in the first method, temperature changes caused by other surface variations are characterized by a steeper temperature gradient at their boundaries compared to temperature changes caused by internal defects. Therefore, the cause of the temperature change can also be inferred based on this characteristic. The method for this is described in the first method, and therefore its description is omitted.

[0169] (Stripping)

[0170] In the absence of peeling, the cause of temperature change is inferred to be an internal defect that does not accompany peeling, such as floating without peeling. In the presence of peeling, the location and shape of the temperature change derived in the temperature change information derivation step (step S3) are compared with the location and shape of the peeling. If the locations of the temperature change and peeling are different, and the shapes of the adjacent boundaries are similar, the cause of the temperature change is inferred to be floating that accompanies peeling.

[0171] For example, given a binarized distribution where pixels with temperature changes are set to 1 and pixels without temperature changes are set to 0, and a binarized distribution where pixels with peeling are set to 1 and pixels without peeling are set to 0, based on the shape of the temperature change, a defined range centered on each point on the boundary of the temperature change is extracted. The similarity between the temperature change shape distribution (the binarized distribution where pixels with temperature changes are set to 1 and pixels without temperature changes are set to 0) and the peeling shape distribution (the binarized distribution where pixels with peeling are set to 1 and pixels without peeling are set to 0) in each range is evaluated. If they are similar in any part, it is inferred that the rise is accompanied by peeling. Alternatively, if the temperature change and peeling are at different locations, but adjacent to each other on a part of the boundary of the temperature change, and the temperature distribution of the temperature change and the brightness distribution of the peeling are similar, for example, based on the shape of the temperature change, the temperature distribution and brightness distribution of a defined range centered on each point on the boundary of the temperature change are extracted. The similarity between the temperature distribution and brightness distribution in each range is evaluated. If they are similar in any part, it is inferred that the rise is accompanied by peeling. Alternatively, if the peeling occurs outside the magnitude of the temperature change derived in the temperature change information derivation step (step S3), and the peeling boundary is near the boundary of the magnitude of the temperature change, it can be inferred that the peeling is accompanied by buoyancy.

[0172] As described in the first method, in the temperature change caused by the buoyancy accompanying peeling, the temperature slope is steep in the portion adjacent to the peeling within the boundary of the temperature change. Therefore, it is also possible to evaluate the similarity between the shape or temperature distribution of the temperature change and the shape or brightness distribution of the peeling within a defined range centered on each point on the boundary of the temperature change, and to evaluate the temperature slope of the boundary of the temperature change within the defined range. If, in any portion, the shape or temperature distribution of the temperature change is similar to the shape or brightness distribution of the peeling and / or there is a portion with a steep temperature slope, it is inferred to be a buoyancy accompanying peeling. Alternatively, it is also possible to infer a buoyancy accompanying peeling if the peeling is outside the magnitude of the temperature change and the boundary of the peeling is near the boundary of the magnitude of the temperature change, and near that boundary, the temperature slope of the temperature change is steep.

[0173] Alternatively, the temperature tilt can be standardized as described in the first approach.

[0174] (crack)

[0175] In the absence of cracks, the cause of temperature changes is inferred to be an internal defect not associated with cracks, such as un-cracked lifting. In the presence of cracks, the similarity between the shape of the temperature change (i.e., the shape of the boundary) derived in the temperature change information derivation step (step S3) and the shape of the crack is evaluated. Furthermore, if similarity exists in any part of the boundary of the temperature change, it is inferred that there is a crack along the boundary in that part, i.e., lifting accompanied by a crack. The similarity evaluation is performed as follows: For a distribution where the pixels of the temperature change boundary are set to 1 and all other pixels are set to 0, and a distribution where the pixels with cracks are set to 1 and the pixels without cracks are set to 0, a defined range centered on each point on the boundary of the temperature change is extracted. Furthermore, the similarity is evaluated between the distribution of the temperature change boundary shape in each range (i.e., the distribution where the pixels of the temperature change boundary are set to 1 and all other pixels are set to 0) and the crack shape distribution (i.e., the distribution where the pixels with cracks are set to 1 and the pixels without cracks are set to 0). Alternatively, the similarity between the temperature distribution for which edge detection of the temperature change was performed and the brightness distribution of the crack can be evaluated. If they are similar in any part of the boundary of the temperature change, it is inferred that there is a crack along the boundary in that part, i.e., a crack-accompanying lift-up. Furthermore, if there is a crack inside the temperature change derived in the temperature change information derivation step (step S3), it is inferred that there is a crack-accompanying lift-up. Alternatively, if there is a crack near or inside the boundary of the magnitude of the temperature change derived in the temperature change information derivation step (step S3), it can also be inferred that there is a crack-accompanying lift-up.

[0176] As explained in the first method, the temperature change caused by the lifting of the accompanying crack has a steep temperature slope in both the portion with cracks along the boundary of the temperature change and the portion with cracks inside the temperature change. Therefore, it is also possible to evaluate the similarity between the shape or temperature distribution of the temperature change after edge detection within a specified range centered on each point on the boundary of the temperature change and the shape or brightness distribution of the crack, and to evaluate the temperature slope of the boundary of the temperature change within the specified range, such as the size of the detected edge. In any part, if the distribution after edge detection of the shape or temperature distribution of the temperature change is similar to the shape or brightness distribution of the crack and / or there is a portion with a steep temperature slope, it is inferred that the lifting of the accompanying crack is occurring. Alternatively, if there is a crack inside the temperature change, the temperature slope of the temperature distribution in that crack portion is evaluated; in the crack portion, if the temperature slope is steep, it is inferred that the lifting of the accompanying crack is occurring. At this point, it is also possible to evaluate the similarity between the temperature distribution after edge detection and the shape or brightness distribution of the crack within the specified range including the part of the crack. In the case of similarity and / or steep temperature gradient, it is inferred that the crack is rising.

[0177] In addition, as explained in the first method, the temperature tilt can also be standardized.

[0178] For cracks, peeling, and other surface changes, explain the process for causal deduction. Regardless of the order of causal deduction, the process for causal deduction will be explained as an example, following the order of other surface changes, peeling, and cracks.

[0179] First, evaluate the presence or absence of other surface changes derived in the surface change information deriving step (step S4), and if other surface changes exist, evaluate the similarity between the surface changes and the temperature changes derived in the temperature change information deriving step (step S3). At this time, the temperature slope of the temperature change boundary can also be evaluated.

[0180] If there are no other surface changes or dissimilarities (and / or the temperature slope of the boundary is gentle), proceed to the next step.

[0181] In similar cases (and / or cases where the temperature slope of the boundary is steep), the cause of the temperature change is inferred to be other surface changes.

[0182] Next, the presence or absence of peeling derived in the surface change information export step (step S4) is evaluated, and if peeling is present, the partial similarity or positional relationship between the peeling and the temperature change derived in the temperature change information export step (step S3) is evaluated. At this time, the temperature slope of the boundary of the temperature change in each part can also be evaluated.

[0183] If there is no peeling, or if the parts are dissimilar in any way (and / or the temperature slope of the boundary is gentle), proceed to the next step.

[0184] In similar cases in any part (and / or where the temperature slope of the boundary is steep), the cause of the temperature change is inferred to be the buoyancy accompanying the peeling.

[0185] Next, the presence or location of cracks derived in the surface change information deriving step (step S4) is evaluated (evaluating the presence or absence of cracks near or inside the temperature change boundary). If cracks are present, the similarity between the cracks and the boundary or interior portion of the temperature change derived in the temperature change information deriving step (step S3) is evaluated. At this time, the temperature inclination of the temperature change in each part can also be evaluated.

[0186] In cases where there are no cracks near the temperature change boundary, or where there are cracks near the temperature change boundary but no part of the temperature change boundary resembles the crack (and / or the temperature slope of the boundary is gentle) and there are no cracks inside the temperature change (or where there are cracks inside the temperature change, but the temperature distribution after edge detection in the crack portion is not similar to the crack, and / or the temperature slope in the crack portion is gentle), the cause of the temperature change is inferred to be buoyancy.

[0187] In cases where there is a crack near the temperature change boundary, a part of the temperature change boundary is similar to a crack (and / or the temperature slope of the boundary is steep), or a crack is inside the temperature change (and the temperature distribution in the cracked part after edge detection is similar to the crack, and / or the temperature slope in the cracked part is steep), the cause of the temperature change is inferred to be the lifting of the crack.

[0188] <Third Method>

[0189] As a third approach, in the surface change information export step (step S4), the presence, type, shape, location, and brightness distribution of surface changes are explicitly exported from the brightness distribution of the area corresponding to the temperature change, further explaining the situation of type discrimination for other surface changes.

[0190] The method for inferring the causes of cracks and peeling is the same as the second method, only the method for inferring the causes of other surface changes is different from the second method.

[0191] As mentioned above, depending on the type of surface change, the information available for inferring the cause varies as follows.

[0192] The difference in visible light absorption between concrete and surface variations is the main reason for the difference in surface temperature. Surface variations such as uneven color, rust liquid or rust brightness distribution are effective information for inferring causes.

[0193] Differences in thermal conductivity or infrared emissivity between the concrete and surface variations such as repair marks and free lime, which are the main causes of surface temperature differences, are also surface variations caused by structural factors such as the shape of joints and steps. These surface variations are effective information for inferring the cause.

[0194] For example, it can be understood that for Figure 5 The brightness distribution of the rust solution is effective in inferring the cause, but for... Figure 7The repair marks, and the brightness distribution of the repair mark surface, become noise, with only the shape being valid. Therefore, in the third approach, the cause inference of other surface changes is basically the same as in the second approach, but the surface change information used in the similarity evaluation is used separately according to the type of surface change, specifically, brightness distribution or shape is used separately.

[0195] As explained in the first method, the temperature change caused by surface variations has a steeper slope at the boundary compared to the temperature change caused by internal defects. However, as... Figure 5 As with rust liquid, there are surface changes where the temperature gradient at the boundary may not be steep. Therefore, depending on the type of surface change, it is advisable to switch between evaluating the temperature gradient at the boundary of the temperature change. For surface changes like rust liquid where the temperature gradient at the boundary may not be steep, it is preferable not to evaluate the temperature gradient.

[0196] <Steps for displaying inference results>

[0197] The information display unit 61 displays the inference result of the cause of the temperature change on the display device 30 via the display control unit 26 (inference result display step: step S6).

[0198] For each temperature change determined and extracted in the temperature change information export step (step S3), the cause inferred in the cause inference step (step S5) is displayed. It can be displayed near each temperature change on the infrared thermal image, or it can be displayed on the visual image along with the location and shape of the temperature change. It can also be displayed on any image obtained by processing the infrared thermal image and the visual image, and / or any other arbitrary image. In the third scenario (determining the type of other surface changes), it is preferable that when the cause inferred in the cause inference step (step S5) is "other surface changes," its type (color unevenness, seams, steps, rust, etc.) is displayed.

[0199] Figure 17 This diagram illustrates an example of displaying the inference result on the display device 30. Figure 17 Display and Figure 10 (B) The corresponding infrared thermal image. In addition, the area determined to be a temperature change and extracted in the temperature change information export step (step S3) is displayed with a white frame, and the cause "lifting accompanied by peeling" inferred in the cause inference step (step S5) is displayed in white text near the white frame.

[0200] exist Figure 17 The system can display only a portion of the infrared thermal image, but it can also display the entire captured infrared thermal image. Furthermore, it can show the reasons for each part that has been identified as a temperature change and extracted, and display it over a larger area.

[0201] Figure 17 The example is just one instance. For the sake of easy observation of the "lifting that accompanies peeling", the background of the text can also be set to white. There are countless variations in how it is displayed, and there are no particular restrictions.

[0202] <Other methods>

[0203] Differences in the surface temperature of a concrete structure caused by internal factors such as buoyancy may be masked by differences in surface temperature caused by surface changes, making it impossible to identify temperature differences caused by internal factors. For example, on a concrete surface, a detached area may be repaired with filler material, but this material may sometimes buoy from the original concrete structure after several years. While it would be desirable to identify the temperature difference caused by this buoyancy, it may be obscured by temperature differences arising from variations in the thermal conductivity or infrared emissivity of the repair material compared to the surrounding concrete.

[0204] In this case, if the cause of the temperature change is inferred to be other surface changes, the temperature distribution based on the surface changes can be inferred from the temperature change information and surface change information, and reduced from the original temperature distribution, so as to be able to distinguish the temperature difference caused by internal factors of the structure. There are various methods for this.

[0205] For example, the brightness distribution of surface changes, which serves as information about surface changes, can be subjected to a specified fuzzing process (the temperature distribution based on surface changes is blurred compared to the brightness distribution due to factors such as heat conduction). The contrast of the blurred brightness distribution can be adjusted to optimize it to best suit the original temperature distribution. Furthermore, the brightness distribution with optimized contrast can be inferred as the temperature distribution based on surface changes and subtracted or removed from the original temperature distribution.

[0206] However, methods that infer temperature distribution based on the brightness distribution of surface changes, as described in this method, are not suitable for surface changes where the brightness and temperature distributions are dissimilar. That is, this method is suitable for surface changes where, as mentioned above, uneven color, or differences in visible light absorption such as rust or rust liquid, are the primary cause of temperature differences compared to concrete.

[0207] On the other hand, it is unsuitable for surface changes such as repair marks, free lime, etc., where the difference in thermal conductivity and infrared emissivity between the surface and concrete is the main cause of temperature differences, or for surface changes such as joints and steps that cause temperature differences due to structural reasons. It is preferable not to use the brightness distribution of such surface changes to infer the temperature distribution. For example, depending on the type of surface change, multiple thermal parameters related to the surface change can be set, and thermal simulations (including simulations of heat conduction, heat radiation, and convection) can be performed based on these parameters and the shape of the surface change to simulate the temperature distribution. The simulated temperature distribution that best suits the original temperature distribution is then inferred as the temperature distribution based on the surface change, and subtracted or removed from the original temperature distribution.

[0208] For surface changes such as repair marks and free lime where thermal conductivity or infrared emissivity is the main cause of temperature differences, thermal conductivity or infrared emissivity is set as a particularly important parameter. For surface changes such as joints and steps where temperature differences are caused by structural reasons, structural parameters such as the depth or height of the joint's unevenness and the height of the step are set as particularly important parameters.

[0209] Alternatively, the original temperature distribution can be represented simply by a lower-order polynomial, such as a linear (planar) or quadratic (surface) expression. The temperature distribution that best fits the original distribution can then be inferred as a surface-variable-based temperature distribution, which is then subtracted from the original distribution. Alternatively, the original temperature distribution can be inferred as a surface-variable-based temperature distribution after undergoing a specified fuzzy processing, which is then subtracted from the original distribution.

[0210] In the above embodiments, the methods for each step of the temperature change determination and temperature change information export step (step S3) according to the rule base, the surface change information export step (step S4) surface change information export, and the cause inference step (step S5) cause inference are described.

[0211] Not limited to this, various machine learning methods can also be used to implement each step of the temperature change information extraction step (step S3), surface change information extraction step (step S4), and cause inference step (step S5).

[0212] For example, in the temperature change information extraction step (step S3), as a method to determine the temperature change based on the infrared thermal image and extract information including its shape, machine learning methods such as FCN (Fully Convolutional Network), SegNet (ADeep Convolutional Encoder-Decoder Architecture for Image Segmentation), and U-Net (Convolutional Networks for Biomedical Image Segmentation) can be used to detect objects and extract their regions based on the image.

[0213] Furthermore, the same machine learning method can also be used in the method for deriving surface change information from the visual image in the surface change information deriving step (step S4). In the case of visual images, when there are multiple images such as RGB images, all types of visual images can be input.

[0214] As long as the temperature change is determined in the temperature change information export step (step S3) and only its magnitude is exported, machine learning methods such as R-CNN (Regions with CNN (Convolutional Neural Network) features), Fast R-CNN, YOLO (You Only Look Once), and SSD (Single Shot MultiBox Detector) can also be used.

[0215] In the surface change information extraction step (step S4), firstly, surface changes are determined, and feature quantities (average brightness, shape, texture of brightness distribution, contrast, spectrum, etc.) are extracted from its brightness distribution. Based on these feature quantities, the type of surface change (other surface changes, peeling, cracks; additionally, other surface change types such as color inhomogeneity, seams, steps, rust, etc.) can be determined. As a discrimination method, machine learning methods that classify objects based on feature quantities, such as logistic regression, linear discriminant analysis, K-nearest neighbor algorithm, decision tree (classification tree), random forest, and support vector machine (SVM), can be used.

[0216] In the cause inference step (step S5), the method for inferring the cause of temperature change based on temperature change information and surface change information can also be implemented using various machine learning methods based on the manner in which the temperature change information and surface change information are obtained. For example, when the temperature distribution of the temperature change is input as temperature change information and the brightness distribution of the part corresponding to the temperature change is input as surface change information, machine learning methods such as DNN (Deep Neural Network) and CNN can be used to classify the cause of temperature change into types such as "other surface changes (when further classifying other surface changes, they are "color inhomogeneity", "seams", "steps", "rust liquid", etc.)", "lifting accompanied by peeling", "lifting accompanied by cracks", and "lifting".

[0217] It can also summarize the temperature change information extraction step (step S3), surface change information extraction step (step S4), and cause inference step (step S5) and implement them through a machine learning method. That is, it can take infrared thermal images and visual images (or all types of visual images when multiple images such as RGB are available) as input, and detect temperature changes using machine learning methods such as FCN, SegNet, U-Net, R-CNN, Fast R-CNN, YOLO, and SSD, and classify their causes. Furthermore, when performing machine learning, it is preferable to have abundant learning data.

[0218] In the above embodiments, cause inference was performed sequentially for each of the other surface changes, peeling, and cracks. An example of inferring one cause as the cause of temperature change was described. Multiple candidates can also be inferred as causes of temperature change. That is, cause inference can be performed for each of the other surface changes, peeling, and cracks, and multiple candidates such as other surface changes, buoyancy accompanying peeling, and buoyancy accompanying cracks can also be inferred as causes of temperature change.

[0219] Alternatively, the cause can be inferred probabilistically rather than through a binary choice. For example, for each of the other surface changes, peeling, and cracks, if the area of ​​the visual image corresponding to the temperature change does not have the aforementioned other surface changes, peeling, or cracks, the probability of it being the cause of the temperature change is set to 0%. If the aforementioned other surface changes, peeling, or cracks are present, the probability of it being the cause of the temperature change can be converted into a probability value by calculating the similarity between its brightness distribution and / or shape and the temperature distribution and / or shape (or the distribution after edge detection) of the temperature change (e.g., the aforementioned Euclidean distance or Pearson's probability correlation coefficient).

[0220] Alternatively, the calculated values ​​of the temperature change boundary and the internal temperature slope can also be included and converted into probability values. In this embodiment, in the inference result display step (step S6), for each temperature change determined and extracted in the temperature change information derivation step (step S3), multiple candidates or their probabilities that are inferred as the cause are displayed.

[0221] The procedures described above can also be implemented using dedicated analysis programs, and there are no limitations on the implementers. For example, a personal computer can also be used. Furthermore, the apparatus or program for implementing each step can be integrated or separate.

[0222] <Other>

[0223] In the above embodiments, the hardware structure of the processing unit that performs various processes is as shown below, including various processors. These processors include general-purpose processors that execute software (programs) and function as various processing units, such as CPUs (Central Processing Units); processors with circuit structures that can be changed after manufacturing, such as FPGAs (Field Programmable Gate Arrays), which are programmable logic devices (PLDs); and processors with circuit structures specifically designed for performing specific processes, such as ASICs (Application Specific Integrated Circuits), which are dedicated circuits.

[0224] A processing unit can be composed of one of these various processors, or it can be composed of two or more processors of the same or different types (e.g., multiple FPGAs or a combination of CPU and FPGA). Alternatively, multiple processing units can be composed of a single processor. As examples of multiple processing units composed of a single processor, firstly, there are computer-like devices such as client or server computers, where a combination of one or more CPUs and software is used to construct a single processor, which functions as multiple processing units. Secondly, there are devices such as System-on-Chip (SoC), where a single IC (Integrated Circuit) chip implements the overall functionality of a system containing multiple processing units. In this way, various processing units are constructed using one or more of the aforementioned processors as hardware structures.

[0225] Furthermore, more specifically, the hardware architecture of these various processors is a circuit composed of circuit elements such as semiconductor components.

[0226] The aforementioned structures and functions can be suitably implemented by any hardware, software, or a combination of both. For example, the present invention can also be applied to a program that causes a computer to perform the above-described processing steps (processing sequence), a computer-readable recording medium (non-transitory recording medium) that records such a program, or a computer capable of installing such a program.

[0227] The examples of the present invention have been described above, but the present invention is not limited to the above embodiments. Of course, various modifications can be made without departing from the spirit of the present invention.

[0228] Symbol Explanation

[0229] 10 Image Analysis Device

[0230] 12 Input / Output Interfaces

[0231] 16 Storage Department

[0232] 18 Operations Department

[0233] 20 CPU

[0234] 22 RAM

[0235] 24 ROM

[0236] 26 Display Control Unit

[0237] 30 display devices

[0238] 32 Infrared camera device

[0239] 34. Visual camera device

[0240] 36 Structures

[0241] 51 Infrared Thermal Image Acquisition Unit

[0242] 53 Visible Image Acquisition Unit

[0243] 55 Temperature Change Information Export Section

[0244] 57 Surface Change Information Export Section

[0245] 59. Causal Inference Department

[0246] 61 Information Display Department

[0247] 101 Infrared thermal image

[0248] 102 Temperature Change Information

[0249] 103 Visual Images

[0250] 104 Surface Change Information

[0251] Step S1

[0252] S2 Step

[0253] S3 Step

[0254] S4 Step

[0255] S5 Step

[0256] S6 Step

Claims

1. An image analysis device comprising a processor, wherein, The processor performs the following processing: Acquire infrared thermal images of the structures of the objects to be inspected; Obtain a visual image of the structure of the object to be inspected; Temperature changes are determined based on the infrared thermal image; Determine the surface changes corresponding to the temperature changes based on the visual image; Determine at least the temperature change information that includes the temperature distribution of the temperature change; Determine the surface change information, which includes at least the brightness distribution of the surface change; For the region corresponding to the overall temperature change, determine the first similarity between the temperature distribution and the brightness distribution; For the region corresponding to the portion of the temperature change, a second similarity between the temperature distribution and the brightness distribution is determined; The surface changes corresponding to the temperature changes are determined to be cracks or peeling. as well as Based on the temperature change information and the surface change information, the cause of the temperature change is inferred. If the temperature change is not similar in the first similarity determination but similar in the second similarity determination, it is inferred that the cause of the temperature change is the lifting that accompanies the crack or peeling.

2. The image analysis device according to claim 1, wherein, The temperature change information includes the temperature distribution of the infrared thermal image and / or information obtained from the temperature distribution.

3. The image analysis device according to claim 2, wherein, The temperature change information includes information about the shape and / or magnitude of the temperature change.

4. The image analysis apparatus according to any one of claims 1 to 3, wherein, The surface change information includes the brightness distribution of the visible image and / or information obtained from the brightness distribution.

5. The image analysis apparatus according to any one of claims 1 to 3, wherein, The surface change information includes at least one of the following: the type, shape, and location of the surface change.

6. The image analysis apparatus according to any one of claims 1 to 3, wherein, When the processor infers that the cause of the temperature change is the surface change, it infers the temperature distribution based on the surface change and reduces it from the infrared thermal image.

7. The image analysis apparatus according to any one of claims 1 to 3, wherein, The processor performs the following processing: If the peeling is outside the magnitude of the temperature change, and the boundary of the peeling is near the boundary of the magnitude of the temperature change, it is inferred that the peeling is accompanied by buoyancy. as well as If the crack is present near or within the boundary of the magnitude of the temperature change, it is inferred that the crack is associated with buoyancy.

8. The image analysis apparatus according to any one of claims 1 to 3, wherein, The processor infers the cause of the temperature change based on the slope of the temperature at the boundary of the temperature change.

9. The image analysis apparatus according to any one of claims 1 to 3, wherein, The surface changes include at least one of repair marks, free lime, seams, steps, cracks, and peeling.

10. The image analysis apparatus according to any one of claims 1 to 3, wherein, The visual image is an image obtained by visualizing the reflection intensity distribution of two or more different wavelength ranges within the visible light wavelength range.

11. The image analysis apparatus according to any one of claims 1 to 3, wherein, It also has a display device. The processor displays the inference results of the cause of the temperature change on the display device.

12. An image analysis method, comprising: The steps for obtaining infrared thermal images of the structures of the objects to be inspected; The steps for obtaining a visual image of the structure of the object to be inspected; The step of determining temperature changes based on the infrared thermal image; The step of determining the surface change corresponding to the temperature change based on the visible image; The step of determining temperature change information that includes at least the temperature distribution of the temperature change; The step of determining surface change information that includes at least the brightness distribution of the surface change; The step of determining the first similarity between the temperature distribution and the brightness distribution for a region corresponding to the overall temperature change; The step of determining the second similarity between the temperature distribution and the brightness distribution for the region corresponding to the portion of the temperature change; The step of determining the surface change corresponding to the temperature change as a crack or peeling; as well as Based on the temperature change information and the surface change information, the cause of the temperature change is inferred. If the temperature change is not similar in the first similarity determination but similar in the second similarity determination, it is inferred that the cause of the temperature change is the lifting step that accompanies the crack or peeling.

13. A non-transitory computer-readable recording medium having a program recorded thereon for causing a computer to perform the following steps: The steps for obtaining infrared thermal images of the structures of the objects to be inspected; The steps for obtaining a visual image of the structure of the object to be inspected; The step of determining temperature changes based on the infrared thermal image; The step of determining the surface change corresponding to the temperature change based on the visible image; The step of determining temperature change information that includes at least the temperature distribution of the temperature change; The step of determining surface change information that includes at least the brightness distribution of the surface change; The step of determining the first similarity between the temperature distribution and the brightness distribution for a region corresponding to the overall temperature change; The step of determining the second similarity between the temperature distribution and the brightness distribution for the region corresponding to the portion of the temperature change; The step of determining the surface change corresponding to the temperature change as a crack or peeling; as well as Based on the temperature change information and the surface change information, the cause of the temperature change is inferred. If the temperature change is not similar in the first similarity determination but similar in the second similarity determination, it is inferred that the cause of the temperature change is the lifting that accompanies the crack or peeling.

Images