Image analysis device, image analysis method, and program

Abstract

Provided are an image analysis device, an image analysis method, and a program with which it is possible to reduce erroneous detection of defects. This image analysis device comprises a processor, the processor acquiring an infrared thermal image in which a structure subject to inspection is imaged, acquiring a visible image in which the structure subject to inspection is imaged, assessing a temperature change from the infrared thermal image, and estimating the origin of the temperature change on the basis of at least temperature change information obtained from the infrared thermal image and surface change information obtained from the visible image.

Description

Technical Field

[0001] The present invention relates to an image analysis apparatus, an image analysis method, and a program.

Background Art

[0002] There is known a technique for discriminating defective parts such as voids and cracks and sound parts included in a structure by using an infrared thermal image obtained when photographing the structure such as concrete with an infrared camera. When there are defective parts in the structure, the surface temperature of the defective parts becomes higher than the surface temperature of the sound parts during temperature rise such as in the daytime, and conversely, the surface temperature of the defective parts becomes lower than the surface temperature of the sound parts during temperature drop such as at night. Therefore, if there is a part in the infrared thermal image where the surface temperature is different from the surroundings, it can be determined that there is a defective part inside that part.

[0003] On the other hand, in infrared inspection, there are parts where the surface temperature is different from the surroundings even though there are no defective parts such as voids inside the structure. For example, when the structure is repaired, the surface temperature of the repair mark may be different from the surface temperature of the surrounding concrete surface due to the difference in the thermal conductivity of the repair material from that of the surrounding concrete. Also, due to the difference in the infrared emissivity of the repair mark from that of the surrounding concrete, the apparent surface temperature of the repair mark in the infrared thermal image may be different from that of the surrounding concrete. When foreign substances such as free lime adhere to the surface, the actual temperature and / or the apparent temperature at that part may be different from that of the surrounding concrete. In addition, color unevenness (such as mold, moss, release agent, water influence, etc.), joints, steps, slime, sand streaks, rust juice, rust, water leakage, surface unevenness, pebbles, etc. can also cause parts where the actual and / or apparent surface temperature is different from the surroundings. Thus, infrared inspection has a problem that there are many parts where the surface temperature is different from the surroundings even though there are no defective parts such as voids inside the structure, that is, there are many false detections.

[0004] To address this issue, Patent Document 1 discloses the following method: It identifies factors that affect the thermal image of a structure and a multivariate analysis relation that uses this factor information to determine the probability that a defect is present in the abnormal area extracted from the thermal image of the structure. Then, it photographs the structure with an infrared camera to acquire a thermal image, extracts abnormal areas with different temperatures from the surroundings from the thermal image, identifies the factor information in those abnormal areas, quantifies the identified factor information, and applies these quantifiable values ​​to the multivariate analysis relation to determine the probability that a defect is present in the extracted abnormal area. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2013-096741 [Overview of the project] [Problems that the invention aims to solve]

[0006] Patent Document 1 discloses a method for determining the probability that defects are present in abnormal areas by extracting abnormal areas with different surface temperatures from the surrounding area in an infrared thermal image, then determining the presence or absence of cracks and the surface condition using an external image (visible image) or human visual inspection, and applying this to a multivariate analysis formula. Here, "surface condition" refers to the presence or absence of deformation on the surface of a concrete structure, such as color unevenness, uneven surfaces, and free lime. Many false detections (abnormal areas with different surface temperatures from the surrounding area despite the absence of defects inside the structure) are caused by deformation on the surface of the structure, so determining the presence or absence of deformation on the surface of the structure is considered effective in reducing false detections.

[0007] However, the invention described in Patent Document 1 only statistically determines the probability of a defect based on information about the presence or absence of deformation (including cracks) on the surface of the structure that has been identified. This method is insufficient to reduce false detections.

[0008] This invention has been made in view of these circumstances, and its purpose is to provide an image analysis device, an image analysis method, and a program that can reduce false detection of defects. [Means for solving the problem]

[0009] The first embodiment of the image analysis device is an image analysis device equipped with a processor, the processor acquires an infrared thermal image of the structure to be inspected, acquires a visible image of the structure to be inspected, determines temperature deformation from the infrared thermal image, and estimates the cause of the temperature deformation based on at least the temperature deformation information obtained from the infrared thermal image and the surface deformation information obtained from the visible image.

[0010] In the image analysis apparatus according to the second embodiment, the temperature deformation information includes information obtained from the temperature distribution and / or temperature distribution of an infrared thermal image regarding the temperature deformation.

[0011] In the image analysis apparatus according to the third embodiment, the temperature deformation information includes information on the shape and / or size of the temperature deformation.

[0012] In the image analysis apparatus according to the fourth embodiment, the surface deformation information includes the luminance distribution of the visible image and / or information obtained from the luminance distribution.

[0013] In the image analysis apparatus according to the fifth embodiment, the surface deformation information includes at least one piece of information regarding the type, shape, and location of the surface deformation.

[0014] In the image analysis apparatus according to the sixth embodiment, the processor estimates the cause of the temperature deformation based on the similarity between the temperature deformation information and the surface deformation information.

[0015] In the image analysis apparatus according to the seventh embodiment, if the processor estimates that the cause of the temperature deformation is surface deformation, it estimates the temperature distribution due to the surface deformation and reduces it from the infrared thermal image.

[0016] In the image analysis apparatus according to the eighth aspect, the similarity includes partial similarity.

[0017] In the image analysis apparatus according to the ninth aspect, the processor determines that a surface deformation corresponding to a temperature deformation is a crack or peeling, evaluates the similarity between the temperature deformation information and the surface deformation information, and when at least a part thereof is similar, estimates that the cause of the temperature deformation is a bulge accompanied by a crack or peeling.

[0018] In the image analysis apparatus according to the tenth aspect, the processor determines that a surface deformation corresponding to a temperature deformation is a crack or peeling, evaluates the presence or absence of a surface deformation within the magnitude of the temperature deformation and / or near the boundary of the temperature deformation, and when present, estimates that the cause of the temperature deformation is a bulge accompanied by a crack or peeling.

[0019] In the image analysis apparatus according to the eleventh aspect, the processor estimates the cause of the temperature deformation based on the temperature gradient of the boundary of the temperature deformation.

[0020] In the image analysis apparatus according to the twelfth aspect, the surface deformation includes at least one of a repair mark, free lime, a joint, a step, a crack, and peeling.

[0021] In the image analysis apparatus according to the thirteenth aspect, the visible image is an image obtained by imaging the reflection intensity distribution in two or more different wavelength ranges in the wavelength range of visible light.

[0022] In the image analysis apparatus according to the fourteenth aspect, further comprising a display device, and the processor displays the estimation result of the cause of the temperature deformation on the display device.

[0023] The image analysis method according to the fifteenth aspect includes the steps of obtaining an infrared thermal image of a structure to be inspected, obtaining a visible image of the structure to be inspected, determining a temperature deformation from the infrared thermal image, and estimating the cause of the temperature deformation based on at least the temperature deformation information obtained from the infrared thermal image and the surface deformation information obtained from the visible image.

[0024] A program for causing a computer to execute according to the 16th aspect includes steps of: acquiring an infrared thermal image obtained by photographing a structure to be inspected; acquiring a visible image obtained by photographing the structure to be inspected; determining a temperature change state from the infrared thermal image; and estimating a cause of the temperature change state based on at least temperature change state information obtained from the infrared thermal image and surface change state information obtained from the visible image, and causing the computer to execute these steps.

Effect of the Invention

[0025] According to the image analysis apparatus, image analysis method, and program of the present invention, false detection of defective portions can be reduced.

Brief Description of the Drawings

[0026] [Figure 1] FIG. 1 is a block diagram showing an example of the hardware configuration of an image analysis apparatus. [Figure 2] FIG. 2 is a block diagram showing processing functions realized by a CPU. [Figure 3] FIG. 3 is a diagram showing information stored in a storage unit. [Figure 4] FIG. 4 is a flowchart showing an analysis method using an image analysis apparatus. [Figure 5] FIG. 5 is an infrared thermal image and a visible image of a structure with rust juice attached. [Figure 6] FIG. 6 is an image showing the shape of a change in binary for the image of FIG. 5. [Figure 7] FIG. 7 is an infrared thermal image and a visible image of a structure including a repaired portion. [Figure 8] FIG. 8 is an image showing the shape of a change in binary for the image of FIG. 7. [Figure 9] FIG. 9 is an infrared thermal image and a visible image of a structure including a peeled portion. [Figure 10] FIG. 10 is an infrared thermal image and a visible image of a structure including two peeled portions. [Figure 11]Figure 11 is a binary image showing the shape of the deformation relative to the image in Figure 9. [Figure 12] Figure 12 is a binary image showing the shape of the deformation relative to the image in Figure 10. [Figure 13] Figure 13 shows infrared thermal and visible images of a structure that includes areas with cracks. [Figure 14] Figure 14 shows infrared thermal and visible images of a structure that includes another area with cracks. [Figure 15] Figure 15 is a binary image showing the shape of the deformation relative to the image in Figure 13. [Figure 16] Figure 16 is a binary image showing the shape of the deformation relative to the image in Figure 14. [Figure 17] Figure 17 shows an example of the display result when the estimation result is shown on a display device. [Modes for carrying out the invention]

[0027] The image analysis apparatus according to the embodiment is an image analysis apparatus equipped with a processor, the processor acquires an infrared thermal image of the structure to be inspected, acquires a visible image of the structure to be inspected, determines temperature deformation based on the infrared thermal image, derives temperature deformation information, derives surface deformation information from surface deformation based on the visible image of the location corresponding to the temperature deformation, and estimates the cause of the temperature deformation based on the temperature deformation information and the surface deformation information.

[0028] With respect to the invention according to the embodiment of the above configuration, the inventors have diligently studied how to reduce false detection of defects and have found the following, leading to the present invention.

[0029] To reduce false positives, it is necessary to determine the influence of identified structural surface deformations (hereinafter also referred to as surface deformations) on infrared thermal images (to determine the cause of abnormal areas). For this purpose, it is essential to analyze the relationship between the location and shape of abnormal areas extracted from infrared thermal images and surface deformations identified from visible images. The inventors of this invention conducted a comparative study of infrared thermal images and visible images of concrete structures and found that the relationship between surface deformations identified in visible images (color unevenness, joints, steps, rust stains, etc.) and temperature deformations identified in infrared thermal images (hereinafter referred to as temperature deformations) differs depending on the thermal environment such as the location on the structure and the time of imaging. In other words, the influence of surface deformations on the concrete surface on the surface temperature differs depending on the thermal environment such as the location on the structure and the time of imaging.

[0030] For example, when photographing the railing of a bridge wall on a sunny day, the location and shape of the color variations and rust stains identified from the visible image clearly matched those of the temperature changes identified from the infrared thermal image, meaning that the color variations and rust stains significantly affected the surface temperature. On the other hand, when photographing the underside of the deck of the same bridge at the same time, the visible image showed the same color variations and rust stains as the railing, but the infrared thermal image did not show corresponding temperature changes. The inventors believe that this difference is because the railing was exposed to sunlight on a sunny day, and the amount of sunlight absorbed differed between the areas with color variations and rust stains and other areas. Thus, the inventors have found that the effect of surface changes on surface temperature varies depending on the thermal environment, and therefore it is necessary to determine that effect (determine the cause of the temperature changes). For this purpose, it is essential to analyze the relationship between the location and shape of surface changes and temperature changes.

[0031] Preferred embodiments of the image analysis apparatus, image analysis method, and program according to the present invention will be described below with reference to the attached drawings.

[0032] [Hardware configuration of the image analysis device] Figure 1 is a block diagram showing an example of the hardware configuration of an image analysis device according to an embodiment.

[0033] As shown in Figure 1, the image analysis device 10 can be a computer or a workstation. In this example, the image analysis device 10 mainly consists of an input / output interface 12, a storage unit 16, an operation unit 18, a CPU (Central Processing Unit) 20, a RAM (Random Access Memory) 22, a ROM (Read Only Memory) 24, and a display control unit 26. A display device 30 is connected to the image analysis device 10, and under the command of the CPU 20, the display control unit 26 controls the display on the display device 30. The display device 30 is, for example, a monitor.

[0034] The input / output interface 12 (input / output I / F in the figure) can input various types of data (information) to the image analysis device 10. For example, data to be stored in the storage unit 16 is input via the input / output interface 12.

[0035] The CPU (processor) 20 reads various programs stored in the memory unit 16 or ROM 24, loads them into the RAM 22, and performs calculations to control each part. The CPU 20 also reads programs stored in the memory unit 16 or ROM 24, uses the RAM 22 to perform calculations, and then performs various processes for the image analysis device 10.

[0036] The infrared camera 32 shown in Figure 1 photographs the structure 36 to be inspected and acquires an infrared thermal image of the structure's surface. The visible camera 34 photographs the structure 36 to be inspected and acquires a visible image of the structure 36.

[0037] The image analysis device 10 can acquire infrared thermal images from the infrared camera 32 via the input / output interface 12. The image analysis device 10 can also acquire visible images from the visible camera 34 via the input / output interface 12. The acquired infrared thermal images and visible images can be stored, for example, in the storage unit 16.

[0038] Figure 2 is a block diagram showing the processing functions implemented by CPU 20.

[0039] The CPU 20 includes an infrared thermal image acquisition unit 51, a visible image acquisition unit 53, a temperature deformation information derivation unit 55, a surface deformation information derivation unit 57, a cause estimation unit 59, and an information display unit 61. The specific processing functions of each unit will be explained later. Since the infrared thermal image acquisition unit 51, the visible image acquisition unit 53, the temperature deformation information derivation unit 55, the surface deformation information derivation unit 57, the cause estimation unit 59, and the information display unit 61 are all parts of the CPU 20, it can also be said that the CPU 20 executes the processing of each unit.

[0040] Returning to Figure 1, the memory unit 16 stores data and programs for operating the image analysis device 10, such as the operating system and programs for executing image analysis methods. The memory unit 16 also stores information used in the embodiments described below.

[0041] Figure 3 shows the information stored in the storage unit 16. The storage unit 16 is a memory composed of various semiconductor memories such as CD (Compact Disk), DVD (Digital Versatile Disk), hard disk (Hard Disk), and flash memory.

[0042] The memory unit 16 primarily stores infrared thermal images 101, temperature deformation information 102, visible images 103, and surface deformation information 104.

[0043] The infrared thermal image 101 is an image taken by the infrared camera 32, which detects the infrared radiation energy emitted from the structure 36, converts that infrared radiation energy into temperature, and shows the temperature distribution on the surface of the structure.

[0044] The temperature change information 102 is information about temperature changes obtained from the temperature distribution and / or temperature distribution of the infrared thermal image 101.

[0045] The visible image 103 is an image captured by the visible camera 34, showing the distribution of visible light reflection intensity from the surface of the structure 36. Typically, a visible image consists of RGB images that visualize the reflection intensity distribution in three different wavelength ranges within the visible light wavelength range, meaning that each pixel has color information (RGB signal value). In this example, it is assumed that the image also has color information. The brightness of the visible image 103, described later, refers to the signal value of the visible image 103, and the brightness of each pixel in the visible image 103 reflects the visible light reflection intensity at the corresponding location on the surface of the structure 36.

[0046] Furthermore, the surface deformation information 104 is information about surface deformation at locations corresponding to temperature deformation in the visible image 103, and is information obtained from the brightness distribution and / or brightness distribution of the visible image 103.

[0047] The control unit 18 shown in Figure 1 includes a keyboard and mouse, allowing the user to perform necessary processing on the image analysis device 10 via these devices. By using a touch panel type device, the display device 30 can function as the control unit.

[0048] The display device 30 is, for example, a liquid crystal display, and can display the results obtained by the image analysis device 10.

[0049] Figure 4 is a flowchart showing an image analysis method using the image analysis device 10. As shown in Figure 4, the image analysis method comprises an infrared thermal image acquisition step (step S1), a visible image acquisition step (step S2), a temperature deformation information derivation step (step S3), a surface deformation information derivation step (step S4), a cause estimation step (step S5), and an estimation result display step (step S6).

[0050] <Infrared thermal image acquisition steps> The infrared thermal image acquisition unit 51 acquires an infrared thermal image of the structure 36 to be inspected (infrared thermal image acquisition step: step S1). The infrared thermal image is the infrared thermal image 101 stored in the storage unit 16. The infrared thermal image 101 is acquired from the storage unit 16 by the infrared thermal image acquisition unit 51. If the infrared thermal image 101 is not stored in the storage unit 16, the infrared thermal image acquisition unit 51 acquires the infrared thermal image 101 from an external source. For example, the infrared thermal image acquisition unit 51 can acquire the infrared thermal image 101 via a network through the input / output interface 12, and the infrared thermal image acquisition unit 51 can also acquire the infrared thermal image 101 from the infrared camera 32 via the input / output interface 12.

[0051] <Visible image acquisition step> The visible image acquisition unit 53 is used to acquire the structure to be inspected. 36 A visible image is acquired (visible image acquisition step: step S2). The visible image is the visible image 103 stored in the storage unit 16. The visible image 103 is acquired from the storage unit 16 by the visible image acquisition unit 53. If the visible image 103 is not stored in the storage unit 16, the visible image acquisition unit 53 acquires the visible image 103 from an external source. For example, the visible image acquisition unit 53 can acquire the visible image 103 via the network through the input / output interface 12, and the visible image acquisition unit 53 can also acquire the visible image 103 from the visible camera 34 via the input / output interface 12.

[0052] <Step to derive temperature change information> Next, the temperature deformation information derivation unit 55 determines the temperature deformation based on the infrared thermal image 101 and derives temperature deformation information 102 (temperature deformation information derivation step: step S3).

[0053] The temperature change information extraction unit 55 extracts from the infrared thermal image 101 areas where the surface temperature differs from the surrounding area in a localized manner, determining these areas as temperature changes, and derives temperature change information 102.

[0054] For example, within a predetermined range of the structural surface in the infrared thermal image 101, areas exceeding a predetermined temperature difference from the average temperature (areas with higher temperatures than the surroundings during temperature increases like daytime, and areas with lower temperatures than the surroundings during temperature decreases like nighttime) can be identified and extracted as temperature variations. Areas where the surface temperature differs from the surroundings and are spatially connected in a cohesive manner, or areas distributed at a distance closer than a predetermined distance even if not connected, can be identified as a single temperature variation. It should be noted that a temperature variation is not necessarily limited to areas where the actual surface temperature differs from the surroundings. In other words, even if the actual surface temperature is the same as the surroundings, a difference in infrared emissivity may cause the surface temperature to differ from the surroundings in the infrared thermal image 101, and this may be identified as a temperature variation.

[0055] Here, the infrared thermal image 101 may be the infrared thermal image itself, taken by the infrared camera 32 of the concrete structure 36. Alternatively, to facilitate the determination of temperature changes and / or the derivation of temperature change information, the original infrared thermal image 101 may be processed. For example, the surface temperature of the structure 36 often has a gradient (temperature gradient) due to differences in the amount of heat received by the surface of the structure 36 or the amount of heat dissipated from the surface of the structure 36. Therefore, the original infrared thermal image 101 may be processed to reduce the temperature gradient, and the temperature changes may be determined and extracted from the processed image to derive temperature change information 102.

[0056] The temperature deformation information 102 represents the temperature distribution (spatial distribution of temperature) of the temperature deformation, and represents the temperature distribution of at least the entire range of the temperature deformation (if the temperature deformation extends to the edge of the infrared thermal image 101, it includes the range up to the edge). In the cause estimation step (step S5) described later, it is preferable for the information to represent the temperature distribution over as wide an area as possible for the purpose of cause estimation.

[0057] The temperature deformation information 102 may be the temperature distribution itself in the original infrared thermal image 101, or it may be a distribution obtained by coarsely quantizing the original temperature distribution, such as a binarized, trinarized, or quaternary distribution. The temperature deformation information 102 may also be information representing the shape of the temperature deformation. For example, a binarized temperature distribution can be said to be information representing the shape of the temperature deformation. The temperature deformation information may also be information representing the size when the temperature deformation is approximated by a rectangle or ellipse.

[0058] Therefore, the temperature change information 102 is information relating to temperature changes obtained from the temperature distribution of the infrared thermal image 101 and / or its temperature distribution.

[0059] The temperature change information derivation unit 55 can determine multiple temperature changes (i=1,2,3,···N) and derive temperature change information 102.

[0060] <Step to derive surface deformation information> The surface deformation information extraction unit 57 extracts surface deformation information 104 based on the visible image 103 (surface deformation information extraction step: step S4).

[0061] The surface deformation information extraction unit 57 extracts surface deformation information 104 from the visible image 103 at locations corresponding to temperature deformation (i=1,2,3,···N). Deformation means "a state that has changed from the original state" or "a state that is different from the normal state," but in this embodiment in particular, surface conditions that can affect the temperature of the concrete surface in the infrared thermal image and cause temperature deformation are called "surface deformations," such as repair marks, adhesion of foreign matter such as free lime, color unevenness (mold, moss, release agent, water effects, etc.), joints, steps, grout, sand streaks, rust stains or rust, water leakage, surface irregularities, and beading. Cracks and peeling are also called "surface deformations."

[0062] Surface deformation information 104 represents information such as the presence, type, shape, and location of surface deformation, and is obtained from the brightness distribution (spatial distribution of brightness) of the visible image 103. Furthermore, in the case of surface deformation such as color unevenness, rust stains, or rust in the visible image 103, the brightness distribution is effective in estimating the cause of temperature deformation.

[0063] The brightness of each pixel in the visible image 103 reflects the reflected intensity of visible light at the corresponding location on the surface of the structure 36. The difference in brightness reflects the difference in reflected intensity to visible light that uniformly illuminates the surface of the structure 36, that is, the difference in reflectance. In other words, it reflects the difference in the absorption rate of visible light. Therefore, the brightness distribution of the surface of the structure 36 in the visible image 103 reflects the distribution of the absorption rate of visible light, such as solar radiation, that uniformly illuminates the surface of the structure 36, that is, the distribution of absorption amount. Similarly, the brightness distribution of each surface deformation on the surface of the structure 36 reflects the distribution of the amount of visible light absorbed at that surface deformation, making it effective for estimating the cause of temperature deformation. For this reason, the surface deformation information 104 may be information representing the brightness distribution of the surface deformation, and may be the brightness distribution of the visible image 103 itself, or a distribution that is a coarsely quantized version of that brightness distribution, for example, a binarized, trinarized, or quaternarized distribution.

[0064] Therefore, the surface deformation information 104 is information about surface deformation at locations corresponding to temperature deformation in the visible image 103, and is information obtained from the brightness distribution and / or brightness distribution of the visible image 103.

[0065] The luminance distribution of the area corresponding to the temperature change (the luminance distribution of the visible image 103 itself, or a coarsely quantized distribution of that luminance distribution) is information (internal information) that represents the presence, type, shape, location, and luminance distribution of the surface change at that area, and this luminance distribution may also be referred to as surface change information 104.

[0066] When explicitly deriving at least one of the presence, type, shape, location, and brightness distribution of surface deformation from the brightness distribution of areas corresponding to temperature deformation as surface deformation information 104, cracks and delamination are distinguished from other surface deformations (surface conditions that affect the temperature of the concrete surface in the infrared thermal image 101 and may cause misdetection of defects inside the structure 36) and extracted. From the visible image 103, areas with brightness lower than a predetermined brightness difference from the surroundings and connected linearly for a length of a predetermined value or more are determined to be cracks and extracted. Various methods using machine learning and various methods focusing on linearity and extracting based on its characteristics have been proposed for crack extraction, and any of these methods may be used.

[0067] The peeled areas clearly differ in brightness from the original concrete surface, and the texture, contrast, and frequency spectrum of the brightness distribution also differ. Therefore, areas that exceed a predetermined brightness difference, brightness distribution texture, contrast, and frequency spectrum difference from the surrounding area in a cohesive manner are identified as peeled areas and extracted. There is a step between the peeled area and the original concrete surface, and since this step is dark (low brightness), the darkness (low brightness) of the boundary area can also be an effective feature for identifying peeling. In addition, as other surface deformations, areas that differ in brightness from the surrounding area in a locally cohesive manner from the visible image 103, and / or areas that differ in brightness distribution texture, and / or areas that differ in brightness distribution contrast are identified and extracted. For example, areas that exceed a predetermined brightness difference from the average brightness are identified and extracted as other surface deformations. Areas that differ from their surroundings in one or more of the following ways: luminance, texture of luminance distribution, frequency spectrum, or contrast, and that are spatially connected or distributed at a distance closer than a predetermined distance, are identified as a single surface deformation and extracted.

[0068] The above process yields at least one of the following for each of the cracks, peeling, and other surface defects: presence, shape, location, and brightness distribution. Here, for the other surface defects, it is not necessarily required to meticulously identify the type of surface defect in the cause estimation step (step S5) described later (e.g., repair marks, free lime, color unevenness, joints, steps, grout, sand streaks, rust stains or rust, water leakage, surface irregularities, beading, etc.). However, identifying the type of surface defect allows for more accurate cause estimation in the cause estimation step (step S5).

[0069] Specifically, for surface deformations where differences in the amount of visible light absorbed compared to the surrounding concrete, such as color unevenness, rust stains, or rust, are the main cause of the difference in surface temperature compared to the surroundings in the infrared thermal image 101, as described above, the luminance distribution reflects the distribution of visible light absorption, and therefore the luminance distribution is effective as surface deformation information 104 for cause estimation in the cause estimation step (step S5).

[0070] On the other hand, there are surface deformations where the luminance distribution is not effective for causal estimation, that is, surfaces where the luminance distribution actually becomes noise. strange In the case of the condition, the following applies: For example, differences in thermal conductivity and infrared emissivity from the surrounding concrete, such as repair marks and free lime, are the main causes of the difference in surface temperature from the surroundings in the infrared thermal image 101. Structural causes such as joints and steps also affect the infrared thermal image. 101 For surface deformations that cause a difference in surface temperature with the surroundings, the shape is used as surface deformation information 104, which is effective for estimating the cause in the cause estimation step (step S5). Therefore, it is preferable to distinguish the types of other surface deformations in detail. Based on the characteristics of the extracted surface deformations, such as average brightness, contrast, brightness dispersion, texture, frequency spectrum, and shape, it is possible to distinguish their types in detail.

[0071] Here, the visible image 103 typically consists of RGB images that visualize the reflectance intensity distribution in three different wavelength ranges within the visible light wavelength range. Cracks, delamination, and other surface deformations may be determined and extracted from any of the RGB brightness distributions, but surface deformations such as rust stains or rust are concrete. surface Since the difference in luminance (difference in reflectance intensity) varies greatly depending on the RGB channel, it is preferable to determine and extract from the luminance distribution of the channel with the largest difference. For example, in the case of rust stains or rust, the luminance of B is particularly low compared to concrete, meaning that the difference in absorption, which is the difference in reflectance intensity in the B wavelength range, is large, so it is preferable to determine from the luminance distribution of B. However, it is thought that the channel with the largest difference in luminance compared to concrete will differ depending on the type of surface deformation. For example, depending on the type of color unevenness, it is possible that the difference in luminance of the R channel may be particularly large, the opposite of rust stains or rust. Therefore, it is preferable to evaluate the magnitude of the variation in luminance in the luminance distribution of each RGB channel and determine and extract cracks, peeling, and other surface deformations from the channel with the largest variation. For example, the coefficient of variation may be calculated by dividing the standard deviation of luminance at the location corresponding to temperature deformation in the luminance distribution of each RGB channel by the average luminance of concrete, and then determining and extracting cracks, peeling, and other surface deformations from the channel with the largest coefficient of variation. As the average luminance of the concrete, the average luminance value at the location corresponding to the temperature change may be used, or the average luminance value over a wider area including the location corresponding to the temperature change may be used. For other surface changes, if their type needs to be determined in detail, they can be determined based on the characteristics of each RGB channel of the extracted surface change, such as average luminance, contrast, luminance dispersion, texture, and frequency spectrum. Similarly, if there are two or more types of visible images 103, cracks, delamination, and other surface changes can be determined and extracted using the channel with the greatest variation, and the types of other surface changes can be determined in detail based on the characteristics of each of the two or more channels, such as average luminance and contrast.

[0072] Here, the area corresponding to the temperature deformation is a wider area than the spatial range corresponding to the temperature deformation when the temperature deformation is determined from the infrared thermal image 101 in the temperature deformation information derivation step (step S3). As will be described later, in the cause estimation step (step S5), surface deformation information covering a wider area than the range corresponding to the temperature deformation is required for the analysis of the cause estimation of the temperature deformation. In particular, for delamination, it is necessary to analyze the relationship between the temperature deformation and delamination at a different location.

[0073] In the above description, the steps were explained in the order of infrared thermal image acquisition (step S1), visible image acquisition (step S2), temperature deformation information derivation (step S3), and surface deformation information derivation (step S4). However, within the scope of the embodiment, these steps can be changed as appropriate.

[0074] <Cause estimation step> The cause estimation unit 59 estimates the cause of the temperature deformation based on the temperature deformation information 102 and the surface deformation information 104 (cause estimation step: step S5).

[0075] Furthermore, since the cause estimation process differs slightly depending on the type of surface deformation information 104 derived in the surface deformation information derivation step (step S4), these will be explained as the first, second, and third embodiments, respectively.

[0076] <First aspect> As a first embodiment, we will explain the case in which the brightness distribution of the area corresponding to the temperature deformation is used as surface deformation information in the surface deformation information derivation step (step S4). We will explain the method for estimating the causes of cracks and delamination and other surface deformations.

[0077] (Other surface deformations) From the luminance distribution derived in the surface deformation information derivation step (step S4), a range corresponding to the temperature deformation derived in the temperature deformation information derivation step (step S3), or a slightly broader range including the temperature deformation, is extracted. The similarity between the luminance distribution and temperature distribution in this range is evaluated, and if they are similar, it is estimated that the cause of the temperature deformation is this surface deformation. On the other hand, if they are not similar, it is estimated that the cause is something other than this surface deformation, for example, an internal defect such as delamination.

[0078] There are many methods for evaluating the similarity between two distributions. For example, one can normalize the luminance distribution and the temperature distribution so that their minimum and maximum values ​​are the same, and then calculate the Euclidean distance given by equation (1) below to evaluate the similarity. Alternatively, one can calculate equation (1) and determine that the distributions are similar if the distance is less than or equal to a predetermined value (the closer to 0, the more similar they are judged to be).

[0079] Furthermore, since the relationship between high and low brightness and high and low temperature can be the same or the opposite depending on the type of surface deformation and the time of imaging, it is necessary to calculate both cases. In other words, the Euclidean distance is calculated for both the original brightness distribution and the distribution with the brightness levels reversed (for example, a distribution obtained by subtracting the original brightness value from 255), and similarity is determined if either distance is less than or equal to a predetermined value.

[0080] sqrt((v(1,1)-t(1,1)) 2 + (v(2,1)-t(2,1)) 2 + ···) Formula (1) Here, v(x,y) represents the pixel value at coordinate (x,y) in the normalized luminance distribution, and t(x,y) represents the pixel value at coordinate (x,y) in the normalized temperature distribution.

[0081] Alternatively, another method for evaluating similarity is to calculate Pearson's product-moment correlation coefficient, given by equation (2) below, and determine similarity if the absolute value of the correlation coefficient is greater than or equal to a predetermined value (the closer to 1, the more similar the results). Here, by calculating the absolute value of the correlation coefficient, similarity can be evaluated whether the relationship between high and low brightness and high and low temperature is the same or the opposite.

[0082] Σ(v(x,y) - v_ave) * (t(x,y) - t_ave) / sqrt(Σ(v(x,y) - v_ave) 2 ) / sqrt(Σ(t(x,y) - t_ave) 2 ) Formula (2) Here, v(x,y) represents the pixel value at coordinate (x,y) in the luminance distribution, v_ave represents the average value of the luminance distribution, t(x,y) represents the pixel value at coordinate (x,y) in the temperature distribution, and t_ave represents the average value of the temperature distribution.

[0083] Figure 5 shows images of a concrete structure taken during the daytime; Figure 5(A) is a visible light image, and Figure 5(B) is an infrared thermal image. Note that in Figure 5, rust stains are adhering to the surface of the concrete structure being inspected.

[0084] In this example, in the temperature deformation information derivation step (step S3), a temperature deformation with a higher surface temperature than the surroundings is determined and extracted from the infrared thermal image (Figure 5(B)). Rust stains are visible in the visible image (Figure 5(A)) of the area corresponding to the temperature deformation, and since the brightness distribution of the rust stains is similar to the temperature distribution of the temperature deformation (the relationship between high brightness and high temperature is inverse), it is estimated that the cause of the temperature deformation is this surface deformation (rust stains).

[0085] Figure 6 shows the shape of the deformation in the image of Figure 5 in binary, with pixels showing deformation represented by 1 (white) and pixels without deformation represented by 0 (black). Figure 6(A) shows the shape of surface deformation (rust stain) derived from the visible image (Figure 5(A)), and Figure 6(B) shows the shape of temperature deformation derived from the infrared thermal image (Figure 5(B)). Comparing Figure 6(A) and Figure 6(B), they are judged to be similar.

[0086] Figure 7 shows images of a concrete structure taken during the daytime; Figure 7(A) is a visible light image, and Figure 7(B) is an infrared thermal image. The surface of the concrete structure being inspected includes areas that have been repaired.

[0087] In this example, in the temperature deformation information derivation step (step S3), a temperature deformation with a higher surface temperature than the surrounding area is determined and extracted from the infrared thermal image (Figure 7(B)). Repair marks are visible in the visible image (Figure 7(A)) of the area corresponding to the temperature deformation, and since the brightness distribution of the repair marks is similar to the temperature distribution of the temperature deformation (the relationship between high brightness and high temperature is the same), it is estimated that the cause of the temperature deformation is this surface deformation (repair marks).

[0088] Figure 8 shows the shape of the deformation relative to the image in Figure 7 in binary, with pixels with deformation represented by 1 (white) and pixels without deformation represented by 0 (black), as described above. Figure 8(A) is the shape of the surface deformation (repair marks) derived from the visible image (Figure 7(A)), and Figure 8(B) is the shape of the temperature deformation derived from the infrared thermal image (Figure 7(B)). Comparing Figure 8(A) and Figure 8(B), they are judged to be similar.

[0089] In the infrared thermal image of the repaired area in Figure 7(B), it can be seen that the temperature gradient is steep at the boundary of the temperature deformation. Thus, temperature deformation caused by surface deformation is characterized by a steep temperature gradient at its boundary (however, this depends on the type of surface deformation). In the case of internal defects such as delamination, heat diffuses between the internal defect and the surface, so the temperature gradient at the boundary of the temperature deformation caused by the internal defect is gentle. The deeper the internal defect, that is, the wider the space between the internal defect and the surface, the more heat is diffused, and the gentler the temperature gradient becomes.

[0090] However, in the case of surface deformation, the difference in surface temperature between the surface deformation and its surroundings, resulting from differences in thermal absorptivity, thermal conductivity, and total infrared emissivity of the surface deformation, is directly captured. Therefore, the slope of this surface temperature difference is steeper than in the case of internal defects. Consequently, the cause of the temperature deformation may be estimated based on this characteristic (the characteristic of a steep temperature slope at the boundary of the temperature deformation).

[0091] Specifically, if the temperature distribution and luminance distribution are similar in the range corresponding to the temperature deformation derived in the temperature deformation information derivation step (step S3) or a slightly wider range, and / or if the temperature slope at the boundary of the temperature deformation is greater than or equal to a predetermined value (a pre-set threshold), then the cause of the temperature deformation may be estimated to be surface deformation.

[0092] As the temperature gradient at the boundary of a temperature change, for example, if the temperature distribution is f(x,y), the magnitude of the gradient vector grad f(x,y) = ∇f(x,y) = (∂f / ∂x, ∂f / ∂y) is sqrt((∂f / ∂x)). 2 + (∂f / ∂y) 2 You can calculate (3) and find the average of the maximum slopes at each point on the boundary. For example, you can calculate equation (3) at each point (x,y) on the boundary and find its average value.

[0093] sqrt((f(x+1,y)- f(x,y)) 2 + (f(x,y+1)- f(x,y)) 2 ) Formula (3) Furthermore, the temperature gradient at the boundary of a temperature deformation varies greatly depending on the thermal environment, such as solar radiation and ambient temperature. Therefore, the absolute value of the difference between the temperature deformation and the ambient temperature, or the absolute value of the difference between the temperature deformation and the average temperature of the structural surface in a wide predetermined range including the temperature deformation, can be calculated. The temperature gradient at the boundary can then be normalized (for example, by division) using the maximum or average value of these absolute difference values, and if the normalized temperature gradient is greater than or equal to a predetermined value, it can be estimated that the cause of the temperature deformation is surface deformation.

[0094] When evaluating the similarity between the temperature distribution and luminance distribution of temperature deformation, the characteristic of a steep temperature gradient at the boundary of the temperature deformation caused by surface deformation is also evaluated. However, this characteristic may be further evaluated as described above. Note that the above method is just one example of a method for evaluating the temperature gradient, and is not limited to this method. For example, the temperature gradient can also be indirectly evaluated using the spatial second derivative of temperature. Any method may be used to evaluate the temperature gradient.

[0095] This characteristic is particularly noticeable in "repair marks," "joints," and "level differences," and is useful for estimating the cause.

[0096] (Peeling) Figures 9 and 10 are images of concrete structures taken during the daytime. Figure 9(A) is a visible light image, and Figure 9(B) is an infrared thermal image. Similarly, Figure 10(A) is a visible light image, and Figure 10(B) is an infrared thermal image. Figures 9 and 10 include areas of peeling on the surface of the concrete structure being inspected.

[0097] As shown in the visible image of Figure 9(A), peeling occurs in the upper right area, and a step can be seen. Also, as shown in the visible image of Figure 10(A), there is a peeling area on the right side, and a smaller peeling area on the left side, and both have a step.

[0098] From the infrared thermal images in Figures 9(B) and 10(B), areas with high surface temperature (lighter colored areas in the infrared thermal images) can be seen adjacent to the delamination area. It is thought that air has entered from the delamination area and caused this area to float. Thus, in areas where the surface has delaminated, many areas where air has entered and caused the surface to float are observed.

[0099] Based on the temperature distribution derived in the temperature deformation information derivation step (step S3) and the brightness distribution derived in the surface deformation information derivation step (step S4), it is possible to determine delamination accompanied by peeling (or delamination accompanied by peeling) in this manner. Specifically, the similarity between the temperature distribution and the brightness distribution is evaluated for both the whole and parts of the temperature deformation, and if the whole is not similar but parts are similar, it is determined to be delamination accompanied by peeling.

[0100] The overall similarity of temperature variations is evaluated by extracting the range corresponding to the temperature variation from the luminance distribution and evaluating the similarity between the luminance distribution and temperature distribution within that range. The similarity of the parts of the temperature variation is evaluated by extracting predetermined ranges from the luminance distribution, based on the shape of the temperature variation, centered on each point on the boundary of the temperature variation, and evaluating the similarity between the luminance distribution and temperature distribution within each range. If similarity is found in any part, it is determined to be delamination accompanied by detachment. This evaluation can be used because the boundary (step) of the delaminating area in the luminance distribution matches the shape of the boundary of the temperature variation in the temperature distribution. As explained earlier, the similarity evaluation is performed to cover both cases where the relationship between the high and low luminance in the luminance distribution and the high and low temperatures in the temperature distribution is the same and the opposite.

[0101] The infrared thermal images in Figures 9 and 10 show that the surface temperature at the delamination site is lower than the surrounding area. When the same area is photographed in infrared at night, the surface temperature at the delamination site is higher than the surrounding area. The reason why the surface temperature of the delamination site differs from the surrounding area is thought to be because the delamination site is located deeper than the surrounding area. Because the surface temperature of the delamination site differs from the surrounding area, it is identified as a temperature variation and extracted. Temperature variations caused by delamination can be distinguished from internal defects such as budding and other surface variations because the temperature relationship with the surrounding area is different. Furthermore, since the areas of delamination in the brightness distribution and the areas of temperature variation in the temperature distribution coincide, the overall temperature distribution and brightness distribution of the temperature variation are clearly similar. Therefore, because delamination can be distinguished from other surface variations by its characteristic brightness distribution accompanied by dark areas due to steps, i.e., areas of low brightness, it can be easily determined that the cause of this temperature variation is delamination. Note that in the case of budding accompanied by delamination, the temperature variation and delamination are adjacent but in different locations.

[0102] As can be seen from Figures 9 and 10, in the case of delamination accompanied by lifting, the temperature gradient at the boundary of the temperature deformation is steep, just as in the case of other surface deformations, and this characteristic can be used to estimate the cause. However, in the case of delamination accompanied by lifting, the temperature gradient is steep only in the part of the temperature deformation boundary adjacent to the delamination, while the temperature gradient in other parts is gentle, as in normal lifting. Therefore, in the case of delamination accompanied by lifting, the temperature gradient is gentle overall along the boundary of the temperature deformation, but it can be determined that lifting accompanied by lifting is caused by delamination if the temperature gradient is steep at a part of the boundary.

[0103] Specifically, the similarity between the temperature distribution and the luminance distribution is evaluated for the whole and part of the temperature deformation, based on the temperature distribution derived in the temperature deformation information derivation step (step S3) and the luminance distribution derived in the surface deformation information derivation step (step S4). If the whole is not similar but the parts are similar, and the slope of the temperature distribution is evaluated for the whole and part of the boundary of the temperature deformation, and the overall slope is less than or equal to a predetermined value and the slope in the part is greater than or equal to a predetermined value, the cause of the temperature deformation may be estimated to be delamination accompanied by peeling. The overall temperature slope of the boundary of the temperature deformation can be calculated as in the case of other surface deformations by finding the maximum slope, i.e., the magnitude of the gradient vector of the temperature distribution, at each point on the boundary of the temperature deformation and calculating its average value (the average value of the maximum slopes of all points on the boundary of the temperature deformation). The temperature slope in the part of the boundary of the temperature deformation can be calculated by extracting a predetermined range centered on each point on the boundary and calculating the average value of the maximum slopes of all points on the boundary included in the extracted predetermined range.

[0104] Here, in the temperature distribution of the temperature variation, the parts that are similar to the luminance distribution and the parts where the temperature gradient at the boundary is steep are the same. In other words, in the temperature distribution of the temperature variation, in the parts adjacent to the delamination, the temperature distribution and the luminance distribution are similar, and the temperature gradient at the boundary is steep. Therefore, it is preferable to simultaneously evaluate the similarity with the luminance distribution and the temperature gradient for each part of the temperature variation, and if there are parts that are similar to the luminance distribution and have a steep temperature gradient, to estimate that the cause of the temperature variation is delamination accompanied by lifting.

[0105] In addition, in the case of delamination accompanied by separation, the temperature gradient may be normalized using the absolute difference between the temperature change and the ambient temperature, or the absolute difference between the temperature change and the average temperature of a wider area of ​​the structure's surface, as in the case of other surface deformations. deformation Similar to the previous case, the evaluation of the similarity with the luminance distribution includes the feature of a steep temperature gradient at the boundary, but the temperature gradient may also be evaluated in the manner described above.

[0106] Figure 11 shows the shape of the deformation relative to the image in Figure 9 in binary, and Figure 12 shows the shape of the deformation relative to the image in Figure 10 in binary. Pixels with deformation are represented by 1 (white), and pixels without deformation are represented by 0 (black). Figure 11(A) shows the shape of surface deformation (peeling) derived from the visible image (Figure 9(A)), and Figure 11(B) shows the shape of temperature deformation derived from the infrared thermal image (Figure 9(B)).

[0107] As shown in Figure 11, a comparison of Figure 11(A) and Figure 11(B) reveals that they are not similar overall. However, the upper right portion of the temperature deformation is adjacent to the delamination, and the shape of the boundary between the temperature deformation and the delamination is similar in the portion adjacent to the delamination. Therefore, it can be inferred that this is a delamination accompanied by lifting.

[0108] Figure 12(A) shows the shape of surface deformation (peeling) derived from the visible image (Figure 10(A)), and Figure 12(B) shows the shape of temperature deformation derived from the infrared thermal image (Figure 10(B)).

[0109] As shown in Figure 12, comparing Figure 12(A) and Figure 12(B), in addition to the delamination on the right, there is also a small delamination on the left, both adjacent to the temperature deformation, and since the shape is similar to the temperature deformation in the adjacent parts, it can be inferred that both temperature deformations are lifting accompanied by delamination.

[0110] (crack) Figures 13 and 14 are images of concrete structures taken during the daytime. Figure 13(A) is a visible light image, and Figure 13(B) is an infrared thermal image. Similarly, Figure 14(A) is a visible light image, and Figure 14(B) is an infrared thermal image. Figures 13 and 14 include areas with cracks on the surface of the concrete structures being inspected.

[0111] From the visible image in Figure 13(A), it can be seen that there are cracks running horizontally, and from the infrared thermal image in Figure 13(B), it can be seen that the surface temperature is high above the cracks (the lighter colored area in the infrared thermal image). This area with a high surface temperature is in a floating state. As shown in Figure 13, many floating areas are accompanied by cracks, and often there are cracks along the boundary of the floating area. Also, as shown in Figure 14, there are cases where cracks are present within the area of ​​floating.

[0112] From the temperature distribution derived in the temperature deformation information derivation step (step S3) and the brightness distribution derived in the surface deformation information derivation step (step S4), it is possible to determine whether there is delamination accompanied by cracks (or cracks accompanied by delamination). First, known edge detection is performed on the temperature distribution to extract the boundary of the temperature deformation. Various known methods such as the Sobel method, Laplacian method, and Canny method can be used for edge detection. Next, based on the shape of the temperature deformation, the brightness distribution in a predetermined range centered on each point on the boundary of the temperature deformation and the distribution extracted from the boundary of the temperature deformation are extracted, and the similarity is evaluated. If there is similarity in any part of the boundary, it is determined that there is a crack along the boundary in that part, that is, delamination accompanied by cracks.

[0113] As shown in Figure 14, the presence of cracks within the buoyancy range can also be determined as follows. As can be seen from Figure 14, the temperature changes abruptly in the cracked area within the buoyancy range. Therefore, in the temperature distribution where edge detection is performed, there are linear parts extracted as edges not only at the boundaries of the temperature changes but also within the temperature changes. If the shape of these linear parts matches the shape of the cracks in the brightness distribution, it can be determined that the buoyancy is accompanied by cracks.

[0114] Specifically, based on the shape of the temperature deformation, the luminance distribution within a predetermined range centered on each point inside the temperature deformation and the distribution extracted from the edges of the temperature deformation are extracted, and the similarity is evaluated. If there is similarity in at least a portion of the inside of the temperature deformation, it is determined that there is a crack inside the temperature deformation, i.e., a delamination accompanied by a crack. The evaluation of similarity is based on the relative height of the values ​​of the two distributions being evaluated, as described above. Relationship The process should cover both cases where the values ​​are the same and cases where they are reversed. Alternatively, the similarity may be evaluated by considering that the luminance of the cracks is lower than that of the surrounding area in the luminance distribution.

[0115] In cases of delamination accompanied by cracks, similar to delamination, the temperature gradient at some boundaries is steeper compared to temperature changes caused by internal defects such as delamination. That is, the temperature gradient is steep in the cracked areas along the boundary of the temperature change. The temperature gradient is also steep in the cracked areas within the temperature change. Therefore, this characteristic can also be used to determine delamination accompanied by cracks. For example, edge detection is performed on the temperature distribution, and after extracting areas where the temperature change is steep at the boundary and inside of the temperature change, areas where the edge size is greater than or equal to a predetermined value, i.e., areas where the temperature gradient is steeper than or equal to a predetermined value, are extracted. If there are no areas with a temperature gradient greater than or equal to a predetermined value at the boundary and inside of the temperature change, the cause of the temperature change is estimated to be something other than delamination accompanied by cracks, and is estimated to be an internal defect such as delamination. If there are areas where the temperature gradient exceeds a predetermined value, the similarity between each of these areas and the luminance distribution may be evaluated. If there is a similarity in any of these areas (for example, if there is a linear area with a steep temperature gradient, and the shape of that linear area matches the shape of the cracks in the luminance distribution), the cause of the temperature deformation may be estimated to be delamination accompanied by cracks.

[0116] In cases of delamination accompanied by cracks, the temperature gradient may be normalized by the absolute difference between the temperature change and the ambient temperature, or by the absolute difference between the temperature change and the average temperature of a wider area of ​​the structure's surface, similar to cases of delamination accompanied by other surface deformations or peeling. In evaluating the similarity between the edge-detected temperature distribution and the brightness distribution, the characteristic of a steep temperature gradient in the cracked area is also included in the evaluation, but the temperature gradient may be further evaluated as described above.

[0117] As explained above, visible images typically consist of RGB images. While the cause can be estimated from the luminance distribution of any of the RGB channels, it is preferable to evaluate the magnitude of the luminance variation in the luminance distribution of each RGB channel and estimate the cause from the channel with the largest variation. The same applies when there are two or more types of visible images.

[0118] Figure 15 shows the shape of the deformation in the image of Figure 13 in binary, and Figure 16 shows the shape of the deformation in the image of Figure 14 in binary. Pixels with deformation are represented by 1 (white), and pixels without deformation are represented by 0 (black).

[0119] Figure 15(A) shows the shape of surface deformation (cracks) derived from the visible image (Figure 13(A)), and Figure 15(B) shows the shape of the boundary of the temperature deformation derived from the infrared thermal image (Figure 13(B)). As shown in Figure 15, the shape of the surface deformation (cracks) in Figure 15(A) and the shape of the boundary of the temperature deformation in Figure 15(B) are at least partially similar, so it can be inferred that this is delamination accompanied by cracks.

[0120] Figure 16(A) shows the shape of surface deformation (cracks) derived from the visible image (Figure 14(A)), and Figure 16(B) shows the shape of the edges (parts where the temperature changes abruptly) inside the temperature deformation derived from the infrared thermal image (Figure 14(B)). Note that in Figure 16, the boundaries of the temperature deformation are also extracted as edges, but the boundaries are omitted. Since the shape of the surface deformation (cracks) in Figure 16(A) and the shape of the edges inside the temperature deformation in Figure 16(B) are at least partially similar, it can be inferred that this is delamination accompanied by cracks.

[0121] This section explains the flow for estimating the causes of cracks, delamination, and other surface defects. While the order in which the causes are estimated for cracks, delamination, and other surface defects is not important, the flow for estimating the causes in the order of other surface defects, delamination, and cracks is explained as an example.

[0122] First, the range corresponding to the temperature deformation derived in the temperature deformation information derivation step (step S3) is extracted from the luminance distribution derived in the surface deformation information derivation step (step S4), and the similarity between the luminance distribution and temperature distribution in this range is evaluated. At this time, the temperature slope at the boundary of the temperature deformation may also be evaluated.

[0123] If similar conditions are observed (and / or if the temperature gradient at the boundary is steep), the cause of the temperature deformation is presumed to be a surface deformation. Furthermore, it may be possible to determine whether the surface deformation is delamination or another type of surface deformation based on the relationship between the temperature deformation and the surrounding area.

[0124] If they are not similar (and / or the temperature gradient at the boundary is gentle), proceed to the next step.

[0125] Next, in the luminance distribution derived in the surface deformation information derivation step (step S4), predetermined ranges are extracted centered on each point on the boundary of the temperature deformation derived in the temperature deformation information derivation step (step S3), and the similarity between the luminance distribution and the temperature distribution in each range is evaluated. At this time, the temperature slope of the boundary of the temperature deformation in each range may also be evaluated.

[0126] If similarities are observed in any part (and the temperature gradient at the boundary is steep), the cause of the temperature change is presumed to be delamination accompanied by peeling. If no similarities are observed in any part (or the temperature gradient at the boundary is gentle), proceed to the next step.

[0127] Next, edge detection is performed on the temperature distribution of the temperature deformation derived in the temperature deformation information derivation step (step S3). In the luminance distribution derived in the surface deformation information derivation step (step S4), predetermined ranges are extracted centered on each point on and inside the boundary of the temperature deformation derived in the temperature deformation information derivation step (step S3), and the similarity between the luminance distribution of each range and the temperature distribution detected by edge detection is evaluated. At this time, only the portion where the edge size is greater than or equal to a predetermined value may be extracted from the temperature distribution detected by edge detection.

[0128] If similarities exist in any part, the cause of the temperature change is presumed to be delamination accompanied by cracking. If they are not similar, delamination is presumed to be the cause.

[0129] From the brightness distribution of areas corresponding to temperature changes, cracks, delamination, or Regarding other surface damages of It is also possible to estimate the cause after explicitly deriving the presence, type, shape, location, and brightness distribution of surface deformation. However, since this method is the same as the method of estimating the cause after explicitly deriving the presence, type, etc., of surface deformation in the surface deformation information derivation step (step S4), the explanation will be omitted.

[0130] <Second aspect> As a second embodiment, we will explain the case in which, in the surface deformation information derivation step (step S4), the presence, type, shape, location, and brightness distribution of surface deformation are explicitly derived from the brightness distribution of the area corresponding to the temperature deformation, and the type of other surface deformation is not determined. We will explain the method for estimating the cause of cracks and delamination, and other surface deformations.

[0131] (Other surface deformations) If there are no other surface deformations, the cause of the temperature deformation is presumed to be something other than this surface deformation, for example, an internal defect such as delamination. If there are other surface deformations, the similarity between the brightness distribution and temperature distribution of the surface deformation is evaluated using the same method as described in the first embodiment. If they are similar, the cause of the temperature deformation is presumed to be this surface deformation; if they are not similar, the cause is presumed to be something other than this surface deformation, for example, an internal defect such as delamination. However, since the shape of the other surface deformations has already been derived, the similarity may be evaluated within a range that includes both the range of this surface deformation and the range corresponding to the temperature deformation derived in the temperature deformation information derivation step (step S3). Alternatively, instead of evaluating the temperature distribution and brightness distribution, the similarity may be evaluated based on the shape, for example, between a distribution in which pixels with temperature deformations are binarized to 1 and pixels without temperature deformations to 0, and a distribution in which pixels with other surface deformations are binarized to 1 and pixels without temperature deformations to 0.

[0132] As explained in the first embodiment, temperature deformation caused by other surface deformations is characterized by a steeper temperature gradient at its boundary compared to temperature deformation caused by internal defects. Therefore, the cause of the temperature deformation may be estimated based on this characteristic. The method for doing so was explained in the first embodiment, so that explanation will be omitted here.

[0133] (Peeling) If there is no delamination, the cause of the temperature change is presumed to be an internal defect that does not involve delamination, such as a non-delamination-related buoyancy. If there is delamination, the location and shape of the temperature change derived in the temperature change information derivation step (step S3) are compared with the location and shape of the delamination. If the locations of the temperature change and the delamination are different, and the shapes of the adjacent boundaries are partially similar, the cause of the temperature change is presumed to be buoyancy accompanied by delamination.

[0134] For example, regarding the distribution obtained by binarizing pixels with temperature deformation to 1 and pixels without to 0, and the distribution obtained by binarizing pixels with delamination to 1 and pixels without to 0, a predetermined range centered on each point on the boundary of the temperature deformation is extracted based on the shape of the temperature deformation, and the similarity between the temperature deformation shape distribution (distribution obtained by binarizing pixels with temperature deformation to 1 and pixels without to 0) and the delamination shape distribution (distribution obtained by binarizing pixels with delamination to 1 and pixels without to 0) in each range is evaluated, and if they are similar in any part, it is estimated to be delamination accompanied by lifting. Alternatively, if the locations of the temperature deformation and delamination are different and adjacent, and the temperature distribution of the temperature deformation and the brightness distribution of the delamination are similar in a part of the boundary of the temperature deformation, for example, the temperature distribution and brightness distribution of a predetermined range centered on each point on the boundary of the temperature deformation are extracted based on the shape of the temperature deformation, the similarity between the temperature distribution and brightness distribution of each range is evaluated, and if they are similar in any part, it is estimated to be delamination accompanied by lifting. Furthermore, if the delamination is outside the range of the temperature deformation magnitude derived in the temperature deformation information derivation step (step S3), and the boundary of the delamination is located near the boundary of the temperature deformation magnitude, it may be estimated that the delamination is accompanied by lifting.

[0135] As described in the first embodiment, temperature changes caused by delamination have a steep temperature gradient in the portion adjacent to the delamination within the boundary of the temperature change. Therefore, the similarity between the shape or temperature distribution of the temperature change and the shape or brightness distribution of the delamination can be evaluated within a predetermined range centered on each point on the boundary of the temperature change, and the temperature gradient of the boundary of the temperature change within the predetermined range can be evaluated. If the shape or temperature distribution of the temperature change and the shape or brightness distribution of the delamination are similar in any portion, and / or if there is a portion with a steep temperature gradient, it can be estimated that there is delamination. Alternatively, if the delamination is outside the range of the temperature change and the boundary of the delamination is located near the boundary of the range of the temperature change, and the temperature gradient of the temperature change is steep near that boundary, it can be estimated that there is delamination.

[0136] Furthermore, as explained in the first embodiment, the temperature gradient may be normalized.

[0137] (crack) If there are no cracks, the cause of the temperature deformation is presumed to be an internal defect that does not involve cracks, such as delamination without cracks. If there are cracks, the similarity between the shape of the temperature deformation derived in the temperature deformation information derivation step (step S3), i.e., the shape of the boundary, and the shape of the cracks is evaluated. If there is a similarity in any part of the boundary of the temperature deformation, it is presumed that there is a crack along the boundary in that part, i.e., delamination accompanied by cracks. The evaluation of similarity is performed, for example, as follows: A predetermined range is extracted centered on each point on the boundary of the temperature deformation for a distribution in which pixels at the boundary of the temperature deformation are set to 1 and other pixels to 0, and a distribution in which pixels with cracks are set to 1 and pixels without cracks are set to 0. Then, the similarity is evaluated between the temperature deformation boundary shape distribution in each range, i.e., the distribution in which pixels at the boundary of the temperature deformation are set to 1 and other pixels to 0, and the crack shape distribution, i.e., the distribution in which pixels with cracks are set to 1 and pixels without cracks are set to 0. Alternatively, the similarity between the temperature distribution obtained by edge detection of the temperature deformation and the brightness distribution of the crack may be evaluated, and if there is a similarity in any part of the boundary of the temperature deformation, it may be estimated that there is a crack along the boundary in that part, i.e., a delamination accompanied by a crack. In addition, if there is a crack inside the temperature deformation derived in the temperature deformation information derivation step (step S3), it may be estimated that there is a delamination accompanied by a crack. Alternatively, if the crack is near or inside the boundary of the size of the temperature deformation derived in the temperature deformation information derivation step (step S3), it may be estimated that there is a delamination accompanied by a crack.

[0138] As described in the first embodiment, temperature deformation caused by cracking has a steep temperature gradient in the cracked portion along the boundary of the temperature deformation, and in the cracked portion within the temperature deformation. Therefore, the similarity between the distribution obtained by edge detection of the shape or temperature distribution of the temperature deformation and the shape or brightness distribution of the crack is evaluated within a predetermined range centered on each point on the boundary of the temperature deformation, and the temperature gradient of the boundary of the temperature deformation within the predetermined range, for example, the size of the detected edge is evaluated, and if the distribution obtained by edge detection of the shape or temperature distribution of the temperature deformation and the shape or brightness distribution of the crack are similar in any portion, and / or if there is a portion with a steep temperature gradient, it may be estimated that there is cracking. Furthermore, if there is a crack inside the temperature deformation, the temperature gradient of the temperature distribution in the cracked portion is evaluated, and if the temperature gradient is steep in the cracked portion, it may be estimated that there is cracking. In this process, the similarity between the temperature distribution detected by edge detection and the shape or brightness distribution of the crack may also be evaluated within a predetermined range including the crack. If the distributions are similar and / or if the temperature gradient is steep, it may be estimated that the crack is causing delamination.

[0139] Furthermore, as explained in the first embodiment, the temperature gradient may be normalized.

[0140] This section explains the flow for estimating the causes of cracks, delamination, and other surface defects. While the order in which the causes are estimated for cracks, delamination, and other surface defects is not important, the flow for estimating the causes in the order of other surface defects, delamination, and cracks is explained as an example.

[0141] First, the presence or absence of other surface deformations derived in the surface deformation information derivation step (step S4) is checked, and if present, the similarity between the surface deformations and the temperature deformations derived in the temperature deformation information derivation step (step S3) is evaluated. At this time, the temperature gradient of the boundary of the temperature deformation may also be evaluated.

[0142] If there are no other surface abnormalities, or if they are not similar (and / or the temperature gradient at the boundary is gentle), proceed to the next step.

[0143] If similar conditions are observed (and / or if the temperature gradient at the boundary is steep), the cause of the temperature deformation is presumed to be another surface deformation.

[0144] Next, the presence or absence of delamination, which was derived in the surface deformation information derivation step (step S4), and, if present, the partial similarity and positional relationship between the delamination and the temperature deformation, which was derived in the temperature deformation information derivation step (step S3), are evaluated. At this time, the temperature gradient of the boundary of the temperature deformation in each part may also be evaluated.

[0145] If there is no delamination, or if no similarities exist in any part (and / or the temperature gradient at the boundary is gentle), proceed to the next step.

[0146] If similarities are observed in any part (and / or if the temperature gradient at the boundary is steep), the cause of the temperature change is presumed to be delamination accompanied by separation.

[0147] Next, the presence and location of cracks derived in the surface deformation information derivation step (step S4) are evaluated (evaluating the presence or absence of cracks near the boundary of the temperature deformation and within the surface). If cracks are present, the partial similarity between the cracks and the boundary and interior of the temperature deformation derived in the temperature deformation information derivation step (step S3) is evaluated. At this time, the temperature gradient of the temperature deformation in each part may also be evaluated.

[0148] If there are no cracks near the temperature deformation boundary, or if there are cracks near the temperature deformation boundary but none of the parts of the temperature deformation boundary resemble the cracks (and / or the temperature gradient of the boundary is gentle), and if there are no cracks inside the temperature deformation (or if there are cracks inside the temperature deformation but the temperature distribution in the cracked area does not resemble the distribution detected by edge detection, and / or the temperature gradient in the cracked area is gentle), then the cause of the temperature deformation is presumed to be delamination.

[0149] If a crack is located near the temperature deformation boundary and resembles the crack in any part of the temperature deformation boundary (and / or the temperature gradient at the boundary is steep), or if the crack is located inside the temperature deformation (and the temperature distribution in the cracked area resembles the distribution detected by edge detection, and / or the temperature gradient in the cracked area is steep), then the cause of the temperature deformation is presumed to be delamination accompanied by cracking.

[0150] <Third aspect> As a third embodiment, we will describe a case in which, in the surface deformation information derivation step (step S4), the presence, type, shape, location, and brightness distribution of surface deformation are explicitly derived from the brightness distribution of the area corresponding to the temperature deformation, and further, the type of other surface deformation is determined.

[0151] The method for estimating the causes of cracks and delamination is the same as in the second embodiment, with only the method for estimating the causes of other surface deformations differing from the second embodiment.

[0152] As mentioned above, the information useful for estimating the cause differs depending on the type of surface deformation, as follows:

[0153] For surface deformations such as discoloration, rust stains, or rust, where the difference in visible light absorption compared to concrete is the main cause of surface temperature differences, the luminance distribution is effective as surface deformation information for estimating the cause.

[0154] Surface deformations where differences in thermal conductivity or infrared emissivity from concrete are the main cause of surface temperature differences, such as repair marks and free lime, and surface deformations that cause surface temperature differences due to structural reasons, such as joints and steps, are effective in estimating the cause as surface deformation information.

[0155] For example, while the luminance distribution is effective for estimating the cause of rust stains in Figure 5, it can be seen that for repair marks in Figure 7, the luminance distribution on the repair mark surface becomes noise, making shape alone more effective. Therefore, in the third embodiment, the cause of other surface deformations is basically estimated using the same method as in the second embodiment, but the surface deformation information used for evaluating similarity is differentiated depending on the type of surface deformation, specifically whether to use luminance distribution or shape.

[0156] As explained in the first embodiment, temperature changes caused by surface deformation have a steeper temperature gradient at the boundary compared to temperature changes caused by internal defects. However, there are surface deformations where the temperature gradient at the boundary is not necessarily steep, such as the rust stain shown in Figure 5. Therefore, it is preferable to switch whether or not to evaluate the temperature gradient at the boundary of the temperature deformation depending on the type of surface deformation, and to not evaluate the temperature gradient for surface deformations where the temperature gradient at the boundary is not necessarily steep, such as the rust stain.

[0157] <Estimated Result Display Step> The information display unit 61 displays the estimated cause of the temperature change on the display device 30 via the display control unit 26 (estimated result display step: step S6).

[0158] For each temperature deformation determined and extracted in the temperature deformation information derivation step (step S3), the cause estimated in the cause estimation step (step S5) is displayed. It may be displayed near each temperature deformation on the infrared thermal image, or it may be displayed on the visible image along with the location and shape of the temperature deformation. It may be displayed on any image processed from the infrared thermal image and / or any other image. In the third embodiment (when other types of surface deformation are determined), if the cause estimated in the cause estimation step (step S5) is "other surface deformation", it is preferable to display its type (color unevenness, joints, steps, rust stains, etc.).

[0159] Figure 17 shows an example of the display result when the estimation results are displayed on the display device 30. Figure 17 displays the infrared thermal image corresponding to Figure 10(B). Furthermore, the areas that were determined to be temperature deformations and extracted in the temperature deformation information derivation step (step S3) are enclosed in white frames and displayed, and the cause estimated in the cause estimation step (step S5), "lifting accompanied by delamination," is displayed in white letters near the white frames.

[0160] Figure 17 shows only a portion of the infrared thermal image, but it is also possible to display the entire captured infrared thermal image and show the cause for each area identified as having temperature changes, or to display a wider area.

[0161] The example in Figure 17 is merely one example; to make the cause "lifting accompanied by delamination" easier to see, the background of the letters could be made white, and there are countless variations in how it can be presented; it is not particularly limited.

[0162] <Other forms> Differences in the surface temperature of a concrete structure caused by internal factors such as delamination can be masked by differences in surface temperature due to surface deformation, making it impossible to distinguish the temperature differences caused by internal factors. For example, when a section of concrete surface that has delaminated is repaired by filling it with a repair material, that repair material may detach from the original concrete structure after several years. It is desirable to be able to distinguish the temperature difference caused by this detachment, but this can be masked by the temperature difference caused by the repair material itself having different thermal conductivity and infrared emissivity than the surrounding concrete, making it impossible to distinguish.

[0163] Even in such cases, in order to distinguish temperature differences caused by internal structural factors, if the cause of the temperature deformation is estimated to be another surface deformation, the temperature distribution due to the surface deformation may be estimated based on the temperature deformation information and the surface deformation information, and reduced from the original temperature distribution. There are various methods for doing this.

[0164] For example, a predetermined blurring process may be applied to the luminance distribution of surface deformation, which is surface deformation information (the temperature distribution due to surface deformation is blurred compared to the luminance distribution due to factors such as heat conduction), the contrast of the blurred luminance distribution may be adjusted to optimize it to best match the original temperature distribution, and then the contrast-optimized luminance distribution may be estimated to be the temperature distribution due to surface deformation and subtracted from or divided from the original temperature distribution.

[0165] However, this method, which estimates the temperature distribution from the brightness distribution of surface deformation, is not suitable for surface deformations where the brightness distribution and temperature distribution are not similar. In other words, as mentioned above, this method is suitable for surface deformations where the difference in the amount of visible light absorbed by the concrete is the main cause of the temperature difference, such as color unevenness, rust stains, or rust.

[0166] On the other hand, this method is not suitable for surface deformations where the main cause of temperature differences is differences in thermal conductivity and infrared emissivity from concrete, such as repair marks and free lime, or for surface deformations where temperature differences occur due to structural causes, such as joints and steps. For such surface deformations, it is preferable to estimate the temperature distribution without using the brightness distribution of the surface deformation. For example, depending on the type of surface deformation, various thermal parameters related to the surface deformation may be set, and a thermal simulation (including heat conduction, thermal radiation, and convection simulations) may be performed based on these parameters and the shape of the surface deformation to simulate the temperature distribution. The simulation temperature distribution that best matches the original temperature distribution may then be estimated as the temperature distribution due to the surface deformation and subtracted from the original temperature distribution.

[0167] For surface deformations where thermal conductivity and infrared emissivity are the main causes of temperature differences, such as repair marks and free lime, thermal conductivity and infrared emissivity are set as particularly important parameters. For surface deformations where temperature differences occur due to structural causes such as joints and steps, structural parameters such as the depth and height of joint irregularities and the height of steps are set as particularly important parameters.

[0168] Alternatively, the original temperature distribution may be approximated by a lower-order mathematical formula, such as a linear (plane) or quadratic (surface) polynomial, and the temperature distribution that best fits the original temperature distribution may be estimated as the temperature distribution due to surface deformation and subtracted from the original temperature distribution.

[0169] In the above-described embodiment, a rule-based method was explained for each of the following steps: temperature deformation determination and temperature deformation information derivation in the temperature deformation information derivation step (step S3), surface deformation information derivation in the surface deformation information derivation step (step S4), and cause estimation in the cause estimation step (step S5).

[0170] However, the process of deriving temperature deformation information (step S3), deriving surface deformation information (step S4), and estimating the cause (step S5) can also be carried out using various machine learning methods.

[0171] For example, in the temperature deformation information derivation step (step S3), machine learning methods that detect objects from images and extract their regions can be used as a method to determine temperature deformation from infrared thermal images and derive information including their shape. These methods include FCN (Fully Convolutional Network), SegNet (A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation), and U-Net (Convolutional Networks for Biomedical Image Segmentation).

[0172] Furthermore, the same machine learning technique can be used to derive surface deformation information from visible images in the surface deformation information derivation step (step S4). In the case of visible images, if there are multiple types of images, such as RGB images, all types of visible images can be used as input.

[0173] In the step for deriving temperature change information (step S3), if the only task is to determine the temperature change and derive its magnitude, machine learning methods such as R-CNN (Regions with CNN (Convolutional Neural Network) features), Fast R-CNN, YOLO (You Only Look Once), and SSD (Single Shot MultiBox Detector) can also be used.

[0174] In the surface deformation information derivation step (step S4), if the surface deformation is first determined and features (such as average brightness, shape, texture of the brightness distribution, contrast, and frequency spectrum) are extracted from its brightness distribution, and if the type of surface deformation (other surface deformation, peeling, cracks, and other types of surface deformation such as color unevenness, joints, steps, rust stains, etc.) is to be determined from these features, machine learning methods that classify objects from features can be used as the method of determination. These methods include logistic regression, linear discriminant analysis, K-nearest neighbors, decision trees (classification trees), random forests, and support vector machines (SVM).

[0175] In the cause estimation step (step S5), the method for estimating the cause of temperature deformation from temperature deformation information and surface deformation information can also be carried out using various machine learning techniques depending on the form of the temperature deformation information and surface deformation information. For example, if the input is the temperature distribution of the temperature deformation as temperature deformation information and the brightness distribution of the area corresponding to the temperature deformation as surface deformation information, the cause of the temperature deformation can be classified into "other surface deformations (if other types of surface deformations are also to be classified, such as "color unevenness," "joints," "steps," "rust stains," etc.)", "lifting accompanied by delamination", "lifting accompanied by cracks", "lifting", etc.) using machine learning techniques such as DNN (Deep Neural Network) and CNN.

[0176] The steps of deriving temperature deformation information (step S3), deriving surface deformation information (step S4), and estimating the cause (step S5) can all be performed together using a single machine learning method. In other words, using infrared thermal images and visible images (all types of visible images if there are multiple types such as RGB) as input, the temperature deformation can be detected and its cause classified using the machine learning methods mentioned earlier, such as FCN, SegNet, U-Net, R-CNN, Fast R-CNN, YOLO, and SSD. It is preferable to have abundant training data when performing machine learning.

[0177] In the above-described embodiment, an example was explained in which the causes of other surface deformations, delamination, and cracks were estimated sequentially, and one cause was estimated as the cause of the temperature deformation. Multiple candidates for the cause of the temperature deformation may be estimated. That is, the causes of other surface deformations, delamination, and cracks may be estimated, and multiple candidates such as other surface deformations, delamination accompanied by budding, and cracking accompanied by budding may be estimated as the cause of the temperature deformation.

[0178] Furthermore, instead of estimating the cause as a binary choice, it may be estimated probabilistically. For example, for each of the other surface deformations, delaminations, and cracks, if they are not located in the area corresponding to the temperature deformation in the visible image, the probability that they are the cause of the temperature deformation is set to 0%. If they are present, the probability that they are the cause of the temperature deformation can be determined by using a calculated value (for example, the Euclidean distance or Pearson's probability correlation coefficient mentioned above) of the similarity between the brightness distribution and / or shape of the deformation and the temperature distribution and / or shape (or the distribution obtained by edge detection), converted into a probability.

[0179] Alternatively, values ​​obtained by converting the calculated temperature gradients at the boundaries and within the temperature deformation into probabilities may also be adopted. In such embodiments, the estimation result display step (step S6) displays multiple candidates estimated as causes for each temperature deformation determined and extracted in the temperature deformation information derivation step (step S3), as well as their probabilities.

[0180] The program of the above embodiment may be implemented using a dedicated analysis program, and the device used for implementation is not limited. For example, it can be implemented on a personal computer. Furthermore, the device or program that implements each step may be integrated or separate.

[0181] <Other> In the above embodiment, the hardware structure of the processing unit that performs various processes is a variety of processors as shown below. These various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as a processing unit; a Programmable Logic Device (PLD), such as an FPGA (Field Programmable Gate Array), which is a processor whose circuit configuration can be changed after manufacturing; and a dedicated electrical circuit, such as an ASIC (Application Specific Integrated Circuit), which has a circuit configuration specifically designed to perform a particular process.

[0182] A single processing unit may be composed of one of these various processors, or it may be composed of two or more processors of the same or different type (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Furthermore, multiple processing units can be composed of a single processor. Examples of composing multiple processing units with a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as multiple processing units, as is typical of computers such as client and server systems. Secondly, a configuration using a processor that realizes the functions of the entire system, including multiple processing units, on a single IC (Integrated Circuit) chip, as is typical of System-on-a-Chip (SoC) systems. Thus, various processing units are configured, in terms of hardware structure, using one or more of the above-mentioned various processors.

[0183] Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit composed of circuit elements such as semiconductor devices.

[0184] Each of the above-described configurations and functions can be appropriately implemented using any hardware, software, or a combination thereof. For example, the present invention can also be applied to a program that causes a computer to execute the above-described processing steps (processing procedures), a computer-readable recording medium (non-temporary recording medium) that records such a program, or a computer on which such a program can be installed.

[0185] Although examples of the present invention have been described above, it goes without saying that the present invention is not limited to the embodiments described above, and various modifications are possible without departing from the spirit of the invention. [Explanation of symbols]

[0186] 10 Image analysis device 12 Input / Output Interfaces 16 Memory section 18 Control section 20 CPU 22 RAM 24 ROM 26 Display Control Unit 30 Display device 32 Infrared Cameras 34 Visible Cameras 36 Structures 51 Infrared thermal image acquisition unit 53 Visible Image Acquisition Unit 55 Temperature change information derivation part 57 Surface deformation information derivation unit 59 Cause estimation part 61 Information display section 101 Infrared thermal imaging 102 Temperature change information 103 Visible Images 104 Surface deformation information S1 Step S2 Step S3 Step S4 Step S5 Step S6 Step

Claims

1. An image analysis device equipped with a processor, The aforementioned processor, We acquire infrared thermal images of the structure to be inspected, A visible image of the aforementioned structure to be inspected is obtained, Temperature changes are determined from the aforementioned infrared thermal image, Information on surface deformation is derived from the aforementioned visible image, Based at least the information on temperature changes obtained from the infrared thermal image and the information on surface changes obtained from the visible image, the cause of the temperature change is estimated. If the cause of the temperature change is estimated to be the surface deformation, the temperature distribution due to the surface deformation is estimated and reduced from the infrared thermal image. Image analysis device.

2. An image analysis device equipped with a processor, The aforementioned processor, We acquire infrared thermal images of the structure to be inspected, A visible image of the aforementioned structure to be inspected is obtained, Temperature changes are determined from the aforementioned infrared thermal image, Information on surface deformation is derived from the aforementioned visible image, The surface deformation corresponding to the temperature deformation is determined to be a crack or delamination. The similarity between the temperature change information and the surface change information is evaluated. If at least a portion is similar, it is presumed that the cause of the temperature change is the delamination accompanied by cracking or peeling. Image analysis device.

3. An image analysis device equipped with a processor, The aforementioned processor, We acquire infrared thermal images of the structure to be inspected, A visible image of the aforementioned structure to be inspected is obtained, Temperature changes are determined from the aforementioned infrared thermal image, Information on surface deformation is derived from the aforementioned visible image, The surface deformation corresponding to the temperature deformation is determined to be a crack or delamination. The presence or absence of surface deformation is evaluated within the magnitude of the temperature deformation and / or near the boundary of the temperature deformation. If present, it is presumed that the cause of the temperature change is the delamination accompanied by the cracks or peeling. Image analysis device.

4. The image analysis apparatus according to any one of claims 1 to 3, wherein the information on the temperature change includes the temperature distribution of the infrared thermal image and / or information obtained from the temperature distribution with respect to the temperature change.

5. The image analysis apparatus according to claim 4, wherein the information on the temperature change includes information on the shape and / or size of the temperature change.

6. The image analysis apparatus according to any one of claims 1 to 5, wherein the surface deformation information includes the luminance distribution of the visible image and / or information obtained from the luminance distribution.

7. The image analysis apparatus according to any one of claims 1 to 6, wherein the surface deformation information includes at least one piece of information on the type, shape, and location of the surface deformation.

8. The aforementioned processor, The image analysis apparatus according to claim 1, which estimates the cause of the temperature change based on the similarity between the temperature change information and the surface change information.

9. The aforementioned processor, If the cause of the temperature change is estimated to be the surface change, The image analysis apparatus according to claim 2 or 3, which estimates the temperature distribution due to the surface deformation and reduces it from the infrared thermal image.

10. The image analysis apparatus according to claim 8, wherein the similarity includes partial similarity.

11. The aforementioned processor, The surface deformation corresponding to the temperature deformation is determined to be a crack or delamination. The similarity between the temperature change information and the surface change information is evaluated. If at least a portion is similar, it is presumed that the cause of the temperature change is the delamination accompanied by cracking or peeling. The image analysis apparatus according to claim 1.

12. The aforementioned processor, The surface deformation corresponding to the temperature deformation is determined to be a crack or delamination. The presence or absence of surface deformation is evaluated within the magnitude of the temperature deformation and / or near the boundary of the temperature deformation. If present, it is presumed that the cause of the temperature change is the delamination accompanied by the cracks or peeling. The image analysis apparatus according to claim 1.

13. The aforementioned processor, The cause of the temperature change is estimated based on the temperature gradient at the boundary of the temperature change. The image analysis apparatus according to any one of claims 1 to 12.

14. The image analysis apparatus according to any one of claims 1 to 13, wherein the surface deformation includes at least one of repair marks, loose lime, joints, steps, cracks, and peeling.

15. The image analysis apparatus according to any one of claims 1 to 14, wherein the visible image is an image obtained by imaging the reflection intensity distribution in two or more different wavelength ranges in the visible light wavelength range.

16. Furthermore, it is equipped with a display device. The image analysis apparatus according to any one of claims 1 to 15, wherein the processor displays the estimated cause of the temperature change on the display device.

17. The steps include: acquiring an infrared thermal image of the structure to be inspected, The steps include: acquiring a visible image of the structure to be inspected, The steps include determining temperature changes from the aforementioned infrared thermal image, The steps include: deriving surface deformation information from the visible image; A step of estimating the cause of the temperature deformation based at least on the information of the temperature deformation obtained from the infrared thermal image and the information of the surface deformation obtained from the visible image, If the cause of the temperature change is estimated to be the surface deformation, the steps include estimating the temperature distribution due to the surface deformation and reducing it from the infrared thermal image, Image analysis methods including [specific methods].

18. The steps include: acquiring an infrared thermal image of the structure to be inspected, The steps include: acquiring a visible image of the structure to be inspected, The steps include determining temperature changes from the aforementioned infrared thermal image, The steps include: deriving surface deformation information from the visible image; A step of determining that the surface deformation corresponding to the temperature deformation is a crack or delamination, A step of evaluating the similarity between the temperature change information and the surface change information, Steps include: If at least a portion is similar, presuming that the cause of the temperature change is delamination accompanied by cracking or peeling; Image analysis methods including [specific methods].

19. The steps include: acquiring an infrared thermal image of the structure to be inspected, The steps include: acquiring a visible image of the structure to be inspected, The steps include determining temperature changes from the aforementioned infrared thermal image, The steps include: deriving surface deformation information from the visible image; A step of determining that the surface deformation corresponding to the temperature deformation is a crack or delamination, A step of evaluating whether the surface deformation is present within the magnitude of the temperature deformation and / or near the boundary of the temperature deformation, If present, the step of estimating that the cause of the temperature change is the crack or delamination accompanied by lifting, Image analysis methods including [specific methods].

20. The steps include: acquiring an infrared thermal image of the structure to be inspected, The steps include: acquiring a visible image of the structure to be inspected, The steps include determining temperature changes from the aforementioned infrared thermal image, The steps include: deriving surface deformation information from the visible image; A step of estimating the cause of the temperature deformation based at least on the information of the temperature deformation obtained from the infrared thermal image and the information of the surface deformation obtained from the visible image, If the cause of the temperature change is estimated to be the surface deformation, the steps include estimating the temperature distribution due to the surface deformation and reducing it from the infrared thermal image, A program that causes a computer to execute something.

21. The steps include: acquiring an infrared thermal image of the structure to be inspected, The steps include: acquiring a visible image of the structure to be inspected, The steps include determining temperature changes from the aforementioned infrared thermal image, The steps include: deriving surface deformation information from the visible image; A step of determining that the surface deformation corresponding to the temperature deformation is a crack or delamination, A step of evaluating the similarity between the temperature change information and the surface change information, Steps include: If at least a portion is similar, presuming that the cause of the temperature change is delamination accompanied by cracking or peeling; A program that causes a computer to execute something.

22. The steps include: acquiring an infrared thermal image of the structure to be inspected, The steps include: acquiring a visible image of the structure to be inspected, The steps include determining temperature changes from the aforementioned infrared thermal image, The steps include: deriving surface deformation information from the visible image; A step of determining that the surface deformation corresponding to the temperature deformation is a crack or delamination, A step of evaluating whether the surface deformation is present within the magnitude of the temperature deformation and / or near the boundary of the temperature deformation, If present, the step of estimating that the cause of the temperature change is the crack or delamination accompanied by lifting, A program that causes a computer to execute something.

Images