Damage detection system and damage detection method

Abstract

To appropriately detect damage to buildings and other objects being monitored. [Solution] The damage detection system 10 includes an acquisition unit 11 that acquires time-series images of the target of damage detection and sensor information other than images detected by sensors provided in relation to the target of detection; a calculation unit 12 that calculates the degree of abnormality of the target of detection from each of the time-series images acquired by the acquisition unit 11; and a detection unit 13 that detects damage to the target of detection based on the change in the time-series degree of abnormality calculated by the calculation unit 12 and the sensor information acquired by the acquisition unit 11.

Description

Technical Field

[0001] The present invention relates to a damage detection system and a damage detection method for detecting damage to a detection target.

Background Art

[0002] Conventionally, a technique for detecting damaged parts of a building from an image of the building has been used (see, for example, Non-Patent Document 1 below).

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When a disaster such as an earthquake occurs, various problems occur if visual inspection of the damage to the building is to be carried out. For example, in a building with many floors, a great deal of time is required for inspection. Also, when patrolling in an unknown damage situation, there is a risk of injury to the inspector due to unexpected damage. In contrast, it is conceivable to automatically detect the damage to the building using the method described in Non-Patent Document 1.

[0005] However, detection using only images may not adequately determine whether or not damage is caused by a disaster. For example, if an image has changed significantly from its original state for reasons other than damage, it may be incorrectly detected as damaged when no damage has actually occurred.

[0006] The present invention has been made in view of the above, and aims to provide a damage detection system and a damage detection method that can appropriately detect damage to a target such as a building. [Means for solving the problem]

[0007] To achieve the above objective, the damage detection system according to the present invention comprises: acquisition means for acquiring a time-series image of the target of damage detection and sensor information other than the image detected by a sensor provided in relation to the target of detection; calculation means for calculating the degree of abnormality of the target of detection from each of the time-series images acquired by the acquisition means; and detection means for detecting damage to the target of detection based on the change in the time-series degree of abnormality calculated by the calculation means and the sensor information acquired by the acquisition means.

[0008] In the damage detection system according to the present invention, damage to the target is detected based on the change in the degree of abnormality calculated from each time-series image and sensor information. Therefore, according to the damage detection system according to the present invention, damage to the target can be appropriately detected.

[0009] The detection means may also detect damage to the target by comparing statistical values ​​of the degree of abnormality over time series since a predetermined starting time with the degree of abnormality at the time of detection. With this configuration, damage to the target can be detected appropriately and reliably based on appropriate criteria.

[0010] The detection means may set the starting time based on changes in the degree of abnormality over time. This configuration allows for more appropriate criteria for detection, and as a result, damage to the target can be detected appropriately and reliably.

[0011] The calculation means may calculate the degree of anomaly using a learning model generated by machine learning. With this configuration, the degree of anomaly can be calculated appropriately and reliably, and as a result, damage to the target can be detected appropriately and reliably.

[0012] Incidentally, in addition to being described as an invention of a damage detection system as described above, the present invention can also be described as an invention of a damage detection method as follows. These are substantially the same invention, differing only in category, and produce similar functions and effects.

[0013] In other words, the damage detection method according to the present invention includes an acquisition step of acquiring a time-series image of the target of damage detection and sensor information other than the image detected by a sensor provided in relation to the target of detection; a calculation step of calculating the degree of abnormality of the target of detection from each of the time-series images acquired in the acquisition step; and a detection step of detecting damage to the target of detection based on the change in the time-series degree of abnormality calculated in the calculation step and the sensor information acquired in the acquisition step. [Effects of the Invention]

[0014] According to the present invention, damage to objects such as buildings can be appropriately detected. [Brief explanation of the drawing]

[0015] [Figure 1] This figure shows the configuration of a damage detection system according to an embodiment of the present invention. [Figure 2] This figure shows an example of image pixel value correction used to calculate the anomaly score. [Figure 3]It is a diagram showing an example of projective transformation of an image used for calculating an abnormality score. [Figure 4] It is a graph of an example of time-series abnormality scores. [Figure 5] It is a graph for explaining the setting of the start time used for determination regarding time-series abnormality scores. [Figure 6] It is a flowchart showing a damage detection method which is a process executed by a damage detection system according to an embodiment of the present invention.

Embodiments for Carrying Out the Invention

[0016] Hereinafter, embodiments of a damage detection system and a damage detection method according to the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same reference numerals are assigned to the same elements, and redundant descriptions are omitted.

[0017] FIG. 1 shows a damage detection system 10 according to the present embodiment. The damage detection system 10 is a system (device) that detects (senses) damage to a detection target 100. As shown in FIG. 1, the detection target 100 is, for example, a building 110. More specifically, the detection target 100 is a non-structural member that constitutes the building 110. The non-structural member that is the detection target 100 is, for example, a ceiling material as shown in FIG. 1. The non-structural member that is the detection target 100 may be other than a ceiling material, for example, a partition wall or a floor surface. Note that the detection target 100 may be other than the non-structural members of the building 110 (for example, an outer wall (more specifically, paint, tiles, or glass)), or may be other than the building 110 (for example, furniture or fixtures inside the building 110).

[0018] The damage detection system 10 detects damage to the detection target 100 during an earthquake. For example, the damage detection system 10 detects the presence or absence of damage to the detection target 100.

[0019] Normally, to assess the extent of indoor damage after an earthquake, the disaster prevention officer of building 110 must visually inspect the building. Damage detection by the damage detection system 10 may be performed, for example, to facilitate the visual inspection by the disaster prevention officer. If the damage detection system 10 detects that damage has occurred to the detection target 100, the disaster prevention officer visually inspects the detection target 100. For example, the damage detection system 10 detects damage to the detection target 100 for each part of building 110 (for example, each floor), and the disaster prevention officer visually inspects the detection target 100 according to the detection for each part. For example, the disaster prevention officer prioritizes and carefully patrols and checks the floors where damage has been detected. Alternatively, the disaster prevention officer checks only the floors where damage has been detected.

[0020] According to the above, visual inspections by disaster prevention personnel can be conducted efficiently. Furthermore, disaster prevention personnel can be aware in advance of any potential hazards during visual inspections or whether it is permissible to enter certain areas. This helps prevent injuries to disaster prevention personnel and allows them to properly prepare for visual inspections.

[0021] Furthermore, the damage detection system 10 may be used for purposes other than visual inspection by disaster prevention personnel. Also, the damage detection system 10 may detect damage to the detection target 100 at times other than when an earthquake occurs. In addition to detecting whether or not the detection target 100 is damaged, the damage detection system 10 may also detect, for example, the degree of damage to the detection target 100.

[0022] Specifically, the damage detection system 10 is a computer including hardware such as a CPU (Central Processing Unit) and memory. The functions of the damage detection system 10 described later are performed by these components operating through programs, etc. The damage detection system 10 may be implemented by a single computer, or by a computer system consisting of multiple computers connected to each other by a network. The damage detection system 10 may have communication functions for acquiring information necessary for the detection functions described below and for outputting information in response to detection.

[0023] Damage detection by the damage detection system 10 is performed using the camera 20 and sensor 30 shown in Figure 1.

[0024] Camera 20 is an imaging device that acquires time-series images used for damage detection by the damage detection system 10. Camera 20 is fixedly positioned in advance at a location where it can image the object to be detected 100. Camera 20 images the object to be detected 100 at predetermined intervals (for example, every hour) to acquire time-series images. Camera 20 has a communication function and transmits the acquired images to the damage detection system 10 using this communication function. As will be described later, camera 20 may also perform imaging at timings controlled by the damage detection system 10. A conventional camera can be used as camera 20.

[0025] Furthermore, camera 20 may be something other than a fixedly installed camera. For example, camera 20 may be mounted on a drone or robot that moves within building 110. Also, camera 20 may be a visible light camera. In addition, if the object to be detected 100 is in a dark place, camera 20 may be an infrared camera.

[0026] Sensor 30 is a device that is provided in relation to the object to be detected 100, performs sensing, and acquires sensor information used for damage detection by the damage detection system 10. Sensor information is information other than images. That is, sensor 30 is a device other than a camera. Sensor 30 is, for example, an acceleration sensor, and the sensor information is the acceleration (maximum acceleration) (Gal) of the object to be detected 100. Sensor 30 is pre-positioned and fixedly installed at a position where it can sense the object to be detected 100. Sensor 30 only needs to be set at a position where it can acquire sensor information that can be used to detect damage to the object to be detected 100. As long as it is at such a position, sensor 30 may be installed at the location of the object to be detected 100, or at a location around the object to be detected 100 that is different from the location of the object to be detected 100. For example, as shown in Figure 1, sensor 30 may be installed at a location different from the location of the object to be detected 100, such as on the floor of the building 110 that contains the object to be detected 100. In other words, the sensor 30 may be directly attached to the object to be detected 100, or it may be attached indirectly to the object to be detected 100.

[0027] Sensor 30 performs sensing of the detection target 100 at a preset timing (for example, continuously) and acquires sensor information. Sensor 30 has a communication function and transmits the acquired sensor information to the damage detection system 10 using this communication function. Any conventional sensor 30 may be used as sensor 30. For example, sensor 30 may be a structural health monitoring (SHM) system (or one included in one). Note that sensor 30 does not have to be an acceleration sensor, as long as it can acquire sensor information used for damage detection by the damage detection system 10. For example, sensor 30 may detect the velocity (maximum velocity), displacement (maximum displacement), seismic intensity at the detection target 100, or acceleration-velocity response spectrum of the detection target 100.

[0028] Furthermore, as described above, the sensor 30 does not have to be directly installed on the object to be detected 100, but rather it is sufficient that it is installed in a way that allows it to acquire sensor information that can estimate (predict) the state (acceleration, etc.) of the object to be detected 100. For example, the sensor 30 is installed at another location within the same building 110 as the object to be detected 100, and acquires sensor information from that other location, provided that the sensor information from the other location can predict the state of the object to be detected 100. In this specification, even in this case, the sensor 30 is defined as a sensor 30 installed in relation to the object to be detected 100.

[0029] Next, the functions of the damage detection system 10 according to this embodiment will be described. As shown in Figure 1, the damage detection system 10 comprises an acquisition unit 11, a calculation unit 12, and a detection unit 13.

[0030] The acquisition unit 11 is an acquisition means that acquires time-series images of the target for damage detection and sensor information other than the images detected by the sensor 30 provided on the target for detection.

[0031] The acquisition unit 11 receives images transmitted from the camera 20 and acquires a time-series image of the damage detection target 100. The acquisition unit 11 may also acquire additional information related to the image. This additional information may include, for example, Exif (Exchangeable image file format) information or information indicating the amount of light at the time the image was captured. This information can be the same as conventional information, for example, generated by the camera 20 when it captures an image and transmitted to the damage detection system 10.

[0032] The acquisition unit 11 receives and acquires sensor information transmitted from the sensor 30. If the frequency of imaging by the camera 20 is less than the frequency of sensor information acquisition by the sensor 30, the acquisition unit 11 may instruct the camera 20 to take an image when the sensor information from the sensor 30 meets certain conditions. For example, if the acceleration indicated by the sensor information is above a preset threshold, the acquisition unit 11 may instruct the camera 20 to take an image. This is because the detection unit 13 uses the image at that time to determine whether the acceleration indicated by the sensor information is above a preset threshold.

[0033] The acquisition unit 11 may also acquire time-series images and sensor information by methods other than those described above. The acquisition unit 11 outputs the acquired time-series images to the calculation unit 12. The acquisition unit 11 outputs the acquired sensor information to the detection unit 13.

[0034] The calculation unit 12 is a calculation means that calculates the degree of anomaly in the detected target from each time-series image acquired by the acquisition unit 11. The calculation unit 12 may calculate the degree of anomaly using a learning model generated by machine learning.

[0035] The calculation unit 12 calculates the degree of abnormality of the detection target 100, for example, as follows: The calculation unit 12 calculates (evaluates) an abnormality score as the degree of abnormality of the detection target 100. The abnormality score is a score (scalar value) that indicates the degree to which the detection target 100 captured in the image has changed from its normal state. The abnormality score considers any change from the normal state as abnormal. For example, the higher the degree of abnormality, the larger the abnormality score. Abnormalities (changes from the normal state) include those caused by damage resulting from earthquakes. In addition, abnormalities may include, for example, changes in the layout of the detection target 100, other than those caused by damage resulting from earthquakes.

[0036] The calculation unit 12 receives an image from the acquisition unit 11. The calculation unit 12 stores rules for calculating an anomaly score in advance and calculates an anomaly score from the image input from the acquisition unit 11 according to these rules. These rules may include a learning model generated by machine learning.

[0037] The calculation unit 12 can calculate the anomaly score from an image in the same way as the calculation of the Anomaly Score shown in, for example, Nitta, Yoshihiro; Fukutomi, Yu; Abe, Masashi; Suzuki, Yoshitaka; Nakajima, Masayoshi; Nishitani, Akira, Detection of Damaged Areas in Ceilings Using Efficient GAN, AI & Data Science Papers, Vol. 5, No. 3, 778-785 (2024) (Non-Patent Literature 2) (where the Anomaly Score in Non-Patent Literature 2 corresponds to the anomaly score). In this case, the learning model takes an image as input and generates (outputs) an image in which there are no changes occurring in the original image (an image considered to represent a normal state). The learning model may be a deep learning model generated by Anomaly Detection with Generative Adversarial Networks (Ano GAN). The calculation unit 12 generates an image in which there are no changes occurring in the original image (an image considered to represent a normal state) from the image using the learning model. The calculation unit 12 calculates the anomaly score from the difference between the original image and the generated image.

[0038] The generation of the above-mentioned learning model can be carried out in the same manner as before, for example, using images as training data. The images used as training data for generating the above-mentioned learning model include images of a normal state (healthy or normal state). In addition, the images used as training data for generating the above-mentioned learning model may also include images of a non-normal state (for example, damaged images). In this case, the number of non-normal images used as training data may be less than the number of normal images used as training data (for example, a few percent of the total). Furthermore, a simulated image may be created and used as a non-normal image by partially coloring over an image of a normal state.

[0039] The training data images may be images of the target object 100, or images other than the target object 100. Both may also be used. By using images of the target object 100 as training data, the generated training model can be made more suitable for detecting damage to the target object 100.

[0040] Furthermore, the learning model used to calculate the anomaly score does not necessarily have to generate an image that does not show any changes from the original image. For example, the learning model may take an image as input and calculate the anomaly score. Alternatively, the anomaly score may be calculated (generated) from an image using image interpretation with a large-scale language model.

[0041] Furthermore, the images used as training data to generate the learning model may be different from those mentioned above, depending on the learning model being generated. For example, only images that are not in a normal state (e.g., images that are damaged) may be used as training data.

[0042] The calculation unit 12 may calculate the anomaly score according to rules that do not use a learning model. Furthermore, the degree of anomaly of the detection target 100 calculated by the calculation unit 12 may be expressed in a way other than a numerical value, such as no anomaly score, for example, small, medium, and large.

[0043] The calculation unit 12 may perform preprocessing on the image input from the acquisition unit 11 so as to enable the calculation of an anomaly score, and use the preprocessed image for calculating the anomaly score. For example, the following image processing may be performed as preprocessing. By performing preprocessing, the calculated anomaly score can be made stable so as not to fluctuate due to factors other than the degree of anomaly.

[0044] The calculation unit 12 may convert the image to grayscale and remove hue information from the image.

[0045] The calculation unit 12 may normalize the histogram by correcting the brightness of the image. The three images in the first row of Figure 2 are images of the same detection target 100 obtained by imaging at different times. Of the three images, the left and middle images are daytime images, and the right image is a nighttime image. The three graphs in the second row of Figure 2 are histograms of pixel values ​​corresponding to the images at the top of the graph. The horizontal axis of the histogram is the pixel value, and the vertical axis is the number of pixels for each pixel value. The differences in the histograms between these images are not due to anomalies related to the anomaly score, but rather to the brightness of the image.

[0046] Taking the above into consideration, the calculation unit 12 may correct the pixel values ​​of the images. For example, the calculation unit 12 corrects the pixel values ​​so that the distribution shapes of the pixel value histograms correspond to each other. Specifically, the calculation unit 12 corrects the pixel values ​​so that the statistical values ​​of the pixel value histograms (e.g., median and standard deviation) match between images. For example, the calculation unit 12 corrects the pixel values ​​so that the statistical values ​​of the images match a preset reference value. The three images in the third row of Figure 2 are the corrected images corresponding to the three images in the first row. The three graphs in the fourth row of Figure 2 are the histograms of pixel values ​​corresponding to the corrected images above the graphs.

[0047] If the image is not obtained by capturing the detection target 100 from the front, that is, if the image is an image of the detection target 100 with an angle (perspective), the calculation unit 12 may use projection transformation to make the detection target 100 appear as if it were viewed from the front. For example, if the detection target 100 is a rectangular ceiling material, depending on the position where the camera 20 is installed, the ceiling material portion (the hatched portion) in the image may appear as a rectangle rather than a rectangle, as shown in the example image on the left side of Figure 3. The calculation unit 12 extracts only this portion from the image and performs projection transformation to generate an image of the ceiling material as if it were viewed from the front, as shown in the example image on the right side of Figure 3.

[0048] The calculation unit 12 stores in advance the position of the part of the image to be detected 100 (the part to be extracted) and what kind of projection transformation to perform, and then performs the extraction and projection transformation described above. Of the parts of the image to be subjected to projection transformation (the part of the image to be detected 100), the part that is behind the camera 20 will have the image stretched when projection transformation is performed. As this will result in different resolutions after projection transformation, the calculation unit 12 adjusts the resolution for each position, such as lowering the resolution of the part of the image that is in front of the camera 20 after projection transformation.

[0049] If the size of the image input to the learning model does not match the size of the image input from the acquisition unit 11 (or the image after image processing), the calculation unit 12 divides the image input from the acquisition unit 11 into multiple parts to match the size of the image input to the learning model. The calculation unit 12 stores in advance how to divide the image and performs the image division. When calculating an anomaly score by dividing the image, subsequent detection of damage to the detection target 100 using the divided images can be performed for each position of the divided image. Alternatively, statistical values ​​such as the average value of the anomaly score of each divided image can be calculated, and subsequent detection of damage to the detection target 100 using the divided images can be performed using these statistical values.

[0050] The calculation unit 12 outputs the calculated degree of abnormality of the detection target 100, for example, the abnormality score, to the detection unit 13.

[0051] The detection unit 13 is a detection means that detects damage to the detection target 100 based on the change in the degree of abnormality over time calculated by the calculation unit 12 and the sensor information acquired by the acquisition unit 11. The detection unit 13 may also detect damage to the detection target 100 by comparing a statistical value of the degree of abnormality over time since a preset start time with the degree of abnormality at the time of detection. The detection unit 13 may also set a start time based on the change in the degree of abnormality over time.

[0052] The detection unit 13 detects damage to the target 100, for example, as follows. The detection unit 13 receives information indicating the degree of abnormality over time, for example, a time-series abnormality score, from the calculation unit 12. Figure 4(a) shows a graph of an example of a time-series abnormality score. The horizontal axis of the graph is the date and time (time), and the vertical axis is the abnormality score. For example, the input of the abnormality score from the calculation unit 12 to the detection unit 13 is performed in real time (or in near real time) each time an image is acquired by the camera 20. The detection unit 13 receives sensor information from the acquisition unit 11. The input of sensor information from the acquisition unit 11 to the detection unit 13 is performed in real time (or in near real time) each time sensing is performed by the sensor 30.

[0053] The detection unit 13 stores rules in advance for detecting (determining) damage to the detection target 100, and detects (determines) damage to the detection target 100 from the information input from the acquisition unit 11 and the calculation unit 12 according to these rules.

[0054] The detection unit 13 determines whether the time-series anomaly score and sensor information each meet the conditions for detecting damage to the detection target 100. If both the time-series anomaly score and sensor information meet the conditions, the detection unit 13 determines that the detection target 100 is damaged. If neither the time-series anomaly score nor the sensor information meets the conditions, the detection unit 13 determines that the detection target 100 is not damaged.

[0055] The following determination is made regarding the time-series anomaly score. The detection unit 13 uses the time-series anomaly score from the start time onward, selected from the anomaly scores input from the calculation unit 12. The detection unit 13 uses the anomaly score (S) at the time of determination. EqThe judgment criteria are generated from the anomaly score of 200 shown in the graph in Figure 4(b), that is, from the time-series anomaly scores since the start time, excluding the most recent anomaly score. The period related to the anomaly score used to generate the judgment criteria (the period since the start time) is considered to be a time when no damage has occurred to the detected target 100, i.e., normal times. The judgment criterion above is whether the anomaly score at the time of judgment deviates significantly from the anomaly score during normal times. The start time above defines the normal times to be compared with the anomaly score at the time of judgment. The start time will be explained later.

[0056] For example, the detection unit 13 calculates (evaluates) the average (μ) of the normal time series anomaly score (the value of the solid line 300 shown in the graph in Figure 4(b)) and the standard deviation (σ) as statistical values ​​of the degree of anomaly in normal times. The anomaly score used to generate the judgment criteria (i.e., the anomaly score used to calculate the average and standard deviation) may be the anomaly score at the time of judgment (the time when the image used to calculate the anomaly score was captured) and the anomaly score at the same time (or the closest time) on the same day (anomaly score 210 shown in the graph in Figure 4(b)). The detection unit 13 calculates the anomaly score at the time of judgment (S Eq The criterion for judgment is whether the value is greater than or equal to the threshold μ+3σ (the value of the dashed line 310 shown in the graph in Figure 4(b)). Note that the threshold for judgment may be a value other than μ+3σ, for example, a value within the range of μ+σ to μ+3σ.

[0057] The detection unit 13 determines the abnormality score (S) at the time of judgment. Eq ) is compared with the threshold μ+3σ. Eq If <μ+3σ, that is, the anomaly score (S) at the time of judgment. Eq If the deviation from the normal anomaly score is considered small, the detection unit 13 determines that the time-series anomaly score conditions are not met and that there is no damage to the detection target 100. Eq If ≥μ+3σ, that is, the abnormality score (S) at the time of judgment. EqIf the deviation from the normal anomaly score is large, the detection unit 13 determines that the conditions for the time-series anomaly score are met. In the example shown in the graph of Figure 4(b), the anomaly score at the time of determination (S Eq )200 satisfies the above conditions.

[0058] If additional information regarding the image related to the anomaly score (for example, Exif information, or information indicating the amount of light when the image was captured) is acquired by the acquisition unit 11, the detection unit 13 may use this information to make a judgment regarding the time-series anomaly score. For example, the detection unit 13 may change the judgment criteria for the above judgment based on the additional information. Specifically, the detection unit 13 may determine whether the time related to the anomaly score at the time of judgment is day or night based on the additional information, and change the threshold according to that determination. In the case of nighttime, since anomalies such as damage are less likely to be reflected in the anomaly score from the image, a threshold for nighttime may be used.

[0059] The following determination is made regarding the sensor information. The detection unit 13 compares the value indicated by the sensor information at the time of determination (for example, the acceleration value (Acc.)) input from the acquisition unit 11 with a preset threshold. The threshold is, for example, a value at which it can be determined that an earthquake has occurred that could cause damage to the detection target 100. If the sensor information is acceleration, the threshold is, for example, 100 Gal. Alternatively, the threshold may be a value within the range of, for example, 100 Gal to 1000 Gal.

[0060] If, as a result of comparing the value indicated by the sensor information at the time of judgment with the threshold value, the detection unit 13 determines that there is no damage to the detection target 100 if the value indicated by the sensor information is such that an earthquake capable of causing damage to the detection target 100 has not occurred, for example, if Acc. < 100Gal, the detection unit 13 determines that the sensor information conditions are not met and that there is no damage to the detection target 100. If, as a result of comparing the value indicated by the sensor information at the time of judgment with the threshold value, the detection unit 13 determines that an earthquake capable of causing damage to the detection target 100 has occurred, for example, if Acc. ≥ 100Gal, the detection unit 13 determines that the sensor information conditions are met.

[0061] The judgments based on the time-series anomaly score and the judgments based on sensor information may be performed sequentially. For example, if one judgment determines that the conditions are met, i.e., that there is a possibility of damage to the detected object 100, then the other judgment may be performed.

[0062] The detection unit 13 can perform the detection (determination) of damage to the detection target 100 in response to a pre-set trigger input or at a pre-set timing. For example, if an SHM system installed in the building 110, which includes the detection target 100, detects the occurrence of an earthquake, it can notify the damage detection system 10 of this fact. The detection unit 13 uses this notification from the SHM system to the damage detection system 10 as a trigger to perform the detection (determination) of damage to the detection target 100.

[0063] The detection unit 13 may also detect damage to the detection target 100 based on changes in the degree of abnormality over time and sensor information, using methods other than those described above.

[0064] The detection unit 13 performs processing according to the detection (judgment). For example, if it detects that there is damage to the detection target 100, the detection unit 13 issues a damage detection alert. As a damage detection alert, the detection unit 13 sends information to that effect to a terminal used by a disaster prevention officer. This information may, for example, illustrate the location of the detection target 100 within the building 110 where damage was detected. Also, if damage to the detection target 100 is detected for each grid into which the image has been divided (i.e., an abnormality score is calculated for each divided image), the detection unit 13 may, in addition to the damage detection alert, calculate (evaluate) and present the percentage of the area where damage was detected. As described above, the disaster prevention officer performs a visual inspection of the detection target 100 using this information as a reference. The detection unit 13 may also perform processing other than that described above as processing according to the detection (judgment).

[0065] This section explains how to set the starting time for generating the criteria for determining the anomaly score over time. As mentioned above, this defines the normal period for comparison with the anomaly score at the time of judgment. Even during periods when no damage has occurred to the detected object 100, the appearance of the detected object 100 may change due to rearrangement of the room in which it is located, or the installation of new equipment (e.g., lighting fixtures). As a result, the anomaly score calculated from the image of the detected object 100 may change even during normal periods. Therefore, it is best to define the normal period as the period immediately preceding the time of judgment during which the anomaly score has not changed.

[0066] The detection unit 13 may set the start time based on the change in the time-series anomaly score, taking the above into consideration. For example, the detection unit 13 sets the start time as follows: The detection unit 13 sets the start time (start date) in days. For each time-series anomaly score, the detection unit 13 compares it with the anomaly score from a predetermined period prior, for example, 3 days prior, and calculates the percentage increase or decrease (error) in the anomaly score compared to the score prior to that period. The anomaly scores being compared are the anomaly scores at the same time on each day. Note that the above period does not necessarily have to be 3 days, and may be any of 1 to 7 days, for example.

[0067] Figure 5(a) shows a graph of an example of a time-series anomaly score, and Figure 5(b) shows a graph of the percentage increase or decrease in the anomaly score compared to the anomaly score three days prior, calculated for the same time-series anomaly score example. In the graph in Figure 5(a), the horizontal axis represents the date and time, and the vertical axis represents the anomaly score. In the graph in Figure 5(b), the horizontal axis represents the date and time, and the vertical axis represents the percentage increase or decrease in the anomaly score compared to the anomaly score three days prior.

[0068] The detection unit 13 calculates daily statistical values ​​(e.g., median) of the rate of increase or decrease and sets the start time based on these daily statistical values. For example, if the absolute value of the daily median rate of increase or decrease exceeds 10% for three consecutive days or more, the detection unit 13 sets the start time to the beginning of the first consecutive day. For example, in the example shown in Figure 5(b), the absolute value of the median exceeds 10% for three consecutive days from the 4th to the 6th, so the start time of the 4th (0:00 on the 4th) is set as the start time. Note that the above rate does not have to be 10%; for example, it could be any percentage between 3% and 50%.

[0069] Furthermore, the detection unit 13 may set the start time based on changes in the time-series anomaly score using methods other than those described above. Also, the detection unit 13 may set the start time without using the time-series anomaly score. For example, the detection unit 13 may update the start time at predetermined intervals.

[0070] If the start time is updated, the learning model used to calculate the anomaly score may be updated by the calculation unit 12. For example, the learning model may be retrained using images of the detection target 100 taken after the start time. In this case, the retraining of the learning model may be performed outside the damage detection system 10, or it may be performed in the damage detection system 10 (for example, in the calculation unit 12). The retraining of the learning model may be performed in the same manner as in the conventional method. The above describes the functions of the damage detection system 10 according to this embodiment.

[0071] Next, the damage detection method, which is a process (operation method performed by the damage detection system 10) executed by the damage detection system 10 according to this embodiment, will be explained using the flowchart in Figure 6. In this process, the acquisition unit 11 acquires a time-series image of the damage detection target 100 and sensor information other than the image detected by the sensor 30 provided in relation to the detection target 100 (S01, acquisition step). Subsequently, the calculation unit 12 calculates an abnormality score, which is the degree of abnormality in the detection target, from each of the time-series images acquired by the acquisition unit (S02, calculation step).

[0072] Next, the detection unit 13 detects damage to the target 100 based on the change in the time-series anomaly score calculated by the calculation unit 12 and the sensor information acquired by the acquisition unit 11 (S03, detection step). Subsequently, the detection unit 13 performs processing according to the detection, for example, an alert for damage detection (S04). The above is the damage detection method according to this embodiment.

[0073] In this embodiment, damage to the target object 100 is detected based on the change in the anomaly score, which is the degree of anomaly calculated from each time-series image, and sensor information. Therefore, according to this embodiment, damage to the target object 100 can be appropriately detected. As a result, as described above, visual inspections by disaster prevention personnel can be performed more efficiently and safely.

[0074] The detection unit 13 may detect damage to the detection target 100 by comparing the statistical value of the anomaly score over a time series since a preset start time with the anomaly score at the time of detection. With this configuration, damage to the detection target 100 can be detected appropriately and reliably using appropriate criteria.

[0075] Furthermore, the detection unit 13 may set the start time based on the change in the time-series anomaly score. This configuration allows for more appropriate criteria for detection, and as a result, damage to the detection target 100 can be detected appropriately and reliably. However, the start time does not necessarily have to be set as described above. Also, the detection of damage to the detection target 100 by the detection unit 13 does not have to be done as described above, as long as it is based on the change in the time-series anomaly score and sensor information.

[0076] The calculation unit 12 may calculate the anomaly score using a learning model generated by machine learning. With this configuration, the degree of anomaly can be calculated appropriately and reliably, and as a result, damage to the target 100 can be detected appropriately and reliably. However, it is not always necessary to use a learning model to calculate the anomaly score.

[0077] The damage detection system and damage detection method of this disclosure have the following configurations. [1] Acquisition means for acquiring a time-series image of the target of damage detection and sensor information other than the image detected by a sensor provided in relation to the target of detection, A calculation means that calculates the degree of abnormality in the detected target from each time-series image acquired by the acquisition means, A detection means for detecting damage to a target based on the change in the degree of abnormality over time calculated by the calculation means and the sensor information acquired by the acquisition means, A damage detection system equipped with the following features. [2] The damage detection system according to [1], wherein the detection means compares a statistical value of the degree of abnormality in a time series since a preset starting time with the degree of abnormality at the time of detection, and detects damage to the target. [3] The damage detection system according to [2], wherein the detection means sets the starting time based on the change in the degree of abnormality over time. [4] The damage detection system according to any one of [1] to [3], wherein the calculation means calculates the degree of anomaly using a learning model generated by machine learning. [5] A method for detecting damage, which is a method for operating a damage detection system, An acquisition step involves acquiring a time-series image of the target of damage detection and sensor information other than the image detected by sensors provided in relation to the target of detection. A calculation step is performed to calculate the degree of abnormality in the detected target from each of the time-series images acquired in the acquisition step, A detection step in which damage to the target is detected is performed based on the change in the degree of abnormality over time calculated in the calculation step and the sensor information acquired in the acquisition step. Damage detection method including [Explanation of symbols]

[0078] 10...Damage detection system, 11...Acquisition unit, 12...Calculation unit, 13...Detection unit, 20...Camera, 30...Sensor.

Claims

1. An acquisition means for acquiring time-series images of the target of damage detection and sensor information other than images detected by sensors provided in relation to the target of detection, A calculation means that calculates the degree of abnormality in the detected target from each time-series image acquired by the acquisition means, A detection means for detecting damage to a target based on the change in the degree of abnormality over time calculated by the calculation means and the sensor information acquired by the acquisition means, Equipped with, The detection means compares the statistical value of the degree of abnormality over time since a predetermined start time with the degree of abnormality at the time of detection, thereby detecting damage to the target. The detection means is a damage detection system that sets the starting time based on changes in the degree of abnormality over time.

2. The damage detection system according to claim 1, wherein the calculation means calculates the degree of anomaly using a learning model generated by machine learning.

3. A damage detection method which is a method for operating a damage detection system, An acquisition step involves acquiring a time-series image of the target of damage detection and sensor information other than the image detected by sensors provided in relation to the target of detection. A calculation step is performed to calculate the degree of abnormality in the detected target from each of the time-series images acquired in the acquisition step, A detection step in which damage to the target is detected is performed based on the change in the degree of abnormality over time calculated in the calculation step and the sensor information acquired in the acquisition step. Includes, In the detection step, the statistical value of the degree of abnormality over time since a predetermined start time is compared with the degree of abnormality at the time of detection, and damage to the target is detected. A damage detection method in which, in the detection step, the start time is set based on the change in the degree of abnormality over time.

Images