Abnormality detection system

Abstract

This invention provides a deformation detection system that improves the accuracy of detecting deformations compared to extracting information from images alone by incorporating structural information of concrete structures. [Solution] This is a deformation detection system 1 that detects deformations appearing on the surface of a concrete structure. The system comprises an image acquisition unit 31 that acquires an image of the surface of the concrete structure to be inspected, a machine learning determination unit 32 that outputs deformation candidates extracted from the image using a machine learning model and outputs a confidence score, a correction value setting unit 33 that sets a correction value for the confidence score according to the type of structural information related to the concrete structure, a score adjustment unit 34 that corrects the confidence score with the correction value and outputs it as an adjusted score, and a deformation determination unit 35 that detects the location of deformation based on the adjusted score.

Description

Technical Field

[0001] The present invention relates to a deformation detection system for detecting deformations that appear on the surface of concrete structures.

Background Art

[0002] Many concrete structures were constructed during the period of high economic growth, and as they age, the deformations are increasing. Here, in urban tunnels, the proportion of material deterioration such as water leakage and rebar corrosion is higher compared to structural deformations such as displacement and deformation. Such deformations have a high affinity with AI technologies such as image analysis, and tunnel integrity determination devices as disclosed in Patent Document 1 have been developed.

[0003] Also, Patent Document 2 discloses a technique for outputting the deterioration level of a building from an image of a deteriorated part of the building. Furthermore, Patent Document 3 discloses a technique for extracting damage regions such as water leakage and free lime, and damage cause sites such as cracks and joints, from inspection images of concrete structures.

[0004] Also, Patent Document 4 discloses a technique for generating partial images of the outer wall of a building and identifying the deterioration situation. In the technologies of these Patent Documents 1 - 4, a machine learning model is created by machine learning, and the object is detected from the image using it.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document Two

Patent Document 3

Patent Document 4

Non - Patent Documents

[0006] [Non-Patent Document 1] Kanzaki, Utilization of 3D Models in Construction Production Systems: Case Studies of Application in Planning, Construction, and Maintenance, Journal of Japan Society of Civil Engineers, Series F4 (Construction Management), Vol. 73, No. 4, I_55-I_63, 2017. [Overview of the project] [Problems that the invention aims to solve]

[0007] However, when using machine learning models for image analysis, information not used in training, especially information that cannot be obtained from images, is not reflected in the detection results.

[0008] On the other hand, in the field of construction and architecture, the recent introduction of CIM (Construction Information Modeling) and BIM (Building Information Modeling) has led to an increase in cases where 3D models are created from the planning, surveying, and design stages.

[0009] Furthermore, the ability to add information at each stage of subsequent construction to the 3D model has led to advancements in the sophistication of structural information. For example, Non-Patent Literature 1 describes an example in which a tunnel was constructed using CIM, and construction data (quality and completed work) and geological models were added to the 3D model before it was handed over to the client upon completion.

[0010] Therefore, the present invention aims to provide a deformation detection system that can improve the accuracy of detecting deformation locations compared to extracting information from images alone by incorporating structural information of concrete structures. [Means for solving the problem]

[0011] To achieve the above objective, the present invention provides a deformation detection system for detecting deformations appearing on the surface of a concrete structure, comprising: an image acquisition unit that acquires an image of the surface of a concrete structure to be inspected; a machine learning determination unit that outputs deformation candidates extracted from the image using a machine learning model and outputs a confidence score indicating the likelihood of the determination result of the deformation candidates; a correction value setting unit that sets a correction value for the confidence score according to the type of structural information relating to the concrete structure; a score adjustment unit that, with respect to the confidence score output from the machine learning determination unit, corrects it using the correction value if the location of the deformation candidate matches the corresponding area of ​​the structural information to obtain an adjusted score, and outputs it as the adjusted score without correction if it does not match; and a deformation determination unit that detects the location of the deformation based on the adjusted score output by the score adjustment unit.

[0012] Here, it is preferable that the structural information is at least one of the following: information registered for the three-dimensional model of the concrete structure, information obtained from inspections already conducted, or information based on document research.

[0013] Furthermore, the structural information is information relating to at least one of the structure, construction, or environment of the concrete structure. More specifically, the structure, construction, or environment of the concrete structure includes at least one of the location of ventilation openings, the location of reinforcing bars, the location of construction joints, the amount of concrete cover, the presence or absence of drying cracks, and the environment of the construction site. [Effects of the Invention]

[0014] The deformation detection system of the present invention, configured as described above, acquires an image of the surface of the concrete structure to be inspected using an image acquisition unit, and extracts deformation candidates from the image using a machine learning model. At that time, the machine learning judgment unit outputs a confidence score indicating the likelihood of the deformation candidate judgment result.

[0015] On the other hand, in the correction value setting unit, a correction value of the reliability score corresponding to the type of structure information regarding the concrete structure is set, and the reliability score output from the machine learning determination unit is corrected by the correction value as needed by the score adjustment unit and output as an adjusted score. Then, in the deformation determination unit, a deformed portion is detected based on the adjusted score.

[0016] By outputting the adjusted score into which the structure information of the concrete structure is incorporated in this way, the detection accuracy of the deformed portion can be improved as compared with the case of detecting deformation only from the image.

Brief Description of Drawings

[0017] [Figure 1] It is a block diagram for explaining the overall configuration of the deformation detection system of this embodiment. [Figure 2] It is a flowchart for explaining the processing flow by the deformation detection system of this embodiment. [Figure 3] It is an explanatory diagram showing an example of a deformed development image of the inner peripheral surface of a tunnel. [Figure 4] It is an explanatory diagram for explaining the extraction result of deformation candidates from a deformed development image using a machine learning model. [Figure 5] It is an explanatory diagram showing an example of a corresponding region of structure information. [Figure 6] It is an explanatory diagram showing an example of a correction value and an adjusted score based on structure information. [Figure 7] It is an explanatory diagram summarizing the concept of the correction value based on structure information in tabular form.

Embodiments for Carrying Out the Invention

[0018] Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram for explaining the overall configuration of the deformation detection system 1 of this embodiment. Further, FIG. 2 is a flowchart for explaining the processing flow by the deformation detection system 1 of this embodiment.

[0019] The deformation detection system 1 of this embodiment is a system for inspecting the inner surface of tunnels where the lining and walls are concrete structures, the surfaces of bridge girders and elevated bridges which are concrete structures, and the surfaces of retaining walls which are concrete structures. In short, it is a system to support maintenance work by detecting deformations that appear on the surface of concrete structures made of reinforced concrete or unreinforced concrete. In the following, the system will be described mainly with the inner surface of tunnels as the target of inspection.

[0020] As shown in Figure 1, the deformation detection system 1 of this embodiment includes an input means for information necessary for calculation processing, such as a shooting device 2 and a structural information DB (database) 6, a calculation processing unit 3, and a display device 4 that displays the results obtained by the calculation processing unit 3.

[0021] The imaging device 2 can use imaging equipment such as digital cameras, including area sensor cameras and line sensor cameras. Images captured by the imaging device 2 are stored in the storage unit 21, which can be a flash memory (such as an SD memory card), hard disk, solid state drive (SSD), magnetic disk, or optical disk.

[0022] The Structural Information DB6 stores structural information about the concrete structures being inspected. For example, if the inspection target is a tunnel, then information related to CIM (Construction Information Modeling), including a 3D model of the tunnel, is recorded in the Structural Information DB6.

[0023] The 3D models stored in the structural information DB6 record information at each stage of construction, linked to the model's location information, and also record maintenance data after construction, linked to the location information.

[0024] In other words, the structural information DB6 records information such as maintenance data obtained from inspections already conducted. Information based on document research can also be recorded in the structural information DB6.

[0025] The processing unit 3 determines the location of deformation on the inner surface of the tunnel based on information about the tunnel obtained from the imaging device 2 and the structural information DB 6. In other words, the processing unit 3 takes in and uses image data obtained from the imaging device 2 and other sources.

[0026] Figure 1 shows an example where the processing unit 3 and the imaging device 2 are connected, but a configuration in which the storage unit 21 and the processing unit 3 are connected is also possible. For example, while imaging is being performed by the imaging device 2, the processing unit 3 can acquire image data each time, or the processing unit 3 can acquire image data stored in the storage unit 21 regardless of the timing of the image acquisition.

[0027] The processing unit 3 of this embodiment includes an image acquisition unit 31 that acquires an image of the inner surface of the tunnel, a machine learning determination unit 32 that extracts deformation candidates from the acquired image using a machine learning model, a correction value setting unit 33 that sets correction values ​​to be described later, a score adjustment unit 34 that calculates an adjustment score to be described later, and a deformation determination unit 35 that determines the location of deformation.

[0028] Furthermore, a storage unit 5, which serves as a storage medium for recording data generated by processing in the arithmetic processing unit 3 and data necessary for arithmetic processing, is connected to the arithmetic processing unit 3. Examples of storage units 5 include solid-state drives (SSDs), flash memory (such as SD memory cards), hard disks, magnetic disks, and optical disks. A cloud server can also be used as the storage unit 5.

[0029] Furthermore, a display device 4, such as a display or printer, is connected to the arithmetic processing unit 3. The configuration of the arithmetic processing unit 3 and the display device 4 can be that of an arithmetic processing unit 3 installed in a personal computer, etc., and a display device 4 such as a liquid crystal display connected to it. Alternatively, the arithmetic processing unit 3 and the display device 4 may be integrated, as in a laptop computer or tablet device.

[0030] The machine learning determination unit 32 extracts candidate anomalies using a machine learning model (AI) and outputs a confidence score. The machine learning model can be one generated separately using machine learning methods such as deep learning (multi-layered neural networks). Alternatively, general-purpose object detection algorithms such as YOLO can be used as the machine learning model.

[0031] Machine learning models output a confidence score (the likelihood of the prediction being correct) along with the detection result. Here, we will refer to this confidence score as the confidence score. In short, the confidence score indicates the confidence level of the prediction of the deformation candidate extracted solely from the image using the machine learning model.

[0032] Figure 3 is an explanatory diagram showing an example of a deformation unfolded image of the inner surface of a tunnel. As illustrated in this image, it is possible to extract deformation areas such as water leaks, rust stains, and exposed reinforcing bars from an image of the inner surface of a tunnel. Here, dirt can be extracted from the image, but it does not correspond to the deformation areas that we want to extract.

[0033] Then, in the machine learning judgment unit 32, the types of abnormalities to be extracted are set, and the machine learning model extracts candidate abnormalities. The candidate abnormalities are then output along with a confidence score, enclosed in a bounding box.

[0034] Figure 4 is an explanatory diagram illustrating the extraction of deformation candidates from deformation unfolded images using a machine learning model. The regions enclosed by bounding boxes in the image of the inner surface of the tunnel indicate the location and size of deformation candidates. For example, if the confidence score threshold is set to 0.15, the regions enclosed by bounding boxes with a confidence score of 0.15 or higher will be extracted as deformation candidates using the machine learning model.

[0035] For example, if the defects to be extracted are water leakage, rust stains, and exposed rebar as shown in Figure 4, then relying solely on a threshold for the confidence score may result in dirt being mistakenly extracted as exposed rebar, or rust stains and water leakage falling below the threshold and being missed.

[0036] Therefore, in the deformation detection system 1 of this embodiment, the reliability score is corrected according to the type of structural information relating to the concrete structure. That is, the correction value setting unit 33 sets the correction value for the reliability score.

[0037] The information recorded in the structural information DB6 is used to set these correction values. As mentioned above, the structural information DB6 records structural information linked to CIM, information obtained from completed inspections, and information based on document research. Therefore, the correction values ​​are set by analyzing this structural information.

[0038] Figure 5 is an explanatory diagram showing an example of a region where structural information is recorded. For example, region A is a region where structural information is recorded indicating that the actual concrete cover is smaller than the design cover. If the concrete cover over the reinforcing bars is small, deterioration factors are more likely to reach the embedded location of the reinforcing bars earlier. Therefore, a positive correction value is set in region A to increase the confidence level of reinforcing bar corrosion and the resulting deformation.

[0039] Furthermore, area B is the area where structural information indicating the location of construction joints is recorded. Construction joints in tunnels are prone to penetrating cracks and water leakage. Therefore, a positive correction value is set to increase the confidence level of water leakage and the resulting deformation.

[0040] On the other hand, region C is a region where structural information is recorded indicating that the actual concrete cover is greater than the design cover. In contrast to region A, if the concrete cover over the reinforcement is large, deterioration factors will have difficulty reaching the embedded location of the reinforcement, so a negative correction value is set to reduce the confidence level of reinforcement corrosion and the resulting deformation.

[0041] The score adjustment unit 34 adjusts the confidence score output by the machine learning judgment unit 32 using the correction value set in the correction value setting unit 33. If the location of the deformation candidate output by the machine learning judgment unit 32 matches the corresponding area of ​​the structural information for which a correction value has been set in the correction value setting unit 33, the score adjustment unit 34 performs a correction using the correction value.

[0042] In contrast, if the area where the structural information correction value is set does not match the location of the deformation candidate, the confidence score is not corrected. However, regardless of whether or not correction was made using the correction value, all confidence scores processed by the score adjustment unit 34 are output as adjusted scores. That is, even confidence scores that did not increase or decrease in the score adjustment unit 34 are treated as adjusted scores in subsequent processing.

[0043] The deformation determination unit 35 detects the deformation location based on the adjustment score output by the score adjustment unit 34. Figure 6 is an explanatory diagram showing an example of the correction value and adjustment score based on structural information.

[0044] For example, the stains explained as mis-extracted in Figure 4 were correctly removed due to a negative correction in area C (correction value -0.03), resulting in an adjustment score of 0.12, which is below the threshold of 0.15. On the other hand, the rust stains explained as being missed during extraction were correctly extracted due to a positive correction in area A (correction value 0.03), resulting in an adjustment score of 0.15, which is above the threshold of 0.15.

[0045] Here, Figure 7 is an explanatory diagram in tabular form that summarizes the concept of correction values ​​based on structural information. Structural information can be classified into information about the structure of concrete structures, information about construction, and information about the environment.

[0046] Furthermore, regarding structural information, correction values ​​are set according to the type of deformation, linked to physical information such as the location of ventilation openings and reinforcing bars. For example, even if there is water leakage, no correction is made to the reliability score for ventilation openings or reinforcing bar locations. On the other hand, a positive correction value is set for areas where rust stains or exposed reinforcing bars are confirmed.

[0047] Regarding construction information, correction values ​​are set according to the type of deformation, in relation to physical information such as the location of construction joints, the amount of concrete cover, and the presence or absence of drying cracks. For example, in areas with construction joints and drying cracks, a positive correction value is set if water leakage, rust stains, or exposed rebar are confirmed. On the other hand, in areas where the concrete cover is less than the design value, no correction value is set for water leakage, but a positive correction value is set if rust stains and exposed rebar are confirmed. Conversely, in areas where the concrete cover is greater than the design value, no correction value is set for water leakage, but a negative correction value is set if rust stains and exposed rebar are confirmed.

[0048] Regarding environmental information, correction values ​​are set according to the type of deformation, linked to physical information such as the environment of the construction site. For example, near tidal rivers and in coastal areas, no correction value is set for water leakage, but a positive correction value is set for areas where rust stains and exposed rebar are confirmed. Also, in cohesive soil, a negative correction value is set for areas where water leakage is confirmed, and no correction value is set for areas where rust stains and exposed rebar are confirmed. On the other hand, in sandy soil, a positive correction value is set for areas where water leakage is confirmed, and no correction value is set for areas where rust stains and exposed rebar are confirmed.

[0049] Figure 2 is a flowchart illustrating the processing flow of the deformation detection system 1 of this embodiment. The following details will be explained in accordance with this flowchart, referring to the examples in Figures 3-6.

[0050] First, in step S1, the image acquisition unit 31 captures an image of the tunnel deformation (see Figure 3). Next, in step S2, a machine learning model is used to extract deformation candidates from the image, and bounding boxes and confidence scores are output as shown in Figure 4.

[0051] On the other hand, regardless of the order of steps S1 and S2, step S3 involves analyzing structural information. This analysis is performed by importing information from each stage of tunnel planning, investigation, design, construction, and maintenance, linked to the 3D CIM model recorded in the structural information DB6, using the correction value setting unit 33. Maintenance data obtained from previously conducted inspections and information based on document surveys can also be used for the analysis.

[0052] Then, in step S4, based on the analysis results from step S3, the region of the inner surface of the tunnel is identified and correction values ​​are set, as shown in Figure 5. Using the correction values ​​set for this region, in step S5, the confidence score calculated by the machine learning model in step S2 is corrected by the score adjustment unit 34 to generate an adjusted score. In other words, post-processing of image analysis using a machine learning model is performed.

[0053] In step S6, the deformation determination unit 35 sets a threshold for the adjustment score to be displayed as a deformation location, and displays bounding boxes where the adjustment score is equal to or greater than the threshold on the image of the inner surface of the tunnel (see Figure 6).

[0054] The bounding boxes displayed in the deformation diagram in Figure 6 represent deformation locations detected by considering structural information, based on deformation candidates extracted using a machine learning model (Figure 4). Therefore, the position coordinates of the displayed bounding boxes are recorded in the storage unit 5 as deformation coordinates, and the deformation diagram, as exemplified in Figure 6, is also saved in the storage unit 5 (step S7).

[0055] Next, the operation of the deformation detection system 1 of this embodiment will be described. The deformation detection system 1 of this embodiment, configured as described above, acquires an image of the surface of the concrete structure to be inspected using the image acquisition unit 31, and extracts deformation candidates from the image using a machine learning model. At that time, the machine learning judgment unit 32 outputs a confidence score indicating the likelihood of the deformation candidate judgment result.

[0056] Meanwhile, the correction value setting unit 33 sets a correction value for the confidence score according to the type of structural information related to the concrete structure. The confidence score output from the machine learning judgment unit 32 is corrected by the correction value as needed in the score adjustment unit 34 and output as an adjusted score. Then, the deformation judgment unit 35 detects the deformation location based on the adjusted score.

[0057] In this way, an adjustment score incorporating structural information of the concrete structure is output, which improves the accuracy of detecting deformations compared to detecting deformations from images alone.

[0058] Machine learning probabilistically predicts physical phenomena through data mining and can be considered an inverse analytical method. Its advantage lies in its ability to derive correct answers with a certain degree of accuracy even when the causal relationships of events are unclear. However, as mentioned above (referencing Figure 4), even with increased accuracy, mis-extractions and omissions can still occur.

[0059] Therefore, in the deformation detection system 1 of this embodiment, in addition to extracting deformation locations using a machine learning model, we decided to improve the detection accuracy by incorporating structural information recorded in the structural information DB6 in order to reflect the technical knowledge that has been accumulated to date.

[0060] In other words, while image analysis using machine learning models alone can only evaluate what is visible in the image, even with high accuracy, post-processing that adds structural information stored in CIM (Construction Information Model) makes it possible to consider information such as cover that cannot be obtained from the image alone. Furthermore, by setting a correction value for the confidence score that reflects the technical knowledge accumulated so far, detection accuracy can also be improved.

[0061] While embodiments of the present invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments, and any design modifications that do not depart from the spirit of the present invention are included in the present invention. [Explanation of symbols]

[0062] 1: Abnormality detection system 31: Image acquisition unit 32: Machine Learning Judgment Unit 33: Correction value setting section 34: Score Adjustment Department 35: Deformation determination section

Claims

1. A deformation detection system that detects deformations appearing on the surface of a concrete structure, An image acquisition unit that acquires images of the surface of the concrete structure to be inspected, A machine learning determination unit outputs candidate deformations extracted from the image using a machine learning model, and also outputs a confidence score indicating the likelihood of the determination result of the candidate deformations. A correction value setting unit sets a correction value for the reliability score according to the type of structural information relating to the concrete structure, A score adjustment unit adjusts the confidence score output from the machine learning determination unit by applying the correction value if the location of the deformation candidate matches the corresponding area of ​​the structural information, and outputs the adjusted score as is without correction if they do not match. A deformation detection system characterized by comprising a deformation determination unit that detects a deformation location based on the adjustment score output by the score adjustment unit.

2. The deformation detection system according to claim 1, characterized in that the structural information is at least one of information registered for a three-dimensional model of the concrete structure, information obtained from inspections already conducted, or information based on document research.

3. The deformation detection system according to claim 1 or 2, characterized in that the structural information is information relating to at least one of the structure, construction, or environment of the concrete structure.

4. The deformation detection system according to claim 3, characterized in that the structure, construction, or environment of the concrete structure includes at least one of the following: the location of ventilation openings, the location of reinforcing bars, the location of construction joints, the size of the concrete cover, the presence or absence of drying cracks, and the environment of the construction site.

Images