Damage assessment device, method and recording medium

Abstract

This invention provides a damage evaluation apparatus, method, and program capable of automatically evaluating surface damage to a structure that occurs in association with its construction. In the damage evaluation apparatus for a structure equipped with a processor, the processor performs the following processes: image acquisition processing to acquire an image of the structure; damage detection processing to detect damage (cracks) to the structure based on the acquired image; feature region detection processing to detect feature regions (areas with P-cone marks) of the structure associated with its construction based on the acquired image; sorting processing to sort specific damages (sinking cracks) that are associated with the detected feature regions of the structure from the detected damages; and information output processing to output information about the sorted specific damages. By outputting information about specific damages in this manner, surface damage to the structure that occurs in association with its construction can be automatically evaluated, and this can be used to verify the appropriateness of construction methods and to improve construction methods.

Description

Technical Field

[0001] This invention relates to a damage assessment device, method, and procedure, and more particularly to a technique for assessing structural damage that occurs in connection with the construction of a structure. Background Technology

[0002] Previously, visual evaluation of defects occurring on the surface of newly constructed structures was proposed for verifying the appropriateness of construction methods and improving construction methods (Non-Patent Literature 1).

[0003] However, visual evaluation results vary from person to person, and there is a need to achieve automated evaluation based on a unified benchmark.

[0004] Furthermore, structural inspection and evaluation are conducted for the purpose of structural maintenance and repair, but there are known techniques for automatically evaluating structural damage based on images of the structure taken (Patent Documents 1 and 2).

[0005] Patent document 1 describes an image processing method in which image analysis is performed on an image representing the surface of a structure to detect cracks on the surface of the structure, and the characteristic quantities of the detected cracks (crack direction, length, width, edge strength and edge density, etc.) are detected. Each crack is grouped according to the detected characteristic quantities, and when displaying the crack image of the crack filling the crack, a different line type or color is used to display the crack image for each group.

[0006] Furthermore, Patent Document 2 describes a method for correcting tunnel lining surface images, wherein the current tunnel lining surface image can be corrected by comparing it with tunnel lining surface images acquired in a time sequence, and in a way that can also identify cracks that have changed by only a few millimeters.

[0007] In the method for correcting tunnel lining surface images described in Patent Document 2, by detecting objects or joints on the tunnel lining surface whose positions remain unchanged, image processing is performed between different time series images to make the positions of the objects or joints consistent, thereby generating a tunnel lining surface image with standardized positions, and thus correcting the tunnel lining surface image used for deformation time series management.

[0008] Previous technical documents

[0009] Patent documents

[0010] Patent Document 1: Japanese Patent Application Publication No. 2020-38227

[0011] Patent Document 2: Japanese Patent Application Publication No. 2015-105905

[0012] Non-patent literature

[0013] Non-patent document 1: Guidelines for Quality Assurance of Concrete Structures (Draft), December 2015, (pp. 17-20, Tohoku Regional Development Station, Ministry of Land, Infrastructure, Transport and Tourism), Internet<http: / / www.thr.mlit.go.jp / road / sesaku / tebiki / kyoukyaku.pdf> Summary of the Invention

[0014] The technical problem to be solved by the invention

[0015] Some of the damage and deterioration of structures are caused by the construction of the structure.

[0016] Patent document 1 describes grouping cracks on the surface of a structure based on the characteristic quantities of the detected cracks, but it does not describe sorting out damage that is associated with the construction of the structure.

[0017] Furthermore, Patent Document 2 describes the detection of cracks and joints in concrete structures, but the joints, whose positions remain unchanged, are used for image processing (affine transformation) to make the previous tunnel lining surface image consistent with the current tunnel lining surface image, not for sorting damage.

[0018] The present invention was made in view of this situation, and its object is to provide a damage evaluation device, method and procedure capable of automatically evaluating damage to the surface of a structure that occurs in connection with the construction of the structure.

[0019] means for solving technical problems

[0020] To achieve the above objectives, the invention involved in the first method is a damage evaluation device for a structure equipped with a processor, wherein the processor performs the following processing: image acquisition processing to acquire an image of the structure; damage detection processing to detect damage to the structure based on the acquired image; feature region detection processing to detect structural feature regions related to the construction of the structure based on the acquired image; sorting processing to sort specific damages that are related to the detected structural feature regions from the detected damages; and information output processing to output information on the sorted specific damages.

[0021] According to a first aspect of the present invention, among the damage detected based on images of the structure, specific damages that are associated with structural feature regions related to the construction of the structure are automatically sorted out, and information on the sorted specific damages is output. Therefore, it is possible to automatically evaluate surface damage of the structure that is associated with the construction of the structure, and it can be used to verify the appropriateness of the construction method and improve the construction method.

[0022] In the damage assessment apparatus according to the second aspect of the present invention, it is preferable to perform the following: in the damage detection process, when an input image is received, a first learned model is executed, wherein the first learned model outputs the region of each damage for each damage to the structure as the identification result.

[0023] In the damage assessment device according to the third aspect of the present invention, the damage to the structure is a crack in the structure, and the specific damage is a specific crack in the structure that is caused by the construction of the structure. Specific cracks include, for example, sinking cracks or crescent cracks that are generated on the surface of a concrete structure.

[0024] In the damage assessment apparatus according to the fourth aspect of the present invention, it is preferable to perform the following: in the feature region detection processing, when an input image is received, a second learned model is executed, and the second learned model outputs the feature region of the structure as the recognition result.

[0025] In the damage assessment device according to the fifth aspect of the present invention, the structural feature area is an area representing construction traces associated with a specific damage, i.e., a specific crack, caused by the construction of the structure. Construction traces associated with a specific crack are, for example, P-cone marks on the surface of a concrete structure, joints, pouring joints, etc.

[0026] In the damage assessment apparatus according to the sixth aspect of the present invention, it is preferable to classify damage that is in contact with or overlaps with the characteristic area of ​​the structure as specific damage during the sorting process.

[0027] In the damage assessment apparatus according to the seventh aspect of the present invention, it is preferable that the sorting process includes an expansion process that expands the size of a structural feature region, and that damage in contact with or overlapping with the expanded structural feature region is sorted as specific damage. The proportion or amount of expansion of the structural feature region can be a preset value or a value appropriately set by the user.

[0028] In the damage assessment apparatus according to the eighth aspect of the present invention, it is preferable that the processor performs a size determination process to determine the size of a specific damage.

[0029] In the damage evaluation device according to the ninth aspect of the present invention, the damage to the structure includes cracks in the structure, and the specific damage is a specific crack in the structure caused by the construction of the structure. The size determination process can calculate the relative length of the length of the specific crack on the image and the length of the feature area of ​​the structure on the image, and use the calculated relative length as the size of the specific damage.

[0030] In the damage evaluation apparatus according to the 10th aspect of the present invention, it is preferable that: the damage to the structure includes cracks in the structure, and the specific damage is a specific crack in the structure caused by the construction of the structure. In the size determination process, the actual size of the specific damage is calculated based on the length of the specific crack in the image, the length of the feature area of ​​the structure in the image, and the actual size of the feature area of ​​the structure.

[0031] In the damage assessment apparatus according to the 11th aspect of the present invention, it is preferable that: the damage to the structure includes cracks in the structure, and the specific damage is a specific crack in the structure caused by the construction of the structure; the structure with a scale reference whose actual size is known is photographed in the image; and in the size determination process, the actual size of the specific damage is calculated based on the length of the specific crack in the image and the length of the scale reference in the image.

[0032] In the damage assessment apparatus according to the 12th aspect of the present invention, it is preferable that: the damage to the structure includes cracks in the structure, and the specific damage is a specific crack in the structure caused by the construction of the structure; in the size determination process, the actual size of the specific damage is calculated based on the length of the specific crack in the image and the photographic conditions and camera information of the camera that took the image. The photographic conditions of the camera include, for example, the distance between the camera and the specific crack, and the camera information includes, for example, the focal length, the size of the image sensor, the number of pixels, or the pixel pitch.

[0033] In the damage evaluation apparatus according to the 13th aspect of the present invention, it is preferable that, during information output processing, each specific damage is output in a recognizable manner based on the attributes of the specific damage. The attributes of the specific damage include the length, width, area, etc., of the damage. Furthermore, it is preferable that each specific damage can be distinguished by color based on its attributes, and identified by differences in line type, etc.

[0034] In the damage assessment apparatus according to the 14th aspect of the present invention, it is preferable to output the structural feature region corresponding to the specific damage in an identifiable manner in the information output processing, based on the attributes of the specific damage.

[0035] In the damage assessment apparatus according to the 15th aspect of the present invention, it is preferable that the processor calculates the ratio of the total number of structural feature regions to the number of structural feature regions corresponding to a specific damage, and outputs the calculated ratio in the information output processing.

[0036] In the damage evaluation apparatus according to the 16th aspect of the present invention, it is preferable that the processor performs the following processing: editing instruction acceptance processing, receiving editing instructions from the operation unit operated by the user for at least one of the detection results of detected damage and the detection results of detected structural feature areas; and editing processing, editing the detection results according to the accepted editing instructions.

[0037] In the damage assessment apparatus according to the 17th aspect of the present invention, it is preferable that, in the information output processing, information on a specific damage is output to a display for display, or stored in a memory in the form of a file.

[0038] In the damage evaluation apparatus according to the 18th aspect of the present invention, it is preferable that the information of a specific damage includes a damage quantity table, which has items such as damage identification information, damage type and size, and records information corresponding to each item for each specific damage.

[0039] The invention involved in Method 19 is a damage evaluation method, in which a processor evaluates the damage of a structure. The processor's processing includes the following steps: acquiring images of the structure; detecting damage to the structure based on the acquired images; detecting structural feature regions related to the construction of the structure based on the acquired images; sorting specific damages that are related to the detected structural feature regions from the detected damages; and outputting information on the sorted specific damages.

[0040] The invention involved in Method 20 is a damage assessment program that enables a computer to perform a method for assessing damage to a structure. The method includes the following steps: acquiring images of the structure; detecting damage to the structure based on the acquired images; detecting structural feature regions related to the construction of the structure based on the acquired images; sorting specific damages that are related to the detected structural feature regions from the detected damages; and outputting information on the sorted specific damages.

[0041] Invention Effects

[0042] According to the present invention, it is possible to automatically evaluate surface damage to a structure that occurs in connection with the construction of the structure. Attached Figure Description

[0043] Figure 1 This is a diagram illustrating an example of damage to a structure.

[0044] Figure 2 This is a diagram illustrating an example of the process of sorting specific damages from damage detection.

[0045] Figure 3 This is a diagram used to illustrate the method for determining whether a crack detected by sorting is a sinking crack.

[0046] Figure 4 This is a block diagram illustrating an example of the hardware structure of the damage assessment device involved in the present invention.

[0047] Figure 5 This is a conceptual diagram illustrating an implementation of a damage detection processing unit and a feature area detection processing unit, which consist of a CPU or similar components.

[0048] Figure 6 This is a first example of a diagram showing images of the structure being evaluated and information about specific damage.

[0049] Figure 7 This is a second example of a diagram showing images of the structure being evaluated and information about specific damage.

[0050] Figure 8 It means that an additional [something] has been added. Figure 7 (B) is an example of a display screen showing a crack image with color differentiation.

[0051] Figure 9 This is another example of a diagram showing specific damage to a structural feature area that is associated with the construction of the structure.

[0052] Figure 10 This is another example of a diagram showing specific damage that is associated with a structural feature area related to the construction of the structure.

[0053] Figure 11 This is another example of a process for sorting specific damages from damage detection.

[0054] Figure 12 It is a damage map that includes crack information.

[0055] Figure 13 This is a chart representing an example of a table showing the number of damages included in a damage detection result.

[0056] Figure 14 This is a diagram illustrating the method of adding vertices along the broken line of a crack.

[0057] Figure 15 This is a diagram illustrating the method of removing vertices from a polyline along a crack.

[0058] Figure 16 This is a flowchart illustrating an implementation method of the damage assessment method involved in this invention. Detailed Implementation

[0059] Hereinafter, with reference to the accompanying drawings, preferred embodiments of the damage assessment device, method, and procedure involved in the present invention will be described.

[0060] [Summary of the Invention]

[0061] Figure 1 This is a diagram illustrating an example of damage to a structure. Figure 1 (A) represents an original image of a concrete structure showing construction traces. Figure 1 (B) represents a composite image that overlays the crack detection results (crack image) onto the original image.

[0062] In this example, the construction marks on the structure are traces of a plastic cone (hereinafter referred to as "P-cone"). Figure 1 The P symbol represents the P-cone mark. The following is a description of the P-cone mark P.

[0063] Construction of concrete structures such as walls, columns, and beams typically involves assembling reinforcing bars and formwork, followed by pouring concrete. The formwork is secured by rod-shaped partitions with threaded ends and P-cones installed at the desired intervals at both ends of the partitions. Concrete flows into the formwork, but the formwork and P-cones are removed once the concrete has hardened.

[0064] When removing the P-cone, a hole was found exposing the threaded portion of the diaphragm. The circular mark left by plugging the hole with mortar is the P-cone mark P.

[0065] Figure 1 (B) C1 to C3 represent cracks around the P-cone mark P. Cracks C1 to C3 are formed by... Figure 1 (A) The cracked areas detected in the original image are represented by crack images filled with specific colors.

[0066] Cracks C1 to C3 around the P-cone mark are classified as settlement cracks, which are one type of crack caused by the construction of the structure. Settlement cracks are caused by displacement due to the settlement or bleeding of concrete after it has been poured, which is constrained by formwork partitions, surface formwork, etc.

[0067] One aspect of this invention involves detecting damage to a structure from images taken of the structure, sorting out damage related to the structure's construction (specific damage) from the detected damage, and outputting information about the sorted specific damage. This can be used to verify the appropriateness of the construction method and to improve the construction method.

[0068] exist Figure 1 In the example shown, the structural feature area associated with the construction of the structure is the area of ​​the P-cone mark P, and the specific damage associated with the area of ​​the P-cone mark is the subsidence cracks C1 to C3.

[0069] Figure 2 This is a diagram illustrating an example of the process of sorting specific damages from damage detection.

[0070] like Figure 2 As shown in (A), damage to the structure (cracks in this example) is detected based on images of the structure. Crack detection can be performed using artificial intelligence (AI) or image processing algorithms.

[0071] And, as Figure 2 As shown in (B), structural feature regions associated with the construction of the structure (in this example, the region of the P-cone mark P) are detected based on the captured images of the structure. The detection of the P-cone mark P can be performed using AI or image processing algorithms. Furthermore, detection can also be performed by accepting manual input instructions from the user.

[0072] Next, as Figure 2 As shown in (C), among the detected cracks, sinking cracks (specific cracks) C1 to C4 that are related to the region of the P cone mark P are sorted out. The sorted sinking cracks C1 to C4 are distinguished by color or have their line shape changed in a way that can be identified from other cracks.

[0073] Figure 3 This is a diagram used to illustrate the method for determining whether a crack detected by sorting is a sinking crack.

[0074] Figure 3 (A) represents the cracks C1 to C3 and the P cone mark P that were detected respectively.

[0075] Regarding the sorting of whether cracks C1 to C3 are sinking cracks related to the region of P-cone mark P, cracks C1 to C3 are sorted as sinking cracks if they are in contact with the region of P-cone mark P or if they overlap with the region of P-cone mark P.

[0076] In this sorting method, cracks C1 to C3 all separated from the region of the P-cone mark P, and were therefore identified as non-sinking cracks. Figure 3 (B)).

[0077] On the other hand, such as Figure 3 As shown in (C), the size of the region of the P-cone mark P is expanded to the size of the region of the P-cone mark P1.

[0078] In the expansion process of the region of the P-cone mark P, the region of the P-cone mark P can be expanded by expanding the region radially from the center of the circular P-cone mark P at a certain proportion, or by expanding the shape of the circular P-cone mark P by only a certain amount of expansion (width). Furthermore, the proportion or amount of expansion of the region of the P-cone mark P can be a preset value or a value appropriately set by the user.

[0079] Furthermore, when cracks C1 to C3 are in contact with the area of ​​the expanded P-cone mark P1, or when they overlap with the area of ​​the P-cone mark P1, they are classified as sinking cracks. Figure 3 (D)).

[0080] In this sorting method, since cracks C1 to C3 are in contact with or overlap with the area of ​​P-cone mark P1, they are sorted as sinking cracks.

[0081] In addition, the region of P-cone traces is not limited to the region of P-cones detected by expanding the P-cone region through dilation processing; a region that is slightly wider than the original P-cone trace region can also be detected as the region of P-cone traces.

[0082] Furthermore, the closest distance between cracks C1 to C3 and the region of P-cone mark P1 can be calculated, and if this distance is within the threshold, it is classified as a sinking crack.

[0083] [Hardware Structure of the Damage Assessment Device]

[0084] Figure 4 This is a block diagram illustrating an example of the hardware structure of the damage assessment device involved in the present invention.

[0085] As Figure 4 The damage assessment device 10 shown can be used with a personal computer or workstation. The damage assessment device 10 in this example mainly consists of an image acquisition unit 12, an image database 14, a storage unit 16, an operation unit 18, a CPU (Central Processing Unit) 20, a RAM (Random Access Memory) 22, a ROM (Read Only Memory) 24, and a display control unit 26.

[0086] The image acquisition unit 12 is equivalent to an input / output interface, which in this example acquires images of the structure being evaluated. The structure being evaluated includes, for example, walls, columns, beams, etc., of bridges, tunnels, and buildings.

[0087] The images acquired by the image acquisition unit 12 are, for example, a large number of images (image sets) of the structure taken by a drone (unmanned aerial vehicle) or robot equipped with a camera, or by human hand. Preferably, the image set covers the entire structure, and adjacent images overlap.

[0088] The image group acquired by the image acquisition unit 12 is stored in the image database 14.

[0089] Storage unit 16 is a memory device composed of a hard disk drive, flash memory, etc. In addition to the operating system and damage evaluation program, storage unit 16 also stores CAD (computer-aided design) data representing the structure and filed damage information. Damage information includes damage images, damage diagrams (CAD data), and other damage evaluation results.

[0090] Regarding the CAD data of the evaluation object structure, if the CAD data exists in advance, it can be used. If the CAD data of the structure does not exist, it can be automatically generated based on the image group stored in the image database 14.

[0091] When the image set stored in the image database 14 is captured by a camera mounted on a drone, a three-dimensional point cloud model can be generated. This three-dimensional point cloud model is generated by extracting feature points between overlapping images in the image set, inferring the position and orientation of the camera mounted on the drone based on the extracted feature points, and simultaneously inferring the three-dimensional position of the feature points based on the inference result of the camera's position and orientation.

[0092] One method is Structure from Motion (SfM: 3D Reconstruction), which tracks the movement of a large number of feature points from a group of images taken by a drone moving from the camera's position, and simultaneously infers the 3D structure of the structure and the camera's pose. In recent years, an optimization calculation method called bundle adjustment has been developed, which can produce high-precision output.

[0093] Furthermore, the camera parameters (focal length, image sensor size, pixel pitch, etc.) required for applying the SfM method can be obtained using parameters stored in the storage unit 16. Moreover, CAD data for the structure can be generated based on the generated 3D point cloud model.

[0094] The operation unit 18 includes a keyboard and mouse that are connected to the computer via wired or wireless connection. Besides functioning as a general computer operation instruction unit, it also functions as an operation unit where the user can edit detection results of structural damage and structural feature areas such as P-cone marks detected based on images of the structure. Further details regarding the editing of damage detection results will be described later.

[0095] CPU20 reads various programs stored in storage unit 16 or ROM24, etc., centrally controls each unit, and performs the following processing: damage detection processing to detect damage to the structure based on the captured images of the structure; feature area detection processing to detect structural feature areas (such as the area of ​​P-cone marks) related to the construction of the structure; sorting processing to sort specific damages that are related to the structural feature areas among the detected damages; and information output processing to output information on the sorted specific damages, etc.

[0096] Damage detection processing based on images of structures and feature region detection processing for detecting characteristic regions of structures can both be performed using AI.

[0097] As AI, for example, it is possible to use a learned model based on a convolutional neural network (CNN).

[0098] Figure 5 This is a conceptual diagram illustrating an implementation of a damage detection processing unit and a feature area detection processing unit, which consist of a CPU or similar components.

[0099] exist Figure 5 In the model, the damage detection processing unit and the feature region detection processing unit are respectively composed of the first learned model 21A and the second learned model 21B.

[0100] The first learned model 21A and the second learned model 21B each have an input layer, an intermediate layer and an output layer, and each layer becomes a structure of multiple "nodes" connected by "edges".

[0101] The input layer of the CNN takes an image of the structure as input (13). The intermediate layers, consisting of multiple groups of convolutional and pooling layers, extract features from the image input from the input layer. In the convolutional layers, nodes located near the previous layer are filtered (convolution operations using filters) to obtain "feature maps." In the pooling layers, the feature maps output from the convolutional layers are scaled down to create new feature maps. The convolutional layers are responsible for extracting features such as edges from the image, while the pooling layers provide robustness so that the extracted features are not affected by parallel shifts, etc.

[0102] The output layer of a CNN is a portion that outputs a feature map representing the features extracted by the intermediate layers. In this example, the output layer of the first learned model 21A outputs, for example, the inference result (recognition result) of classifying (segmenting) the region of each damage to the structure reflected in the image in pixels or in blocks of several pixels as the damage detection result 27A. Similarly, the output layer of the second learned model 21B outputs, for example, the inference result of classifying the structural feature regions associated with the construction of the structure reflected in the image in pixels or in blocks of several pixels as the structural feature region detection result 27B.

[0103] For example, the first learned model 21A is a learned model that has undergone machine learning for detecting cracks, and the second learned model 21B is a learned model that has undergone machine learning for detecting P-cone marks.

[0104] Alternatively, the first learned model 21A and the second learned model 21B can also be composed of a single learned model, which can output damage detection result 27A and structural feature region detection result 27B, respectively.

[0105] Return to Figure 4 The CPU 20, based on the damage detection results 27A and structural feature region detection results 27B detected by the first learned model 21A and the second learned model 21B respectively, performs sorting processing on specific damages that are correlated with the detected structural feature regions. In this example, among the detected cracks, sinking cracks (specific cracks) correlated with P-cone marks are sorted. The sorting of sinking cracks can be achieved by using... Figure 3 The method of explanation will be provided, but detailed explanations are omitted here.

[0106] The CPU 20 outputs information about the specific damages selected via the display control unit 26 to the display unit (monitor) 30 for display, or saves it in the storage unit (memory) 16 as a file. Preferably, the CPU 20 also outputs information about the structural feature areas via the display control unit 26 to the display unit 30 for display, or saves it in the storage unit 16 as a file.

[0107] RAM22 serves as the working area of ​​CPU20, acting as a temporary storage unit for read programs or various data.

[0108] The display control unit 26 is the part that generates display data to be displayed on the display unit 30 and outputs it to the display unit 30. In this example, information such as specific damage detected and sorted by the CPU 20 is displayed on the display unit 30, and editing screens such as information on specific damage based on user operations from the operation unit 18 are displayed on the display unit 30.

[0109] The display unit 30 uses various displays such as an LCD monitor that can be connected to a computer. It displays information such as specific damage detected from the image, along with the image of the structure input from the display control unit 26, and is used as part of the user interface together with the operation unit 18.

[0110] The damage evaluation device 10 with the above structure includes a processor with a CPU 20, which reads the damage evaluation program stored in the storage unit 16 or ROM 24 and executes the damage evaluation program to perform the above-mentioned processes.

[0111] <The Role of Damage Assessment Devices>

[0112] Next, regarding Figure 4 The function of the damage assessment device 10 shown will be explained using a bridge as an example of a structure.

[0113] Figure 6 This is a first example of a diagram showing images of the structure being evaluated and information about specific damage.

[0114] The damage evaluation device 10 comprises a CPU 20, a damage evaluation program stored in a storage unit 16, RAM 22 and ROM 24, a display control unit 26, etc., which constitute a processor. The processor performs various processes as shown below.

[0115] The processor performs image acquisition processing, acquiring images of the evaluation target structure (the surface of the concrete structure) captured by the image acquisition unit 12. If images of the structure are stored in the image database 14, the processor reads the image of the evaluation target structure from the image database 14. In this case, the processor acquires multiple images and performs panoramic composite image processing to ensure consistency in the overlapping areas of the acquired multiple images.

[0116] like Figure 6 As shown in (A), the panoramic composite image is preferably an orthophoto image projected onto the surface of the concrete structure.

[0117] To detect cracks smaller than 0.1 mm from an image, a high-resolution image is required, thus reducing the photographic range of a single image. For obtaining images of structures of a certain size, panoramic synthesis of multiple images is preferred; however, a single image may suffice if a high-resolution camera is used or if the crack width to be detected is large.

[0118] Regarding Figure 5 The first learned model 21A and the second learned model 21B, which are the damage detection processing unit and the feature region detection processing unit shown, are in operation. When the panoramic composite image (image 13) is input, the first learned model 21A detects cracks based on the input image 13 and outputs a damage detection result 27A representing the detected cracks. The second learned model 21B detects P-cone marks based on the input image 13 and outputs a structural feature region detection result 27B representing the detected P-cone marks.

[0119] Next, the processor, based on damage detection result 27A and structural feature area detection result 27B, performs sorting processing on subsidence cracks in damage detection result 27A (cracks) that are correlated with structural feature area detection result 27B (P-cone mark). This sorting process, if using... Figure 3 As explained, the area of ​​the crack is detected to be in contact with the area of ​​the P-cone mark, or the area of ​​the crack overlaps with the area of ​​the P-cone mark. Cracks that are in contact with or overlap with the area of ​​the P-cone mark are classified as sinking cracks.

[0120] Next, the processor performs a dimensional determination process, which determines the size of the specific damage (sinking crack) for each corresponding P-cone mark.

[0121] The following section explains the dimensional determination process for determining the size of the sinking crack.

[0122] <First Dimension Determination Process>

[0123] In the first size determination process, the relative length of the length of the sinking crack on the image and the length of the P-cone mark corresponding to the sinking crack on the image (in this example, the diameter of the P-cone mark) are calculated, and the calculated relative length is used as the size of the sinking crack.

[0124] Based on the first dimension determination process, the size of the sinking crack is determined to be X times the diameter of the P-cone mark. Alternatively, X times can also be expressed in four levels, such as zero (no sinking crack), less than 3 times, more than 3 times but less than 5 times, and more than 5 times.

[0125] <Second Dimension Determination Process>

[0126] In the second size determination process, the actual size of the sinking crack is calculated based on the length of the sinking crack on the image, the diameter of the P-cone mark corresponding to the sinking crack on the image, and the actual size of the P-cone mark.

[0127] That is, the actual size of the sinking crack can be calculated by the above (relative length (X times)) × (actual size of the P-cone mark).

[0128] The actual size of the P-cone mark can be adapted to the value input by the user through the operation unit 18 or the specified value.

[0129] <Third Dimension Determination Process>

[0130] The third dimension determination process applies to situations where images of structures with known actual dimensions have been captured. The scale reference can be a graduated ruler pasted on the surface of the structure, or it can be a piece of steel, bolt head, or other material with known actual dimensions that is set on the surface of the structure.

[0131] In the third dimension determination process, the actual dimension of the sinking crack is calculated based on the length of the sinking crack on the image and the length of the known actual dimension scale reference on the image.

[0132] <4th Dimension Determination Process>

[0133] In the fourth dimension determination process, the actual dimension of the sinking crack is calculated based on the length of the sinking crack in the image and the photographic conditions and camera information of the camera that took the image.

[0134] The photographic conditions of the camera include, for example, the distance between the camera and the sinking crack (photographic distance), the angle between the camera's photographic direction and the surface of the structure, etc. Camera information includes, for example, the focal length of the photographic lens, the size of the image sensor, the number of pixels or the pixel pitch, etc.

[0135] Currently, when the camera's shooting direction is orthogonal to the surface of the structure, with a shooting distance of D, a focal length of f, a length u (converted from the image length of the sinking crack to the length on the image sensor), and a pixel pitch of p, the actual size L of the sinking crack can be calculated using the following formula:

[0136] [Formula 1]

[0137] L=D×u×p / f.

[0138] Furthermore, in the above-mentioned first to fourth dimension determination processes, when multiple sinking cracks are sorted out in association with the region of a P-cone mark, the length of the longest sinking crack among the multiple sinking cracks is taken as the length (representative length) of the sinking crack corresponding to the P-cone mark.

[0139] Furthermore, the information on sinking cracks can include not only the length of the sinking crack, but also the width and area of ​​the sinking crack.

[0140] Figure 6 (B) is a first display example showing information such as specific damage displayed on the screen of the display unit.

[0141] like Figure 6 As shown in (B), the display screen shows the area of ​​the P-cone mark corresponding to the sinking crack, differentiated by color according to the attributes (size, etc.) of the corresponding sinking crack, and superimposed on the panoramic composite image. Figure 6 (B)) images, and displays information related to various P-cone marks, etc.

[0142] exist Figure 6 In the example shown in (B), the regions with 24 P-cone marks are determined by the first dimension to determine the processing result, and the colors are divided into 4 colors (red, yellow, green, and blue).

[0143] exist Figure 6 (B) represents the P-cone mark. R P Y P G P B It has the following sinking cracks.

[0144] P R (Red): P-cone marks that produce sinking cracks more than 5 times the diameter of the P-cone mark.

[0145] P Y (Yellow): P-cone marks that produce sinking cracks with a length greater than 3 times but less than 5 times the diameter of the P-cone mark.

[0146] P G (Green): P-cone mark producing a sinking crack with a length less than 3 times the diameter of the P-cone mark.

[0147] P B (Blue): P-cone traces without sinking cracks

[0148] like Figure 6 As shown in (B), the total number of P-cone marks is 24, and the P-cone marks are distinguished by color as described above. R P Y P G P B The numbers are 2, 6, 7, and 9 respectively. The processor calculates the total number of P-cone marks and the P-cone mark corresponding to the sinking crack. R P Y P G The proportion of the number of (generation ratio) is also shown (63% in this example).

[0149] according to Figure 6 The display screen (B) shows that by observing the color of the P-cone marks, the size of the sinking crack can be easily determined based on the presence or absence of sinking cracks in each P-cone mark and the color of the P-cone mark that caused the sinking crack. It also provides information such as the proportion of sinking cracks.

[0150] In addition, Figure 6 In (B), the P-cone traces are displayed by color, but this is not limited to. The crack images can also be displayed by adding color to all detected cracks, or only the sorted sinking cracks can be displayed by adding color to the crack images, or by switching between them appropriately.

[0151] Figure 7 This is a second example of a diagram showing images of the structure being evaluated and information about specific damage.

[0152] Figure 7 (A) represents six images of the structure being evaluated. These six images were taken with a P-cone mark roughly at the center of the image.

[0153] Figure 7 (B) is a second display example showing information such as specific damage displayed on the screen of the display unit.

[0154] like Figure 7 As shown in (B), six images are displayed side-by-side on the screen of the display unit, and each image's P-cone mark is marked with a color corresponding to the properties (size, etc.) of the sinking crack. The color differentiation of the P-cone mark can be used to distinguish... Figure 6 (B) The same procedure applies.

[0155] Furthermore, crack detection results can also be displayed. For example, crack images can be differentiated by color based on crack length, crack images can be differentiated by color based on crack width, all detected cracks can be colored to display crack images, or only the sorted subsidence cracks can be colored to display crack images, or the display can be achieved by switching between them appropriately.

[0156] Figure 8 It means that an additional [something] has been added. Figure 7 (B) is an example of a display screen showing a crack image with color differentiation.

[0157] exist Figure 8 In the P-cone mark P R Three sinking cracks C were generated in the middle. R C G C Y In the P cone mark P Y Two sinking cracks C were generated in the middle. G C Y In the P cone mark P G Two sinking cracks C were generated in the middle. G .

[0158] In addition, sinking crack C R C G CY Based on the properties (length) of each sinking crack, for example, they are distinguished by color as follows.

[0159] C R (Red): A sinking crack with a length more than 5 times the diameter of the P-cone mark.

[0160] C Y (Yellow): A sinking crack with a length greater than 3 times but less than 5 times the diameter of the P-cone mark.

[0161] C G (Green): A sinking crack less than three times the diameter of the P-cone mark.

[0162] <Another example of specific damage associated with characteristic areas of a structure and related to its construction>

[0163] Figure 9 This is another example of a diagram showing specific damage to a structural feature area that is associated with the construction of the structure.

[0164] Figure 9 (A) is the original image of the evaluation object structure, including its seams and cracks.

[0165] Concrete joints are created by adding cuts at constant intervals to the surface of concrete, which is prone to cracking due to shrinkage and expansion caused by temperature. This helps prevent cracking in other areas, but it can also induce cracking in the joints. Additionally, joint material (cushioning material) is used to fill the joints.

[0166] from Figure 9 In the original image shown in (A), cracks are detected as damage, and the areas of joints are detected as structural feature areas associated with the construction of the structure.

[0167] Among the detected cracks, those whose ends meet or overlap with the seam area are classified as specific cracks (the so-called "crescent cracks").

[0168] In addition, regarding the seam area, the detected seam area can be enlarged through expansion processing, or an area slightly wider than the original seam area can be detected as the seam area.

[0169] Furthermore, when sorting crescent cracks, the shortest distance between the two ends of the crack and the seam area can be calculated separately. If the calculated distances are within the threshold, the crack is sorted as a crescent crack.

[0170] exist Figure 9 In (B), J represents the seam, and C... Y This indicates a crescent-shaped crack where both ends connect to the seam J. Figure 9(B) indicates that in Figure 9 (A) Overlays the original image showing the area of ​​seam J filled with a specific color and the crescent crack C. Y Images of seams and cracks in the area.

[0171] Figure 10 This is another example of a diagram showing specific damage that is associated with a structural feature area related to the construction of the structure.

[0172] Figure 10 (A) is another original image of the evaluation object structure, including seams and cracks. Figure 10 (B) indicates that in Figure 10 (A) Overlays the original image showing the area of ​​seam J filled with a specific color and the crescent crack C. Y Images of seams and cracks in the area.

[0173] User through Figure 9 (B) and Figure 10 (B) The display shown can easily identify the crescent-shaped crack C caused by seam J. Y It can be used to verify the appropriateness of joint construction methods and to improve joint construction methods.

[0174] Figure 11 This is another example of a process for sorting specific damages from damage detection.

[0175] like Figure 11 As shown in (A), damage to the structure (cracks in this example) is detected based on images of the structure. Crack detection can be performed using AI or image processing algorithms.

[0176] And, as Figure 11 As shown in (B), based on the images of the structure taken, structural feature regions associated with the construction of the structure are detected (in... Figure 11 In the example, this refers to the region of seam J. The detection of seam J can be performed using AI or image processing algorithms. Furthermore, it can also be performed by accepting manual input instructions from the user.

[0177] Next, as Figure 11 As shown in (C), among the detected cracks, crescent-shaped cracks (specific cracks) that are associated with the region of joint J were selected. Y The sorted crescent-shaped cracks C Y It can be output by color differentiation or by changing the line type in a way that can be distinguished from other cracks.

[0178] Additionally, crescent-shaped crack C YThe two ends of it connect with the area of ​​seam J, curving into a crescent shape, as shown. Figure 11 Like the crack shown above (C), only the crack whose one end is connected to the area of ​​joint J is not a crescent crack.

[0179] Figure 12 It is a damage map that includes crack information.

[0180] exist Figure 12 The damage diagram shows cracks C1 to C5, and P-cone marks P1 and P2. In particular, crack C1 is a sinking crack that originated on P-cone mark P1, and cracks C4 and C5 are sinking cracks that originated on P-cone mark P2.

[0181] Furthermore, the damage diagram is represented by a pattern drawn along the broken lines of each crack C1 to C5, and can be used as CAD data.

[0182] Figure 13 This is an example chart showing the number of damages included in the damage detection results, and... Figure 12 The damage diagram shown corresponds to this.

[0183] Figure 13 The damage quantity table shown includes items such as damage identification information (ID: identification code), damage type, size (width), size (length), and size (area), and records the information corresponding to each item for each damage.

[0184] In the case of cracks, the length or width of each crack C1 to C5 is quantified, and this information is recorded in the damage quantity table in association with the damage ID.

[0185] [Editing damage detection results]

[0186] Figure 5 The first learned model 21A, when inputting an image 13 of the structure, outputs each damaged area as a damage detection result 27A. However, the damage detection result 27A is sometimes detected incorrectly or inaccurately.

[0187] For example, damage areas are classified by pixel unit or by grouping several pixels into blocks, which sometimes lacks accuracy. Furthermore, cracks detected as two separate cracks are sometimes preferred as a single, connected crack. This is because it can sometimes be inferred that the crack is connected within the concrete.

[0188] Therefore, the CPU20 performs editing instruction acceptance processing, which accepts editing instructions for the damage detection results by using the operation unit 18 (e.g., mouse) operated by the user, and performs editing processing to edit the damage detection results according to the accepted editing instructions.

[0189] As an example of editing damage detection results, in the case of linear damage (cracks) where the endpoints along the crack's broken line are close to each other, editing to connect the endpoints can be considered. Regarding this editing, the distance between the endpoints of the crack's broken line can be measured after damage detection processing. If the measured distance is below a threshold, the endpoints will be automatically connected; alternatively, they can be automatically connected based on user instructions. The threshold can use a default value or can be set by the user.

[0190] Furthermore, a threshold relative to the crack length or width can be set to automatically delete damage detection results smaller than the threshold. The deletion of these damage detection results can be done automatically after damage detection processing or based on user instructions. The threshold can use a default value or be set by the user.

[0191] Figure 14 and Figure 15 These are images showing edited examples of damage detection results. Furthermore, when editing damage detection results, it is preferable to set the transparency of the color used to fill the crack image to a high level, making it an image where the structure is easily visually identifiable.

[0192] Figure 14 This is a diagram illustrating the method of adding vertices along the broken line of a crack.

[0193] A broken line is a line that connects multiple vertices along a crack (in...) Figure 14 The vertices are represented by quadrilaterals.

[0194] When adding vertices to this polyline, such as Figure 14 As shown in (A), position the mouse cursor on the polyline to which you want to add vertices, right-click, and select [Append] from the context menu. Thus, as... Figure 14 As shown in (B), new vertices can be added to the line of the polyline.

[0195] Furthermore, by dragging the added vertex and moving it to the original crack area, it is possible to edit the polyline along the crack.

[0196] Figure 15 This is a diagram illustrating the method of removing vertices from a polyline along a crack.

[0197] In the case of deleting vertices from the polyline, such as Figure 15 As shown in (A), position the mouse cursor over the vertex you want to delete, right-click (to select the vertex), and select [Delete] from the context menu. Thus, as... Figure 15 As shown in (B), vertices can be deleted from polylines.

[0198] like Figure 15 As shown in (B), when deleting a vertex from a polyline, a line connecting the vertices before and after the deleted vertex is used to edit the polyline along the crack.

[0199] The aforementioned editing functions include the following: the ability to select the entire polyline by clicking on the lines connecting the vertices and then delete the entire polyline; and the ability to manually add new polylines to areas of cracks that were missed during inspection.

[0200] [Damage Assessment Methods]

[0201] Figure 16 This is a flowchart illustrating an implementation method of the damage assessment method involved in this invention.

[0202] Figure 16 The processing of each step shown is, for example, through... Figure 4 The damage assessment device 10 shown is performed by a processor consisting of a CPU 20 or the like.

[0203] exist Figure 16 In step S10, the processor acquires an image of the structure being evaluated from the image acquisition unit 12 or the image database 14.

[0204] The first learned model 21A (which functions as a damage detection and processing unit) Figure 5 Detect damage (cracks) in the structure based on the image obtained in step S10 (step S12).

[0205] The second learned model 21B (which plays a role in feature region detection processing) Figure 5 Based on the image obtained in step S10, detect the structural feature region (the region of the P-cone mark) associated with the construction of the structure (step S14).

[0206] The processor determines whether damage is detected by the damage detection process performed in step S12 (step S16). If damage is detected ("yes"), the process proceeds to step S18.

[0207] In step S18, among the damages (cracks) detected in step S12, specific damages (sinking cracks) that are related to the characteristic areas of the structure are sorted out, and the process proceeds to step S20.

[0208] In step S20, it is determined whether a specific damage has been sorted out. If it is determined that a specific damage has been sorted out ("yes"), the processor outputs information about the specific damage (step S22).

[0209] Information on specific damage can be output, for example, by overlaying a damage image onto an image, displaying the damage image separately on a display unit, or outputting CAD data representing the damage diagram as a file. Furthermore, it is preferable to determine the size (length) of the specific damage (sinking crack), and output the sinking crack image by color differentiation or changing the line type based on the determined size.

[0210] Furthermore, in step S20, if it is determined that a specific damage has not been sorted out ("No"), for the damage (crack) detected in step S12, crack information is output in a way that can identify it as not a sinking crack (step S24). For example, the crack image is output with a different color or line type than that of a sinking crack. Alternatively, step S24 can be omitted, and information other than the specific damage (sinking crack) is not output.

[0211] [other]

[0212] In this embodiment, examples of structural feature areas related to the construction of the structure include P-cone marks and joints, but the invention is not limited to these; other structural feature areas such as pouring joints can also be detected. Furthermore, examples of specific damage related to structural feature areas include subsidence cracks and crescent cracks. However, damage such as water leakage from joints, pouring joints, etc., or water leakage from concrete components, resulting in free lime exposed on the surface when the water evaporates, is also specific damage related to structural feature areas (the areas of joints and pouring joints).

[0213] The hardware for implementing the damage assessment device according to this invention can be composed of various processors. These processors include general-purpose processors that execute programs and function as various processing units, such as CPUs (Central Processing Units); processors whose circuit structure can be changed after manufacturing, such as FPGAs (Field Programmable Gate Arrays), which are programmable logic devices (PLDs); and processors with circuit structures specifically designed for performing specific processes, such as ASICs (Application Specific Integrated Circuits), which are dedicated circuits. One processing unit constituting the damage assessment device can be composed of one of the aforementioned processors, or it can be composed of two or more processors of the same or different types. For example, one processing unit can also be composed of multiple FPGAs or a combination of a CPU and an FPGA. Furthermore, one processor can also constitute multiple processing units. As an example of one processor constituting multiple processing units, firstly, there is a method where, as represented by a computer such as a client or server, one or more CPUs and software are combined to form one processor, which functions as multiple processing units. Secondly, there is the following approach: For example, System-on-Chip (SoC) uses a processor that implements the overall system functionality, including multiple processing units, using a single IC (Integrated Circuit) chip. In this way, as a hardware structure, various processing units are constructed using one or more of the aforementioned processors. More specifically, the hardware structure of these various processors is a circuit composed of semiconductor elements and other circuitry.

[0214] Furthermore, the present invention includes a damage assessment program and a recording medium on which the damage assessment program is recorded. The damage assessment program is installed in a computer, enabling the computer to function as the damage assessment device involved in the present invention.

[0215] Furthermore, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention.

[0216] Symbol Explanation

[0217] 10-Damage assessment device, 12-Image acquisition unit, 13-Image, 14-Image database, 16-Storage unit, 18-Operation unit, 20-CPU, 21A-First learned model, 21B-Second learned model, 22-RAM, 24-ROM, 26-Display control unit, 27A-Damage detection result, 27B-Structural feature area detection result, 30-Display unit, C1~C5, C R C G C Y - Crack, J- Joint, P, P1, P2, P B P G P R P Y -P cone mark, S10~S24- steps.

Claims

1. A damage assessment device for a structure equipped with a processor, wherein, The processor performs the following processing: Image acquisition and processing: acquiring images of the structure that have been captured. Damage detection processing, which detects damage to the structure based on the acquired images; Structural feature region detection processing: Based on the acquired image, detect structural feature regions associated with the construction of the structure; The sorting process involves separating specific damages from the detected damage that are correlated with the detected structural feature regions; and Information output processing outputs information about the specific damage identified through sorting. In the damage detection process, when the image is input, a first fully learned model is executed. This first fully learned model outputs the region of each damage for each damage to the structure as the recognition result. In the structural feature region detection process, when the image is input, the second learned model is executed, and the second learned model outputs the structural feature region as the recognition result.

2. A damage assessment device for a structure equipped with a processor, wherein, The processor performs the following processing: Image acquisition and processing: acquiring images of the structure that have been captured. Damage detection processing, which detects damage to the structure based on the acquired images; Structural feature region detection processing: Based on the acquired image, detect structural feature regions associated with the construction of the structure; The sorting process involves separating specific damages from the detected damage that are correlated with the detected structural feature regions; and Information output processing outputs information about the specific damage identified through sorting. The sorting process includes an expansion process that expands the size of the structural feature region, and sorts damage that is in contact with or overlaps with the structural feature region that has undergone the expansion process as the specific damage.

3. A damage assessment device for a structure equipped with a processor, wherein, The processor performs the following processing: Image acquisition and processing: acquiring images of the structure that have been captured. Damage detection processing, which detects damage to the structure based on the acquired images; Feature region detection processing involves detecting structural feature regions associated with the construction of the structure based on the acquired image. The sorting process involves separating specific damages that are correlated with the detected structural feature regions from the detected damages. Information output processing, outputting information on the specific damage identified through sorting; and Size determination process, determining the size of the specific damage. Damage to the structure includes cracks in the structure. The specific damage refers to a specific crack in the structure that was caused during the construction of the structure. In the size determination process, the relative length of the specific crack on the image and the length of the structural feature region on the image are calculated, and the calculated relative length is used as the size of the specific damage.

4. A damage assessment device for a structure equipped with a processor, wherein, The processor performs the following processing: Image acquisition and processing: acquiring images of the structure that have been captured. Damage detection processing, which detects damage to the structure based on the acquired images; Feature region detection processing involves detecting structural feature regions associated with the construction of the structure based on the acquired image. The sorting process involves separating specific damages that are correlated with the detected structural feature regions from the detected damages. Information output processing, outputting information on the specific damage identified through sorting; and Size determination process, determining the size of the specific damage. Damage to the structure includes cracks in the structure. The specific damage refers to a specific crack in the structure that was caused during the construction of the structure. In the size determination process, the actual size of the specific damage is calculated based on the length of the specific crack on the image, the length of the structural feature region on the image, and the actual size of the structural feature region.

5. The damage assessment device according to claim 3 or 4, wherein, The image shows the structure with a known actual size as a scale reference. In the size determination process, the actual size of the specific damage is calculated based on the length of the specific crack on the image and the length of the scale reference on the image.

6. The damage assessment device according to claim 3 or 4, wherein, In the size determination process, the actual size of the specific damage is calculated based on the length of the specific crack in the image and the photographic conditions and camera information of the camera that took the image.

7. A damage assessment device for a structure equipped with a processor, wherein, The processor performs the following processing: Image acquisition and processing: acquiring images of the structure that have been captured. Damage detection processing, which detects damage to the structure based on the acquired images; Feature region detection processing involves detecting structural feature regions associated with the construction of the structure based on the acquired image. The sorting process involves separating specific damages that are correlated with the detected structural feature regions from the detected damages. Information output processing, outputting information on the specific damage identified through sorting; and The process of calculating the ratio of the total number of structural feature regions to the number of structural feature regions corresponding to the specific damage. In the information output processing, the calculated ratio is also output.

8. The damage assessment device according to any one of claims 1, 2, and 7, wherein, The damage to the structure is a crack in the structure. The specific damage is a specific crack in the structure that is caused by the construction of the structure.

9. The damage assessment device according to any one of claims 1 to 4 and 7, wherein, The structural feature area is a region representing construction traces associated with the specific damage, i.e., the specific crack, caused by the construction of the structure.

10. The damage assessment device according to any one of claims 1 to 4 and 7, wherein, In the information output processing, each specific damage is output in a recognizable manner according to the attributes of the specific damage.

11. The damage assessment device according to any one of claims 1 to 4 and 7, wherein, In the information output processing, based on the attributes of the specific damage, the structural feature region corresponding to the specific damage is output in an identifiable manner.

12. The damage assessment device according to any one of claims 1 to 4 and 7, wherein, The processor performs the following processing: The editing instruction is processed, and at least one of the detection results of the detected damage and the detection results of the detected structural feature area is received from the operation unit operated by the user. and Edit the test results according to the accepted editing instructions.

13. The damage assessment device according to any one of claims 1 to 4 and 7, wherein, In the information output processing, the information of the specific damage is output to a display for display, or saved in memory as a file.

14. The damage assessment device according to any one of claims 1 to 4 and 7, wherein, The information on the specific damage includes a damage quantity table, which has items such as damage identification information, damage type and size, and records information corresponding to each item for each specific damage.

15. A damage assessment method, wherein a processor performs damage assessment on a structure, wherein, The processor's processing includes the following steps: Images of the structure were captured. Damage to the structure is detected based on the acquired images; Based on the acquired images, structural feature regions associated with the construction of the structure are detected; From the detected damage, specific damages that are correlated with the detected structural feature regions are sorted out; and Output information about the specific damage identified in the sorting. In the step of detecting damage to the structure, when the image is input, a first fully learned model is executed. This first fully learned model outputs the region of each damage as a recognition result for each type of damage to the structure. In the step of detecting the feature region of the structure, when the image is input, the second learned model is executed, and the second learned model outputs the feature region of the structure as the recognition result.

16. A recording medium, which is non-transitory and computer-readable, recording a program that causes a computer to execute the damage assessment method of claim 15.

Images