Maintenance aids, methods and procedures

Abstract

The present invention provides an inspection assistance device, method, and program, which can output the damage detection results well when two or more types of damage are detected from a structure, especially when two or more types of damage are detected from the same or adjacent positions of the structure. The processor of the inspection assistance device acquires an image of the structure to be inspected and detects the damage of the structure based on the acquired image. When two or more types of damage (crack B and linear free lime C2) of the structure are detected, the processor determines whether two or more types of damage are detected from the same or adjacent positions. When outputting the damage detection results (damage images, damage maps, etc.), when it is determined that crack B and linear free lime C2 are detected from the same or adjacent positions, the damage detection results of linear free lime C2 are preferentially output in the order of priority of damage types ((A) and (B) in Fig. 16).

Description

Technical Field

[0001] The present invention relates to a maintenance auxiliary device, method and program, and in particular to a maintenance auxiliary technology for a structure. Background Art

[0002] Social infrastructure structures such as bridges require regular inspections for maintenance and repair.

[0003] Patent document 1 discloses a crack detection method and a crack display method, in which a camera is used to photograph the inner wall surface of a tunnel, and the photographed image is processed to extract and quantify cracks in each partition of the inner wall surface, and crack information is displayed for each partition. For example, according to the degree of cracks in each partition, the cracks are displayed in different colors for each partition, thereby making it easy to grasp the degree of cracks.

[0004] Previous technical literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2002-188998 Summary of the invention

[0007] Technical issues to be solved by the invention

[0008] However, there are many types of damage to structures besides cracks. For example, in the concrete part of a structure, in addition to cracks, there are many types of damage such as water leakage, free lime, rust, peeling, exposed steel bars, etc. Similarly, in the steel parts of a structure, there are many types of damage such as cracking, corrosion, and deterioration of anti-corrosion function.

[0009] Patent Document 1 describes displaying cracks in different colors for each partition according to the degree of the cracks in each partition, but does not describe detecting two or more (multiple) types of damage from a structure, nor does it describe a method for outputting the detection results of multiple damages.

[0010] The present invention is completed in view of such a situation, and its purpose is to provide a maintenance assistance device, method and program that can output the damage detection results well when more than two types of damage are detected from a structure, especially when more than two types of damage are detected from the same or close positions of the structure.

[0011] Means for solving technical problems

[0012] In order to achieve the above-mentioned purpose, the invention involved in the first mode is a maintenance assistance device, which has a processor, and the processor performs the following processing: image acquisition processing, acquiring an image of a structure of a maintenance object; damage detection processing, detecting damage to the structure based on the acquired image; judgment processing, when more than two types of damage to the structure are detected by the damage detection processing, judging whether more than two types of damage are detected from the same or close positions among the more than two types of damage; and output processing, outputting the damage detection result detected by the damage detection processing, when it is determined by the judgment processing that more than two types of damage are detected from the same or close positions, the damage detection result is output according to the priority order of the damage type.

[0013] According to the first aspect of the present invention, when two or more types of damage to a structure are detected based on an image of the structure to be inspected, and in particular, when two or more types of damage are detected from the same or close positions of the structure, damage detection results are output as damage detection results in order of priority. Thus, when two or more types of damage are detected from the same or close positions of the structure, by outputting damage detection results in order of priority of damage types, it is possible to cope with a situation where two or more types of damage are detected from the same or close positions of the structure. In addition, when two or more types of damage are not at the same or close positions of the structure, two or more types of damage detection results can be directly output.

[0014] In the maintenance support device according to the second aspect of the present invention, it is preferable that: in the damage detection process, the damage area and the damage type of each damage area are detected based on the image, the determination process determines whether two or more damage types are detected in the same or adjacent damage area by the damage detection process, and in the output process, when it is determined by the determination process that two or more damage types are detected in the same or adjacent damage area, the damage detection result of the damage type with the highest priority is output as the damage detection result of the same or adjacent damage area. This is because, in the case where two or more damage types are detected from the same or adjacent positions of the structure, it is effective to output the damage detection result of the damage type with the highest priority and notify the user.

[0015] In the maintenance support device according to the third aspect of the present invention, it is preferable that the close position is a position where a distance between two or more types of damage is equal to or smaller than a threshold value.

[0016] In the maintenance support device according to the fourth aspect of the present invention, it is preferable that in the damage detection process, when an image is input, a learned model is executed that outputs a damaged area and a damage type of each damaged area as a recognition result.

[0017] In the maintenance support device according to the fifth aspect of the present invention, it is preferable that in the output process, different drawing patterns are output when the damage type is linear damage and when the damage type is planar damage.

[0018] In the maintenance support device according to the sixth aspect of the present invention, it is preferable that, in the output process, when the damage type is linear damage, a damage map indicating lines that do not close the linear damage is output, and when the damage type is planar damage, a damage map indicating closed lines that surround the planar damage is output. When the damage map is output in a line drawing pattern, in the case of linear damage, a damage map indicating lines that do not close the linear damage is output, and when the damage type is planar damage, a damage map indicating closed lines that surround the planar damage is output.

[0019] In the maintenance assistance device involved in the 7th mode of the present invention, it is preferred as follows: during the output processing, when the damage type is linear damage, a damage image filled with at least linear damage is output; when the damage type is planar damage, a damage image filled with at least planar damage is output.

[0020] In the maintenance support device according to the eighth aspect of the present invention, it is preferable that the output processing outputs the damage detection result and displays it on a display, or stores the damage detection result in a file format in a memory.

[0021] In the maintenance support device according to the ninth aspect of the present invention, it is preferable that the priority order of the damage type is a priority order set in advance according to the severity of the damage.

[0022] In the maintenance support device according to the tenth aspect of the present invention, in the case of linear damage including linear free lime and cracks as the damage type, the linear free lime has a higher priority than the crack.

[0023] In the maintenance auxiliary device involved in the 11th mode of the present invention, as the damage type, in the case of surface damage including exposed steel bars, peeling, rust, surface free lime and water leakage, the priority is set to become lower in the order of exposed steel bars, peeling, rust, surface free lime and water leakage.

[0024] In the maintenance support device according to the twelfth aspect of the present invention, the processor performs a priority order acceptance process of accepting the priority order of damage types of the structure from the operation unit operated by the user, and the priority order of damage types is the priority order accepted from the user via the operation unit.

[0025] In the maintenance assistance device involved in the 13th mode of the present invention, it is preferred as follows: the processor performs the following processing: editing instruction acceptance processing, accepting editing instructions for damage detection results from an operating unit operated by a user; and editing processing, editing the damage detection results according to the accepted editing instructions.

[0026] In the maintenance support device involved in the 14th aspect of the present invention, it is preferred that the damage detection result includes a damage quantity table having items of damage identification information, damage type and size, and recording information corresponding to each item for each detected damage.

[0027] The invention involved in the 15th embodiment is a maintenance assistance method, which performs maintenance assistance on a structure to be maintained by a processor, and each processing of the processor includes the following steps: acquiring an image of the structure to be maintained; detecting two or more types of damage to the structure based on the acquired image; determining whether two or more types of damage to the detected structure are detected from the same or close positions; and outputting the detected damage detection results. When it is determined by the determination step that two or more types of damage are detected from the same or close positions, the damage detection results are output according to the priority order of the damage types.

[0028] The invention involved in the 16th embodiment is a maintenance assistance program, which enables a computer to execute a method for assisting maintenance of a structure to be maintained, the method comprising the following steps: acquiring an image of the structure to be maintained; detecting two or more types of damage to the structure based on the acquired image; determining whether two or more types of damage to the detected structure are detected from the same or close positions; and outputting the detected damage detection results. When it is determined by the determination step that two or more types of damage are detected from the same or close positions, the damage detection results are output according to the priority order of the damage types.

[0029] Effects of the Invention

[0030] According to the present invention, when two or more types of damage are detected from a structure, especially when two or more types of damage are detected from the same or close positions of the structure, the damage detection results can be outputted satisfactorily. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a diagram showing an example of damage to a structure.

[0032] Figure 2 This is a diagram showing an example of linear free lime.

[0033] Figure 3 This is a diagram showing an example of planar free lime.

[0034] Figure 4 is a diagram showing the types of damage representation methods corresponding to the types of damage, Figure 4 (A) is a diagram showing an image including a crack, Figure 4 (B) is a diagram showing an image in which a broken line along a crack is drawn.

[0035] Figure 5 is a diagram showing the types of damage representation methods corresponding to the types of damage, Figure 5 (A) is a diagram showing an image including peeling and exposed steel bars. Figure 5 (B) is a diagram showing an image in which a polygon surrounding the peeling and steel bar exposed region is drawn.

[0036] Figure 6 is a diagram showing the types of damage representation methods corresponding to the types of damage, Figure 6 (A) is a diagram showing an image including planar free lime. Figure 6 (B) is a diagram showing an image in which a polygon surrounding a planar region of free lime is drawn.

[0037] Figure 7 This is a diagram showing the respective broken lines when cracks as linear damage and linear free lime are detected.

[0038] Figure 8 It is used to illustrate Figure 7 The diagram shown is a diagram showing the proximity of cracks, which are linear damage, and linear free lime.

[0039] Fig. 9 This is a block diagram showing an example of the hardware configuration of the maintenance support device according to the present invention.

[0040] Fig.10 This is a conceptual diagram showing an implementation of a damage detection processing unit composed of a CPU, etc.

[0041] Fig.11 This is a perspective view showing an example of a bridge to be inspected.

[0042] Fig.12 This figure shows an example of an ortho image corresponding to a coffer which is one of the inspection units of a bridge.

[0043] Fig.13 It means according to Fig.12 An example of damage detection results detected by orthophoto images is shown in the figure.

[0044] Fig.14 This is a diagram showing an example of an orthophoto image on which a damage map corresponding to the caisson is superimposed.

[0045] Fig.15 This is a graph showing an example of a damage quantity table included in the damage detection result.

[0046] Fig.16 This is a schematic diagram showing an example of the damage detection result of cracks and linear free lime by the damage detection processing unit and its output processing.

[0047] Fig.17 This is a schematic diagram showing another example of the damage detection result of cracks and linear free lime by the damage detection processing unit and its output processing.

[0048] Fig.18 It is a schematic diagram showing an example of the damage detection result of the planar free lime and the linear free lime by the damage detection processing unit and the output processing thereof.

[0049] Fig.19 This is a schematic diagram showing another example of the damage detection result of planar free lime and linear free lime by the damage detection processing unit and its output processing.

[0050] Fig. 20 This is a schematic diagram showing an example of damage detection results of rust juice, planar free lime and water leakage by the damage detection processing unit and its output processing.

[0051] Fig.21 This is a schematic diagram showing another example of damage detection results of rust juice, planar free lime and water leakage by the damage detection processing unit and its output processing.

[0052] Fig. 22 It is a schematic diagram showing a GUI of the second embodiment that indicates the output of damage detection results, and is a diagram showing an example of a screen displayed on the display unit.

[0053] Fig.23 It is a schematic diagram showing a GUI of the second embodiment that indicates the output of damage detection results, and is a diagram showing another example of a screen displayed on the display unit.

[0054] Fig.24 is a schematic diagram of a GUI of a third embodiment showing the output of damage detection results. Fig.24 (A) shows a case where the transparency of the fill color of the damaged image is set to "10". Fig.24 (B) is a diagram showing a composite image in which a damaged image with a transparency of "10" is superimposed on an image of a photographed structure.

[0055] Fig.25 is a schematic diagram of a GUI of a third embodiment showing the output of damage detection results. Fig.25(A) shows a case where the transparency of the fill color of the damaged image is set to "50". Fig.25 (B) is a diagram showing a composite image in which a damaged image with a transparency of "50" is superimposed on an image of a photographed structure.

[0056] Fig.26 is a schematic diagram of a GUI of a third embodiment showing the output of damage detection results. Fig.26 (A) shows a case where the transparency of the fill color of the damaged image is set to "100". Fig.26 (B) is a diagram showing a composite image in which a damaged image with a transparency of "100" is superimposed on an image of a photographed structure.

[0057] Fig. 27 This figure shows a method of adding vertices to a polygon surrounding a damaged area.

[0058] Fig.28 A diagram showing a method of deleting vertices from a polygon surrounding a damaged area.

[0059] Fig.29 It is a flowchart showing an embodiment of the maintenance support method according to the present invention. DETAILED DESCRIPTION

[0060] Hereinafter, preferred embodiments of the maintenance support device, method, and program according to the present invention will be described with reference to the accompanying drawings.

[0061] [Summary of the Invention]

[0062] Figure 1 This is a diagram showing an example of damage to a structure, and in particular shows damage to a concrete member constituting the structure.

[0063] Figure 1 (A) indicates water leakage A, which is one of the phenomena caused by damage to concrete parts. Water leakage A is caused by water leaking from the damaged part due to damage to concrete parts (cracks, cracks in joints, defective joint fillers, etc.).

[0064] Figure 1 (B) indicates cracks B, free lime C1, and rust D generated in the concrete part. Free lime C1 refers to the phenomenon that lime components flow out from the concrete part due to water leakage, etc., and appear on the surface when the water evaporates. Rust D refers to the corrosion of steel materials such as steel bars inside the concrete part, and the brown corrosion products seep out on the concrete surface.

[0065] Figure 1(C) indicates peeling E and exposed steel bars F generated in the concrete member. Peeling E refers to a state where a floating concrete piece is peeled off, and exposed steel bars F refers to a state where, as a result of peeling E, steel bars in the concrete are exposed.

[0066] Although not shown in the figure, damage to the steel members constituting the structure includes types of damage such as cracking, corrosion, breakage, and deterioration of the anti-corrosion function.

[0067] Figure 2 is a diagram showing an example of linear free lime. Figure 3 This is a diagram showing an example of planar free lime.

[0068] Figure 2 The linear free lime C2 shown in (A) and (B) is a state where the lime component is clogged in the cracks generated in the concrete member. Therefore, the linear free lime C2 and the cracks have approximately the same shape, and the cracks are generated in the linear free lime C2 at the same position (region) as the linear free lime C2.

[0069] Figure 3 The planar free lime shown in (A) spreads downward through the cracks as water leaks from the cracks extending in the horizontal direction. Figure 3 The planar free lime C1 shown in (B) expands around the cracks in the joint. Figure 3 The planar free lime C1 shown in (C) spreads around the concrete crack.

[0070] One aspect of the present invention is to detect damage to a structure based on an image of the structure to be inspected, and output a damage detection result according to the type of the detected damage.

[0071] Figures 4 to 6 This is a diagram showing the types of damage expression methods corresponding to the types of damage.

[0072] like Figure 4 As shown, in the case of crack detection ( Figure 4 (A)) As a method of expressing cracks, a pattern formed by non-closed lines (broken lines) along the cracks is used to express ( Figure 4 (B)). This is because, in the case of linear damage such as cracks, it is necessary to quantify the length.

[0073] Therefore, if Figure 2As shown, when linear free lime is detected, as a method of expressing the linear free lime, it is also output as a drawing pattern along the line of the linear free lime. Moreover, cracks and linear free lime are both linear damages, but since the types of damages are different, it is preferable to express them identifiably with broken lines of different line types (for example, colors).

[0074] like Figure 5 As shown, when peeling E and steel bar exposure F are detected ( Figure 5 As a method of expressing the peeling E and the steel bar exposure F, a drawing pattern formed by closed lines (polygons) surrounding the planar damage area is used to express ( Figure 5 (B)). This is because, in the case of planar damage such as peeling E, quantification of the area is required.

[0075] like Figure 6 As shown, when the surface-like free lime C1 is detected ( Figure 6 As a method of expressing the planar free lime C1, a drawing pattern formed by closed lines (polygons) surrounding the planar damaged area is used to express ( Figure 6 (B)).

[0076] Furthermore, planar free lime C1, peeling E, and exposed steel bar F are all planar damages, but since the types of damages are different, they are preferably expressed identifiably using polygons with different line types (for example, colors).

[0077] In one embodiment of the present invention, even if the damage is classified as the same, if the damage has different shapes, the damage type is different and is represented by different drawing patterns according to the shape. For example, linear free lime is represented by a broken line, and planar free lime is represented by a polygon.

[0078] In addition, on a structure (image), two or more types of damage may be detected from the same or close positions. Here, the close position refers to a position where the distance between the two or more types of damage is less than a threshold. The threshold may be determined by default or may be set by the user.

[0079] For example, in the case of linear free lime, the linear free lime overlaps with the concrete crack, and two damages are detected at the same position. Figure 5 As shown, in the case of the peeling E and the steel bar exposure F, the steel bar exposure F exists in the peeling E. Therefore, the distance between the two damages, the steel bar exposure F and the peeling E, is less than the threshold value, and the two are damages at close positions.

[0080] Figure 7 This is a diagram showing the respective broken lines when cracks as linear damage and linear free lime are detected.

[0081] exist Figure 7 In the figure, X is a broken line representing cracks, and Y is a broken line representing linear free lime.

[0082] Figure 8 It is used to illustrate Figure 7 The diagram shown is a diagram showing the proximity of cracks, which are linear damage, and linear free lime.

[0083] Below, reference Figure 8 , an example of judging the “closeness” of two types of damage, cracks and linear free lime, is described.

[0084] like Figure 8 As shown in (A), the shortest distance between the focus point P1 of the broken line X representing the crack (the first vertex as the end point of the broken line X) and the broken line of the linear free lime is set to L1.

[0085] Likewise, if Figure 8 As shown in (B), the shortest distance between the focus point P2 (the second vertex) of the polyline X and the polyline Y is set to L2, as shown in Figure 8 As shown in (C), the shortest distance between the focus point P3 (the third vertex) of the polyline X and the polyline Y is set to L3.

[0086] And, if Figure 8 As shown in (D) to (F) of FIG. 1 , the shortest distances of the two broken lines X and Y are set to L4, L5, and L6, respectively. In addition, the shortest distance L4 is the shortest distance of the focus point 4 of the broken line Y (the vertex as the end point of the broken line Y), the shortest distance L5 is the shortest distance of the focus point 5 of the broken lines X and Y (the focus point between the focus point 4 and the focus point 6), and the shortest distance L6 is the shortest distance of the focus point 6 of the broken line X (the vertex as the other end point of the broken line X).

[0087] Furthermore, when L1, L2, L3>threshold and L4, L5, L6≤threshold, it is determined that the two broken lines Y and X (two lesions) are "close" within the range of L4 to L6. The number of focus points P1 to P6 is not limited to the above example.

[0088] Furthermore, the above example describes the distance between two types of linear damage, but the distance between linear damage and planar damage can also be determined in the same way by finding the shortest distance between each focus point of the polyline and the polygon to make a "close" judgment.

[0089] In one aspect of the present invention, priority is defined according to the type of damage, and when two or more types of damage are detected from the same or close positions, the damage is displayed in the order of priority. In addition, details of the method of displaying damage in the order of priority will be described later.

[0090] [Hardware structure of maintenance auxiliary device]

[0091] Fig. 9 This is a block diagram showing an example of the hardware configuration of the maintenance support device according to the present invention.

[0092] As Fig. 9 The maintenance support device 10 shown can be used with a personal computer or a workstation. The maintenance support device 10 of this example is mainly composed of an image acquisition unit 12, an image database 14, a storage unit 16, an operation unit 18, a CPU (Central Processing Unit) 20, a RAM (Random Access Memory) 22, a ROM (Read Only Memory) 24 and a display control unit 26.

[0093] The image acquisition unit 12 corresponds to an input / output interface, and in this example acquires a photographic image of a structure to be inspected, etc. The structure to be inspected includes, for example, structures such as bridges and tunnels.

[0094] The images acquired by the image acquisition unit 12 are, for example, a large number of images (photographic image group) of the structure taken by a drone (unmanned aerial vehicle) or robot equipped with a camera, or by human hands. Preferably, the photographic image group covers the entire structure, and adjacent photographic images overlap.

[0095] The photographic image group acquired by the image acquisition unit 12 is stored in the image database 14 .

[0096] The storage unit 16 is a memory composed of a hard disk device, a flash memory, etc. In addition to the operating system and the maintenance support program, the storage unit 16 also stores information indicating the priority of damage types, CAD (computer-aided design) data indicating the structure, and file-based damage maintenance results. The damage maintenance results can be stored in different layers according to the type of damage as damage information. In addition, the damage information includes a damage map.

[0097] Regarding the CAD data, if there is CAD data of the structure to be inspected, the data can be used. If there is no CAD data of the structure, the CAD data can be automatically generated from the image group stored in the image database 14.

[0098] When the photographic image group stored in the image database 14 is taken by a camera mounted on an unmanned aerial vehicle, a three-dimensional point cloud model can be generated. The three-dimensional point cloud model extracts feature points between overlapping photographic images of the photographic image group, estimates the position and posture of the camera mounted on the unmanned aerial vehicle based on the extracted feature points, and simultaneously estimates the three-dimensional position of the feature points based on the estimated results of the position and posture of the camera.

[0099] There is a Structure from Motion (SfM: 3D reconstruction) method that tracks the movement of a large number of feature points in a group of photographic images taken by a drone from the camera's shooting position, and simultaneously estimates the 3D structure (Structure) and camera posture (Motion) of the structure. In recent years, an optimization calculation method called bundle adjustment has been developed, which enables high-precision output.

[0100] In addition, as the camera parameters (focal length, image sensor image size, pixel pitch, etc.) required when applying the SfM method, the parameters stored in the storage unit 16 can be used. Furthermore, CAD data of the structure can be generated based on the generated three-dimensional point cloud model.

[0101] The operation unit 18 includes a keyboard and a mouse connected to the computer by wire or wirelessly, and in addition to functioning as an operation unit for performing normal operation instructions of the computer, it also functions as an operation unit for editing the damage detection result of the structure detected based on the image of the structure through user operation and setting the priority of multiple damage types of the structure through user operation. In addition, details of editing the damage detection result and setting the priority of the damage type will be described later.

[0102] CPU 20 performs the following processing: damage detection processing, reading various programs stored in the storage unit 16 or ROM 24, etc., centrally controlling each unit, and detecting damage to the structure (more than two types of damage) based on the image of the structure; judgment processing, judging whether more than two types of damage are detected from the same or close positions; and output processing, outputting the damage detection results detected by the damage detection processing; etc.

[0103] The damage detection process of detecting two or more types of damage from an image of a structure can be performed by artificial intelligence (AI).

[0104] As AI, for example, a learned model based on a convolutional neural network (CNN) can be used.

[0105] Fig.10 This is a conceptual diagram showing an implementation of a damage detection processing unit composed of a CPU, etc.

[0106] exist Fig.10 In the embodiment, the damage detection processing unit 21 is composed of a plurality of (three in this example) learned models 21A, 21B, and 21C corresponding to a plurality of types of damage.

[0107] Each of the learned models 21A, 21B, and 21C includes an input layer, an intermediate layer, and an output layer, and each layer has a structure in which a plurality of “nodes” are connected by “edges”.

[0108] An image 13 of a photographed structure is input to the input layer of CNN. The intermediate layer has multiple groups of convolutional layers and pooling layers as one group, and is a part that extracts features from the image input by the input layer. In the convolutional layer, the nodes located nearby in the previous layer are filtered (convolution operation using filters is performed) to obtain a "feature map". In the pooling layer, the feature map output from the convolutional layer is reduced as a new feature map. The "convolutional layer" is responsible for extracting features such as edges from the image, and the "pooling layer" is responsible for giving robustness so that the extracted features are not affected by parallel movement, etc.

[0109] The output layer of CNN is a part that outputs a feature map representing the features extracted by the intermediate layer. The output layer of the learned models 21A, 21B, and 21C in this example outputs the inference results obtained by, for example, classifying (segmenting) each damaged area of ​​the structure shown in the image in units of pixels or units of several pixels as a block as damage detection results 27A, 27B, and 27C.

[0110] For example, the learned model 21A is a learned model that has been machine-learned to detect damage caused by water leakage / planar free lime / rust juice, and the damage area of ​​each of the water leakage / planar free lime / rust juice and the damage type of each damage area are output as damage detection results (identification results) 27A. The learned model 21B is a learned model that has been machine-learned to detect damage caused by peeling / rebar exposure, and the damage area of ​​each of the peeling / rebar exposure and the damage type of each damage area are output as damage detection results 27B. The learned model 21C is a learned model that has been machine-learned to detect damage caused by cracks / linear free lime, and the damage area of ​​each of the cracks / linear free lime and the damage type of each damage area are output as damage detection results 27C.

[0111] In addition, the damage detection processing unit 21 is not limited to the above-mentioned embodiment. For example, it can also be configured to have a separate learned model for each damage type, and each learned model outputs a damage area corresponding to each damage type as a damage detection result. In this case, the same number of learned models as the number of damage types of the inspection object are provided. In addition, it can also be configured to have a learned model that can correspond to all damage types, and output the damage area and the damage type of each damage area as the damage detection result.

[0112] Return to Fig. 9 The CPU 20 outputs the damage detection result detected by the damage detection process via the display control unit 26 and displays it on the display unit (display) 30, or saves the damage detection result in the form of a file in the storage unit (memory) 16.

[0113] The RAM 22 is used as a work area of ​​the CPU 20 and as a storage unit for temporarily storing read programs and various data.

[0114] The display control unit 26 is a part that generates display data to be displayed on the display unit 30 and outputs it to the display unit 30. In this example, the damage detection result detected by the CPU 20 is displayed on the display unit 30, and an editing screen for the damage detection result based on the user operation from the operation unit 18 is displayed on the display unit 30.

[0115] The display unit 30 uses various displays such as a liquid crystal monitor that can be connected to a computer, and displays damage detection results detected from the image together with the image of the structure input from the display control unit 26, and is used as a part of the user interface together with the operation unit 18.

[0116] The processor including the CPU 20 of the maintenance support device 10 having the above-described configuration performs the above-described processing by reading out the maintenance support program stored in the storage unit 16 or the ROM 24 and executing the maintenance support program.

[0117] <Function of maintenance auxiliary device>

[0118] Next, for Fig. 9 The function of the maintenance assisting device 10 shown is described by taking a bridge as an example of a structure.

[0119] Fig.11 This is a perspective view showing an example of a bridge to be inspected.

[0120] like Fig.11As shown, the bridge 1 is composed of various components including a main beam 2 erected between piers 7, a cross beam 3 arranged in a direction perpendicular to the main beam 2 and connecting the main beams, a sway bracing 4 and a lateral bracing 5 connecting the main beams 2 to each other, and a bridge deck 6 for vehicles to travel is cast on the upper part of the main beams, etc. The bridge deck 6 is usually made of reinforced concrete.

[0121] The bridge deck 6 is usually based on a rectangular caisson divided by the main beam 2 and the cross beam 3 as a basic unit. When the damage (cracks, concrete peeling, etc.) of the bridge deck is inspected, it is carried out in the caisson unit.

[0122] Each caisson on the bridge deck is one of the components (maintenance units) that constitute the structure (bridge). In addition, the maintenance units of the bridge include not only the bridge deck (caisson), but also the parts / components that constitute the structure (main beam 2, cross beam 3, cross brace 4, cross brace 5, bridge pier 7 (column / wall, beam, corner / joint)).

[0123] The CPU 20 of the maintenance support device 10 , the maintenance support program stored in the storage unit 16 , the RAM 22 and the ROM 24 , the display control unit 26 , and the like constitute a processor, and the processor performs various processes described below.

[0124] The processor performs an image acquisition process of acquiring an image of the inspection unit from a plurality of images of the structure (bridge 1 ) to be inspected stored in the image database 14 .

[0125] Fig.12 This figure shows an example of an orthophoto image corresponding to a caisson, which is one of the inspection units of a bridge.

[0126] An orthophoto is an image obtained by projecting an image of a structure (caisson) onto the surface of the caisson. An orthophoto of the caisson can be generated by extracting a plurality of images corresponding to the caisson from a photographic image group stored in the image database 14, performing panoramic synthesis on the extracted plurality of images, and projecting the panoramic synthesized image onto the surface of the caisson.

[0127] Fig.10 When an orthophoto image (image 13) of the caisson is input, the damage detection processing unit 21 shown detects damage of the caisson based on the input image 13 and outputs the damage detection results 27A to 27C.

[0128] Fig.13 It means according to Fig.12 An example of damage detection results detected by orthophoto images is shown in the figure.

[0129] Fig.13The damage detection result shown shows a damage map indicating the damage of the caisson to be inspected.

[0130] exist Fig.13 The damage diagram shown shows five cracks C1 to C5 and concrete peeling H1.

[0131] Fig.13 The damage map shown is expressed by a drawing pattern formed by broken lines along each crack C1 to C5 (linear damage) detected on the orthoimage, a drawing pattern formed by a polygon surrounding the area of ​​detachment H1 (planar damage), or an image filled in the polygon.

[0132] Fig.14 This is a diagram showing an example of an orthophoto image on which a damage map corresponding to the caisson is superimposed.

[0133] Fig.14 The orthophoto with the damage map superimposed on it can be viewed in Fig.12 Overlay on the orthophoto shown Fig.13 The damage map shown is generated.

[0134] The damage map can be generated by assigning colors corresponding to the damage types to the damaged parts, and the damaged parts can be easily visually recognized by superimposing the damage map on the orthoimage.

[0135] Fig.15 This is a graph showing an example of a damage quantity table included in the damage detection result.

[0136] Fig.15 The damage quantity table shown has items of damage identification information (ID: identification code), damage type, size (width), size (length), and size (area), and information corresponding to each item is described for each damage.

[0137] In the case of cracks as linear damage, the length or width of each crack C1 to C5 is quantified, and in the case of peeling as planar damage, the area of ​​the peeling region H1 is quantified, and this information is recorded in the damage quantity table in association with the damage ID.

[0138] [First Embodiment of Damage Detection Result Output]

[0139] Fig.16 This is a schematic diagram showing an example of the damage detection result of cracks and linear free lime by the damage detection processing unit and its output processing.

[0140] Fig.16 (A) indicates Fig.10The damage detection processing unit 21 (learned model 21C) shown in the figure inputs the image 13, and detects damage detection results indicating the damage areas of the crack B and the linear free lime C2 and the damage types of each damage area by the learned model 21C. In this case, the CPU 20 performs a determination process to determine whether the crack B and the linear free lime C2 are detected from the same or close positions.

[0141] exist Fig.16 In the example shown in (A), for the sake of convenience, the crack B and the linear free lime C2 are shown side by side, but the linear free lime C2 is a state in which the lime component is blocked in the crack B generated in the concrete member. Therefore, the linear free lime C2 and the crack B have approximately the same shape, and the crack B is generated in the linear free lime C2 at the same position (region) as the linear free lime C2.

[0142] exist Fig.16 In the case of the crack B and the linear free lime C2 shown in (A), the CPU 20 determines that the crack B and the linear free lime C2 are detected from the same or close positions. Furthermore, when it is determined that the crack B and the linear free lime C2 are detected from the same or close positions, the CPU 20 performs output processing of outputting the damage detection results in the priority order of the damage types.

[0143] In this example, as the priority of damage types, the priority of linear free lime C2 is set higher than that of crack B. Fig.16 As shown in (B), the CPU 20 displays the damage image of the area filled with linear free lime C2 on the display unit 30 via the display control unit 26, or outputs the CAD data of the damage map showing the broken line of the linear free lime C2 in the form of a file. The file of the CAD data of the damage map is preferably stored in the storage unit 16 in association with the image in which the damage is detected.

[0144] Fig.17 This is a schematic diagram showing another example of the damage detection result of cracks and linear free lime by the damage detection processing unit and its output processing.

[0145] exist Fig.17 In the example shown in (A), cracks B and linear free lime C2 are detected, and a part of the cracks B and a part of the linear free lime C2 occur at the same position.

[0146] In this case, the CPU 20 determines that a portion of the crack B and a portion of the linear free lime C2 are detected from the same position. Fig.17As shown in (B), the CPU 20 displays the damage image of the area filled with linear free lime C2 on the display unit 30 via the display control unit 26, or outputs the CAD data of the damage map representing the broken line of the linear free lime C2 in the form of a file.

[0147] In addition, regarding the remaining portion of the crack B that does not overlap with the linear free lime C2, the CPU 20 directly displays the damage image of the region filling the crack B on the display unit 30 via the display control unit 26, or outputs the CAD data of the damage diagram representing the broken line of the crack B in the form of a file. It is also preferable that the damage image and CAD data representing the crack B and the damage image and CAD data representing the linear free lime C2 can be distinguished by, for example, changing the line type (for example, color).

[0148] Fig.18 It is a schematic diagram showing an example of the damage detection result of the planar free lime and the linear free lime by the damage detection processing unit and the output processing thereof.

[0149] Fig.18 (A) indicates that Fig.10 The damage detection processing unit 21 (learned models 21A and 21C) shown in the figure receives the image 13, detects the damaged area of ​​the planar free lime C1 by the learned model 21A, and detects the damaged area of ​​the linear free lime C2 by the learned model 21C. In this case, the CPU 20 determines whether the planar free lime C1 and the linear free lime C2 are detected from the same or close positions.

[0150] exist Fig.18 In the example shown in (A), the planar free lime C1 and the linear free lime C2 are partially overlapped with each other, so the CPU 20 determines that the planar free lime C1 and the linear free lime C2 are detected from the same or close positions. And, when it is determined that the planar free lime C1 and the linear free lime C2 are detected from the same or close positions, the CPU 20 outputs the damage detection result according to the priority order of the damage type.

[0151] exist Fig.18 In the example shown, the priority is set to be lower in the order of planar free lime C1 and linear free lime C2. Fig.18As shown in (B), the CPU 20 performs the following operations: with respect to the overlapped portion of the planar free lime C1 and the linear free lime C2, the planar free lime C1 is given priority, and a damaged image of the damaged region of the planar free lime C1 filled with a specific color is displayed on the display unit 30 via the display control unit 26, and a damaged image of a partial region of the linear free lime C2 that does not overlap is displayed on the display unit 30 via the display control unit 26. Fig.18 As shown in (C), for the overlapping portion of the planar free lime C1 and the linear free lime C2, the CPU 20 outputs the CAD data of the polygon surrounding the damaged area of ​​the planar free lime C1 together with the CAD data of a part of the broken line of the non-overlapping linear free lime C2 in the form of a file.

[0152] Fig.19 This is a schematic diagram showing another example of the damage detection result of planar free lime and linear free lime by the damage detection processing unit and its output processing.

[0153] Fig.19 The example shown is similar to Fig.18 Compared with the example shown, the priority of the planar free lime C1 and the priority of the linear free lime C2 are reversed, and the priority of the linear free lime C2 is higher than the priority of the planar free lime C1.

[0154] like Fig.19 As shown in (A), when the planar free lime C1 and the linear free lime C2 partially overlap each other, as shown in Fig.19 As shown in (B), for the overlapped portion of the planar free lime C1 and the linear free lime C2, the CPU 20 preferentially displays the damaged image of the region filled with the linear free lime C2, and similarly preferentially outputs the CAD data of the broken line of the linear free lime C2 in the form of a file. In this case, neither the damaged image nor the CAD data is output for the planar free lime C1.

[0155] Fig. 20 This is a schematic diagram showing an example of damage detection results of rust juice, planar free lime and water leakage by the damage detection processing unit and its output processing.

[0156] Fig. 20 (A) indicates Fig.10 The damage detection processing unit 21 (learned model 21A) shown in the figure inputs the image 13 and detects the damaged areas of rust juice D, planar free lime C1 and water leakage A by using the learned model 21A. In this case, the CPU 20 determines whether the rust juice D, planar free lime C1 and water leakage A are detected from the same or close positions.

[0157] exist Fig. 20 In the example shown in (A), planar free lime C1 is generated inside the area of ​​water leakage A, and rust D is generated inside the planar free lime C1, so these damages overlap each other in whole or in part.

[0158] exist Fig. 20 In the case of rust juice D, planar free lime C1, and water leakage A shown in (A), the CPU 20 determines that these damages are detected from the same or close positions. Furthermore, when it is determined that rust juice D, planar free lime C1, and water leakage A are detected from the same or close positions, the CPU 20 outputs the damage detection result according to the priority order of the damage types.

[0159] exist Fig. 20 In the example shown, the priority is set to be lower in the order of rust juice D, surface free lime C1 and leaking water A. Fig. 20 As shown in (B), the CPU 20 overlaps the area of ​​planar free lime C1 on the area of ​​water leakage A, and further overlaps the area of ​​planar free lime C1 on the area of ​​rust juice D, and displays the damage image of each area filled with a different color for each damage type on the display unit 30 via the display control unit 26. Fig. 20 As shown in (C) of FIG. 1 , the CPU 20 outputs the CAD data of the polygons surrounding the rust juice D, the planar free lime C1 and the leaking water A in the form of a file.

[0160] Fig.21 This is a schematic diagram showing another example of damage detection results of rust juice, planar free lime and water leakage by the damage detection processing unit and its output processing.

[0161] Fig.21 The example shown is similar to Fig. 20 Compared with the example shown, the priority order of rust juice D, planar free lime C1 and leaking water A is reversed, and the priority order is set to become lower in the order of leaking water A, planar free lime C1 and rust juice D.

[0162] like Fig.21 As shown in (A), when all or part of the rust juice D, the surface free lime C1 and the leaking water A overlap each other, as shown in Fig.21 As shown in (B), the CPU 20 preferentially displays the damaged image of the area of ​​the water leak A with the highest filling priority. In this case, the damaged image of the rust juice D and the planar free lime C1 existing inside the area of ​​the water leak A is not displayed. Fig.21As shown in (C), the CPU 20 preferentially outputs the CAD data of the polygon surrounding the area of ​​the highest priority water leak A in the file format. In this case, the CAD data of the polygon surrounding the rust juice D and the planar free lime C1 existing inside the area of ​​the water leak A are not output.

[0163] In addition, the priority of the damage type is not limited to the above example, and it is preferable to set the priority according to the severity of the damage (more serious damage). For example, in the case of linear damage including linear free lime and cracks as the damage type, the priority of linear free lime is set higher than that of cracks. And, in the case of planar damage including exposed steel bar, peeling, rust, planar free lime and water leakage as the damage type, the priority is set to be lower in the order of exposed steel bar, peeling, rust, planar free lime and water leakage.

[0164] Furthermore, the priority of the damage type can also be appropriately set by the user using the operation unit 18. In this case, the CPU 20 can perform a priority acceptance process of accepting the priority of the damage type of the structure from the operation unit 18 operated by the user, store the accepted priority in the storage unit 16, etc., and read the priority from the storage unit 16 for use as needed.

[0165] [Second Embodiment of Damage Detection Result Output]

[0166] Fig. 22 and Fig.23 Each of them is a schematic diagram of a GUI (Graphical User Interface) according to a second embodiment of the damage detection result output.

[0167] Fig. 22 2 is a diagram showing an example of a screen 40 displayed on the display unit 30 .

[0168] The screen 40 displays a composite image in which a damage image is superimposed on an image of a photographed structure, a check box 42 for selecting a damage type to be displayed, and various icon buttons used in editing, etc. In addition, the damage detection result for each damage type detected by the damage detection processing unit 21 based on the image of the photographed structure can be stored as CAD data representing the layer structure of the damage area for each damage type.

[0169] exist Fig. 22 In the example shown, in the check box 42, the five damage types of water leakage, free lime, rust, peeling and exposed steel bars are all checked (five damage types are selected), so the damage images of the above five damage types are overlapped on the image of the structure displayed on the screen 40.

[0170] Here, the damage image corresponding to the damage type can be generated by filling the damage area with a color corresponding to the damage type based on the CAD data of the layer corresponding to the damage type. The color corresponding to the damage type can be a color pre-set according to the damage type or a color set by the user.

[0171] Fig.23 It is a diagram showing another example of the screen 40 displayed on the display unit 30 .

[0172] exist Fig.23 In the example shown, Fig. 22 Compared to the example shown, the damage image displayed on the screen 40 is different.

[0173] exist Fig.23 In the illustrated screen 40 , three types of damage, namely, water leakage, rust, and exposed reinforcing bars, are checked in check boxes 42 , and therefore damage images of the three types of damage are displayed superimposed on the image of the structure displayed on the screen 40 .

[0174] Therefore, in Fig.23 In the screen 40 shown, Fig. 22 Compared with the picture 40 shown, the difference is that the damaged images corresponding to free lime and peeling are eliminated.

[0175] According to the second embodiment of the damage detection result output, when the user selects one or more damage types desired, a damage image showing damage of the selected damage type can be displayed. In addition, it is preferred that only one or more damage types detected from the image are displayed in the check box 42. Furthermore, the display method of the damage area for each damage type is not limited to Fig. 22 and Fig.23 The embodiment shown.

[0176] [Third Embodiment of Damage Detection Result Output]

[0177] Figure 24 to Figure 26 Each of them is a schematic diagram of a GUI according to a third embodiment of the damage detection result output.

[0178] Fig.24 (A) is a diagram showing an example of a setting screen 44 for performing various settings.

[0179] exist Fig.24 In the setting screen 44 shown in (A) of FIG. 1 , a "label" for setting the color of damage, etc. is selected. By using this setting screen, the user can set the color of the damage, etc. Fig. 22 The color of the damage image corresponds to the damage type shown in the figure.

[0180] And, in Fig.24The setting screen 44 shown in (A) of FIG. 1 includes a slider 45A used to set the transparency of the color (fill color) of the damaged image and a dialog box 45B for displaying the transparency.

[0181] exist Fig.24 In the example shown in (A), the transparency of the fill color of the damaged image is set to "10". In this example, the transparency is "0" when it is opaque, and the transparency is "100" when it is completely transparent.

[0182] Fig.24 (B) is a diagram showing a composite image in which a damaged image with a transparency of "10" is superimposed on an image of a photographed structure.

[0183] can be achieved through Fig.24 After setting the transparency to "10" in the setting screen shown in (A), close the setting screen to display Fig.24 The composite image shown in (B).

[0184] Fig.25 (A) is a diagram showing a setting screen in which the transparency is set to "50". Fig.25 (B) is a diagram showing a composite image in which a damaged image with a transparency of "50" is superimposed on an image of a photographed structure.

[0185] and, Fig.26 (A) is a diagram showing a setting screen in which the transparency is set to "100". Fig.26 (B) is a diagram showing a composite image in which a damaged image with a transparency of "100" is superimposed on an image of a photographed structure.

[0186] In addition, Fig.26 In (B), the transparency of the fill color is "100", so the fill color is completely transparent, but the closed polygon surrounding the damage area of ​​each damage type is displayed with the color set according to the damage type.

[0187] By setting the transparency of the color filled with the damage image and displaying the damage image in this way, the user can visually recognize the image (damage) of the structure covered by the damage image.

[0188] In addition, the second embodiment and the third embodiment of the damage detection result output can be used in combination.

[0189] [EDIT of damage detection results]

[0190] Fig.10The damage detection processing unit 21 shown outputs the damage type and the damage area for each damage type as a damage detection result when the image 13 of the structure is input. However, the damage detection result may be erroneously detected or inaccurately detected.

[0191] For example, the damaged area is classified in pixel units or by taking several pixels as one unit, which sometimes lacks accuracy. Also, cracks detected as two cracks are sometimes preferably connected as one crack. This is because it can be inferred that the cracks are connected inside the concrete.

[0192] Therefore, the CPU 20 performs an editing instruction acceptance process of accepting an editing instruction of the damage detection result by utilizing an operation of the operating unit 18 (eg, a mouse) operated by the user, and performs an editing process of editing the damage detection result according to the accepted editing instruction.

[0193] As an example of editing the damage detection result, in the case of linear damages of the same type and the end points of the broken lines are close to each other, editing to connect the end points can be considered. In this case, the distance between the end points of the broken lines of the same type of linear damage can be measured after the damage detection process, and when the measured distance is less than a threshold, the end points can be automatically connected, or they can be automatically connected according to the user's instruction. The threshold value can use a default value or can be set by the user.

[0194] Furthermore, a threshold value for the length or width of linear damage or a threshold value for the area of ​​planar damage may be set to automatically delete damage detection results that are smaller than the threshold value. The damage detection results may be deleted automatically after damage detection processing or according to user instructions. The threshold value may use a default value or may be set by the user.

[0195] Fig. 27 and Fig.28 In addition, when editing the damage detection result, as shown in the third embodiment of the damage detection result output, it is preferable to set the transparency of the color filled with the damage image to a higher level so that the image of the structure is easily visually recognized.

[0196] Fig. 27 This figure shows a method of adding vertices to a polygon surrounding a damaged area.

[0197] A polygon is a connection of multiple vertices along the damaged area (in Fig. 27 , the vertices represented by quadrilaterals).

[0198] In the case of appending vertices to the polygon, such as Fig. 27As shown in (A), place the mouse cursor on the line of the polygon to which you want to add a vertex, right-click the mouse, and select [Add] from the context menu. Fig. 27 As shown in (B), new vertices can be added to the lines of the polygon.

[0199] Furthermore, by dragging the added vertices and moving them to the edge of the original damaged area, the polygon surrounding the damaged area can be edited.

[0200] Fig.28 A diagram showing a method of deleting vertices from a polygon surrounding a damaged area.

[0201] In the case of deleting vertices from this polygon, such as Fig.28 As shown in (A), place the mouse cursor on the vertex you want to delete, right-click the mouse (to select the vertex), and select [Delete] from the context menu. Fig.28 As shown in (B), it is possible to delete vertices from a polygon.

[0202] like Fig.28 As shown in (B) of FIG. 1 , when a vertex is deleted from a polygon, a line of the polygon is connected between the vertices before and after the deleted vertex, thereby editing a polygon surrounding the damaged area.

[0203] The above editing example describes editing such as adding and deleting vertices in a polygon of planar damage, but editing such as adding and deleting vertices in a polyline of linear damage can also be performed in the same manner.

[0204] In addition, the editing function includes the following functions: the function of selecting a broken line or polygon as a whole by clicking a line connecting vertices, the function of deleting the broken line or polygon as a whole, and the function of manually adding a broken line or polygon to a missed damaged area.

[0205] [Maintenance assistance method]

[0206] Fig.29 It is a flowchart showing an embodiment of the maintenance support method according to the present invention.

[0207] Fig.29 The processing of each step shown is performed, for example, by Fig. 9 The maintenance support device 10 shown is performed by a processor constituted by the CPU 20 or the like.

[0208] exist Fig.29 In step S10, the processor acquires an image of the structure to be inspected from the image acquisition unit 12 or the image database 14 or the like (step S10).

[0209] Damage detection processing unit 21 ( Fig.10) Detect damage to the structure based on the image acquired in step S10 (step S12).

[0210] The processor determines whether damage is detected by the damage detection performed in step S12 (step S14), and when damage is detected (in the case of "yes"), determines whether two or more types of damage are detected (step S16).

[0211] When it is determined in step S16 that two or more types of damage are detected (in the case of "YES"), the processor further determines whether two or more types of damage are detected from the same or close positions among the two or more types of damage (step S18).

[0212] Furthermore, when it is determined in step S18 that two or more types of damage are detected from the same or close positions (in the case of "yes"), the processor outputs the damage detection result in the priority order of the damage types (step S20). The output of the damage detection result is performed, for example, by superimposing the damage image on the image, displaying the damage image alone on the display unit, or outputting CAD data representing the damage map in the form of a file.

[0213] On the other hand, when more than two types of damage are not detected in step S16 (the "No" case), that is, when only one type of damage is detected, or when it is determined in step S18 that more than two types of damage are not detected from the same or close positions (the "No" case), transfer to step S22, in which one or more types of damage detection results are directly output.

[0214] [other]

[0215] The hardware for realizing the maintenance assistance device involved in the present invention can be composed of various processors. Various processors include general-purpose processors that execute programs and function as various processing units, namely CPUs (Central Processing Units), FPGAs (Field Programmable Gate Arrays), and other processors whose circuit structures can be changed after manufacturing, namely programmable logic devices (PLDs), ASICs (Application Specific Integrated Circuits), and other processors that have circuit structures specially designed for performing specific processing, namely dedicated circuits, etc. A processing unit constituting the maintenance assistance device can be composed of one of the above-mentioned various processors, or it can be composed of two or more processors of the same or different types. For example, one processing unit can also be composed of a plurality of FPGAs or a combination of CPUs and FPGAs. In addition, multiple processing units can also be composed of one processor. As an example of multiple processing units composed of one processor, first, there is the following method: as represented by computers such as clients or servers, one processor is composed of a combination of more than one CPU and software, and the processor functions as multiple processing units. Second, there is the following method: as represented by the system on chip (System On Chip; SoC), a processor is used that realizes the functions of the entire system including multiple processing units by one IC (Integrated Circuit) chip. In this way, as a hardware structure, various processing units are composed of one or more of the above-mentioned various processors. Moreover, more specifically, the hardware structure of these various processors is a circuit (circuitry) composed of circuit elements such as semiconductor elements.

[0216] Furthermore, the present invention includes a maintenance support program and a storage medium storing the maintenance support program. The maintenance support program, when installed in a computer, causes the computer to function as the maintenance support device according to the present invention.

[0217] In addition, the present invention is not limited to the above-mentioned embodiment, and various modifications can be made without departing from the spirit of the present invention.

[0218] Explanation of symbols

[0219] 1-bridge, 2-main beam, 3-cross beam, 4-cross connection, 5-cross brace, 6-bridge deck, 7-bridge pier, 10-maintenance auxiliary device, 12-image acquisition unit, 13-image, 14-image database, 16-storage unit, 18-operation unit, 20-CPU, 21-damage detection processing unit, 21A~21C-learning completed model, 22-RAM, 24-ROM, 26-display control unit, 27A~27C-damage Detection results, 30-display unit, 40-screen, 42-check box, 44-setting screen, 45A-slide button, 45B-dialog box, A-water leakage, B, C1~C5-cracks, C1-free lime (surface), C2-free lime (line), D-rust, E-peeling, F-exposed steel bar, H1-peeling, L1~L6-shortest distance, P1~P6-focus points, S10~S22-steps, X, Y-broken lines.

Claims

1. A maintenance assisting device comprising a processor, The processor performs the following processing: Image acquisition processing, acquiring an image of the structure of the inspection object; Damage detection processing, detecting damage to the structure based on the acquired image; A determination process of, when two or more types of damage to the structure are detected by the damage detection process, determining whether the two or more types of damage are detected from the same or close positions among the two or more types of damage; and Output processing outputs the damage detection results detected by the damage detection processing. When the judgment processing determines that more than two types of damage are detected from the same or close positions, the damage detection results are output according to the priority order of the damage types.

2. The maintenance auxiliary device according to claim 1, wherein: In the damage detection process, the damaged area and the damage type of each damaged area are detected based on the image. The determination process determines whether two or more types of damage are detected in the same or adjacent damage areas. In the output processing, when the determination processing determines that two or more types of damage are detected in the same or adjacent damage areas, the damage detection result of the damage type with the highest priority is output as the damage detection result of the same or adjacent damage areas.

3. The maintenance auxiliary device according to claim 1 or 2, wherein: The close position is a position where the distance between the two or more types of damage is equal to or smaller than a threshold value.

4. The maintenance auxiliary device according to claim 1 or 2, wherein: In the damage detection process, when the image is input, a learned model is executed to output a damaged area and a damage type of each damaged area as a recognition result.

5. The maintenance auxiliary device according to claim 1 or 2, wherein: In the output process, different drawing patterns are output when the damage type is linear damage and when the damage type is planar damage.

6. The maintenance auxiliary device according to claim 5, wherein: In the output processing, when the damage type is linear damage, a damage map showing lines that do not close the linear damage is output, and when the damage type is planar damage, a damage map showing closed lines that surround the planar damage is output.

7. The maintenance auxiliary device according to claim 5, wherein: In the output process, when the damage type is linear damage, a damage image at least filling the linear damage is output, and when the damage type is planar damage, a damage image at least filling the planar damage is output.

8. The maintenance auxiliary device according to claim 1 or 2, wherein: The output processing outputs the damage detection result and displays it on a display, or saves the damage detection result in a memory in the form of a file.

9. The maintenance auxiliary device according to claim 1 or 2, wherein: The priority order of the injury types is a priority order preset according to the severity of the injury.

10. The maintenance auxiliary device according to claim 9, wherein: In the case of linear damage including linear free lime and cracks as the damage types, the linear free lime has a higher priority than the cracks.

11. The maintenance auxiliary device according to claim 9, wherein: In the case of surface damage including exposed steel bars, peeling, rust, planar free lime and water leakage as the damage types, the priority is set to be lower in the order of exposed steel bars, peeling, rust, planar free lime and water leakage.

12. The maintenance auxiliary device according to claim 1 or 2, wherein: The processor performs a priority order receiving process of receiving the priority order of the damage types of the structure from the operation unit operated by the user, The priority order of the damage types is a priority order received from the user via the operation unit.

13. The maintenance auxiliary device according to claim 1 or 2, wherein: The processor performs the following processing: An editing instruction acceptance process for accepting an editing instruction of the damage detection result from an operation unit operated by a user; and Editing processing, editing the damage detection results according to the accepted editing instructions.

14. The maintenance auxiliary device according to claim 1 or 2, wherein: The damage detection result includes a damage quantity table having items of damage identification information, damage type, and size, and information corresponding to each item is recorded for each of the detected damages.

15. A maintenance assistance method, wherein a processor performs maintenance assistance on a structure to be maintained, Each process of the processor includes the following steps: Acquire an image of the structure of the inspection object; detecting damage to the structure based on the acquired image; determining whether, among the two or more types of damages detected on the structure, two or more types of damages are detected from the same or close positions; and The detected damage detection result is outputted. When it is determined by the determination step that two or more types of damage are detected from the same or close positions, the damage detection result is outputted in the priority order of the damage types.

16. A recording medium having a maintenance assistance program recorded thereon, the maintenance assistance program causing a computer to execute a method for assisting maintenance of a structure to be maintained, the method comprising the following steps: Acquire an image of the structure of the inspection object; detecting damage to the structure based on the acquired image; determining whether, among the two or more types of damages detected on the structure, two or more types of damages are detected from the same or close positions; and The detected damage detection result is outputted. When it is determined by the determination step that two or more types of damage are detected from the same or close positions, the damage detection result is outputted in the priority order of the damage types.

Images