Structure inspection support device, structure inspection support method, and program

Abstract

To provide a structure inspection support device, a structure inspection support method, and a program with which comment operations can be made efficient and comments such as observations can be standardized.SOLUTION: A structure inspection support device comprises a processor, and the processor executes selection processing of receiving selection of information on a target structure including at least one of a photographed image and damage information of the target structure; preparation processing of preparing a comment on damage to the target structure on the basis of the selected information on the target structure; and display processing of displaying the comment on a display. The preparation processing is to prepare the comment on the damage to the target structure by using machine learning.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a structure inspection support device, a structure inspection support method, and a program. [Background technology]

[0002] Social infrastructure includes structures such as bridges and tunnels. These structures are prone to damage, and because this damage tends to progress, they require regular inspections.

[0003] Inspectors who inspect a structure are required to prepare an inspection report in a prescribed format based on the inspection procedures established by the structure's manager, etc., as a document showing the results of the inspection. By looking at the damage diagram prepared in the prescribed format, even experts other than the inspector who actually inspected the structure can understand the progress of the damage to the structure and formulate a maintenance plan for the structure.

[0004] Similarly, the condition of structures such as apartment buildings and office buildings is inspected periodically, and repairs and maintenance are carried out based on the inspection results. An inspection report is prepared during the inspection. Regarding the preparation of the inspection report, Patent Document 1 discloses a system that can reduce the time required to prepare the inspection report. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2019-082933 Summary of the Invention [Problem to be solved by the invention]

[0006] Inspection reports for structures include comments in the observations section, explaining the classification of countermeasures for damage, the basis for determining soundness, and other factors, as well as the thinking behind those decisions. However, the process is tedious, as it is necessary to create comments while referring to various data. Furthermore, variations in comments can occur due to human factors such as the level of expertise of the person creating them.

[0007] The present invention has been made in consideration of the above circumstances, and aims to provide a structure inspection support device, a structure inspection support method, and a program that can improve the efficiency of commenting work and standardize comments such as findings. [Means for solving the problem]

[0008] A first aspect of the structure inspection support device is a structure inspection support device that includes a processor, and the processor performs a selection process to accept a selection of information related to the target structure, including at least one of a photographed image and damage information of the target structure, a creation process to create a comment regarding the damage to the target structure based on the selected information about the target structure, and a display process to display the comment on a display.

[0009] In the structure inspection support device of the second aspect, the creation process creates at least one comment on the damage to the target structure based on at least one of the captured image and the damage information.

[0010] In the structure inspection support device of the third aspect, the creation process uses machine learning to create comments on damage to the target structure.

[0011] In a fourth aspect of the structure inspection support device, a database is provided that stores information about past target structures, including at least one of photographed images and damage information of the structure, in association with comments on the damage to the structure, and the creation process creates comments on the damage to the target structure regarding similar damage to the target structure, based on the information about the target structure and the information about the structure stored in the database.

[0012] In the structure inspection support device of the fifth aspect, the selection process accepts selection of information about the target structure by selecting a three-dimensional model of the structure associated with information about the target structure.

[0013] In the structure inspection support device of the sixth aspect, the selection process automatically receives a selection of information on a target structure for which a comment is to be created from the target structure.

[0014] In the structure inspection support device of the seventh aspect, the selection process automatically accepts selection of information about the target structure based on at least one of the captured image and the damage information.

[0015] In the structure inspection support device of the eighth aspect, the processor accepts an edit to the comment and executes an edit process to change the comment.

[0016] In the structure inspection support device of the ninth aspect, the editing process accepts a comment candidate selected from a plurality of comment candidates corresponding to the comment as an edit to the comment.

[0017] In the structure inspection support device of the tenth aspect, the processor executes a related information extraction process to extract related information relating to damage to the target structure, and the display process displays the related information on the display.

[0018] The eleventh aspect of the method for supporting inspection of structures includes a selection step of accepting a selection of information about a target structure including at least one of a photographed image and damage information of the target structure, a creation step of creating a comment regarding damage to the target structure based on the selected information about the target structure, and a display step of displaying the comment on a display.

[0019] The 12th aspect of the structure inspection support program is implemented by a computer with a selection function that accepts selection of information about the target structure, including at least one of a photographed image and damage information about the target structure, a creation function that creates comments regarding damage to the target structure based on the selected information about the target structure, and a display function that displays the comments on a display. [Effects of the Invention]

[0020] The structure inspection support device, structure inspection support method, and program of the present invention make it possible to improve the efficiency of commenting work and standardize comments such as findings. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 1 is a block diagram showing an example of a hardware configuration of a structure inspection support device. [Figure 2] FIG. 2 is a block diagram showing the processing functions realized by the CPU. [Figure 3] FIG. 3 is a diagram showing information stored in the storage unit. [Figure 4] FIG. 4 is a flow diagram showing an inspection support method using the structure inspection support device. [Figure 5] FIG. 5 is a diagram showing an example of a captured image. [Figure 6] FIG. 6 is a diagram illustrating an example of damage information. [Figure 7] FIG. 7 is a diagram illustrating an example of a three-dimensional model. [Figure 8] FIG. 8 is a diagram showing how information about a target structure is selected from a three-dimensional model. [Figure 9] FIG. 9 is a block diagram of a creation processing unit that executes processing using a trained model. [Figure 10] FIG. 10 is a block diagram of a creation processing unit that executes the process of extracting similar damage. [Figure 11] FIG. 11 is a diagram showing an example of a method for extracting similar damage. [Figure 12]FIG. 12 is a diagram showing an example of a screen displayed on the display device in the selection step. [Figure 13] FIG. 13 is a diagram showing another example of a screen displayed on the display device in the selection step. [Figure 14] FIG. 14 is a diagram showing an example of a template of inspection record data displayed on a display device. [Figure 15] FIG. 15 is a diagram showing an example of inspection record data into which text data other than comments such as findings has been input, displayed on a display device. [Figure 16] FIG. 16 is a diagram showing an example of inspection record data into which comment text data has been input, displayed on a display device. [Figure 17] FIG. 17 is a diagram illustrating an example of the editing process. DETAILED DESCRIPTION OF THE INVENTION

[0022] Hereinafter, preferred embodiments of the structure inspection support device, structure inspection support method, and program according to the present invention will be described with reference to the accompanying drawings. Here, the term "structure" includes civil engineering structures such as buildings, bridges, tunnels, and dams, as well as buildings, houses, and architectural structures such as walls, columns, and beams of buildings.

[0023] [Hardware configuration of the structural inspection support device] FIG. 1 is a block diagram showing an example of the hardware configuration of a structure inspection support device according to the present invention.

[0024] A computer or a workstation can be used as the structure inspection support device 10 shown in Fig. 1. The structure inspection support device 10 in this example is mainly composed of an input / output interface 12, a storage unit 16, an operation unit 18, a CPU (Central Processing Unit) 20, a RAM (Random Access Memory) 22, a ROM (Read Only Memory) 24, and a display control unit 26. A display device 30 constituting a display is connected to the structure inspection support device 10, and under the command of the CPU 20, an image is displayed on the display device 30 under the control of the display control unit 26. The display device 30 is composed of, for example, a monitor.

[0025] The input / output interface 12 can input various data (information) to the structure inspection support device 10. For example, data to be stored in the storage unit 16 is input via the input / output interface 12.

[0026] The CPU (processor) 20 reads out various programs including the structure inspection support program of the embodiment stored in the storage unit 16 or the ROM 24, expands them in the RAM 22, and performs calculations to control each unit. The CPU 20 also reads out programs stored in the storage unit 16 or the ROM 24, performs calculations using the RAM 22, and performs various processes of the structure inspection support device 10.

[0027] FIG. 2 is a block diagram showing the processing functions realized by the CPU 20. As shown in FIG.

[0028] The CPU 20 includes a selection processing unit 51, a creation processing unit 53, a display processing unit 55, etc. Specific processing functions of each unit will be described later. Since the selection processing unit 51, the creation processing unit 53, and the display processing unit 55 are part of the CPU 20, it can also be said that the CPU 20 executes the processing of each unit.

[0029] Returning to Fig. 1, the storage unit (memory) 16 is a memory configured from a hard disk drive, flash memory, etc. The storage unit 16 stores data and programs for operating the structure inspection support device 10, such as an operating system and a program for executing a structure inspection support method. The storage unit 16 also stores information, etc., used in the present embodiment, which will be described below.

[0030] 3 is a diagram showing information etc. stored in the storage unit 16. The storage unit 16 is composed of non-transitory recording media such as a CD (Compact Disk), a DVD (Digital Versatile Disk), a hard disk, various semiconductor memories, etc., and a control unit thereof.

[0031] The storage unit 16 mainly stores information 101 about the target structure, a three-dimensional model 103, and inspection report data 105.

[0032] The information 101 about the target structure includes at least one of a photographed image of the target structure and damage information. The photographed image is an image of the structure. The damage information includes at least one of the location of damage to the target structure, the type of damage, and the extent of damage. The information 101 about the target structure may also include an image (damage image) showing damage detected from the photographed image of the structure. The damage information may be obtained automatically, such as by image analysis, or manually by the user.

[0033] The information 101 about the target structure may include multiple types of data, such as a panoramic composite image and a two-dimensional drawing. A panoramic composite image is a group of images corresponding to a specific component, synthesized from photographed images. Damage information (damage images) may also be panoramic synthesized.

[0034] The 3D model 103 is, for example, data of a 3D model of a structure created based on a plurality of captured images. The 3D model 103 includes data on the regions and names of components that constitute the structure. Each region and name of a component may be specified on the 3D model 103.

[0035] The component region and component name may be automatically identified for the three-dimensional model 103 from information on the shape, dimensions, etc. of the component. Alternatively, the component region and component name may be identified for the three-dimensional model 103 based on a user operation.

[0036] The information 101 about the target structure and the three-dimensional model 103 may be associated with each other. For example, the information 101 about the target structure is associated with positions and components on the three-dimensional model 103 and stored in the storage unit 16. By specifying a position on the three-dimensional model 103, the information 101 about the target structure may be displayed on the three-dimensional model 103. Furthermore, by specifying the information 101 about the target structure, the three-dimensional model 103 may be displayed together with the information 101 about the target structure.

[0037] The inspection report data 105 is, for example, a template of a two-dimensional inspection report (a document file in a specified format). The template may be in a format specified by the Ministry of Land, Infrastructure, Transport and Tourism or a local government.

[0038] 1 includes a keyboard and a mouse, and a user can use these devices to cause the inspection support device 10 to perform necessary processing. By using a touch panel type device, the display device 30 can function as the operation unit.

[0039] The display device 30 is, for example, a device such as a liquid crystal display, and is capable of displaying three-dimensional model data, information 101 relating to the target structure, inspection report data 105, and comments.

[0040] FIG. 4 is a flow diagram showing a structure inspection support method using the structure inspection support device.

[0041] <Selection Step> The selection processing unit 51 receives a selection of information 101 relating to the target structure (selection step: step S1). As described above, the information 101 relating to the target structure includes at least one of a photographed image and damage information.

[0042] Information 101 about the target structure is acquired from the storage unit 16. If information 101 about the target structure is not stored in the storage unit 16, information 101 about the target structure may be acquired from another storage unit via the input / output interface 12 over a network.

[0043] 5 is a diagram showing an example of a captured image 107 included in information 101 about a target structure. Captured images 107A and 107B are multiple images of the structure. A captured image group 107C is made up of multiple captured images 107A, 107B, etc., captured at multiple locations on the structure. Note that in the specification, they may be simply referred to as captured images 107 as necessary.

[0044] FIG. 6 is a diagram showing an example of damage information 109 included in information 101 related to the target structure. The damage information 109 shown in FIG. 6 includes the component (location of damage), type of damage, damage dimensions, extent of damage, and change over time. The damage information 109 may include at least one of the location of damage to the target structure, the type of damage, and the extent of damage. The damage information 109 may also include damage information other than the damage information 109 shown in FIG. 6, such as the cause of damage.

[0045] Next, a preferred selection process in the selection step (step S1) will be described.

[0046] In the first selection process, the selection processing unit 51 may select the three-dimensional model 103 of the structure associated with the information 101 about the target structure, thereby accepting the selection of the information 101 about the target structure.

[0047] First, a three-dimensional model will be described. Fig. 7 is a diagram showing an example of a three-dimensional model 103. The three-dimensional model 103 can be displayed as a point cloud, a polygon (mesh), a solid model, or the like. The three-dimensional model 103 in Fig. 7 is a diagram in which a photographed image (texture) of a structure is texture-mapped onto a polygonal polygon.

[0048] 7, the three-dimensional model 103 includes component names and component regions. The three-dimensional model 103 is composed of, for example, a deck 131, a pier 133, and an abutment 135. Information 101 about the target structure is correlated with positions and components on the three-dimensional model 103.

[0049] The method for creating the 3D model 103 is not limited. Various models exist, and the 3D model 103 can be created using, for example, the SfM (Structure from Motion) method. SfM is a method for restoring a 3D shape from multi-viewpoint images. For example, feature points are calculated using an algorithm such as SIFT (Scale-Invariant Feature Transform), and the 3D positions of a point cloud are calculated using the feature points as clues based on the principle of triangulation. Specifically, a straight line is drawn from the camera to the feature points using the principle of triangulation, and the intersection of two lines passing through the corresponding feature points becomes the restored 3D point. Then, by performing this process for each detected feature point, the 3D positions of the point cloud can be obtained. The 3D model 103 may be created using the captured image 107 (captured image group 107C) shown in FIG. 5.

[0050] Although size is not calculated in SfM, it is possible to associate it with the actual scale by, for example, placing a scaler with known dimensions on the subject and taking a photograph.

[0051] Next, the first selection process will be described with reference to Fig. 8. As shown in Fig. 8, a three-dimensional model 103 showing an overall bird's-eye view is displayed on the display device 30 (not shown) by the display processing unit 55 (not shown). The user manually selects on the three-dimensional model 103 via the operation unit 18, and the display processing unit 55 displays an enlarged or reduced three-dimensional model 103A, or a three-dimensional model 103A with any of the viewpoint, line of sight, and field of view changed. The overall bird's-eye view and the enlarged or otherwise changed three-dimensional model may be collectively referred to as the three-dimensional model 103.

[0052] When the user specifies a specific position on the three-dimensional model 103A via the operation unit 18, the selection processing unit 51 can accept the selection of the photographed image 107, which is information 101 about the target structure, or the damage information 109. The selection processing unit 51 may also accept the selection of both the photographed image 107 and the damage information 109, which are information 101 about the target structure. The three-dimensional model 103 on which the damage is mapped can be displayed, and the user can select the damage to comment on on the three-dimensional model 103.

[0053] Next, the second selection process will be described. In the second selection process, the selection processing unit 51 may automatically select and accept information 101 relating to the target structure for which a comment is to be created from among the target structures.

[0054] In one aspect of the second selection process, the user selects the target component (bridge) or span (tunnel) via the operation unit 18, and the selection processing unit 51 automatically accepts the selection of information 101 about the target structure for each component.

[0055] As shown in Figure 8, for example, a user can specify a specific component on the three-dimensional model 103, and the selection processing unit 51 can automatically accept the selection of at least one of the captured image 107, which is information 101 about the target structure, or damage information 109, from the component specified on the three-dimensional model 103.

[0056] In addition to the method of specifying a specific component on the 3D model 103, the user may select a component from a component list displayed on the display device. The selection processing unit 51 can automatically accept the selection of at least one of the photographed image 107 or damage information 109, which is information 101 about the target structure, from the component specified from the component list.

[0057] That is, the selection processing unit 51 automatically selects a predetermined number of pieces of information 101 about the target structure corresponding to the target damage for each damage type from among the target members and spans, and accepts the selection. However, if there is no damage in the entire structure, the information 101 about the target structure corresponding to the damage is not selected.

[0058] In another aspect of the second selection process, the selection processing unit 51 automatically selects a predetermined number of pieces of information 101 about the target structure corresponding to the target damage from the entire target structure, and accepts the selection. However, if there is no damage in the entire structure, the information 101 about the target structure corresponding to the damage is not selected.

[0059] Next, a preferred criterion for automatically selecting information 101 about a target structure will be described.

[0060] As a first selection criterion, the selection processing unit 51 may select the most advanced damage from the information 101 about the target structure. For example, the selection processing unit 51 selects the most advanced damage from the results of the damage level (ranks a, b, c, d, and e) in the damage information 109, and accepts the selection of the information 101 about the target structure.

[0061] As a second selection criterion, the selection processing unit 51 may select the damage with the fastest progression rate from the information 101 about the target structure. For example, the selection processing unit 51 selects the damage with the fastest progression rate based on the results of time-dependent changes in the damage information 109, and selects the information 101 about the target structure. The progression rate may be calculated from the change in length / year, width / year, or area / year. The progression rate may also be calculated from the change in damage size per year. For example, it may be calculated as the change in crack length per year, i.e., the change in length / year. Alternatively, it may be calculated as the change in width of damage per year, such as the change in crack width per year, or as the change in width / year from the change in damage area per year. Alternatively, it may be calculated as the change in area / year from the change in damage area per year, or as the change in depth / year from the change in damage depth per year, such as the change in the depth of metal thinning due to corrosion of steel members.

[0062] As a third selection criterion, the selection processing unit 51 may select the damage with the largest size from the information 101 about the target structure. For example, the selection processing unit 51 may select the longest or widest damage, such as a crack or fissure, from the results of the captured image 107 or the damage information 109, or may select the damage with the largest area, such as spalling, water leakage, free lime, or corrosion. The selection processing unit 51 accepts the selection of the information 101 about the target structure regarding the selected damage.

[0063] As a fourth selection criterion, the selection processing unit 51 may select, from the information 101 about the target structure, damage that corresponds to the cause of damage specified by the user from among fatigue, salt damage, carbonation, alkali-aggregate reaction, frost damage, poor construction, excessive external force, etc. in the case of concrete members, and fatigue, salt damage, water leakage, material deterioration, paint film deterioration, poor construction, excessive external force, etc. in the case of steel members. The selection processing unit 51 accepts the selection of the information 101 about the target structure about damage selected from the results of the damage information 109, for example.

[0064] As a fifth selection criterion, the selection processing unit 51 may select damage present in a target location. For example, the Ministry of Land, Infrastructure, Transport and Tourism's "Guidelines for Periodic Inspection of Bridges" (March 2019) lists examples of target locations that require special attention when performing periodic inspections of concrete bridges. The target locations include (1) end supports, (2) intermediate supports, (3) the center of the span, (4) one-quarter of the span, (5) construction joints, (6) segment joints, (7) anchorages, and (8) notches. The selection processing unit 51 may automatically select and receive information 101 about the target structure for damage for each target location.

[0065] For example, the Ministry of Land, Infrastructure, Transport and Tourism's "Road Tunnel Periodic Inspection Guidelines" (March 2019) lists examples of target locations where similar deformations occur depending on the road tunnel construction method, etc. The target locations include (1) lining joints and construction joints, (2) near the top of the lining, and (3) near the middle of the lining span. The selection processing unit 51 may automatically select and receive information 101 about the target structure for damage for each target location.

[0066] Note that if the first to fifth selection criteria alone cannot limit the damage to a predetermined number, the first to fifth selection criteria may be applied in combination. The selection processing unit 51 may, for example, automatically accept the selection of information 101 relating to damaged target structures that satisfy the first and second selection criteria. The selection processing unit 51 may also automatically accept the selection of information 101 relating to damaged target structures that satisfy the first and third selection criteria. Note that although the case where two selection criteria are combined has been described, three or more selection criteria may also be combined.

[0067] As described above, the CPU 20 functions as the selection processing unit 51.

[0068] <Creation steps> As shown in FIG. 4, the creation processing unit 53 creates a comment on the damage to the target structure based on the information 101 about the selected target structure (creation step: step S2).

[0069] Next, a preferable comment creation process in the creation step (step S2) will be described.

[0070] The first comment creation process may involve the creation processing unit 53 using artificial intelligence (AI) to create at least one comment regarding damage to the target structure based on information 101 about the selected target structure.

[0071] As the AI, for example, a trained model using a convolutional neural network (CNN) can be used.

[0072] 9 shows a block diagram of the creation processing unit 53 using a trained model. The creation processing unit 53 using a trained model is configured with a CPU and the like.

[0073] In FIG. 9, a creation processing unit 53 includes a plurality of (three in this example) trained models 53A, 53B, and 53C corresponding to a plurality of types of damage.

[0074] Each of the trained models 53A, 53B, and 53C includes an input layer, an intermediate layer, and an output layer, and each layer has a structure in which multiple "nodes" are connected by "edges."

[0075] Information 101 about the selected target structure is input to the input layer of the CNN. The information 101 about the target structure is a captured image 107 or damage information 109 (for example, at least one of the type, degree, progression of damage, cause of damage, etc.).

[0076] The intermediate layer has multiple sets of convolutional layers and pooling layers, and is the part that extracts features from the captured image 107 or damage information 109 input from the input layer. The convolutional layer filters nearby nodes in the previous layer (performing a convolution operation using a filter) to obtain a "feature map." The pooling layer reduces the feature map output from the convolutional layer to create a new feature map. The "convolutional layer" plays a role in extracting features such as edge extraction from the captured image 107, or in extracting features from the damage information 109, for example, natural language processing.

[0077] The output layer of the CNN is the part that outputs a feature map that indicates the features extracted by the intermediate layer. In this example, the output layers of the trained models 53A, 53B, and 53C output inference results as damage detection results 53D, 53E, and 53F. The damage detection results 53D, 53E, and 53F include at least one comment on the damage derived from each of the trained models 53A, 53B, and 53C.

[0078] For example, trained model 53A is a trained model trained by machine learning to detect damage caused by water leakage, planar free lime, and rust solution, and outputs the damage areas for each of the damage caused by water leakage, planar free lime, and rust solution, as well as the damage type and comments for each damage area, as damage detection result 53D. Trained model 53B is a trained model trained by machine learning to detect damage caused by peeling and exposed rebar, and outputs the damage areas for each of the damage caused by peeling and exposed rebar, as well as the damage type and comments for each damage area, as damage detection result 53E. Trained model 53C is a trained model trained by machine learning to detect damage caused by cracks and linear free lime, and outputs the damage areas for each of the damage caused by cracks and linear free lime, as well as the damage type and comments for each damage area, as damage detection result 53F.

[0079] As described above, the output damage detection results 53D, 53E, and 53F are created as comments on the damage to the target structure. The trained models 53A, 53B, and 53C of the creation processing unit 53 are not limited to the above embodiment. For example, the trained models 53A, 53B, and 53C may be configured to have individual trained models for each damage type, and each trained model may output a damage area and a comment corresponding to the respective damage type as the damage detection result. In this case, the number of trained models is the same as the number of damage types to be inspected. Alternatively, the system may be configured to have one trained model that can handle all damage types, and output a damage area, a damage type, and a comment for each damage area as the damage detection result. Appropriate, i.e., accurate and error-free, comments can be generated based on information 101 about the target structure. While the case where a damage area, a damage type, and a comment for each damage area are output as the damage detection result has been described, it is sufficient if a comment is output.

[0080] The second comment creation process will be described with reference to Figures 10 and 11. In addition to the first creation process described above, the second comment creation process may also involve the creation processing unit 53 creating a comment on damage to the structure related to similar damage to the target structure based on information 101 related to the target structure and information 140 related to past structures stored in the database 60. The database 60 may be stored in the storage unit 16, or may be stored in another storage unit.

[0081] 10, database 60 is a part that stores and manages information 140 about structures that have been inspected in the past and comments 142 about damage to the structures that were made at that time, in association with each other. Information 140 about the structures includes at least one of photographed images and damage information.

[0082] The creation processing unit 53 further includes a similar damage extraction unit 53G.

[0083] The information 101 about the target structure selected in the selection step (step S1) is output to the similar damage extraction unit 53G. Based on the information 101 about the target structure, the similar damage extraction unit 53G extracts information 140 and comments 142 about structures similar to the information 101 about the target structure from information 140 about past structures stored in the database 60 and comments 142 about damage to the structures that were created at that time.

[0084] The similar damage extraction unit 53G may make a similarity determination based on damage information such as the type of damage, the position of the damage, and the extent of the damage (length, width, area, density, depth, etc.) (average value, maximum value, etc.).

[0085] Furthermore, the similar damage extraction unit 53G may determine similarity based on the location and degree of damage change over time in addition to the damage information. In this case, it is preferable that the database 60 stores at least one of images taken at multiple points in time of the same location of the structure, damage information at multiple points in time detected from the images taken at multiple points in time, and information indicating the change over time in the damage information.

[0086] The similar damage extraction process by the similar damage extraction unit 53G detects changes in damage information over time based on information 101 about the target structure at multiple points in time, and the information indicating this change over time can be used as one of the pieces of information when extracting similar damage that is similar to the damage of the target structure from the database 60.

[0087] Furthermore, the similar damage extraction unit 53G may determine similarity by taking into consideration information other than damage information, for example, at least one of structural information, environmental information, history information, and inspection information of the structure.

[0088] The similar damage extraction unit 53G extracts damage areas of similar damage, and damage types and comments for each damage area as similar damage detection results 53H.

[0089] In addition to the information 101 about the target structure, the following other information may be output to the similar damage extraction unit 53G. The similar damage extraction unit 53G may extract information 140 and comments 142 about structures similar to the information 101 about the target structure based on one or more of the following other information, and extract them as the similar damage detection result 53H.

[0090] <Other information> The other information includes at least one of the following: structural information of the structure, environmental information, history information, and inspection information. Structural information: structural type (for bridges: girder bridge, rigid frame bridge, truss bridge, arch bridge, cable-stayed bridge, suspension bridge), component type (for bridges: deck, pier, abutment, girder, etc.), material (steel, reinforced concrete, PC (Prestressed Concrete), etc.) Environmental information: traffic volume (daily, monthly, annual, cumulative, etc.), distance from the sea, climate (average temperature, average humidity, rainfall, snowfall, etc.) Historical information: Construction conditions (temperature during construction, etc.), age (completion date, start date, and age since then), repair history, disaster history (earthquakes, typhoons, floods, etc.) Inspection information: monitoring information (deflection of the structure, vibration amplitude, vibration period, etc.), core extraction test information, non-destructive testing information (ultrasonic, radar, infrared, hammering, etc.) Other information may also include the diagnostic purpose of the target structure, such as determining the degree of damage, determining the classification of countermeasures, determining soundness, estimating the cause of damage, determining whether repairs are necessary, and selecting repair methods.

[0091] FIG. 11 is a diagram showing an example of a method for extracting similar damage by the similar damage extracting unit.

[0092] In FIG. 11, information 140 about structures stored in the database 60 and information 101 about the target structure are plotted in a feature space defined by feature vectors A and B.

[0093] In Figure 11, information 140 about structures stored in database 60 is indicated by an x, and information 101 about the target structure is indicated by a ●. Furthermore, feature vector A indicates the maximum crack width (mm), and feature vector B indicates the number of years since the structure began operation. The feature space based on feature vectors can be a multidimensional space consisting of three or more feature vectors, but for simplicity's sake, Figure 11 illustrates a two-dimensional space consisting of two feature vectors.

[0094] The similar damage extraction unit 53G calculates the distance between the feature vector (first feature vector) of the damage information of the current diagnosis target indicated by a ● mark and the feature vector (second feature vector) of the damage information indicated by an × mark in the feature space shown in Fig. 11, and extracts the damage information indicated by an × mark when this distance is equal to or less than a threshold (within the dotted circle in Fig. 11) as similar damage. This threshold can be optimized by a statistical method.

[0095] The distance may be a distance (Euclidean distance) when the multiple parameters of the first feature vector and the second feature vector are not weighted, or a distance (Mahalanobis distance) when they are weighted. The weights to be assigned to each parameter may be determined by a statistical method such as principal component analysis.

[0096] In addition to the above judgments, additional search conditions can be specified as points or ranges in the feature space. For example, if you specify bridges with a completion date of January 1, 1990 or later and a basic structure of a girder bridge, you can extract similar damage to structures within the specified range.

[0097] In addition to the above, similar damage can be extracted by setting damage information, structural information, environmental information, history information, and inspection information contained in other information as axes of the feature space.

[0098] The method for extracting similar damage may be a method other than the method that judges based on distance in feature space. For example, extraction may be performed using AI (Artificial Intelligence) that judges similarity from images, or AI that judges similarity by combining multiple pieces of information, such as images, damage information, and other information.

[0099] By creating and referencing comments based on similar damage in the past, it is possible to refer to the know-how of experienced engineers and train young engineers.

[0100] <Display steps> As shown in FIG. 4, the display processing unit 55 displays the comment created in the creation step (step S2) on the display device 30 that constitutes the display (display step: step S3).

[0101] Users can make the comment creation work more efficient by using comments created by the structure inspection support device 10. Comments such as findings can be standardized by the structure inspection support device 10 without being affected by human factors. The inspection support program causes the CPU 20 or the like to realize a selection function corresponding to the selection step, a creation function corresponding to the creation step, and a display function corresponding to the display step.

[0102] Next, preferred embodiments of the selection step (step S1), creation step (step S2), and display step (step S3) will be described.

[0103] Fig. 12 is a diagram showing an example of a screen displayed on the display device in the selection step (step S1). As shown in Fig. 12, a three-dimensional model 103, an enlarged three-dimensional model 103A, and information 101 related to the target structure are displayed on one screen on the display device 30. As information 101 related to the target structure, a photographed image 107 and damage information 109 are displayed. It is sufficient if either the photographed image 107 or the damage information 109 is displayed.

[0104] 13 is a diagram showing another example of a screen displayed on the display device 30. As shown in FIG. 13(A), only the three-dimensional model 103 is displayed on the display device 30. As shown in FIG. 13(B), an enlarged three-dimensional model 103A is displayed on the display device 30. As shown in FIG. 13(C), only information 101 about the target structure is displayed on the display device 30. In FIG. 13, a captured image 107 is displayed as information 101 about the target structure.

[0105] As shown in Fig. 13, the display may be switched from a display of a three-dimensional model 103 showing an overall bird's-eye view (Fig. 13(A)), via a display of an enlarged three-dimensional model 103A (Fig. 13(B)), to a display of information 101 relating to the target structure (Fig. 13(C)). Also, the display may be switched from a display of information 101 relating to the target structure (Fig. 13(C)) to a display of an enlarged three-dimensional model 103A (Fig. 13(B)), and then to a display of the three-dimensional model 103 (Fig. 13(A)).

[0106] As shown in FIGS. 12 and 13, the selection processing unit 51 of the inspection support device 10 is executed to select information 101 relating to the target structure (selection step: step S1).

[0107] FIG. 14 is a diagram showing an example of a template of inspection record data 105 displayed on display device 30. The template inspection record data 105 shows a state in which no text data has been input. Text data corresponding to comments such as material names, symbols, component symbols, degree of damage, necessity of repairs, necessity of detailed investigations, causes, and findings is input into inspection record data 105. In the inspection record data 105 of FIG. 14, text data for comments such as findings is input in the rightmost column.

[0108] 15 is a diagram showing an example of inspection record data 105 into which text data other than comments such as findings has been input, displayed on the display device 30. The user manually inputs text data into the relevant sections of the inspection record data 105 other than comments such as findings, based on the information 101 about the target structure. The inspection support device 10 may automatically input text data into the relevant sections of the inspection record data 105 other than comments such as findings, based on the information 101 about the target structure.

[0109] Fig. 16 is a diagram showing an example of inspection record data 105 into which text data of comments such as findings has been input, displayed on the display device 30. The creation processing unit 53 of the inspection support device 10 executes a creation step (step S2), and the display processing unit 55 executes a display step (step S3), whereby the display device 30 displays a column of findings including automatically created comments. Although only three lines are shown in Figs. 15 and 16, the number of lines is not limited to three.

[0110] As shown in Fig. 16, the automatically created comment is displayed in the observations column. The user may modify the content of the created comment via the operation unit 18 (not shown). The CPU 20 accepts edits to the comment and executes an editing process to change the comment.

[0111] To facilitate the editing process, the creation processing unit 53 may create a comment template in accordance with the selected damage, and the display processing unit 55 may display the comment template on the display device 30. The user may reselect an appropriate comment from the comment template.

[0112] 17A and 17B are diagrams showing an example of editing processing. As shown in Fig. 17A, comments created by the inspection support device 10 are displayed in the observations column. The comments are created using a template corresponding to the selected type of damage.

[0113] For example, the comments include information such as "It is estimated that (type of damage) is due to (cause of damage). (Type of damage) has occurred in (location / range), (comments regarding progression) (comments regarding response)." Figure 17(A) displays "It is estimated that (cracks) are due to (fatigue). (Cracks) have occurred in (main girder) and (are progressing rapidly). (A detailed investigation will be carried out.)" The comments in parentheses are automatically created, and the results are entered.

[0114] In the inspection support device 10, as shown in FIG. 17(B), comment candidates may be displayed in a pull-down menu, and another comment candidate may be selected from the comment candidates.

[0115] For example, the candidates in the (Cause of Damage) pull-down menu may include, for concrete components, "fatigue," "salt damage," "neutralization," "alkali-aggregate reaction," "frost damage," "poor construction," and "excessive external force," and for steel components, "fatigue," "salt damage," "water leakage," "material deterioration," "paint deterioration," "poor construction," and "excessive external force."

[0116] Candidates in the (Damage Type) pull-down menu may include, for example, "crack," "deck crack," "water leakage," "free lime," "peeling," "exposed rebar," "crack," "corrosion," and the like.

[0117] The options in the (Position / Range) pull-down menu may include "Main Girder," "Cross Girder," "Pier," "Abutment," "Bearing," etc., as well as "Whole," "End," etc.

[0118] The options for the (Comments on Progression) pull-down menu may include "Progression is fast," "Progression is slow," "No concerns about progression," and the like.

[0119] The options for the (Comments on Response) pull-down menu may include "Conduct a detailed investigation," "Conduct a follow-up investigation," "Conduct repairs," "Conduct countermeasures," and the like.

[0120] The candidates in the pull-down menu are selected appropriately depending on the damage. The user may edit the comments of the candidates in the pull-down menu.

[0121] Comments on similar damage extracted in the similar damage extraction process may be displayed as a template of findings on the display device 30. As in the editing process shown in Fig. 17, comments on similar damage may be edited using a pull-down menu.

[0122] The CPU 20 may execute a related information extraction process for extracting related information relating to damage to the target structure, and the display processing unit 55 of the CPU 20 may display the related information on the display device 30.

[0123] As related information related to the damage to the target structure, inspection data of the selected damage, past inspection data of the selected damage, data of other closely related damage (damage existing in a position close to the selected damage, damage existing on the back side, etc.), and inspection data of similar damage (which may be of other structures) may be displayed on the display device 30. By displaying related information related to the damage to the target structure, the user may edit comments.

[0124] <Other> In the above embodiment, the hardware structure of the processing unit that executes various processes is the following various processors: The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units, a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array), whose circuit configuration can be changed after manufacture, and a dedicated electrical circuit, such as an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing specific processes.

[0125] A single processing unit may be configured with one of these various processors, or may be configured with two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Furthermore, multiple processing units can be configured with a single processor. Examples of multiple processing units configured with a single processor include, first, a configuration in which one processor is configured with a combination of one or more CPUs and software, as typified by client or server computers, and this processor functions as multiple processing units. Second, a configuration in which a processor is used to realize the functions of an entire system including multiple processing units on a single IC (Integrated Circuit) chip, as typified by a System on Chip (SoC). In this way, the various processing units are configured with one or more of the above-mentioned various processors as a hardware structure.

[0126] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit made up of a combination of circuit elements such as semiconductor elements.

[0127] The above-described configurations and functions can be realized by any hardware, software, or a combination of both. For example, the present invention can be applied to a program that causes a computer to execute the above-described processing steps (processing procedures), a computer-readable recording medium (non-transitory recording medium) on which such a program is recorded, or a computer on which such a program can be installed.

[0128] Although examples of the present invention have been described above, it goes without saying that the present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the present invention. [Explanation of symbols]

[0129] 10 Inspection support device 12 Input / Output Interface 16 Memory section 18 Control section 20 CPU 22 RAM 24 ROM 26 Display control unit 30 Display device 51 Selection processing section 53 Creation processing section 53A Pre-trained model 53B Pre-trained model 53C Pre-trained model 53D damage detection results 53E Damage detection results 53F Damage detection results 55 Display processing section 60 databases 101 Information about the target structure 103 3D Model 103A 3D model 105 Inspection report data 107 images 107A Photo 107B Image 107C Images 109 Damage information 131 Floor slab 133 Bridge Pier 135 Abutment 140 Information about structures 142 comments S1, S2, S3 steps

Claims

1. A structure inspection support device including a processor, The processor: a selection process of accepting a selection of information about the target structure including at least one of a photographed image and damage information about the target structure; a creation process for creating a comment on the damage to the target structure based on information about the selected target structure; a display process for displaying the comment on a display; and The creation process is a structure inspection support device that uses machine learning to create the comments regarding damage to the target structure.

2. the processor performs processing to display a three-dimensional model of the target structure; The structure inspection support device according to claim 1 , wherein the selection process accepts a selection of a specific position of the three-dimensional model designated by a user as information about the target structure.

3. A structure inspection support device including a processor, The processor: a selection process of accepting a selection of information about the target structure including at least one of a photographed image and damage information about the target structure; a creation process for creating a comment on the damage to the target structure based on information about the selected target structure; a display process for displaying the comment on a display; a process of displaying a three-dimensional model of the target structure; and The selection process receives a selection of a specific position of the three-dimensional model designated by the user as information about the target structure, in the structure inspection support device.

4. The structure inspection support device according to claim 1 , wherein the selection process accepts a selection of damage that is most advanced.

5. A structure inspection support device including a processor, The processor: a selection process of accepting a selection of information about the target structure including at least one of a photographed image and damage information about the target structure; a creation process for creating a comment on the damage to the target structure based on information about the selected target structure; a display process for displaying the comment on a display; and The selection process receives the selection of damage with the most advanced degree of damage.

6. The structure inspection support device according to claim 4 or 5, wherein the selection process selects damage that is the most severe.

7. The structure inspection support device according to claim 4 or 5, wherein the selection process selects damage that progresses at the fastest rate.

8. The structure inspection support device according to claim 4 or 5, wherein the selection process selects the damage with the largest size.

9. The structure inspection support device according to claim 1 , wherein the selection process selects a damage cause determined for each material.

10. A structure inspection support device including a processor, The processor: a selection process of accepting a selection of information about the target structure including at least one of a photographed image and damage information about the target structure; a creation process for creating a comment on the damage to the target structure based on information about the selected target structure; a display process for displaying the comment on a display; and The selection process is a structure inspection support device that selects a damage cause determined for each material.

11. The structure inspection support device according to claim 1 , wherein the display process displays at least one of a comment regarding progress and a comment regarding a response as the comment.

12. A structure inspection support device including a processor, The processor: a selection process of accepting a selection of information about the target structure including at least one of a photographed image and damage information about the target structure; a creation process for creating a comment on the damage to the target structure based on information about the selected target structure; a display process for displaying the comment on a display; and The display process displays at least one of a comment regarding progress and a comment regarding response as the comment, in this structure inspection support device.

13. The structure inspection support device according to claim 11 or 12, wherein the display process displays a plurality of comment candidates as comments regarding progress and comments regarding responses.

14. The structure inspection support device according to claim 13 , wherein the plurality of comment candidates for the comment regarding progress and the comment regarding response are displayed as a pull-down menu.

15. The structure inspection support device according to claim 2 , wherein the creation process creates the comment on the damage to the target structure using machine learning.

16. The structure inspection support device according to claim 1 , wherein the creation process creates at least one comment regarding damage to the target structure based on at least one of the captured image and the damage information.

17. a database that stores information about a target structure in the past, including at least one of photographed images of the structure and damage information, and comments on damage to the structure, in association with each other; A structure inspection support device described in any one of claims 1 to 16, wherein the creation process creates comments on damage to the structure regarding similar damage similar to the damage to the target structure based on information about the target structure and information about the structure stored in the database.

18. 18. A structure inspection support device according to claim 1, wherein the selection process accepts a selection of information about the target structure by selecting a three-dimensional model of the structure associated with information about the target structure.

19. The structure inspection support device according to claim 1 , wherein the selection process automatically selects and accepts information about the target structure for which the comment is to be created from the target structure.

20. 20. The structure inspection support device according to claim 19, wherein the selection process automatically accepts a selection of information about the target structure based on at least one of the photographed image and the damage information.

21. The structure inspection support device according to claim 1 , wherein the processor accepts an edit to the comment and executes an edit process to change the comment.

22. The structure inspection support device according to claim 21 , wherein the editing process accepts a comment candidate selected from a plurality of comment candidates corresponding to the comment as an edit to the comment.

23. the processor executes a related information extraction process to extract related information related to damage to the target structure; The structure inspection support device according to claim 1 , wherein the display processing displays the related information on a display.

24. a selection step of accepting a selection of information about the target structure including at least one of a photographed image and damage information about the target structure; a creating step of creating a comment on the damage to the target structure based on information about the selected target structure; a display step of displaying the comment on a display; by a processor, A structure inspection support method in which the creation step creates the comments on damage to the target structure using machine learning.

25. a selection step of accepting a selection of information about the target structure including at least one of a photographed image and damage information about the target structure; a creating step of creating a comment on the damage to the target structure based on information about the selected target structure; a display step of displaying the comment on a display; displaying a three-dimensional model of the target structure; by a processor, The method for supporting inspection of a structure, wherein the selection step accepts a selection of a specific position of the three-dimensional model designated by a user as information about the target structure.

26. a selection step of accepting a selection of information about the target structure including at least one of a photographed image and damage information about the target structure; a creating step of creating a comment on the damage to the target structure based on information about the selected target structure; a display step of displaying the comment on a display; by a processor, The method for supporting inspection of a structure, wherein the selection step receives a selection of damage that is most advanced.

27. a selection step of accepting a selection of information about the target structure including at least one of a photographed image and damage information about the target structure; a creating step of creating a comment on the damage to the target structure based on information about the selected target structure; a display step of displaying the comment on a display; by a processor, The method for supporting inspection of a structure includes selecting a damage cause determined for each material in the selection step.

28. a selection step of accepting a selection of information about the target structure including at least one of a photographed image and damage information about the target structure; a creating step of creating a comment on the damage to the target structure based on information about the selected target structure; a display step of displaying the comment on a display; by a processor, The display step displays at least one of a comment regarding progress and a comment regarding a response as the comment, in accordance with a structure inspection support method.

29. a selection function for receiving a selection of information about the target structure including at least one of a photographed image and damage information about the target structure; a creation function for creating a comment on the damage to the target structure based on information about the selected target structure; a display function for displaying the comment on a display; A structure inspection support program that realizes the above by a computer, The creation function is a structure inspection support program that uses machine learning to create the comments regarding damage to the target structure.

30. a selection function for receiving a selection of information about the target structure including at least one of a photographed image and damage information about the target structure; a creation function for creating a comment on the damage to the target structure based on information about the selected target structure; a display function for displaying the comment on a display; a function of displaying a three-dimensional model of the target structure; A structure inspection support program that realizes the above by a computer, The selection function is a structure inspection support program that accepts the selection of a specific position on the three-dimensional model designated by the user as information about the target structure.

31. a selection function for receiving a selection of information about the target structure including at least one of a photographed image and damage information about the target structure; a creation function for creating a comment on the damage to the target structure based on information about the selected target structure; a display function for displaying the comment on a display; A structure inspection support program that realizes the above by a computer, The selection function is a structural inspection support program that accepts the selection of damage with the most advanced degree of damage.

32. a selection function for receiving a selection of information about the target structure including at least one of a photographed image and damage information about the target structure; a creation function for creating a comment on the damage to the target structure based on information about the selected target structure; a display function for displaying the comment on a display; A structure inspection support program that realizes the above by a computer, The selection function is a structural inspection support program that selects the cause of damage determined for each material.

33. a selection function for receiving a selection of information about the target structure including at least one of a photographed image and damage information about the target structure; a creation function for creating a comment on the damage to the target structure based on information about the selected target structure; a display function for displaying the comment on a display; A structure inspection support program that realizes the above by a computer, The display function displays at least one of comments regarding progress and comments regarding response as the comments, in this structure inspection support program.

Images