Inspection Support Device, Inspection Support Method, and Program

Abstract

To provide an inspection support method and a program which associate a photographed image with damage identification information to automatically reorganize them.SOLUTION: An inspection support device comprises a processor which organizes images including damage to a structure. The processor is configured to: acquire image data of information including a structural drawing of an object structure on a medium, and damage identification information and photographed image identification information about the damage added by a user on the medium; recognize the damage identification information by image recognition from the acquired image data; recognize the photographed image identification information by the image recognition from the acquired image data; associate the damage identification information with the photographed image identification information with respect to the same damage; acquire a photographed image corresponding to the photographed image identification information; and associate the damage identification information with the photographed image.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an inspection support device for a structure, an inspection support method, and a program.

Background Art

[0002] As social infrastructure, there are structures such as bridges and tunnels. Since damage occurs to these structures and the damage has a progressive nature, it is required to conduct inspections regularly. An inspector who inspects a structure creates an inspection report in a predetermined format as a form indicating the results of the inspection.

[0003] Patent Document 1 discloses performing an operation of associating a file number of image data corresponding to a specific position on CAD drawing data based on information on a photographing position entered in a field book during work at the site.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] By the way, at the inspection site of a structure, for example, a printed matter (so-called field book) including a structural drawing of the target structure is prepared based on a design drawing or CAD data, and damage identification information of the structure (information capable of classifying damage attribute information (damage type, damage size, damage degree, etc.)) and photographing image identification information (file name, etc.) are recorded in a handwritten memo on the printed matter of the drawing. After the inspection, the damage and the corresponding photos are sorted out at the office, but there is a problem that sorting out a large number of photos while looking at the field book is a cumbersome task.

[0006] The present invention has been made in view of such circumstances, and an object thereof is to provide an inspection support device, an inspection support method, and a program that can automatically organize a photographed image taken at an inspection site in association with damage identification information.

Means for Solving the Problems

[0007] In an inspection support device including a processor that organizes an image including damage to a structure according to a first aspect, the processor acquires image data of information including a structure diagram of a target structure on a medium, damage identification information regarding damage added by a user on the medium, and photographed image identification information, recognizes the damage identification information by image recognition from the acquired image data, recognizes the photographed image identification information by image recognition from the acquired image data, associates the damage identification information and the photographed image identification information for the same damage, acquires the photographed image corresponding to the photographed image identification information, and associates the damage identification information and the photographed image.

[0008] In a second aspect, the processor recognizes damage identification information regarding an attribute of the damage.

[0009] In a third aspect, the information includes auxiliary information that specifies the photographed image identification information, and the processor specifies the position of the photographed image identification information based on the auxiliary information in recognizing the photographed image identification information.

[0010] In a fourth aspect, the processor associates the damage identification information and the photographed image identification information based on a predetermined condition.

[0011] In a fifth aspect, the processor records the associated damage identification information and photographed image in a database.

[0012] In a sixth aspect, the processor creates a photo ledger from the associated damage identification information and photographed image.

[0013] In a seventh aspect, the processor recognizes member identification information of the target structure and associates the member identification information, the damage identification information, and the photographed image.

[0014] In the eighth aspect, the processor recognizes additional damage attribute information from the image data and associates the damage identification information with the additional damage attribute information.

[0015] In the ninth aspect, the processor recognizes the characters included in the additional damage attribute information by character recognition.

[0016] In the tenth aspect, the processor obtains complementary information for complementing the additional damage attribute information from the captured image.

[0017] In the eleventh aspect, the processor receives any one or more edits to the damage identification information, the captured image identification information, the additional damage attribute information, and the complementary information, and allows any one or more edits to the received damage identification information, the captured image identification information, the additional damage attribute information, and the complementary information.

[0018] In the twelfth aspect, the damage identification information and the additional damage attribute information are associated based on predetermined conditions.

[0019] In the thirteenth aspect, the processor obtains drawing data corresponding to the structural diagram, aligns the structural diagram of the image data with the drawing data, and excludes information regarding the structural diagram based on the drawing data in the association between the damage identification information and the captured image identification information, or in the association between the damage identification information and the additional damage attribute information.

[0020] In the inspection support method for assisting in organizing images including damage to a structure by the processor according to the 14th aspect, the processor includes: a step of acquiring image data of information including a structural diagram of a target structure on a medium, damage identification information regarding damage added by a user, and photographing image identification information; a step of recognizing the damage identification information by image recognition from the acquired image data; a step of recognizing the photographing image identification information by image recognition from the acquired image data; a step of associating the damage identification information and the photographing image identification information for the same damage; a step of acquiring a photographing image corresponding to the photographing image identification information; and a step of associating the damage identification information and the photographing image.

[0021] In the program for executing the inspection support method for assisting in organizing images including damage to a structure by the processor according to the 15th aspect, the processor causes to execute: a step of acquiring image data of information including a structural diagram of a target structure on a medium, damage identification information regarding damage added by a user on the medium, and photographing image identification information; a step of recognizing the damage identification information by image recognition from the acquired image data; a step of recognizing the photographing image identification information by image recognition from the acquired image data; a step of associating the damage identification information and the photographing image identification information for the same damage; a step of acquiring a photographing image corresponding to the photographing image identification information; and a step of associating the damage identification information and the photographing image.

Advantages of the Invention

[0022] According to the inspection support device, inspection support method, and program of the present invention, they are automatically organized in association with the photographed image and the damage identification information.

Brief Description of the Drawings

[0023]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Best Mode for Carrying Out the Invention

[0024] Hereinafter, preferred embodiments of a structure inspection support device, a structure inspection support method, and a program according to the present invention will be described with reference to the accompanying drawings. Here, the "structure" includes civil engineering structures such as buildings, for example, bridges, tunnels, dams, etc., and also includes buildings such as buildings, houses, building walls, columns, beams, etc.

[0025] [Hardware Configuration of Structure Inspection Support Device] FIG. 1 is a block diagram showing an example of the hardware configuration of a structure inspection support device according to the present invention.

[0026] As the structure inspection support device 10 shown in FIG. 1, a computer or a workstation can be used. The structure inspection support device 10 in this example mainly includes an input / output interface 12, a storage unit 16, an operation unit 18, a CPU (Central Processing Unit) 20, a RAM (Random Access Memory) 22, a ROM (Read Only Memory) 24, and a display control unit 26. A display device 30 constituting a display is connected to the structure inspection support device 10, and display is performed on the display device 30 under the control of the display control unit 26 according to the instructions of the CPU 20. The display device 30 is constituted by, for example, a monitor.

[0027] The input / output interface 12 can input various data (information) to the structure inspection support device 10. For example, the data stored in the storage unit 16 is input via the input / output interface 12.

[0028] The CPU (processor) 20 reads various programs including the inspection support program for the structure of the embodiment stored in the storage unit 16 or the ROM 24, etc., expands them in the RAM 22, and performs calculations to comprehensively control each unit. Further, the CPU 20 reads the programs stored in the storage unit 16 or the ROM 24, performs calculations using the RAM 22, and performs various processes of the structure inspection support device 10.

[0029] FIG. 2 is a block diagram showing the processing functions realized by the CPU 20.

[0030] The CPU 20 includes an image data acquisition unit 51, a damage identification information recognition unit 52, a photographed image identification information recognition unit 53, a first association unit 54, a photographed image acquisition unit 55, a second association unit 56, etc. The specific processing functions of each unit will be described later. The image data acquisition unit 51, the damage identification information recognition unit 52, the photographed image identification information recognition unit 53, the first association unit 54, the photographed image acquisition unit 55, and the second association unit 56 are part of the CPU 20, and the CPU 20 executes the processing of each unit.

[0031] Returning to FIG. 1, the storage unit (memory) 16 is a memory composed of a hard disk device, a flash memory, etc. The storage unit 16 stores data and programs for operating the structure inspection support device 10, such as an operating system and a program for executing the structure inspection support method. Further, the storage unit 16 stores information, etc. used in the embodiment described below.

[0032] FIG. 3 is a diagram showing the information, etc. stored in the storage unit 16. The storage unit 16 is composed of a non-temporary recording medium such as a CD (Compact Disk), a DVD (Digital Versatile Disk), a hard disk, various semiconductor memories, etc., and its control unit.

[0033] The storage unit 16 mainly stores the drawing data 101 of the target structure, the photographed image 103 of the target structure, the association table 105, etc.

[0034] The drawing data 101 of the target structure is data (drawings, CAD data) including the shape, dimensions, and members of the target structure at the time of design. The drawing data 101 is stored in association with drawing data identification information (for example, file name) that can identify the drawing data 101.

[0035] The captured image 103 of the target structure is data of an image obtained by photographing the current target structure at the inspection site. The captured image 103 is stored in association with captured image identification information (for example, file name) that can identify the captured image 103.

[0036] The association table 105 constitutes a database that stores by associating damage identification information, which will be described later, with captured image identification information.

[0037] Returning to FIG. 1, the operation unit 18 includes a keyboard and a mouse, and the user can cause the inspection support device 10 to perform necessary processing via these devices. By using a touch panel type device, the display device 30 also functions as an operation unit.

[0038] The display device 30 is a device such as a liquid crystal display, and can display various information.

[0039] Next, a flow of an inspection support method using the inspection support device 10 will be described. FIG. 4 is a flowchart showing the inspection support method using the inspection support device 10.

[0040] As shown in FIG. 4, the inspection support method includes a step of acquiring image data including a structural diagram, damage identification information, and captured image identification information (step S1), a step of recognizing damage identification information from the image data (step S2), a step of recognizing captured image identification information from the image data (step S3), a step of associating damage identification information and captured image identification information for the same damage (step S4), a step of acquiring a captured image corresponding to the captured image identification information (step S5), and a step of associating the damage identification information and the captured image identification information (step S6).

[0041] First, the inspection at the inspection site by the user will be described, and then the medium including the structure diagram and the information including damage identification information that are the basis of the image data will be described.

[0042] Before the user conducts an inspection at the inspection site, the user prepares a medium including the damage diagram of the object to be inspected. As shown in FIG. 5, the user U inputs an operation command to the inspection support device 10 by operating the operation unit 18. Here, the user U inputs the conditions for specifying the target structure from the operation unit 18, and extracts the drawing data 101 of the current inspection target from the storage unit 16 (see FIG. 3). The drawing data 101 of the inspection target is displayed on the display device 30. The user U checks whether the displayed drawing data 101 corresponds to the target structure. When the drawing data 101 corresponds to the target structure, the user U inputs an output instruction from the operation unit 18. The printing device 32 prints the drawing data 101 as the structure diagram 42 of the target structure on the paper medium 41. Thereby, the ledger 40 with the structure diagram 42 printed on the paper medium 41 is created. The name of the bridge and the member name are described in the ledger 40.

[0043] FIG. 6 is a diagram for explaining the case where the user U inspects a bridge at the inspection site. A bridge, which is an example of the structure 70 to which the present invention is applied, the bridge 71 has a main girder 72, a cross girder 73, an inclined strut 74, and a lateral structure 75. On the upper part of the main girder 72, a floor slab 76, which is a concrete member, is provided. The main girder 72 is a member that spans between abutments or piers and supports the load on the floor slab 76. The cross girder 73 is a member that connects the main girders 72 in order to support the load by a plurality of main girders 72. The inclined strut 74 and the lateral structure 75 are members that connect the main girders 72, particularly to resist lateral loads.

[0044] As shown in 0600A, at the inspection site, while carrying the field notebook 40 and the pen 43, the user U visually inspects the appearance (appearance properties) of the structure 70 to be inspected, which is a bridge. The user U adds, by hand with the pen 43, damage identification information 60 (see FIGS. 7 to 12) regarding the damage 77 of the structure 70 obtained by visual inspection to the paper medium 41 of the field notebook 40. A structure diagram 42 is printed on the paper medium 41. The user U photographs the damage 77 with the digital camera 45 and obtains a photographed image 103 (not shown) as objective data. In order to indicate the positional relationship of a plurality of photographed images 103 with respect to the structure 70, the user U adds, by hand, photographed image identification information (file name) corresponding to the photographed image 103 to the paper medium 41 of the field notebook 40 (see FIGS. 7 to 12).

[0045] In 0600B, instead of the field notebook 40 and the pen 43 on the paper medium 41, the user U holds the field notebook 40A on the electronic medium 46 such as a tablet and the electronic pen 47, and visually inspects the state of the bridge, which is the structure 70 to be inspected. The user U adds, by hand with the electronic pen 47, damage identification information 60 regarding the damage 77 of the structure 70 obtained by visual inspection to the electronic medium 46 of the field notebook 40A. Also, the user U photographs the damage 77 with the digital camera 45 and obtains a photographed image 103 (not shown) as objective data. In order to indicate the positional relationship of a plurality of photographed images 103 with respect to the structure 70, the user U adds, by hand, photographed image identification information (file name) corresponding to the photographed image 103 to the electronic medium 46 of the field notebook 40A.

[0046] FIG. 7 shows an example of a paper medium 41 (field notebook 40) on which the structure diagram 42 of the floor slab A is printed, and the paper medium 41, so-called inspection memo 48, to which information including damage identification information 60 is added by hand. Note that FIG. 7 includes a plurality of description formats.

[0047] As shown in FIG. 7, damage identification information 60 is added by the user on the paper medium 41 on which the structural diagram 42 of the floor slab A is printed. On the paper medium 41 of the field book 40, imaging identification information 61 at the time of photographing damage 77 as necessary is added together with the damage identification information 60. Further, it includes association information 62 and 63 indicating the relevance between the imaging identification information 61 and the damage identification information 60, auxiliary information 64 (see FIG. 9), damage additional attribute information 65, and the like. Handwritten damage identification information 60 and imaging identification information 61 are added to the field book 40 to create an inspection memo 48.

[0048] FIGS. 8 to 12 are diagrams showing individual description examples of the inspection memo.

[0049] 0800A in FIG. 8 is the first example, and 0800B shows the second example.

[0050] The first example of 0800A is shown in the inspection memo 48 including damage identification information 60 and imaging identification information 61 added by the user regarding the damage 77 (see FIG. 6) of the structure 70. The damage identification information 60 illustrates damages such as cracks, water leakage, and free lime by diagrams, line types, or regions. The damage identification information 60 includes damage identification information 60A, 60B, 60C, 60D, 60E, and 60F. The damage identification information 60A, 60B, 60C, 60D, and 60E show the state of the damage 77 related to cracks by diagrams. The damage identification information 60A is drawn with an orange line and indicates a crack with a width of 0.2 mm or more and less than 0.3 mm. The damage identification information 60B, 60D, and 60E are drawn with a blue line and indicate cracks with a width of less than 0.2 mm. The damage identification information 60C is drawn with a red line and indicates a crack with a width of 0.3 mm or more. The damage identification information 60F shows the area-based damage 77 by a closed curve of a black dotted line. Area-based damages include, for example, peeling, exposed reinforcement, free lime, water leakage, and floating, and others not belonging to these.

[0051] The captured image identification information 61 is the file name of the captured image 103 that captured the damage 77 or a part of the file name (hereinafter referred to as "file name"), and is composed of characters such as numbers, letters, or symbols, or a combination thereof. The captured image identification information 61 includes captured image identification information 61A, 61B, 61C, 61D, 61E, and 61F. For example, the captured image identification information 61A includes the character string "1510" and indicates a part of the file name "DSCF1510.jpg". Similarly, the captured image identification information 61B, 61C, 61D, 61E, and 61F include "1511", "1512", "1513", "1514", and "1515", respectively. As long as the captured image identification information 61A, 61B, 61C, 61D, 61E, and 61F can be identified, the length of the file name included in the captured image identification information 61 can be freely determined. When the captured image identification information 61 includes a file name, the captured image 103 is extracted from the storage unit 16 based on the captured image identification information 61.

[0052] Also, instead of the file name, the captured image identification information 61 can include the shooting order. The captured image identification information 61A and 61B can include "1001" and "1002", respectively, for example. The shooting order is composed of characters such as numbers, letters, or a combination thereof, and is preferably configured according to some rules so that the order can be specified. Furthermore, "1001" of the shooting order is associated with "1510" of the file name. Therefore, the file name is specified from the shooting order included in the captured image identification information 61, and the captured image 103 is extracted from the storage unit 16 based on the file name.

[0053] The second example of 0800B includes damage identification information 60 and captured image identification information 61, similar to the first example of 0800A. On the other hand, the second example of 0800B further includes association information 62 that associates the damage identification information 60 and the captured image identification information 61.

[0054] The association information 62 is a diagram that connects the corresponding damage identification information 60 and the photographed image identification information 61. The association information 62 includes association information 62A, association information 62B, association information 62C, association information 62D, association information 62E, and association information 62F. For example, the association information 62A associates the damage identification information 60A and the photographed image identification information 61A. The association information 62A is composed of a straight line located below the photographed image identification information 61A and a straight line that extends obliquely downward from the right end of this straight line and reaches the damage identification information 60A. Similarly, the association information 62B associates the damage identification information 60B and the photographed image identification information 61B, the association information 62C associates the damage identification information 60C and the photographed image identification information 61C, the association information 62D associates the damage identification information 60D and the photographed image identification information 61D, the association information 62E associates the damage identification information 60E and the photographed image identification information 61E, and the association information 62F associates the damage identification information 60F and the photographed image identification information 61F. Note that as long as the association information 62 can associate the damage identification information 60 and the photographed image identification information 61, the shape and type of the line are not limited.

[0055] 0900A in FIG. 9 is the third example, and 0900B shows the fourth example.

[0056] The third example of 0900A, like the second example of 0800B, includes the damage identification information 60 and the photographed image identification information 61. On the other hand, it includes association information 63 different from the association information 62 of the second example of 0800B. The association information 63 of the third example is a closed diagram that surrounds the corresponding damage identification information 60 and the photographed image identification information 61. The association information 63 does not necessarily need to surround all of the damage identification information 60 and the photographed image identification information 61, as long as the damage identification information 60 and the photographed image identification information 61 corresponding to the association information 63 are included.

[0057] The association information 63A associates the damage identification information 60A with the captured image identification information 61A. Similarly, the association information 63B associates the damage identification information 60B with the captured image identification information 61B, the association information 63C associates the damage identification information 60C with the captured image identification information 61C, the association information 63D associates the damage identification information 60D with the captured image identification information 61D, the association information 63E associates the damage identification information 60E with the captured image identification information 61E, and the association information 63F associates the damage identification information 60F with the captured image identification information 61F. Note that as long as the association information 63 can associate the damage identification information 60 with the captured image identification information 61, the shape and type of the line are not limited.

[0058] The fourth example of 0900B includes the damage identification information 60, the captured image identification information 61, and the association information 62, similar to the second example of 0800B. On the other hand, the fourth example of 0900B further includes auxiliary information 64 for specifying the position of the captured image identification information 61. The auxiliary information 64 surrounds the captured image identification information 61 with a predetermined line. The auxiliary information 64A surrounds the captured image identification information 61A. Similarly, the auxiliary information 64B surrounds the captured image identification information 61B, the auxiliary information 64C surrounds the captured image identification information 61C, the auxiliary information 64D surrounds the captured image identification information 61D, the auxiliary information 64E surrounds the captured image identification information 61E, and the auxiliary information 64F surrounds the captured image identification information 61F. Note that as long as the auxiliary information 64 can surround the captured image identification information 61, the shape and type of the line are not limited.

[0059] 1000A in FIG. 10 is the fifth example, and 1000B shows the sixth example.

[0060] The fifth example of 1000A includes the damage identification information 60, the captured image identification information 61, and the association information 62.

[0061] The damage identification information 60 includes damage identification information 60A, 60B, 60C, 60D, 60E, and 60F. The damage identification information 60A, 60B, and 60C indicate the state of the crack damage 77, and the damage identification information 60D, 60E, and 60F indicate the state of the damage 77 related to the area system. The damage identification information 60A is drawn in an orange line, the damage identification information 60B is drawn in a blue line, and the damage identification information 60C is drawn in a red line. The photographed image identification information 61 includes photographed image identification information 61A, 61B, 61C, 61D, 61E, and 61F. Also, the association information 62 includes association information 62A, association information 62B, association information 62C, association information 62D, association information 62E, and association information 62F.

[0062] The fifth example of 1000A further includes damage additional attribute information 65. The damage additional attribute information 65 is composed of characters including numbers and symbols indicating the types of area-based damage. Examples of the characters of the damage types include "ha": peeling, "yu": free lime, "ro": water leakage, and "u": floating. The damage additional attribute information 65A includes "u", the damage additional attribute information 65B includes "yu", and the damage additional attribute information 65C includes "ha". The damage additional attribute information 65A is associated with the damage identification information 60D by the association information 62D together with the photographed image identification information 61D. Similarly, the damage additional attribute information 65B is associated with the damage identification information 60E by the association information 62E together with the photographed image identification information 61E, and the damage additional attribute information 65C is associated with the damage identification information 60F by the association information 62F together with the photographed image identification information 61F.

[0063] The sixth example of 1000B, similar to the fifth example of 1000A, includes damage identification information 60, photographed image identification information 61, and association information 62. On the other hand, in the sixth example of 1000B, in addition to the damage additional attribute information 65 of the fifth example of 1000A, information corresponding to the damage additional attribute information 65 is included as a sketch pattern indicating the types of area-based damage in the damage identification information 60D, 60E, and 60F.

[0064] FIG. 11 is an explanatory diagram of a sketch pattern showing the types of area-based damage included in the damage identification information 60. As shown in FIG. 11, the damage type and the pattern are associated with each other. The damage types include six types: "peeling", "exposed reinforcement", "free lime", "leakage", "others", and "blistering", and different patterns correspond to each of the six damage types.

[0065] The sixth example of 1000B in FIG. 10, the damage identification information 60D, 60E, and 60F indicating the state of area-based damage, includes a pattern (see FIG. 11). The damage identification information 60D includes a pattern corresponding to "blistering", the damage identification information 60E includes a pattern corresponding to "free lime", and the damage identification information 60F includes a pattern corresponding to "peeling".

[0066] The fifth example of 1000A shows the damage additional attribute information 65 including characters, and the sixth example of 1000B shows an example including the damage additional attribute information 65 and the pattern of the damage identification information 60 for one area-based damage. However, it is not limited to this. The sixth example of 1000B in FIG. 10 may not include the damage additional attribute information 65.

[0067] 1200A in FIG. 12 is the seventh example, and 1200B shows the eighth example.

[0068] The seventh example of 1200A, similar to the second example of 0800B, includes the damage identification information 60 and the association information 62. The damage identification information 60A, 60B, 60C, 60D, and 60E indicate the state of the crack damage 77, and the damage identification information 60F indicates the state of the area-based damage 77.

[0069] On the other hand, different from the second example of 0800B, in the seventh example of 1200A, two imaging image identification information 61 corresponds to one damage identification information 60. The two imaging image identification information 61 respectively correspond to different imaging images 103. For example, one imaging image 103 is a close-up image where the state of the damage 77 can be seen, and the other imaging image 103 is a distant view image where the position of the damage 77 can be seen.

[0070] As shown in 1200A, in the damage identification information 60A, the captured image identification information 61A and 61B are associated by the association information 62A. Similarly, in the damage identification information 60B, the captured image identification information 61C and 61D are associated by the association information 62B. In the damage identification information 60C, the captured image identification information 61E and 61F are associated by the association information 62C. In the damage identification information 60D, the captured image identification information 61G and 61H are associated by the association information 62D. In the damage identification information 60E, the captured image identification information 61I and 61J are associated by the association information 62E. In the damage identification information 60F, the captured image identification information 61K and 61L are associated by the association information 62F.

[0071] The eighth example of 1200B, similar to the second example of 0800B, includes damage identification information 60, captured image identification information 61, and association information 62. The damage identification information 60A, 60B, 60C, 60D, and 60E indicate the state of the crack damage 77, and the damage identification information 60F indicates the state of the area-based damage 77. The captured image identification information 61 includes the captured image identification information 61A, 61B, 61C, 61D, 61E, and 61F.

[0072] The eighth example of 1200B further includes damage additional attribute information 67. The damage additional attribute information 67 is composed of numbers indicating the size of the damage 77. In the damage identification information 60A, the captured image identification information 61A and the damage additional attribute information 67A are associated by the association information 62A. The damage additional attribute information 67A includes the number "0.2(1.5)" regarding the size corresponding to the damage identification information 60A. Here, the number means a width of 0.2 mm and a length of 1.5 m. For the crack damage, the first number indicates the width (mm), and the number within the parentheses indicates the length (m). Therefore, the numbers of the damage additional attribute information 67B to 67E indicate the width (mm) and the length (m).

[0073] The damage additional attribute information 67B includes the number "0.1(0.7)" regarding the size corresponding to the damage identification information 60B. The damage additional attribute information 67C includes the number "0.3(2.0)" regarding the size corresponding to the damage identification information 60C. The damage additional attribute information 67D includes the number "0.15(0.8)" regarding the size corresponding to the damage identification information 60D. The damage additional attribute information 67E includes the number "0.15(0.8)" regarding the size corresponding to the damage identification information 60E.

[0074] The damage additional attribute information 67F includes the number "0.5×0.3" regarding the size corresponding to the damage identification information 60F. Here, the numbers mean a long side of 0.5 m × a short side of 0.3 m. For damage related to area systems, the first number indicates the long side (m), and the next number indicates the short side (m).

[0075] Examples 1 to 8 of the inspection memo 48 are illustrated in FIGS. 8 to 12, but are not limited thereto.

[0076] Returning to FIG. 4, in step S1, image data 80 (see FIG. 13) of information including the structure diagram 42 of the target structure on the medium, the damage identification information 60 added by the user on the medium, and the photographed image identification information 61 is acquired. The image data acquisition unit 51 of the CPU 20 acquires the image data 80 of information including the structure diagram 42 of the target structure on the medium, the damage identification information 60 added by the user on the medium, and the photographed image identification information 61 (step S1).

[0077] In step S1, as shown in FIG. 13, for example, the inspection memo 48 of the sixth example of the paper medium 41 shown at 1000B is read by the scanner device 33, and the read image data 80 is acquired by the inspection support device 10 from the image data acquisition unit 51. Note that the printing device 32 and the scanner device 33 can be configured as a single device having both functions.

[0078] The inspection memo 48 includes the structure diagram 42, the damage identification information 60 added by the user U, the photographed image identification information 61, the association information 62, and the damage additional attribute information 65. The acquired image data 80 of the inspection memo 48 is displayed on, for example, the display device 30. In the case of the ledger 40A of the electronic medium 46, the image data acquisition unit 51 acquires the ledger 40A from the electronic medium 46 as the image data 80.

[0079] Next, in step S2, the damage identification information is recognized by image recognition from the acquired image data 80. The damage identification information recognition unit 52 of the CPU 20 recognizes the damage identification information 60 by image recognition from the acquired image data 80 (step S2).

[0080] In step S2, for the damage identification information 60 added by the user U, a diagram or a region of a sketch line representing the position, shape, or attribute (damage type, size, damage degree, etc.) of the damage in the damage identification information 60 is recognized by image recognition.

[0081] The damage identification information recognition unit 52 recognizes the difference in the attributes of the damage identification information 60 from different colors, line types, and patterns drawn according to the attributes of the damage identification information 60 in the image data 80. The damage identification information recognition unit 52 can determine the difference in attributes based on the difference in colors and line types. Note that the colors, line types, and patterns applied to the damage identification information 60 are determined in advance by the user U.

[0082] The attributes in the damage identification information 60 include: (1) Damage type: Linear damage such as cracks and fissures. Area-based damage such as water leakage, free lime, delamination, steel bar exposure, rust juice, corrosion, and deterioration of the anticorrosion function are classified. (2) Size: The width of the crack is classified into less than 0.2 mm, 0.2 mm or more and less than 0.3 mm, 0.3 mm or more, etc. (3) Damage degree: Ranks a, b, c, d, and e of the progress of the damage (classified according to the criteria of the inspection guidelines determined by the structure manager, etc.). The attributes of the damage identification information 60 are useful for grasping the status and diagnosing the soundness in regular inspections, and become basic data regarding the state of the damage. It can be used for repairs to structures, etc. Also, it can be used to recognize the progress state of the damage at the next regular inspection.

[0083] The damage identification information recognition unit 52 preferably recognizes the damage identification information 60 from the image data 80 by image recognition using at least one of a machine learning model or an image recognition algorithm. The recognition accuracy of the damage identification information 60 can be improved. The machine learning model is a machine learning model trained by known techniques. For example, teacher data regarding the damage identification information 60 can be prepared and a machine learning model trained using the teacher data can be applied. Known algorithms can be applied to the image recognition algorithm.

[0084] In the embodiment, the damage identification information recognition unit 52 recognizes that the damage identification information 60A, 60B, and 60C indicate the state of damage related to cracks from the image data 80, and recognizes that the damage identification information 60D, 60E, and 60F indicate the state of damage related to the area system (for each reference numeral of the damage identification information 60A, 60B, 60C, 60D, 60E, and 60F, refer to 1000B in FIG. 10). Further, the damage identification information recognition unit 52 recognizes that the line types of the damage identification information 60D, 60E, and 60F, the damage identification information 60A is drawn in an orange line, the damage identification information 60B is drawn in a blue line, and the damage identification information 60C is drawn in a red line. The damage identification information recognition unit 52 recognizes the classification of the width of the damage (less than 0.2 mm / 0.2 mm or more and less than 0.3 mm / 0.3 mm or more) from the colors of the line types of the damage identification information 60A, 60B, and 60C. The relationship between the color of the line type and the width of the damage is stored in the storage unit 16 as, for example, attribute reference information, and the damage identification information recognition unit 52 recognizes the width of the damage from the attribute reference information.

[0085] Further, from the patterns included in the damage identification information 60D, 60E, and 60F, it is recognized that the damage type of the damage identification information 60D is "floating", the damage type of the damage identification information 60E is "free lime", and the damage type of the damage identification information 60F is "peeling". Information indicating the relationship between the damage type and the pattern shown in FIG. 11 is stored in the storage unit 16 as attribute reference information, and the damage identification information recognition unit 52 recognizes the attributes of the damage based on the reference information.

[0086] Further, the damage identification information recognition unit 52 recognizes the position of the damage identification information 60 on the image data 80. The damage identification information recognition unit 52 recognizes, for example, the relative position of the damage identification information 60 with respect to the structure diagram 42 included in the image data 80. The position of the damage identification information 60 can be specified based on the structure diagram 42.

[0087] Next, in step S3, the captured image identification information 61 is recognized from the acquired image data 80 by image recognition. The captured image identification information recognition unit 53 of the CPU 20 recognizes the captured image identification information 61 from the acquired image data 80 by image recognition (step S3).

[0088] As shown in FIG. 13, the captured image identification information 61 included in the inspection memo 48 is basically composed of characters such as a file name. Therefore, the captured image identification information recognition unit 53 recognizes the characters of the captured image identification information 61 by image recognition. The captured image identification information recognition unit 53 recognizes, for example, "1510", "1511", "1512", "1513", "1514", and "1515" indicating the file name as the captured image identification information 61A, 61B, 61C, 61D, 61E, and 61F. As described above, when the captured image identification information 61 is the shooting order, the captured image identification information recognition unit 53 recognizes the character string indicating the shooting order.

[0089] For example, as shown in the fourth example of 0900B in FIG. 9, when the auxiliary information 64 is included in the image data 80, the captured image identification information recognition unit 53 detects the auxiliary information 64 included in the image data 80 and recognizes the characters of the captured image identification information 61 based on the auxiliary information 64. By surrounding the captured image identification information 61 with the auxiliary information 64, the captured image identification information recognition unit 53 specifies the position of the captured image identification information 61 based on the auxiliary information 64. By specifying the position of the captured image identification information 61, the accuracy of the captured image identification information recognition unit 53 recognizing the characters of the captured image identification information 61 can be improved.

[0090] Next, in step S4, the damage identification information 60 and the captured image identification information 61 for the same damage 77 are associated. The first association unit 54 of the CPU 20 associates the damage identification information 60 and the captured image identification information 61 (step S4).

[0091] As described above (see FIG. 6), in the inspection of the structure 70, the user U describes the appearance characteristics of the damage 77 as the damage identification information 60 in the inspection memo 48 and acquires the captured image 103 (not shown) as objective data. In the confirmation of the inspection result and the creation of a report regarding the inspection result, it is premised that the damage identification information 60 and the captured image identification information 61 target the same damage 77.

[0092] Therefore, the first association unit 54 associates the damage identification information 60 included in the image data 80 and the captured image identification information 61 based on a predetermined condition. When the inspection memo 48 is in the sixth example (1000B in FIG. 10), the first association unit 54 associates the damage identification information 60 and the captured image identification information 61 based on the association information 62 of the image data 80. The association information 62 is a diagram connecting the damage identification information 60 and the captured image identification information 61. The first association unit 54 detects the association information 62 and associates the character specifying the corresponding captured image identification information 61 with the damage identification information 60. The association information 62 is not limited to a diagram, and may be association information 63 (a closed diagram) as shown in the third example (0900A in FIG. 9) of the inspection memo 48.

[0093] Also, in the case of the first example (0800A in FIG. 8) where the image data 80 does not include the association information 62 and 63, the first association unit 54 associates the captured image identification information 61 closest to the damage identification information 60 as the captured image identification information 61 corresponding to the damage identification information 60. The user U can determine in advance the condition for associating the damage identification information 60 and the captured image identification information 61, and can cause the first association unit 54 to perform processing based on the condition.

[0094] Along with FIG. 4, steps S1, S2, S3, and S4 have been described, but a preferred embodiment will be described.

[0095] For example, it is preferable to recognize the damage additional attribute information 65(67) from the image data 80 and associate the damage identification information 60 and the damage additional attribute information 65(67).

[0096] The CPU 20 is provided with a damage additional attribute information recognition unit (not shown). When the damage additional attribute information 65 is included in the image data 80, the damage additional attribute information recognition unit and the damage identification information recognition unit 52 recognize, by character recognition, the damage additional attribute information 65 associated with the damage identification information 60 among the damage additional attribute information 65. The attribute information of the damage not included in the damage identification information 60 is additionally recognized by the character string of the damage additional attribute information 65. Further, the more detailed attributes of the attributes recognized from the damage identification information 60 are additionally recognized by the character string of the damage additional attribute information 65.

[0097] For example, in the sixth example (1000B in FIG. 10), the damage additional attribute information recognition unit and the damage identification information recognition unit 52 recognize the character "ウ" of the damage additional attribute information 65A corresponding to the damage identification information 60D, recognize the character "ユ" of the damage additional attribute information 65B corresponding to the damage identification information 60E, and recognize the character "ハ" of the damage additional attribute information 65C corresponding to the damage identification information 60F.

[0098] The damage identification information recognition unit 52 recognizes, by recognizing the damage identification information 60, that it is any one of the area-based damages (such as water leakage, free lime, peeling, and floating) (the major classification of the damage type), and recognizes the specific damage type (such as water leakage, free lime, peeling, and floating) from the character string of the associated damage additional attribute information 65. Thereby, the attributes of the damage identification information 60 can be recognized with higher accuracy.

[0099] Also, when the inspection memo 48 is in the eighth example (1200B in FIG. 12), the damage additional attribute information recognition unit and the damage identification information recognition unit 52 recognize the damage additional attribute information 67 associated with the damage identification information 60 included in the image data 80 by character recognition. The damage additional attribute information recognition unit and the damage identification information recognition unit 52 recognize the number "0.2(1.5)" regarding the size as a character from the damage additional attribute information 67A associated with the damage identification information 60A. Similarly, the damage identification information recognition unit 52 and the damage additional attribute information recognition unit recognize the number "0.1(0.7)" regarding the size as a character from the damage additional attribute information 67B, recognize the number "0.3(2.0)" regarding the size as a character from the damage additional attribute information 67C, recognize the number "0.15(0.8)" regarding the size as a character from the damage additional attribute information 67D, and recognize the number "0.15(0.8)" regarding the size as a character from the damage additional attribute information 67E. As described above, it is predetermined that for the crack damage, the first number indicates the width (mm), and the number in the parentheses indicates the length (m).

[0100] When the damage additional attribute information recognition unit recognizes the damage additional attribute information 65(67), the first association unit 54 associates the damage identification information 60 with the damage additional attribute information 65(67).

[0101] The first association unit 54 associates the damage identification information 60 with the damage additional attribute information 65(67) based on predetermined conditions, in the same manner as described in step S4. The two can be associated based on the association information 62 and 63, or the two with the closest positions can be associated.

[0102] By recognizing the damage identification information 60, it is recognized that it is any of the area-based damages (such as water leakage, free lime, peeling, floating, etc.) (the major classification of damage types), and the specific damage type (such as water leakage, free lime, peeling, floating, etc.) can be recognized from the character string of the damage additional attribute information 65.

[0103] Also, based on this, the damage identification information recognition unit 52 recognizes the classification of the crack width from the color of the damage identification information 60 (less than 0.2 mm / 0.2 mm or more and less than 0.3 mm / 0.3 mm or more, etc.), and recognizes the detailed crack width and length from the characters (numbers) of the damage additional attribute information 67.

[0104] Next, in the association of the damage identification information 60 with the photographed image identification information 61 and / or the association of the damage identification information 60 with the damage additional attribute information 65, 67, it is preferable to exclude the diagram (information) of the structure diagram 42. The first association unit 54 can surely detect the association information 62, 63, and can accurately process the association of the damage identification information 60 with the photographed image identification information 61 and / or the association of the damage identification information 60 with the damage additional attribute information 65, 67.

[0105] Regarding the above-mentioned association process, the case where the inspection memo 48 is the fifth example (1000A in FIG. 10) will be described with reference to FIG. 14. As shown in 1400A, the inspection memo 48 of the fifth example has been acquired by the inspection support device 10 (not shown) as the image data 80. The image data 80 includes the drawing data 101, the damage identification information 60 (60A, 60B, 60C, 60D, 60E, and 60F), the photographed image identification information 61 (61A, 61B, 61C, 61D, 61E, and 61F), the association information 62 (62A, 62B, 62C, 62D, 62E, and 62F), and the damage additional attribute information 65 (65A, 65B, and 65C).

[0106] The CPU 20 includes a drawing data acquisition unit (not shown). The drawing data acquisition unit acquires the drawing data 101 corresponding to the structure diagram 42 and aligns the structure diagram 42 with the drawing data 101. The drawing data acquisition unit, for example, sets an origin for the structure diagram 42 and the drawing data 101 and aligns them based on this origin.

[0107] As shown in 1400B, when making an association, the first association unit 54 can more reliably detect the association information 62 by excluding the diagram corresponding to the drawing data 101, and can accurately process the association between the damage identification information 60 and the photographed image identification information 61, and / or the association between the damage identification information 60 and the additional damage attribute information 65, 67.

[0108] Since the photographed image identification information recognition unit 53 recognizes the photographed image identification information 61 of the photographed image 103 of the target structure taken, for example, it is preferable to obtain the photographed image 103 from the photographed image identification information 61 and obtain the complementary information for complementing the damage identification information 60 from the photographed image 103.

[0109] By obtaining complementary information from not only the damage identification information 60 and the additional damage attribute information 65(67) but also the photographed image 103, the state of the damage 77 can be recognized more accurately.

[0110] Based on the image recognition (machine learning model or image analysis) from the photographed image 103, the damage identification information recognition unit 52 recognizes the shape and / or its attributes (type, size, degree, etc.) of the damage 77.

[0111] When also recognizing the size based on the photographed image 103, the size of the damage 77 can be recognized by inputting the resolution information (mm / pixel) of the photographed image 103 or the actual size of a predetermined region or object in the image data 80. The attributes of the outer shape obtained from the photographed image 103 complement the damage identification information 60 or the additional damage attribute information 65, and the attributes of the size obtained from the photographed image 103 complement the additional damage attribute information 67.

[0112] Next, in step S5, the photographed image 103 corresponding to the photographed image identification information 61 is obtained. The photographed image acquisition unit 55 of the CPU 20 obtains the photographed image 103 from the storage unit 16 (step S5).

[0113] As a result, as shown in FIG. 15, the image data 80 based on the inspection memo 48 includes damage identification information 60, and all the captured images 103 corresponding to the captured image identification information 61 associated with the damage identification information 60 are acquired. Although there are a plurality of damage identification information 60, captured image identification information 61, association information 62, and damage additional attribute information 65, only some of them are illustrated. Seven captured images 103 are acquired for the combination of seven pieces of damage identification information 60 and captured image identification information 61.

[0114] Next, in step S6, the damage identification information 60 and the captured image 103 are associated with each other. The second association unit 56 of the CPU 20 associates the damage identification information 60 and the captured image 103 (step S6).

[0115] FIG. 16 illustrates the association between the damage identification information 60 and the captured image 103. For each damage identification information 60 (damage number, damage type, and damage size) indicating one damage, the captured image identification information 61 (image file name) is associated and organized. Specifically, the damage number: 1, damage type: crack, damage size: width 0.2 mm or more and less than 0.3 mm, and image file name: DSCF1510.jpg are associated. On the other hand, when the inspection memo 48 is the seventh example (1200A in FIG. 12), two or more captured image identification information 61 (image file name) may be associated with one damage identification information 60. Note that the damage identification information 60 and the damage number are linked one-to-one, for example, damage identification information 60A and damage number: 1, damage identification information 60B and damage number: 2, and so on.

[0116] In addition, for each piece of imaging image identification information 61 (image file name) that identifies one captured image 103, damage identification information 60 (type of damage and size of damage) is associated and organized. Specifically, photograph number: 1, image file name: DSCF1510.jpg, type of damage: crack, and size of damage: width 0.2 mm or more and less than 0.3 mm are associated. Note that the imaging image identification information 61 and the photograph number are linked one-to-one, for example, imaging image identification information 61A and photograph number 1, imaging image identification information 61B and photograph number 2, and so on. Basically, one piece of damage identification information 60 is associated with the imaging image identification information 61.

[0117] In 1600A or 1600B, the type of damage, the size of damage, etc. may be supplemented from the damage additional attribute information 65 and 67 and the captured image 103.

[0118] In 1600C, the imaging image identification information 61 (image file name) is associated and organized for each damage category (type of damage and size of damage). Specifically, taking the type of damage: crack and the size of damage: width less than 0.2 mm as one damage category, the image file names: DSCF1511.jpg, DSCF1513.jpg, and DSCF1514.jpg are associated.

[0119] In FIG. 16, 1600A, 1600B, and 1600C are illustrated, but the present invention is not limited thereto. Note that 1600A, 1600B, and 1600C are recorded in the association table 105 of the storage unit 16, for example.

[0120] Next, the organization of the captured images using the above-described inspection support device 10 will be described. As described with reference to FIG. 13, in step S1, the inspection memo 48 on the paper medium 41 is read by the scanner device 33, and the read image data 80 is acquired by the inspection support device 10 from the image data acquisition unit 51.

[0121] As shown in FIG. 17, a damage diagram 84 is created from the image data 80 by the inspection support device 10. For example, the CPU 20 acquires the drawing data 101 corresponding to the structure diagram 42 of the image data 80, and aligns the structure diagram 42 with the drawing data 101. Based on the relative positions of the structure diagram 42 and the damage identification information 60 by the CPU 20, a damage graphic 82 corresponding to the damage identification information 60 is drawn on the drawing data 101. A predetermined line type, color, pattern, or a combination thereof corresponding to the damage identification information 60 is drawn at the corresponding position on the drawing data 101, and the damage diagram 84 corresponding to the image data 80 is created. The damage graphic 82 can be drawn by vectorizing or polygonizing the recognized damage identification information 60. The vectorization can be drawn as a line diagram showing the start point and the end point of the damage.

[0122] When acquiring the drawing data 101 to be inspected from the drawing data group stored in the storage unit 16, it is preferable to acquire, by the CPU 20, the member identification information corresponding to the structure diagram 42 by pattern matching between the structure diagram 42 of the image data 80 and the drawing data 101. The member identification information includes member names such as floor slabs and main girders, member IDs for specifying members, etc. However, the member identification information is not limited to the member name and the member ID. Alternatively, the member identification information (such as the member name and / or the member ID) can be described in the inspection memo 48, and the member identification information can be acquired by image recognition from the image data 80.

[0123] The damage graphic 82 is added to the drawing data 101 as damage graphic data, and a damage diagram 84 including the damage graphic data and the drawing data 101 is created. Since the damage diagram 84 includes the damage graphic data and the drawing data 101, it can be printed on a paper medium from the printing device 32. The damage diagram 84 is output as electronic data (damage diagram data) and stored in the storage unit 16. At the next regular inspection, the damage diagram 84 can be used as past damage information.

[0124] As described above, based on the length of the diagram or the size of the area recognized from the damage identification information 60, the damage figure 82 is drawn on the drawing data 101 having the actual size information of the target structure. Therefore, the actual size of the damage can be calculated from the correspondence between the actual size information and the damage figure 82, and the damage figure 82 can be converted into more accurate actual size information.

[0125] As shown in FIG. 17, the inspection support device 10 can automatically create a damage diagram 84 from the image data 80 of the inspection memo 48 read by the scanner device 33. As shown on the right side of the display device 30, the user U can edit, add, and delete from the edit menu 85 for the damage diagram 84. In this edit menu 85, all the information acquired in relation to the damage 77 can be edited, including the damage identification information 60, the photographed image identification information 61, the damage additional attribute information 65, and the complementary information that complements the damage additional attribute information 65. It may be possible to edit not only on the damage diagram 84 but also on the data organized as an association table as shown in FIG. 16 and on the data of the photo ledger 90 of FIG. 19 described later. By preparing the edit menu 85, it becomes possible to manually edit in the case of misrecognition or non-recognition by character recognition. For example, by selecting the edit menu 85, the reception of editing is executed. Editing is permitted by selecting the locations to be edited, added, and deleted and performing specific editing, addition, and deletion. Note that the reception of editing is executed, but there may be cases where no editing is performed and the editing is not permitted.

[0126] Next, the flag-raising function will be described with reference to FIG. 18. The determined damage diagram 84 is displayed on the display device 30. The damage diagram 84 includes the damage figure 82 and the drawing data 101.

[0127] The damage diagram creation unit 59 of the CPU 20 automatically flags the damage diagram shape 82 of the determined damage diagram 84. For the damage diagram shape 82, a character string related to the line type or pattern recognized by the damage identification information recognition unit 52 from the damage identification information 60 is displayed in association. For example, a character string indicating an attribute (such as type, size, degree of damage, or a combination thereof) with respect to the position and shape of the damage diagram shape 82 may be displayed in association. Alternatively, at least one of the attributes (expressed by differences in color or line type) with respect to the position and shape of the damage diagram shape 82 is displayed, and character strings are displayed in association for the remaining attributes or all attributes. Differences in the type and size of damage can be represented in the damage diagram shape 82 by differences in the color and pattern of the line representing the damage identification information 60. Prepare a database of correspondence relationships in advance (such as the relationship between the color and width of the line, the relationship between the sketch and area-based damage, etc.), and by interpreting the meaning of the line representing the damage identification information 60 from the correspondence relationship, it can be replaced with a character string for representation. The database of correspondence relationships can be edited.

[0128] In FIG. 18, an example of flagging for one damage diagram shape 82 is shown. In the flagging display 86, a character string related to the attribute recognized from the damage identification information 60 is displayed. The flagging display 86 includes an arbitrarily determined damage number: "NO8", the type of damage: "crack", the size of the damage: "W = 0.15 mm, L = 0.8 m", and an arbitrarily determined photo number: "photo - 1004".

[0129] Note that the user U can edit, add, and delete from the flagging menu 87. The edited result is reflected in real time in the damage diagram data of the damage diagram 84.

[0130] Next, the process of creating a photo ledger from the created damage diagram 84 will be described. FIG. 19 is a diagram for explaining the process of creating a photo ledger from the damage diagram. In 1900A, a display device 30 is shown that displays the damage diagram 84 on the left side of the screen and the photo ledger 90 on the right side of the screen. In 1900B, a flow for creating the photo ledger 90 from the inspection memo 48 is shown.

[0131] Step S10 is a conceptual diagram showing the inspection work at the site. As described above, an inspection memo 48 is created by the user at the inspection site. The file name "2354" of the captured image 103 is handwritten in the inspection memo 48 as the captured image identification information 61. The "2354" of the captured image identification information 61 corresponds to DSCF2354JPG which is the file name of the captured image 103.

[0132] Step S11 is a conceptual diagram of the process of creating a damage diagram 84 from the image data 80. As described above, the image data 80 of the inspection memo 48 is acquired, and a damage diagram 84 including a damage graphic 82 corresponding to the damage identification information 60 and the drawing data 101 is created based on the image data 80. Also, "2354" which is the captured image identification information 61 is added to the damage diagram 84 by character recognition. In step S11, damage identification information 60 other than "2354", damage addition attribute information 65, etc. are added as a flag-raising display 86.

[0133] Step S12 is a conceptual diagram of the process of creating a photo ledger 90 from the damage diagram 84, and 1900A shows an example on the display device 30 when the inspection support device 10 executes the process.

[0134] In the 1900A, the damage diagram 84 and the photo ledger 90 are displayed. In the process of creating the photo ledger 90, first, the damage diagram 84 is displayed. For example, a photo ledger creation button (not shown) is prepared, and by pressing the photo ledger creation button, the photo ledger 90 is automatically created from the damage diagram 84. Step S12 shows an enlarged view of the photo ledger 90. Information on the flag-raising display 86 associated with the photographed image identification information 61 is automatically transferred to the photo ledger 90. For example, the photo number, radial number, member name, damage type, and damage degree are transferred to the photo ledger 90. In the example on the left side of the photo ledger 90, the photo number: 1004, radial number: 1, member name: floor slab, damage type: crack, and damage situation: D are transferred. Here, the photo number is transferred as "1004" as the serial number corresponding to the damage number (here, the serial number) assigned to the damage figure 82, rather than "2354" which is the photographed image identification information 61. Since "2354" which is the photographed image identification information 61 and "1004" which is the photo number are associated with each other, the photo number corresponds to the photographed image 103 via the photographed image identification information 61. Also in the flag-raising display 86, it is changed from "photo - 2354" shown in step S11 to "photo - 1004" shown in step S12.

[0135] Next, the link process from the created damage diagram 84 to the photographed image 103 will be described. FIG. 20 is a diagram for explaining the link process from the damage diagram to the photographed image. As shown in FIG. 20, the damage diagram 84 is displayed on the left side of the display device 30. The flag-raising display 86 is displayed on the damage diagram 84. When the flag-raising display 86 corresponding to the damage figure 82 is selected, the photographed image 103 (here, DSCF1513.jpg) is displayed on the right side of the display device 30 via the linked photographed image identification information 61. By associating and linking the photographed image identification information 61 in the damage diagram 84, the photographed images 103 can be organized.

[0136] <Others> In the above-described embodiment, the hardware structure of the processing unit that executes various processes is various processors as described below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (program) and functions as various processing units, a programmable logic device (PLD) such as an FPGA (Field Programmable Gate Array), which is a processor whose circuit configuration can be changed after manufacturing, and an application-specific electric circuit, which is a processor having a circuit configuration specifically designed to execute specific processes such as an ASIC (Application Specific Integrated Circuit).

[0137] One processing unit may be composed of one of these various processors, or may be composed of two or more processors of the same or different types (for example, a plurality of FPGAs, or a combination of a CPU and an FPGA). Also, a plurality of processing units can be composed of one processor. As an example of configuring a plurality of processing units with one processor, first, as represented by a computer such as a client or a server, one processor is configured by a combination of one or more CPUs and software, and this processor functions as a plurality of processing units. Second, as represented by a system on chip (SoC), there is a form in which a processor that realizes the functions of the entire system including a plurality of processing units with one IC (Integrated Circuit) chip is used. Thus, the various processing units are configured using one or more of the above-described various processors as a hardware structure.

[0138] Furthermore, the hardware structure of these various processors is more specifically an electric circuit (circuitry) that combines circuit elements such as semiconductor elements.

[0139] Each of the above configurations and functions can be appropriately implemented by any hardware, software, or a combination of both. For example, the present invention can also be applied to a program that causes a computer to execute the above-described processing steps (processing procedures), a computer-readable recording medium (non-temporary recording medium) that records such a program, or a computer to which such a program can be installed.

[0140] Although the examples of the present invention have been described above, it goes without saying that the present invention is not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present invention.

Explanation of Reference Numerals

[0141] 10 Inspection Support Device 12 Input / Output Interface 16 Storage Unit 18 Operation Unit 26 Display Control Unit 30 Display Device 32 Printing Device 33 Scanner Device 40 Field Book 40A Field Book 41 Paper Medium 42 Structure Diagram 43 Pen 45 Digital Camera 46 Electronic Medium 47 Electronic Pen 48 Inspection Memo 51 Image Data Acquisition Unit 52 Damage Identification Information Recognition Unit 53 Captured Image Identification Information Recognition Unit 54 First Association Unit 55 Captured Image Acquisition Unit 56 Second Association Unit 59 Damage Diagram Creation Unit 60 Damage Identification Information 60A Damage Identification Information 60B Damage Identification Information 60C Damage Identification Information 60D Damage Identification Information 60E Damage Identification Information 60F Damage Identification Information 61 Photograph Image Identification Information 61A Photograph Image Identification Information 61B Photograph Image Identification Information 61C Photograph Image Identification Information 61D Photograph Image Identification Information 61E Photograph Image Identification Information 61F Photograph Image Identification Information 61G Photograph Image Identification Information 61H Photograph Image Identification Information 61I Photograph Image Identification Information 61J Photograph Image Identification Information 61K Photograph Image Identification Information 61L Photograph Image Identification Information 62 Association Information 62A Association Information 62B Association Information 62C Association Information 62D Association Information 62E Association Information 62F Association Information 63 Association Information 63A Association Information 63B Association Information 63C Association Information 63D Association Information 63E Association Information 63F Association Information 64 Auxiliary Information 64A Auxiliary Information 64B Auxiliary Information 64C Auxiliary Information 64D Auxiliary Information 64E Auxiliary Information 64F Auxiliary Information 65 Damage Additional Attribute Information 65A Damage Additional Attribute Information 65B Damage Additional Attribute Information 65C Damage Additional Attribute Information 67 Damage Additional Attribute Information 67A Damage Additional Attribute Information 67B Damage Additional Attribute Information 67C Additional Damage Attribute Information 67D Additional Damage Attribute Information 67E Additional Damage Attribute Information 67F Additional Damage Attribute Information 70 Structure 71 Bridge 72 Main Girder 73 Cross Girder 74 Anti-Inclination Structure 75 Lateral Structure 76 Floor Slab 77 Damage 80 Image Data 82 Damage Graphic 84 Damage Diagram 85 Edit Menu 86 Hoisting Display 87 Hoisting Menu 90 Photo Album 101 Drawing Data 103 Captured Image 105 Association Table S1 Step S2 Step S3 Step S4 Step S5 Step S6 Step S10 Step S11 Step S12 Step U User

Claims

1. An inspection support device comprising a processor for sorting images including damage to a structure, wherein the processor acquires image data of information including a structural diagram of a target structure on a medium, damage identification information regarding damage added by a user on the medium, and photographing image identification information composed of characters, recognizes the damage identification information from the acquired image data by image recognition using at least one of a machine learning model or an image recognition algorithm, recognizes the photographing image identification information from the acquired image data by character recognition, associates the damage identification information and the photographing image identification information for the same damage, acquires a photographing image corresponding to the photographing image identification information, associates the damage identification information and the photographing image, an inspection support device.

2. The processor recognizes the damage identification information regarding the attribute of the damage, The inspection support device according to claim 1.

3. The information includes auxiliary information for specifying the photographing image identification information, The processor specifies the position of the photographing image identification information based on the auxiliary information in the recognition of the photographing image identification information, The inspection support device according to claim 1 or 2.

4. The processor associates the damage identification information and the photographing image identification information based on predetermined conditions, The inspection support device according to any one of claims 1 to 3.

5. The processor records the associated damage identification information and the photographing image in a database. The inspection support device according to any one of claims 1 to 4.

6. The processor creates a photo ledger from the associated damage identification information and the photographing image. The inspection support device according to any one of claims 1 to 5.

7. The processor recognizes member identification information of the target structure, associates the member identification information, the damage identification information, and the photographing image, The inspection support device according to any one of claims 1 to 6.

8. The processor recognizes damage additional attribute information from the image data, associates the damage identification information and the damage additional attribute information, The inspection support device according to any one of claims 1 to 7.

9. The processor recognizes characters included in the damage additional attribute information by character recognition, The inspection support device according to claim 8.

10. The processor Obtaining complementary information for supplementing the damage additional attribute information from the captured image The inspection support device according to claim 8 or 9

11. The processor Receiving any one or more edits to the damage identification information, the captured image identification information, the damage additional attribute information, and the complementary information, and allowing any one or more edits to the received damage identification information, the captured image identification information, the damage additional attribute information, and the complementary information The inspection support device according to claim 10

12. Associating the damage identification information and the damage additional attribute information based on a predetermined condition The inspection support device according to any one of claims 8 to 11

13. The processor Obtaining drawing data corresponding to the structural diagram Aligning the structural diagram of the image data with the drawing data In the association between the damage identification information and the captured image identification information, or the association between the damage identification information and the damage additional attribute information Excluding information regarding the structural diagram based on the drawing data The inspection support device according to any one of claims 8 to 12

14. An inspection support method for assisting in organizing images including damage to a structure by a processor, wherein The processor Obtaining image data of information including a structural diagram of a target structure on a medium, damage identification information regarding damage added by a user on the medium, and captured image identification information composed of characters Recognizing the damage identification information from the obtained image data by image recognition using at least one of a machine learning model or an image recognition algorithm Recognizing the captured image identification information from the obtained image data by character recognition Associating the damage identification information and the captured image identification information for the same damage Obtaining a captured image corresponding to the captured image identification information Associating the damage identification information and the captured image An inspection support method for executing

15. A program for executing an inspection support method for assisting in organizing images including damage to a structure by a processor, wherein The processor Obtaining image data of information including a structural diagram of a target structure on a medium, damage identification information regarding damage added by a user on the medium, and captured image identification information composed of characters recognizing the damage identification information from the acquired image data by image recognition using at least one of a machine learning model or an image recognition algorithm; recognizing the photographed image identification information from the acquired image data by character recognition; associating the damage identification information and the photographed image identification information for the same damage; acquiring a photographed image corresponding to the photographed image identification information; associating the damage identification information and the photographed image; A program for causing the above to be executed.

Images