Information processing device, information processing method, and program

Abstract

The goal is to support efficient inspections, even for damage that is difficult to inspect visually. [Solution] The information processing device includes an extraction unit that, when the inspection image of the target includes an image of damage to a structure that is difficult to visually inspect, refers to the location of the damage included in the inspection history information which stores past inspection history, and extracts past inspection images taken in past inspections where the distance from the inspection imaging position representing the position where the inspection image was taken or the location of the damage to the target is within a preset distance range, and a first calculation unit that calculates a first score for each extracted past inspection image based on the proximity of the distance.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program for assisting inspections.

Background Art

[0002] For damage to structures, there are damages that are difficult to visually inspect, such as, for example, the peeling of concrete. In the case of damage that is difficult to visually inspect, sufficient features for collation cannot be extracted from each of the image captured at the time of inspection and the image captured at the time of past inspections.

[0003] As a related technique, Patent Document 1 discloses a technique for performing image collation processing after narrowing down collation target images, and specifying the position of an inspection target object captured in a target image in a short time and with high accuracy. The geographical location information specifying system of Patent Document 1 includes a first database including a collation target image for a target image in which an inspection target object is captured, position information of the collation target image, and attribute information of the collation target image, and uses a narrowing-down condition that is attribute information of the inspection target object to search for a collation target image from the first database, compares the target image with the retrieved collation target image, and extracts a collation target image similar to the target image and position information of the similar collation target image from the first database.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the geographical location information specifying system of Patent Document 1, collation is performed using an image captured at the time of inspection and an image captured at the time of past inspections, but it does not mention the case of damage to a structure that is difficult to visually inspect.

[0006] One example of the purpose of this disclosure is to support efficient inspection even for damage that is difficult to inspect visually. [Means for solving the problem]

[0007] To achieve the above objective, the information processing device in one aspect of this disclosure is: If the target inspection image includes images of structural damage that is difficult to visually inspect, the extraction unit refers to the damage location included in the inspection history information, which stores past inspection history, and extracts past inspection images taken in past inspections where the distance from the inspection imaging position representing the position where the inspection image was taken or the target damage location to the past damage location is within a preset distance range. A first calculation unit calculates a first score for each of the extracted past inspection images based on the proximity of the distance, It is characterized by having the following features.

[0008] Furthermore, in order to achieve the above objectives, the information processing method in one aspect of this disclosure is: Computers If the target inspection image includes images of structural damage that is difficult to visually inspect, the system refers to the damage location included in the inspection history information, which stores past inspection history, and extracts past inspection images taken during past inspections where the distance from the inspection imaging position representing the location where the inspection image was taken or the target damage location to the past damage location is within a predetermined distance range. For each of the extracted past inspection images, a first score is calculated based on the proximity of the distance. The system generates display information for displaying on a display device a display that associates the extracted past inspection images with the first score. It is characterized by the following:

[0009] Furthermore, in order to achieve the above objectives, the program in one aspect of this disclosure is On the computer, If the target inspection image includes images of structural damage that is difficult to visually inspect, the system refers to the damage location included in the inspection history information, which stores past inspection history, and extracts past inspection images taken during past inspections where the distance from the inspection imaging position representing the location where the inspection image was taken or the target damage location to the past damage location is within a predetermined distance range. For each of the extracted past inspection images, a first score is calculated based on the proximity of the distance. The system generates display information for displaying on a display device a display that associates the extracted past inspection images with the first score. The characteristic feature is that it causes the process to be executed. [Effects of the Invention]

[0010] As described above, this disclosure can help to efficiently inspect even damage that is difficult to inspect visually. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 is a diagram illustrating an example of an information processing device. [Figure 2] Figure 2 shows an example of a system having an information processing device. [Figure 3] Figure 3 is a diagram illustrating an example of bridge inspection. [Figure 4] Figure 4 is a diagram illustrating the exposure of a specific component. [Figure 5] Figure 5 illustrates an example of how to display a score in the case of structural damage that is difficult to visually inspect. [Figure 6] Figure 6 illustrates an example of how scores are displayed in the case of damage to a structure that can be visually inspected. [Figure 7] Figure 7 is a diagram illustrating an example of the operation of an information processing device. [Figure 8] Figure 8 is a diagram illustrating an example of the operation of an information processing device. [Figure 9]FIG. 9 is a diagram for explaining an example of a computer that realizes the information processing apparatus according to the embodiment.

Embodiment for Implementing the Invention

[0012] Hereinafter, embodiments will be described with reference to the drawings. In the drawings described below, elements having the same function or corresponding functions are denoted by the same reference numerals, and repeated descriptions thereof may be omitted.

[0013] (Embodiment) The configuration of the information processing apparatus according to the embodiment will be described using FIG. 1. FIG. 1 is a diagram for explaining an example of the information processing apparatus.

[0014] [Device Configuration] The information processing apparatus shown in FIG. 1 is a device (inspection support device) that supports efficient inspection even for damages that are difficult to visually inspect. Also, as shown in FIG. 1, the information processing apparatus 10 includes an extraction unit 11 and a first calculation unit 12.

[0015] When the target inspection image includes a damage image of a structure that is difficult to visually inspect, the extraction unit 11 refers to the damage position (two-dimensional or three-dimensional coordinates) included in the inspection history information in which the past inspection history is stored, and from the inspection imaging position (two-dimensional or three-dimensional coordinates) representing the position where the target inspection image was taken or the target damage position to the past damage position, extracts past inspection images (candidate images) taken in past inspections that are within a preset distance range.

[0016] Note that since the inspection imaging position and the past damage position can be obtained in two-dimensional or three-dimensional coordinates respectively, when both are in three-dimensional coordinates, the distance in the three-dimensional space is used, when both are in two-dimensional coordinates, the distance in the two-dimensional space is used, and when one is in three-dimensional and the other is in two-dimensional, the distance in the two-dimensional space is represented after ignoring the height axis of the three-dimensional coordinates in advance.

[0017] A structure is a hardened material (such as concrete or mortar) solidified using at least sand, water, and cement, or a metal, or a structure constructed using these materials. Furthermore, a structure can be an entire building or a part thereof. Additionally, a structure can be an entire machine or a part thereof.

[0018] Inspection images show structural damage that is difficult to detect visually, such as concrete delamination, which cannot be judged by visual inspection (and can only be judged by other tests such as tapping tests). However, this is not limited to concrete delamination.

[0019] The damage location represents the position of the damaged area (two-dimensional or three-dimensional position coordinates). Alternatively, it may be the position of the terminal device used by the worker who captured the inspection image during a previous inspection.

[0020] The first calculation unit 12 calculates a first score for each extracted past inspection image (candidate image) based on the proximity of the distance from the inspection imaging location to the damage location.

[0021] Thus, in this embodiment, by presenting the score of each extracted past inspection image (candidate image) to the worker, inspection support can be provided so that even damage that is difficult to visually inspect can be efficiently correlated with past inspection results.

[0022] [System Configuration] Next, the configuration of the information processing device 10 in the embodiment will be described in more detail using Figure 2. Figure 2 is a diagram showing an example of a system having an information processing device. As shown in Figure 2, the system 100 in the embodiment includes an information processing device 10, a storage device 20, a terminal device 30, and a network 40.

[0023] The information processing device 10 is, for example, a CPU (Central Processing Unit), a programmable device such as an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or a circuit equipped with one or more of these, a server computer, a personal computer, a mobile terminal, or other information processing device.

[0024] The storage device 20 includes a database for storing past inspection history, a server computer, and a circuit with memory. In the example shown in Figure 2, the storage device 20 is located outside the information processing device 10, but it may also be located inside the information processing device 10.

[0025] Furthermore, as shown in Figure 2, the storage device 20 stores inspection history information 21 and the like. The inspection history information 21 is information that associates at least past inspection images, past inspection imaging locations (two-dimensional or three-dimensional position coordinates) representing the locations where past inspection images were captured, damage information representing the type of damage, and the location of the damaged area (two-dimensional or three-dimensional position coordinates) representing the location of the damaged area. However, the storage device 20 may also store information other than inspection history information.

[0026] The terminal device 30 is, for example, an information processing device such as a personal computer or mobile terminal equipped with a CPU, an FPGA, or both. The terminal device 30 may further be equipped with an imaging device 31, an input device 32, and an output device 33. The imaging device 31, input device 32, and output device 33 may be provided outside the terminal device 30.

[0027] The imaging device 31 is, for example, a camera, or a device equipped with a camera and LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging). Examples of cameras include monocular cameras (e.g., wide-angle cameras, fisheye cameras, 360-degree cameras), compound cameras (e.g., stereo cameras, multi-cameras), and RGB-D cameras (e.g., depth cameras, ToF cameras).

[0028] The input device 32 is, for example, a device such as a touch panel, mouse, or keyboard.

[0029] The output device 33 acquires output information, which has been converted into an outputtable format, and outputs generated images and sounds based on that output information. The output device 33 is, for example, an image display device using liquid crystal, organic EL (Electro Luminescence), or CRT (Cathode Ray Tube). Furthermore, the image display device may also be equipped with sound output devices such as speakers. The output device 33 may also be a printing device such as a printer.

[0030] Network 40 is a general network constructed using communication lines such as the Internet, LAN (Local Area Network), dedicated lines, telephone lines, corporate networks, mobile communication networks, Bluetooth (registered trademark), and Wi-Fi (Wireless Fidelity) (registered trademark).

[0031] ●The information processing device will be explained in detail. The information processing device 10 will be described using Figures 2 and 3. As shown in Figure 2, the information processing device 10 includes an acquisition unit 13, a first determination unit 14, an extraction unit 11, a first calculation unit 12, a matching unit 15, a second calculation unit 16, a second determination unit 17, and a generation unit 18.

[0032] Figure 3 illustrates an example of bridge inspection. The lower part of Figure 3 shows worker 2 using terminal device 30 to image damage 4 to the bridge deck 1. The upper part of Figure 3 shows damage diagram 5 representing the damage to the bridge deck 1.

[0033] Damage diagram 5 (or unfolded diagram) is information generated during inspection, and is a diagram that records, for example, the location of the damaged area, the type of damage, its size, and its direction. Damage diagram 5 is stored as information in the inspection history information 21.

[0034] The dashed line area 3' in Figure 3 indicates the imaging range of the inspection image when damage 4 was captured. Damage image 4' in Figure 3 is the image corresponding to damage 4 (damage image). The seven black circles 6 (●) in Figure 3 indicate the locations and sites of past damage. Note that the structure is not limited to the bridge deck 1.

[0035] The acquisition unit 13 acquires inspection images (the images corresponding to 3 in Figure 3) captured using the imaging device 31 installed on the terminal device 30 during the inspection of the structure, via the network 40. The acquisition unit 13 also acquires the inspection imaging position (information representing the target inspection imaging position), which indicates the location where the worker 2 captured the inspection image. Alternatively, the damage location may be the location of the damage 4 (two-dimensional or three-dimensional position coordinates).

[0036] The inspection imaging location is, for example, location information (two-dimensional or three-dimensional position coordinates) obtained using GPS (Global Positioning System). Alternatively, the inspection imaging location information may be manually set by the operator. Note that the acquisition of the inspection imaging location is not limited to the methods described above.

[0037] The first determination unit 14 uses a machine learning model to determine whether the inspection image contains images of damage to the structure that are difficult to visually inspect.

[0038] Structural damage includes damage that cannot be detected by visual inspection, such as concrete delamination. Damage images corresponding to structural damage that is difficult to visually inspect do not contain sufficient features, making it difficult to determine whether or not damage is present. Therefore, a machine learning model capable of detecting visually visible damage such as cracks and rust is used to determine that an inspection image is a damage image that is difficult to visually inspect when the result shows that no damage is present in the inspection image. Specifically, the first determination unit 14 inputs the inspection image into a trained machine learning model during estimation to determine whether the inspection image is a damage image that is difficult to visually inspect.

[0039] Alternatively, worker 2 may identify structural damage that is difficult to visually inspect through other inspections such as tapping tests, and manually specify on the inspection image in question that the damage is difficult to visually inspect, or the type of damage, or both, using an input device 32 or the like.

[0040] If the target inspection image includes images of structural damage that is difficult to visually inspect, the extraction unit 11 refers to the inspection history information 21 and extracts past inspection images (candidate images) taken in past inspections where the distance from the inspection imaging position (two-dimensional or three-dimensional coordinates) where the target inspection image was taken to the damage position (two-dimensional or three-dimensional coordinates) falls within the preset distance range 7 shown in Figure 3.

[0041] Furthermore, the extraction unit 11 may, from among the past inspection images within the distance range 7, further refer to the inspection history information 21 to extract past inspection images that include structural damage 4 that is difficult to visually inspect.

[0042] Furthermore, the location may be determined based on the information in the forms that record whether the damage was to a kilometer post or to either the up or down line, or both, as included in the inspection history information 21. In addition, when extracting past inspection images (candidate images), metadata (material type, digit number) associated with the past inspection images may be used to further narrow down the results.

[0043] The first calculation unit 12 calculates a first score for each extracted past inspection image (candidate image) based on its proximity to the previous image. Specifically, the first calculation unit 12 calculates the first score S as the reciprocal of the distance L from the inspection imaging position to the damage position (1 / L). Alternatively, the first calculation unit 12 may calculate the first score S by multiplying the reciprocal of the distance L (1 / L) by a coefficient K that has been set in advance for each type of structural damage that is difficult to visually inspect, such as delamination, peeling, and cavities (K × (1 / L)).

[0044] Furthermore, even if the target is damage to a structure that is difficult to inspect visually, it is possible to determine whether past and present inspection images were taken at the same location by using the shape of the structure, the arrangement of components, or both, contained in the image. Therefore, this coefficient may also be the number of feature point pairs that have been matched between the inspection image and past inspection images, for example, by identifying feature points extracted from the shape of the structure, the arrangement of components, etc., contained in the image.

[0045] If the target inspection image contains images of visible structural damage, the matching unit 15 performs a matching process between the target inspection image and the extracted past inspection images (candidate images).

[0046] Specifically, the matching unit 15 extracts feature points from the target inspection image and the extracted past inspection images (candidate images). Next, the matching unit 15 extracts pairs of feature points that match. The above process is performed for each of the extracted past inspection images (candidate images).

[0047] As a preprocessing step, the target inspection image, or the extracted past inspection image (candidate image), may be transformed so that the matching points are in the same position before the matching process is performed.

[0048] As a preprocessing step, inspection images typically include images taken from a distance to check the surrounding environment and images taken up close to check the details of the object. However, it may be possible to first compare past distance images with the current distance image, and only if a match is found, then compare it with the corresponding close-up image. This can reduce the probability of mismatches in areas such as repeating structures.

[0049] As a preprocessing step, if the target inspection image or extracted past inspection images (candidate images) contains anything other than the image to be inspected, the parts other than the image to be inspected may be masked before performing the matching process.

[0050] The second calculation unit 16 calculates a second score based on the results of the matching process. Specifically, the second calculation unit 16 uses the number of feature point pairs that match the target inspection image in each of the extracted past inspection images (candidate images) as the second score.

[0051] The second determination unit 17 sets a new deformation flag in the inspection image (the inspection image containing the image of the exposed specific member) if the inspection image includes a damaged image in which a specific member is exposed from the structure, and the damaged image is not included in past inspection images corresponding to the inspection image. The determination of whether or not a specific member is exposed from the structure in the inspection image is made using a machine learning model that has been trained in advance to detect specific members.

[0052] Figure 4 is a diagram illustrating the exposure of a specific structural member. In the example shown in Figure 4, reinforcing steel is exposed from the concrete. Note that the method of addressing a specific structural member varies depending on the progression of the damage, so even in the same location, it may be necessary to count it as a new deformation depending on the degree of damage. For example, concrete deformation progresses from spalling to reinforcing steel exposure. In the case of spalling, work such as knocking it down is performed to prevent further detachment. In contrast, when reinforcing steel is exposed, repairs are necessary to prevent corrosion of the reinforcing steel.

[0053] The generation unit 18 generates output information to output device 33, which associates extracted past inspection images (candidate images) with a first score, in the case of damage to a structure that is difficult to visually inspect. Next, the generation unit 18 transmits the output information to terminal device 30. Subsequently, terminal device 30 receives the output information and displays the inspection image 51, the extracted past inspection images (candidate images), and the first score on the output device 33 (monitor) of terminal device 30.

[0054] Figure 5 illustrates an example of score display in the case of structural damage that is difficult to visually inspect. Specifically, as shown in Figure 5, the user interface 50 of the output device 33 (monitor) displays the inspection image 51, the extracted past inspection image 52 (candidate image), and the first score 53. However, the number of candidate images is not limited to one.

[0055] Furthermore, in the case of damage to a structure that can be visually inspected, the generation unit 18 generates output information to the output device 33 that associates the second score with a past inspection image (candidate image) that has a high second score. Next, the generation unit 18 transmits the output information to the terminal device 30. The terminal device 30 then receives the output information and displays the inspection image, the extracted past inspection image (candidate image), and the second score on the output device 33 (monitor) of the terminal device 30.

[0056] Figure 6 illustrates an example of score display in the case of damage to a structure that can be visually inspected. Specifically, as shown in Figure 6, the user interface 50 of the output device 33 (monitor) displays the inspection image 61, the extracted past inspection images (candidate images) 62a, 62b, 62c, and the second scores 63a, 63b, 63c. However, the number of candidate images is not limited to three.

[0057] Furthermore, if all secondary scores are below a predetermined threshold, the secondary scores do not need to be displayed. Specifically, a message such as "No matching past inspection images found" may be displayed.

[0058] In addition to the score, to make the identification of identical locations visually easier to understand, candidate images with matching field angles may be displayed using homography transformation with matching points.

[0059] The coordinates of the structure in the candidate images, the type of damage, and other information may be displayed. This makes it easier for workers to distinguish between current and past inspection images, taking into account information other than the images themselves.

[0060] [Device operation] Next, the operation of the information processing device in the embodiment will be described using Figures 7 and 8. Figures 7 and 8 are diagrams illustrating an example of the operation of the information processing device. In the following description, the diagrams will be referred to as appropriate. In this embodiment, the information processing method is implemented by operating the information processing device. Therefore, the explanation of the information processing method in this embodiment will be replaced by the following explanation of the operation of the information processing device.

[0061] As shown in Figure 7, first, the acquisition unit 13 acquires inspection images captured using the imaging device 31 provided on the terminal device 30 during the inspection of the structure via the network 40 (Step A1). Next, it calculates a score (Step A2). Step A2 will be explained in detail using Figure 8.

[0062] As shown in Figure 8, in calculating the score, first, the first determination unit 14 uses a machine learning model to determine whether or not there are images of damage to the structure that are difficult to visually inspect in the inspection images (step B1).

[0063] Next, if the inspection images include images of structural damage that is difficult to visually inspect (Step B2: Yes), calculate the score for damage that is difficult to visually confirm (Step B3).

[0064] Specifically, in step B3, the extraction unit 11 first refers to the inspection history information 21 and extracts past inspection images (candidate images) taken during past inspections in which the distance from the inspection imaging position or the target damage position (two-dimensional or three-dimensional coordinates) where the target inspection image was taken to the past damage position (two-dimensional or three-dimensional coordinates) falls within a predetermined distance range 7.

[0065] In step B3, the extraction unit 11 may further extract past inspection images from the distance range 7 that include structural damage 4 that is difficult to visually inspect, by referring to the inspection history information 21.

[0066] Next, in step B3, the first calculation unit 12 calculates a first score for each extracted past inspection image (candidate image) based on the proximity of the distance. Specifically, the first calculation unit 12 calculates the first score S as the reciprocal of the distance L from the inspection imaging position to the damage position (1 / L). Alternatively, the first calculation unit 12 may calculate the first score S by multiplying the reciprocal of the distance L (1 / L) by a coefficient K that has been set in advance for each type of structural damage that is difficult to visually inspect (K × (1 / L)).

[0067] Next, if the target inspection image contains images of visible structural damage (Step B2: No), the extraction unit 11 calculates a score with past inspection images (Step B4).

[0068] Specifically, in step B4, the matching unit 15 first performs a matching process between the target inspection image and the extracted past inspection images (candidate images) if the target inspection image includes images of visible structural damage.

[0069] Specifically, the matching unit 15 extracts feature points from the target inspection image and the extracted past inspection images (candidate images). Next, the matching unit 15 extracts pairs of feature points that match. The above process is performed for each of the extracted past inspection images (candidate images).

[0070] Next, in step B4, the second calculation unit 16 calculates a second score based on the results of the matching process. Specifically, the second calculation unit 16 uses the number of pairs of each extracted past inspection image (candidate image) as the second score.

[0071] Next, the second determination unit 17 determines that if the inspection image includes a damaged image in which a specific member is exposed from the structure (Step B5: Yes), and that the damaged image is not included in past inspection images corresponding to the inspection image (Step B6: Yes), it sets a new deformation flag in the inspection image (the inspection image containing the damaged image in which the specific member is exposed) to indicate that there is a new deformation (Step B7).

[0072] Next, if there is no other damage in the inspection image (Step B8: Yes), the process ends and proceeds to Step A3 in Figure 7. If there is other damage in the inspection image (Step B8: No), proceeds to Step B1.

[0073] Next, if the first score indicates damage that is difficult to visually confirm (Step A3: Yes), the generation unit 18 generates output information to output device 33 that associates the extracted past inspection image (candidate image) with the first score in the case of damage to a structure that is difficult to visually inspect (Step A4). Subsequently, in Step A4, the generation unit 18 transmits the output information to terminal device 30. The terminal device 30 then receives the output information and displays the inspection image 51, the extracted past inspection image (candidate image), and the first score on the output device 33 (monitor) of the terminal device 30.

[0074] Furthermore, in the case of damage to a structure that can be visually inspected (Step A3: No), if all second scores are below a preset threshold (Step A5: Yes), output information is generated to indicate that there are no corresponding past inspection images (Step A6). Subsequently, in Step A6, the generation unit 18 transmits the output information to the terminal device 30. The terminal device 30 then receives the output information and may display a message such as "No corresponding past inspection images" on the output device 33 (monitor) of the terminal device 30.

[0075] Furthermore, in the case of damage to a structure that can be visually inspected (Step A3: No), if the second score exceeds a preset threshold (Step A5: No), the generation unit 18 generates output information to cause the output device 33 to output an output associating the second score with a past inspection image (candidate image) that has a high second score. Subsequently, in Step A4, the generation unit 18 transmits the output information to the terminal device 30. The terminal device 30 then receives the output information and displays the inspection image, the extracted past inspection image (candidate image), and the second score on the output device 33 (monitor) of the terminal device 30.

[0076] Furthermore, if a specific component is exposed from the structure in the inspection image, the inspection image (the inspection image containing the damaged image in which the specific component is exposed) may be marked to indicate that there is a new deformation.

[0077] [Effects of the Embodiment] As described above, according to the embodiment, even for damage that is difficult to visually inspect, the inspection can be supported to enable efficient inspection by presenting the operator with the first score for each extracted past inspection image (candidate image). Specifically, it enables efficient correspondence with past inspection results.

[0078] Even for damage that can be visually inspected, the system can support inspections by presenting the operator with a secondary score for each extracted past inspection image (candidate image), enabling more efficient inspections.

[0079] If a specific component is exposed from a structure in an inspection image, the system can support inspections by indicating the presence of new deformation in that inspection image (the inspection image containing the damaged image showing the exposed specific component).

[0080] [program] The program in the embodiment can be any program that causes a computer to execute steps A1 to A6 and B1 to B8 shown in Figures 7 and 8. By installing and executing this program on a computer, the information processing apparatus and information processing method in the embodiment can be realized. In this case, the computer's processor functions as an acquisition unit 13, a first determination unit 14, an extraction unit 11, a first calculation unit 12, a matching unit 15, a second calculation unit 16, a second determination unit 17, and a generation unit 18, and performs the processing.

[0081] Furthermore, the program in the embodiment may be executed by a computer system constructed by multiple computers. In this case, for example, each computer may function as one of the following: acquisition unit 13, first determination unit 14, extraction unit 11, first calculation unit 12, matching unit 15, second calculation unit 16, second determination unit 17, or generation unit 18.

[0082] [Physical configuration] Here, a computer that implements an information processing device by executing the program in the embodiment will be described using Figure 9. Figure 9 is a diagram illustrating an example of a computer that implements an information processing device in the embodiment.

[0083] As shown in Figure 9, the computer 110 comprises a CPU (Central Processing Unit) 111, main memory 112, storage device 113, input interface 114, display controller 115, data reader / writer 116, and communication interface 117. These components are connected to each other via a bus 121, enabling data communication. In addition to the CPU 111, or in place of the CPU 111, the computer 110 may also include a GPU or FPGA.

[0084] The CPU 111 loads the program in the embodiment, which consists of a set of codes stored in the storage device 113, into the main memory 112, and performs various calculations by executing each code in a predetermined order. The main memory 112 is typically a volatile storage device such as DRAM (Dynamic Random Access Memory).

[0085] Furthermore, the program in this embodiment is provided stored on a computer-readable recording medium 120. The program in this embodiment may also be distributed over the Internet via a communication interface 117.

[0086] Specific examples of the storage device 113 include hard disk drives and semiconductor storage devices such as flash memory. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and mouse. The display controller 115 is connected to the display device 119 and controls the display on the display device 119.

[0087] The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, reads programs from the recording medium 120, and writes processing results from the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and other computers.

[0088] Specific examples of the recording medium 120 include general-purpose semiconductor memory devices such as CF (Compact Flash®) and SD (Secure Digital), magnetic recording media such as Flexible Disks, and optical recording media such as CD-ROMs (Compact Disk Read Only Memory).

[0089] Furthermore, the information processing device 10 in this embodiment can be implemented not by a computer on which a program is installed, but by using hardware corresponding to each part, such as electronic circuits. Moreover, the information processing device 10 may be partially implemented by a program and the remaining part by hardware. In this embodiment, the computer is not limited to the computer shown in Figure 9.

[0090] [Note] The following additional notes are disclosed regarding the embodiments described above. Some or all of the embodiments described above can be expressed by (Note 1) to (Note 27) below, but are not limited to the following descriptions.

[0091] (Note 1) If the target inspection image includes images of structural damage that is difficult to visually inspect, the extraction unit refers to the damage location included in the inspection history information, which stores past inspection history, and extracts past inspection images taken in past inspections where the distance from the inspection imaging position representing the position where the inspection image was taken or the target damage location to the past damage location is within a preset distance range. A first calculation unit calculates a first score for each of the extracted past inspection images based on the proximity of the distance, An information processing device having

[0092] (Note 2) The system includes a generation unit that generates output information for displaying on an output device an output that associates the extracted past inspection images with the first score. The information processing device described in Appendix 1.

[0093] (Note 3) The extraction unit further extracts past inspection images from the distance range that include images of damage to structures that are difficult to visually inspect, by referring to the inspection history information. The information processing device described in Appendix 1.

[0094] (Note 4) The first calculation unit calculates the reciprocal of the distance from the inspection imaging position to the damage position as the first score. The information processing device described in Appendix 1.

[0095] (Note 5) The first calculation unit calculates the first score by multiplying the reciprocal of the distance by a coefficient predetermined for each type of structural damage that is difficult to visually inspect. The information processing device described in Appendix 4.

[0096] (Note 6) The first calculation unit calculates the first score by multiplying the reciprocal of the distance by the number of feature point pairs that have been matched as being identical based on the shape of the structure in the image, the arrangement of the members, or both, extracted from the inspection image and the past inspection image, respectively, using this number as a coefficient. The information processing device described in Appendix 4.

[0097] (Note 7) The inspection image includes a determination unit that uses a machine learning model to determine whether or not there is damage to a structure that is difficult to inspect visually. The information processing device described in Appendix 1.

[0098] (Note 8) If the inspection image of the subject includes an image of visible structural damage, a matching unit performs a matching process between the inspection image and past inspection images taken in past inspections. A second calculation unit calculates a second score based on the results of the matching process, The generation unit generates output information to cause the output device to output an output that associates the past inspection image with the second score with the second score. The information processing device described in Appendix 2.

[0099] (Note 9) If the inspection image includes an image of damage showing a specific component exposed from the structure, and the corresponding past inspection image does not include such damage image, then a new deformation flag is set in the inspection image to indicate the presence of a new deformation. The information processing device described in Appendix 1.

[0100] (Note 10) Computers If the target inspection image includes images of structural damage that is difficult to visually inspect, the system refers to the damage location included in the inspection history information, which stores past inspection history, and extracts past inspection images taken during past inspections where the distance from the inspection imaging position representing the location where the inspection image was taken or the target damage location to the past damage location is within a predetermined distance range. For each of the extracted past inspection images, a first score is calculated based on the proximity of the distance. The system generates display information for displaying on a display device a display that associates the extracted past inspection images with the first score. Information processing methods.

[0101] (Note 11) The aforementioned computer, The system generates output information for displaying on an output device an output that associates the extracted past inspection images with the first score. The information processing method described in Appendix 10.

[0102] (Note 12) The aforementioned computer, From the past inspection images within the distance range, further, by referring to the inspection history information, past inspection images that include images of damage to structures that are difficult to visually inspect are extracted. The information processing method described in Appendix 10.

[0103] (Note 13) The aforementioned computer, The reciprocal of the distance from the inspection imaging position to the damage position is calculated as the first score. The information processing method described in Appendix 10.

[0104] (Note 14) The aforementioned computer, The first score is calculated by multiplying the reciprocal of the distance by a coefficient predetermined for each type of structural damage that is difficult to visually inspect. The information processing method described in Appendix 13.

[0105] (Note 15) The aforementioned computer, The first score is calculated by multiplying the reciprocal of the distance by the number of matching feature point pairs, which are determined to be identical based on the shape of the structure in the image, the arrangement of components, or both, extracted from the inspection image and the past inspection image, respectively. The information processing method described in Appendix 13.

[0106] (Note 16) The aforementioned computer, The inspection images are used to determine, using a machine learning model, whether or not there is damage to structures that are difficult to inspect visually. The information processing method described in Appendix 10.

[0107] (Note 17) The aforementioned computer, If the aforementioned inspection image includes images of visible structural damage, a matching process is performed between the aforementioned inspection image and past inspection images taken in previous inspections. A second score is calculated based on the results of the matching process described above. The generation unit generates output information to cause the output device to output an output that associates the past inspection image with the second score with the second score. The information processing method described in Appendix 11.

[0108] (Note 18) The aforementioned computer, If the inspection image includes an image of damage showing a specific component exposed from the structure, and the corresponding past inspection image does not include such damage image, then a new deformation flag is set in the inspection image to indicate the presence of a new deformation. The information processing method described in Appendix 10.

[0109] (Note 19) On the computer, If the target inspection image includes images of structural damage that is difficult to visually inspect, the system refers to the damage location included in the inspection history information, which stores past inspection history, and extracts past inspection images taken during past inspections where the distance from the inspection imaging position representing the location where the inspection image was taken or the target damage location to the past damage location is within a predetermined distance range. For each of the extracted past inspection images, a first score is calculated based on the proximity of the distance. The system generates display information for displaying on a display device a display that associates the extracted past inspection images with the first score. A program that executes a process.

[0110] (Note 20) To the aforementioned computer, The system generates output information for displaying on an output device an output that associates the extracted past inspection images with the first score. The program described in Appendix 19 that executes the process. Information processing methods.

[0111] (Note 21) To the aforementioned computer, From the past inspection images within the distance range, further, by referring to the inspection history information, past inspection images that include images of damage to structures that are difficult to visually inspect are extracted. The program described in Appendix 19 that executes the process.

[0112] (Note 22) To the aforementioned computer, The reciprocal of the distance from the inspection imaging position to the damage position is calculated as the first score. The program described in Appendix 21 that executes the process.

[0113] (Note 23) To the aforementioned computer, The first score is calculated by multiplying the reciprocal of the distance by a coefficient predetermined for each type of structural damage that is difficult to visually inspect. The program described in Appendix 22 that executes the process.

[0114] (Note 24) To the aforementioned computer, The first score is calculated by multiplying the reciprocal of the distance by the number of matching feature point pairs, which are determined to be identical based on the shape of the structure in the image, the arrangement of components, or both, extracted from the inspection image and the past inspection image, respectively. The program described in Appendix 22 that executes the process.

[0115] (Note 25) To the aforementioned computer, The inspection images are used to determine, using a machine learning model, whether or not there is damage to structures that are difficult to inspect visually. The program described in Appendix 119 that executes the process.

[0116] (Note 26) To the aforementioned computer, If the aforementioned inspection image includes images of visible structural damage, a matching process is performed between the aforementioned inspection image and past inspection images taken in previous inspections. A second score is calculated based on the results of the matching process described above. The generation unit generates output information to cause the output device to output an output that associates the past inspection image with the second score with the second score. The program described in Appendix 20 that executes the process.

[0117] (Note 27) To the aforementioned computer, If the inspection image includes an image of damage showing a specific component exposed from the structure, and the corresponding past inspection image does not include such damage image, then a new deformation flag is set in the inspection image to indicate the presence of a new deformation. The program described in Appendix 19 that executes the process.

[0118] Although the invention has been described above with reference to embodiments, the invention is not limited to the embodiments described above. Various modifications to the structure and details of the invention can be made that will be understood by those skilled in the art within the scope of the invention. [Industrial applicability]

[0119] As described above, this system can help efficiently inspect damage that is difficult to visually inspect. It is also useful in fields where structural inspection is required. [Explanation of Symbols]

[0120] 1. Structural deck 4 damage 10 Information Processing Devices 11 Extraction part 12 First Calculation Unit 13 Acquisition Department 14 First determination unit 15. Verification section 16 Second Calculation Unit 17 Second Judgment Unit 18 Generation part 20 Storage device 30 Terminal devices 31 Imaging device 32 Input devices 33 Output device 40 Networks 100 Systems 110 Computer 111 CPU 112 Main Memory 113 Storage device 114 Input Interface 115 Display Controller 116 Data Readers / Writers 117 Communication Interface 118 Input devices 119 Display device 120 recording media 121 Bus

Claims

1. If the target inspection image includes images of damage to a structure that is difficult to visually inspect, the extraction means refers to the damage location included in the inspection history information that stores past inspection history, and extracts past inspection images taken in past inspections where the distance from the inspection imaging position representing the position where the inspection image was taken or the target damage location to the past damage location is within a predetermined distance range. A first calculation means calculates a first score for each of the extracted past inspection images based on the proximity of the distance, An information processing device having

2. The system includes a generation means for generating output information to display on an output device an output that associates the extracted past inspection images with the first score. The information processing apparatus according to claim 1.

3. The first calculation means calculates the reciprocal of the distance from the inspection imaging position to the damage position as the first score. The information processing apparatus according to claim 1.

4. The first calculation means calculates the first score by multiplying the reciprocal of the distance by a coefficient predetermined for each type of structural damage that is difficult to visually inspect. The information processing apparatus according to claim 1.

5. The first calculation means calculates the first score by multiplying the reciprocal of the distance by the number of feature point pairs that have been matched as being identical based on the shape of the structure in the image, the arrangement of the members, or both, extracted from the inspection image and the past inspection image, respectively, using the number of feature point pairs as a coefficient. The information processing apparatus according to claim 4.

6. The inspection image includes a determination means that uses a machine learning model to determine whether or not there is damage to a structure that is difficult to inspect visually. The information processing apparatus according to claim 1.

7. If the aforementioned inspection image includes an image of visible structural damage, a matching means is provided to perform a matching process between the aforementioned inspection image and past inspection images taken in past inspections. A second calculation means for calculating a second score based on the results of the matching process, The generation means generates output information to cause the output device to output an output that associates the past inspection image with the second score with the second score. The information processing apparatus according to claim 2.

8. If the inspection image includes an image of damage showing a specific component exposed from the structure, and the corresponding past inspection image does not include such damage image, then a new deformation flag is set in the inspection image to indicate the presence of a new deformation. The information processing apparatus according to claim 1.

9. Computers If the target inspection image includes images of structural damage that is difficult to visually inspect, the system refers to the damage location included in the inspection history information, which stores past inspection history, and extracts past inspection images taken during past inspections where the distance from the inspection imaging position representing the location where the inspection image was taken or the target damage location to the past damage location is within a predetermined distance range. For each of the extracted past inspection images, a first score is calculated based on the proximity of the distance. The system generates display information for displaying on a display device a display that associates the extracted past inspection images with the first score. Information processing methods.

10. On the computer, If the target inspection image includes images of structural damage that is difficult to visually inspect, the system refers to the damage location included in the inspection history information, which stores past inspection history, and extracts past inspection images taken during past inspections where the distance from the inspection imaging position representing the location where the inspection image was taken or the target damage location to the past damage location is within a predetermined distance range. For each of the extracted past inspection images, a first score is calculated based on the proximity of the distance. The system generates display information for displaying on a display device a display that associates the extracted past inspection images with the first score. A program that executes a process.

Images