Information processing apparatus, information processing method, and program

Abstract

To reduce a user's labor in deformation analysis operations.SOLUTION: An information processing apparatus detects deformation occurring in a subject to be inspected from an image obtained by photographing the subject, and acquires a parameter based on the attribute of the detected deformation. According to a parameter for each of division areas obtained by dividing a display area set to the image obtained by photographing the subject into plurality, the information processing apparatus determines a display mode for each of the division areas.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to information processing technology for analyzing and displaying images.

Background Art

[0002] When inspecting structures such as highway, railway bridges, and tunnel walls, it is possible to investigate the presence, distribution, etc. of deformations such as concrete cracks by analyzing images of these structures taken by a camera. The detection results of deformations obtained from the images are associated with the structures to be inspected and stored as data, and it is also possible to analyze the progress and transition of deterioration due to aging by comparing past data with current data. Furthermore, in order to improve the efficiency of deformation analysis work, various techniques for automatically extracting deformations from images by image analysis processing have been proposed. In particular, attributes such as width are important for deformations such as cracks, and the ability to accurately determine the attributes of deformations such as cracks by image analysis processing is highly relevant to the efficiency of the analysis work. Note that Patent Document 1 discloses a method of associating the inspection results of roads with coordinates on a map using GPS position information and presenting the degree of deterioration and aging of the roads to the user.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, bridges, walls, etc. to be investigated are generally large structures and there are a large number of them. Therefore, when analyzing the deformation of structures, a large number of high-resolution images are handled as images taken of them and images of detection results obtained by image analysis. For this reason, for users who investigate the deformation, etc. of structures, a great deal of labor is required for tasks such as checking the images of those detection results and analyzing the comparison of deformation from the past to the present, and the burden on the users is extremely large.

[0005] Therefore, an object of the present invention is to make it possible to reduce the labor of the deformation analysis work by the user.

Means for Solving the Problems

[0006] The information processing apparatus of the present invention includes: detection means for detecting deformation occurring in a subject from an image of the subject to be inspected; parameter acquisition means for acquiring parameters based on the attributes of the deformation detected from the image; and display processing means for determining a display mode for each divided region by dividing a display region set for the image of the subject into a plurality of divided regions according to the parameters for each divided region.

Effects of the Invention

[0007] According to the present invention, it is possible to reduce the labor of the deformation analysis work by the user.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Each of the embodiments described hereinafter does not limit the present invention, and not all of the plurality of features described in this embodiment are essential for the solution means of the present invention, and those plurality of features may be arbitrarily combined. The configuration of the embodiment can be appropriately modified or changed according to the specifications of the device to which the present invention is applied and various conditions (usage conditions, usage environment, etc.). Also, a configuration may be formed by appropriately combining a part of each of the embodiments described later.

[0010] <Workflow at the time of detecting deformation of a structure> Before explaining the configuration and operation of the information processing apparatus according to the present embodiment, a general flow from analyzing an image obtained by photographing a structure to be inspected, such as a highway, a bridge or a tunnel of a railway, as a subject to detecting a deformation of the structure and analyzing the deformation will be described. In the present embodiment, it is assumed that an image obtained by photographing a structure to be inspected as a subject with a digital camera is analyzed to detect a deformation occurring in the inspection target area of the structure, and the size of the deformation, the progress and transition of the deformation, etc. will be analyzed. Note that in the present embodiment, the deformation refers to, for example, cracks, bulges, peeling, exposed reinforcing bars, rust, rust juice, water leakage, dripping water, sand streaks, corrosion, damage (defects), efflorescence, cold joints, precipitates, etc. occurring on the surface of the structure. In the present embodiment, in particular, cracks are cited as an example of the deformation, and a case where the secular change of the cracks is analyzed will be described. Also, in the present embodiment, the pier of a bridge such as a highway or a railway is cited as an example of the structure to be inspected, and the concrete wall surface of the pier is set as the inspection target area, and a case where a deformation is inspected using an image taken with a general digital camera will be described.

[0011] For example, when a field worker photographs the wall surface of a structure as an inspection target area, it is rare to be able to photograph with a sufficient image resolution that allows recognition of a deformation on the image and to include the entire wall surface, which is the inspection target area, in a single image. Therefore, in many cases, a part of the wall surface of the inspection target area is photographed in a large view, and the photographing is repeated while moving the photographing range of the large view little by little in the vertical and horizontal directions. Then, an image processing operation of joining together a plurality of images obtained by such repeated photographing is performed while appropriately performing scaling, rotation, projective transformation, color adjustment, removal of obstacles, etc., to generate a single combined image, which is saved as an image of the structure to be inspected.

[0012] In addition, the image processing operations of multiple shootings and stitching as described above are repeated according to the number of components in the drawing of the structure to be investigated. For example, when a pier with a square cross-section is the object of investigation, for each of the four wall surfaces that make up the pier, a set of four images generated by repeating the above-described multiple shootings and stitching image processing operations is obtained as an image of the pier being investigated. Note that for some structures to be investigated, a standard image resolution for the drawing is defined, and shooting is performed so as to satisfy that image resolution. However, for example, when a focused investigation is performed, even higher-definition images may be taken. And those images are saved together with the shooting date and time, the name of the structure, the position information of the structure, etc. at the time of shooting, at the time of image registration, and at the time of performing the deformation inspection by image analysis described later. Information such as these shooting date and time, the name of the structure, and the position of the structure is used as a clue when identifying images of each structure and comparing with past investigation results.

[0013] In this embodiment, by performing the image analysis process described later on the image generated by shooting the inspection target area of the structure as described above, the deformation occurring in the inspection target area of the structure is detected. However, unlike the method of manually finding and recording the deformation while looking at the image, when detecting the deformation by image analysis, there is a possibility of false detection or missed detection of the deformation. For this reason, an inspector (hereinafter referred to as the user) who inspects the deformation of the structure etc. displays the image of the structure on a monitor screen etc., visually checks the image of the structure, and corrects false detection or missed detection of the deformation as necessary. For example, when confirming the state of deformation at the time of inspection of the structure or the secular change of the deformation compared with the past, the attributes of the deformation are confirmed. For example, when the deformation is a crack, the attributes of the crack are the length and width of the crack. Therefore, the user identifies the length and width of the falsely detected or undetected crack on the image, and corrects them by adding them to the drawing or image, etc., to create a report on the inspection results of the deformation. However, the work of visually checking the image of the structure and confirming the detection result of the deformation, or the analysis work of comparing the deformation from the past to the present is a very burdensome work for the user.

[0014] In addition, when a high-resolution image of the entire structure to be inspected is displayed so as to fit on the monitor screen, the high-resolution image is often displayed in a reduced size according to the display resolution of the monitor screen. However, when the high-resolution image is displayed in a reduced size, small deformations and the like are flattened on the screen and cannot be visually recognized, so that deformations may not be found in the overhead display state where the high-resolution image of the entire structure is displayed on the monitor screen. For this reason, by enlarging and displaying a partial area of the image of the structure, small deformations and the like can be made visible. However, in this case, it is necessary to repeat the operation of enlarging and displaying a partial area in the image of the entire inspection target area of the structure while changing the position of the partial area to be enlarged, which is also a task that places a heavy burden on the user.

[0015] Therefore, in this embodiment, when the display area set for the inspection target area of the structure is displayed on the monitor screen, the display area is divided into a grid, and a display mode for highlighting each divided area is determined according to the parameters based on the attributes of the deformation in each divided area. As the parameters based on the attributes of the deformation, any one of the absolute amount of the attributes of the deformation, the change amount of the attributes of the deformation, the density of the deformation, the maximum value, the average value, the variance, etc., or a value obtained by combining two or more of them, the state of repair when the deformation is repaired, etc. can be used. In the division of the display area, the display area may be actually divided, but in this embodiment, an example of pseudo-division by drawing a grid on the display area is given. Also, in this embodiment, as the highlighting according to the display mode determined for each divided area, for example, a display that makes at least one of the display color, brightness, pattern, etc. different for each divided area, or a display of the value of the parameter based on the attributes of the deformation for each divided area, etc. can be cited. Also, in the highlighting for each divided area, the display that makes the display color different, etc. and the display of the value of the parameter based on the attributes of the deformation, etc. may be performed individually or may be combined. In this embodiment, by performing highlighting according to the parameters based on the attributes of the deformation for each divided area (for each grid), it is easy for the user to overlook and find important parts to be compared and confirmed in the entire inspection target area of the structure, and the labor of the user's deformation analysis work can be reduced.

[0016] <Functional Configuration> FIG. 1 is a diagram showing the functional configuration of an information processing apparatus 101 that detects a deformation in the inspection target area of a structure from an image obtained by photographing the structure to be inspected as described above, and performs highlighting for each divided area according to the parameters based on the attributes of the detected deformation. The functional configuration shown in FIG. 1 may be realized by an information processing apparatus such as a single computer, or may be realized as an information processing system including a plurality of computers, servers, etc. connected by a LAN or the like. In the following description, an example in which the information processing apparatus 101 of this embodiment is realized by a computer will be given for explanation.

[0017] In the information processing apparatus 101 shown in FIG. 1, the image storage unit 112 stores an image of a structure to be investigated taken with a digital camera. The user input unit 115 acquires information such as various instructions input by the user. In the following description, unless otherwise explicitly stated, the description of the input of instructions or the like from the user being performed through the user input unit 115 is omitted. The image management unit 111 performs storage of an image of a structure to be investigated in the image storage unit 112 and management of the image stored in the image storage unit 112. That is, the image management unit 111 has functions such as creating a folder, registering an image in the folder, deleting a folder or an image, creating a list of folders and images, etc., and stores the information of those folders and the data of the images in the image storage unit 112. Also, based on an instruction from the user, the image management unit 111 reads out the image stored in the image storage unit 112 and sends it to the image analysis unit 113. For example, when analyzing the secular change of deformation of a specific structure, the image management unit 111 reads out a first image and a second image of the specific structure taken at different times from the image storage unit 112 and sends them to the image analysis unit 113.

[0018] Here, in the present embodiment, the first image and the second image are images of the same structure to be investigated although taken at different times. Also, in the present embodiment, it is assumed that the second image was taken earlier than the first image, and the time difference from the time of taking the second image to the time of taking the second image is a time difference in units of years such as 1 year or several years. Note that in the present embodiment, it is assumed to be a time difference in units of years, but in the case of an object to be investigated where the change in deformation occurs in a short time, different from, for example, the bridge pier of a bridge, it may be in units of days, weeks, or even seconds. In the following embodiments, it is assumed that the first image is an image taken at the time closest to the present, but of course, the first image is not limited to an image taken at the time closest to the present, and it may be an image taken at any past time as seen from the present.

[0019] The image analysis unit 113 performs image analysis for detecting deformation of a structure on the images managed in units of folders by the image management unit 111. In the present embodiment, an example of using an AI model for image analysis for detecting deformation of a structure from an image is given. Of course, image analysis may be performed by other methods than using an AI model. When an image to be analyzed is specified by the user and an instruction to start execution is further input, the image analysis unit 113 performs image analysis on the image to detect deformation. For example, when investigating the secular change of deformation in a specific structure, the image analysis unit 113 acquires a first image and a second image of the specific structure taken at different times from the image storage unit 112, and performs image analysis on the first and second images to detect deformation. Then, the image analysis unit 113 stores the detection result of deformation by image analysis in the analysis result storage unit 114.

[0020] The region extraction unit 121 performs a process of extracting an image of a region to be inspected for deformation from the image on which image analysis has been performed by the image analysis unit 113 and the detection result has been stored in the analysis result storage unit 114. In the present embodiment, for example, when the pier of a bridge is the object of investigation, the region to be inspected for deformation is assumed to be the region of the wall surface of the pier, and the region extraction unit 121 extracts an image region corresponding to the wall surface of the pier from the image. Note that the region extraction unit 121 may extract a region to be inspected that is specified in advance according to the object of investigation, or may extract a region to be inspected arbitrarily specified by the user. For example, when inspecting the secular change of deformation of a specific structure, the region extraction unit 121 extracts a region to be inspected for deformation from the first and second images of the specific structure taken at different times. Details of the process of extracting the region to be inspected performed by the region extraction unit 121 will be described later. Then, the region extraction unit 121 sends the image of the extracted region to be inspected to the region alignment unit 122.

[0021] The area alignment unit 122 performs a process of aligning at least the positions and shapes of two inspection target areas respectively extracted from first and second images of a specific structure taken at different times, and further, if necessary, also aligning resolution, brightness, etc. That is, when comparing the first and second images, it is desirable that these two images are images taken while facing the inspection target area (such as the wall surface of a pier) of the structure, with an appropriate resolution, a constant brightness, and further, without camera shake, defocus, etc. However, the on-site operator who takes pictures of the structure takes pictures of the inspection target area of the structure by adjusting to the distance and angle of view calculated in advance according to the characteristics of the digital camera used for shooting, but it is not always possible to take pictures from an appropriate position. Moreover, when taking pictures for the purpose of comparing the secular changes of deformation, it is practically impossible to take pictures of the inspection target area of the structure under exactly the same conditions as when it was taken in the past. That is, although the first and second images are images of the same structure, since they are images taken at different times, they may not be images taken from the same position, in the same direction, and with the same angle of view, and there is a possibility that the positions and shapes of the inspection target areas are shifted. For this reason, the area alignment unit 122 performs an area alignment process to align the positions and shapes of the corresponding two inspection target areas, and further, resolution, brightness, etc. The details of the area alignment process performed by the area alignment unit 122 will be described later.

[0022] The parameter acquisition unit 130 acquires parameters based on the attributes of deformation detected from an image of a specific structure. In the present embodiment, the attributes of deformation vary depending on the type of deformation. For example, when the deformation is a crack, the width and length of the crack, etc. are the attributes. In addition, for deformations other than cracks, such as floating, peeling, rebar exposure, rust, rust juice, water leakage, water dripping, sand streaks, corrosion, damage (defect), efflorescence, cold joint, precipitate, etc., it is also possible to acquire the attributes of each deformation. In the case of the configuration example shown in FIG. 1, the parameter acquisition unit 130 is configured to include an absolute quantity acquisition unit 131 and a relative quantity acquisition unit 132.

[0023] The absolute quantity acquisition unit 131 acquires, as parameters, the absolute quantities of the attributes of each change state within the inspection target area described above, that is, the values obtained by quantifying the attributes of the change state. In this embodiment, cracks are cited as an example of the change state, and the attributes of the cracks are related to the size of the cracks, such as the width and length of the cracks as described above. The absolute quantity acquisition unit 131 acquires, as a parameter, the change state absolute quantity obtained by quantifying the size of the crack. Although the description is omitted in this embodiment, the attributes of other change states other than cracks can also be quantified. In the case of this embodiment, the absolute quantity acquisition unit 131 acquires, as change state absolute quantities, the values obtained by quantifying the attributes of each change state detected from the first image and the values obtained by quantifying the attributes of each change state detected from the second image, respectively.

[0024] The relative quantity acquisition unit 132 acquires, as the relative quantity of the change state, the difference between the change state absolute quantities respectively acquired from two corresponding inspection target areas of the first image and the second image. The relative quantity of the change state corresponds to the amount of change when the change state changes (increases or decreases) over time.

[0025] The display processing unit 133 generates display data such as the images managed by the image management unit 111, the display data of the management screen related to the start of execution of image analysis by the image analysis unit 113, the detection results by image analysis, and the display data of the change state inspection results of the structure. Then, the display processing unit 133 switches the management screen and the display data of the change state inspection results according to an instruction from the user and displays them on the monitor screen of the display device. Thereby, the user can view the management screen and the change state inspection results. Thus, the display processing unit 133 generates display data and displays it on the monitor screen of the display device. However, in the following description, for the sake of simplicity, the description of the generation of display data and the display device is omitted, and it is described as if the display processing unit 133 displays.

[0026] Further, the display processing unit 133 performs processing to determine the display mode for each of the divided regions obtained by dividing a display region set for an image of a specific structure into a plurality of regions, according to parameters based on the deformation attributes of each divided region. The display region may be set based on an instruction from the user, or an inspection target region estimated from an image of the structure by an estimation process using an AI model or the like, or a partial region within the inspection target region may be automatically set. In the case of the present embodiment, the display processing unit 133 sets a display region from an image of a specific structure by performing enlarged or reduced display of the image according to an instruction from the user. For example, when enlarging and displaying a part of an image of a structure, the display processing unit 133 enlarges and displays the display region set by the user for the image, and divides the display region into a plurality of regions according to the magnification at the time of display. Further, the display processing unit 133 of the present embodiment determines different display modes for each divided region according to the absolute amount of deformation and performs highlighting display, and also determines different display modes for each divided region according to the relative amount of deformation and performs highlighting display. Details such as the management screen in the display processing unit 133, the display of the deformation inspection result, the region division according to the magnification at the time of display, and the highlighting display for each divided region will be described later.

[0027] The connection unit 141 connects the above-described respective functional units. For example, when each functional unit in FIG. 1 is realized by one computer or the like, the connection unit 141 corresponds to an internal bus or the like. Also, for example, when each functional unit in FIG. 1 is realized by a system including a plurality of computers, servers, etc., the connection unit 141 corresponds to a network such as a LAN.

[0028] FIG. 2 is a diagram showing an example of the hardware configuration of the information processing apparatus 101 according to the present embodiment. For example, when each functional unit in FIG. 1 is realized by one computer, FIG. 2 corresponds to the configuration of the computer, and when realized by a system including a plurality of computers, servers, etc., FIG. 2 corresponds to the respective configurations of those computers and servers.

[0029] In FIG. 2, the CPU 205 controls the operation of the entire computer. The RAM 204 temporarily stores the information processing program, images, image analysis results, AI models, etc. according to this embodiment. The large-capacity memory 202 is, for example, an HDD or an SSD, etc., and stores programs, image data, image analysis results, AI models, etc. in a long term. Various programs including the OS (Operating System) and the information processing program according to this embodiment are stored in the large-capacity memory 202 or the ROM 203. The information processing program of this embodiment stored in the large-capacity memory 202 or the ROM 203 is loaded into the RAM 204, and when the CPU 205 executes the program, each functional unit shown in FIG. 1 is realized. Also, viewing of images and analysis results, acquisition of instructions such as various settings by the user, and corresponding operations, etc. are realized by the CPU 205 executing the application program operating on the OS and loaded into the RAM 204.

[0030] The input device 206 is a device including a mouse, a keyboard, a touch panel, etc. for the user to input various instructions, etc. The CPU 205 acquires the user instructions input from the user via the input device 206 and performs processing according to the user instructions. The output device 207 is a device for outputting data and information to an external device. The output device 207 may include a display device having a monitor screen such as a liquid crystal display or an organic EL display. The display processing unit 133 shown in FIG. 1 causes the display device of the output device 207 to display the management screen and the deformation inspection results of the structure to be inspected described above.

[0031] The communication device 208 is a device for acquiring data, programs, etc. from an external device. Also, the communication device 208 communicates with other computers, servers, devices, etc. connected to the network. The bus 201 connects the above-described components to each other and enables data exchange between the components. In this embodiment, an example is given in which each functional unit shown in FIG. 1 is realized by the CPU 205 in FIG. 2 executing the information processing program of this embodiment. However, part or all of each functional unit in FIG. 1 may also be realized by a hardware configuration such as a circuit.

[0032] <Execution of Image Registration and Image Analysis> FIG. 3 is a schematic diagram showing an example of a management screen 301 related to image registration and management by the image management unit 111 and execution of image analysis by the image analysis unit 113. In the information processing apparatus 101 of this embodiment, the display processing unit 133 generates display data of the management screen and sends it to the display device, whereby the management screen 301 in FIG. 3 is displayed.

[0033] The management screen 301 is divided into a folder list panel 302 and an image list panel 303. Also, on the management screen 301, a new creation button 312, a registration button 322, an analysis button 323, etc. are arranged, and these buttons can be virtually pressed by a user operation through the user input unit 115. Note that the pressing operation for these buttons is performed, for example, by a mouse click or a touch on a touch panel.

[0034] In this embodiment, the image management unit 111 manages images, for example, for each folder, and the display processing unit 133 displays the folder list 311 of the images managed by the image management unit 111 in the folder list panel 302. Also, a new creation button 312 is arranged in the folder list panel 302. When the new creation button 312 is pressed by the user, the display processing unit 133 pops up and displays a folder new creation window (not shown). Then, when information necessary for creating a new folder, such as a folder name, is input by the user to the folder new creation window, the image management unit 111 adds the folder as a newly created folder.

[0035] Also, when the user designates a folder to be used for deformation inspection from among the folder list 311 in the folder list panel 302, the display processing unit 133 displays the image list 321 of each image registered in that folder within the image list panel 303. A check box is associated with each image in the image list 321, and the user can select that image as an object for image analysis by checking the check box. Also, a registration button 322 and an analysis button 323 are arranged within the image list panel 303. When the user presses the registration button 322 while one or more of the images in the image list 321 are checked, the image management unit 111 accepts the input of an image file from the user through the user input unit 115 and registers it as a new image. Also, when the user presses the analysis button 323 while one or more of the images in the image list 321 are checked, the image analysis unit 113 executes image analysis processing on the checked images.

[0036] <Extraction Process and Region Matching Process for Inspection Target Region> In the present embodiment, in order to enable comparison of secular changes in deformation in the inspection target regions of the same structure shown in the first and second images taken of a specific structure at different times, the region extraction unit 121 extracts the corresponding inspection target regions of the structure from those two images. Also, the region matching unit 122 performs a region matching process for matching the sizes, positions, and shapes of the two corresponding inspection target regions extracted by the region extraction unit 121 from the first and second images. Note that since it is difficult to completely match the sizes, positions, and shapes of the two corresponding inspection target regions, in the present embodiment, a simple method of performing approximate region matching is adopted within a range where there is no inconvenience for the purpose of comparing and confirming secular changes from an overhead view.

[0037] FIG. 4 is a diagram used to explain a case where, for example, from an image 401 of a bridge, an image area of one wall surface of a pier 402 is extracted as an inspection target area 412. In the image 401 of the bridge, in addition to the pier 402, unnecessary objects such as the ground and floor slab are reflected, and further, a pier 403 that is not an inspection target is also reflected right behind the pier 402 that is the inspection target. Since this pier 403 that is not an inspection target is a structure similar to the pier 402 that is the inspection target, it is likely to cause false detection. Therefore, the area extraction unit 121 extracts only the image area of the wall surface corresponding to the front of the pier 402 as the inspection target area 412, and generates an image 411 in a state where the other areas are masked by a mask image 413 as the background image area.

[0038] Note that the mask image 413 may be not only an image that completely masks the background other than the inspection target area 412, but also a semi-transparent image that slightly transmits the background so that the user can recognize which pier of the bridge the inspection target area 412 corresponds to. Known techniques can be used for the image processing of extracting only the inspection target area 412 from the image 401 and masking the others as the background with the mask image 413. As an example, AI technology may be used, or pattern image processing technology using, as feature amounts, the color and texture of the concrete of the wall surface that is the inspection target area, the focus degree of the camera focus, the size of the texture due to perspective, etc. may also be used. In addition, the user may manually specify the range to be the inspection target area through the user input unit 115 while viewing the image.

[0039] FIG. 5 is a diagram used to explain a case of performing projective transformation as an example of area alignment processing for an inspection target area 501 and an inspection target area 502 extracted from an image of a bridge. Note that the inspection target area 501 shows an example of an inspection target area extracted from a first image of a pier taken from an oblique direction, and the inspection target area 502 shows an example of an inspection target area extracted from a second image of a pier taken from a looking-up direction. The area alignment unit 122 performs projective transformation processing on the inspection target areas 501 and 502 respectively to generate an inspection target area 503 in a direction substantially facing the wall surface of the pier.

[0040] Here, an example of projective transformation of an inspection target region 501 extracted from a first image taken of a bridge from an oblique direction to an inspection target region 503 in a direction substantially facing the pier wall surface will be described. For example, the region matching unit 122 generates an inspection target region 503 in a direction substantially facing the pier wall surface by normalizing a quadrangle distorted obliquely based on four points 511 to 514 surrounding the inspection target region 501 into a rectangle by projective transformation. Processing based on such four points of a quadrangle is known. The four points 511 to 514 of the quadrangle may be estimated using, for example, AI technology, or may use, for example, four intersection points of straight lines 521 to 514 in the image detected by Hough transformation. Also, for example, the four points 511 to 514 of the quadrangle may be specified by a user's operation of a cursor 530 through the user input unit 115. These processes can be similarly applied to the inspection target region 502 extracted from the second image taken from the upward-looking direction of the pier.

[0041] The region matching unit 122 performs the above-described normalization on the inspection target regions extracted by the region extraction unit 121 from the first and second images, and performs a process of aligning the positions of the normalized inspection target regions. FIG. 6 is a diagram showing a series of processes from extracting an inspection target region from an image, normalizing it, and performing alignment. In FIG. 6, an image 601 is an example of a first image taken of a bridge, which is an example of a structure to be inspected, and an image 611 is an example of a second image taken of the same bridge in the past. Note that the wall surface of the pier shown in the first image 601 has more deformation due to aging compared to the wall surface of the pier shown in the second image 611.

[0042] The region extraction unit 121 extracts an image region of the wall surface of the pier from the first image 601 as an inspection target region 602. Similarly, the region extraction unit 121 extracts an image region of the wall surface of the pier from the second image 611 as an inspection target region 612. The region alignment unit 122 normalizes the inspection target region 602 extracted from the first image 601, and similarly normalizes the inspection target region 612 extracted from the second image 611. Further, the region alignment unit 122 makes these inspection target regions have a certain size, resulting in the inspection target region 603 and the inspection target region 613.

[0043] These normalized and sized inspection target regions 603 and inspection target region 613 are each composed of quadrilaterals of approximately the same size that enclose their respective regions. Therefore, by performing the overlay 620, approximate alignment can be achieved. That is, the region extraction unit 121 performs approximate alignment of the inspection target regions 602 and inspection target region 612 by the overlay 620. In the overlay 620, the deformations within the inspection target region 603 are represented by dotted lines, and the deformations within the inspection target region 612 are represented by solid lines. In actuality, due to the conditions during shooting, the accuracy of region extraction, and the accumulation of errors due to projective transformation, there may be some slight deviations in the contours of the inspection target regions and the positions of the deformations. However, for the purpose of comparing and confirming secular changes from an aerial view, there are no inconveniences.

[0044] <Region division according to the magnification at the time of display> FIG. 7 is a diagram used to explain the region division according to the magnification at the time of display and the display process of the divided regions divided by the region division. As described above, in order to view the deformation of the structure from an aerial view, it is necessary to display it in a reduced scale so that the entire inspection target region of the structure fits within the screen. On the other hand, in order to confirm the details of the deformation grasped by the aerial observation, it is necessary to display the image in a stepwise enlarged scale.

[0045] The image 701 in FIG. 7 shows an example of an image displayed at a magnification such that the entire inspection target region extracted by the region extraction unit 121 can be viewed from an aerial view and the entire inspection target region fits within the screen. In the present embodiment, when the entire inspection target region is displayed at a magnification that fits within the screen, the image 701 is divided into a grid pattern of, for example, 4 vertical divisions and 4 horizontal divisions.

[0046] The user can input an instruction to enlarge a part of the image through the user input unit 115. For example, the user designates the area 702 enclosed by the dotted line in the image 701 as the display area, and when enlarging and displaying the area 702, the display processing unit 133 displays an image 711 obtained by enlarging the area 702 by a factor of 2 on the screen. At this time, the display processing unit 133 also divides the enlarged image 711 by a factor of 2 vertically and horizontally in the same manner as the image 701. Further, when the user designates the area 712 enclosed by the dotted line in the image 711 as the display area and enlarges and displays the area 712, the display processing unit 133 displays an image 721 obtained by enlarging the inside of the area 712 by a factor of 2. Also in this case, the display processing unit 133 divides the image 721 vertically and horizontally in the same manner. That is, the display processing unit 133 always divides the display area into a fixed number of grids regardless of the magnification at the time of display. Note that the vertical and horizontal divisions into four are just examples, and it may be divided more finely, or the number of divisions may be different between the vertical and horizontal directions.

[0047] <Highlight display for each divided region according to parameters based on the attributes of the change> FIG. 8 is a diagram used to explain an example in which, as parameters based on the attributes of the change, the absolute amount of change and the relative amount of change, which is the amount of change in the attributes of the change, are obtained, and highlighting is performed in different display modes for each divided region according to these absolute amounts of change and relative amounts of change. As described above, the display processing unit 133 performs highlighting with different display modes for each divided region according to the absolute amount of change, and also performs highlighting with different display modes for each divided region according to the relative amount of change.

[0048] In FIG. 8, image 803 shows an example of an image obtained by dividing the inspection target area after extraction from the first image, normalization, and region alignment. Similarly, image 813 shows an example of an image obtained by dividing the inspection target area after extraction from the second image, normalization, and region alignment. Further, image 822 shows an image enlarged by a factor of 2 because a part of image 803 (e.g., the lower right part of the wall surface) is designated as the display area. Similarly, image 832 shows an example of an image enlarged by a factor of 2 because a part of image 813 (the lower right part of the wall surface) is designated as the display area. The image 822 obtained by enlarging a part of image 803 by a factor of 2 is divided into regions (vertically and horizontally divided into 4 in the above example) as described above. Similarly, the image 832 obtained by enlarging a part of image 813 by a factor of 2 is also divided into regions (vertically and horizontally divided into 4).

[0049] In FIG. 8, image 840 shows a heatmap image in which highlighting is performed in different display modes for each divided region according to the absolute amount of deformation of image 803. Similarly, image 841 shows a heatmap image in which highlighting is performed in different display modes for each divided region according to the absolute amount of deformation of image 813. Further, image 850 shows a heatmap image in which highlighting is performed in different display modes for each divided region according to the absolute amount of deformation of the doubled image 822. Similarly, image 851 shows a heatmap image in which highlighting is performed in different display modes for each divided region according to the absolute amount of deformation of the doubled image 813.

[0050] In the images 840, 841, 850, and 851 shown in FIG. 8, as examples of highlighting for each divided region, examples are shown in which the display color, etc. is made different for each divided region, that is, highlighting by color coding is combined with the display of numerical values representing the absolute amount of deformation. Here, for example, when performing highlighting using numerical values representing the absolute amount of deformation for each divided region, the display processing unit 133 acquires numerical values representing the absolute amount of deformation for each divided region based on the absolute amount of deformation acquired by the absolute amount acquisition unit 131 regarding the deformation of the inspection target region. As an example, the numerical values representing the absolute amount of deformation for each divided region can include the total extension or total area value of all deformations existing within the divided region with respect to the size of the divided region. Additionally, for example, when the deformation is a crack, the numerical values representing the absolute amount of deformation for each divided region may adopt values such as the width of the crack or the degree of damage that quantifies the necessity of repair considering the influence of other surrounding deformations. Assume that the numerical values described in each divided region of images 840, 841, 850, and 851 in FIG. 8 are numerical values representing the absolute amount of deformation for each divided region, such as the value of the total extension.

[0051] Also in FIG. 8, image 870 shows a heatmap image with highlighting performed in different display modes for each divided region according to the relative amount of deformation between image 803 and image 813. Similarly, image 880 shows a heatmap image with highlighting performed in different display modes for each divided region according to the relative amount of deformation between image 850 and image 851.

[0052] In images 870 and 880 shown in FIG. 8, as examples of highlighting for each divided region, examples are shown that combine highlighting by color-coding with different display colors for each divided region and the display of numerical values representing the relative amount of deformation. Here, when performing highlighting using numerical values representing the relative amount of deformation for each divided region, the display processing unit 133 obtains the difference between the numerical values representing the absolute amount of deformation for each divided region as the numerical value representing the relative amount of deformation for each divided region. That is, the display processing unit 133 calculates the difference between the numerical values representing the absolute amount of deformation of the corresponding divided regions of the image 840 and the image 841 in FIG. 8, and obtains it as the numerical value representing the relative amount of deformation for each divided region of the image 870. Similarly, the display processing unit 133 calculates the difference between the numerical values representing the absolute amount of deformation of the corresponding divided regions of the image 850 and the image 851 in FIG. 8, and obtains it as the numerical value representing the relative amount of deformation for each divided region of the image 880. When calculating the difference between the numerical values representing the absolute amount of deformation of the corresponding divided regions, the display processing unit 133 takes the difference between the numerical value of the divided region corresponding to the first image as the minuend and the numerical value of the divided region corresponding to the second image, considering the time series when the image was taken. As described above, highlighting based on the relative amount of deformation due to aging for each divided region is realized.

[0053] <Example of Selection of Detection Results, Deformation Confirmation, and Highlighting> FIG. 9 is a schematic diagram showing an example of a detection result selection screen 901 in which a user can select a detection result to be the target of deformation confirmation from among a plurality of detection results stored in the analysis result storage unit 114. In the information processing apparatus 101 of the present embodiment, the display processing unit 133 generates the display data of the detection result selection screen 901 and sends it to the display device, thereby displaying the detection result selection screen 901 in FIG. 9.

[0054] The detection result selection screen 901 displays folders corresponding to each detection result stored in the analysis result storage unit 114 in the form of a list, and the list of folders is divided and displayed in a first folder list panel 911 and a second folder list panel 921. The folders of a plurality of detection results displayed in the first folder list panel 911 and the second folder list panel 921 are displayed in a selectable state by the user. Note that a folder selected by the user within the first folder list panel 911 cannot be selected within the second folder list panel 921, and conversely, a folder selected within the second folder list panel 921 cannot be selected within the first folder list panel 911. Also, a detection result confirmation button 931 is arranged on the detection result selection screen 901, and the button can be pressed by the user.

[0055] When the user selects one folder from the first folder list panel 911, the display processing unit 133 acquires the detection result recorded in the selected folder from the analysis result storage unit 114. Then, the display processing unit 133 displays the acquired detection result in the first detection result list panel 912 arranged on the right side. Radio buttons are associated with each detection result in the detection result list panel 912, and the user can select a detection result by selecting any one of the radio buttons.

[0056] Similarly, when the user selects one folder from the second folder list panel 921, the display processing unit 133 acquires the detection result of the selected folder from the analysis result storage unit 114 and displays it in the second detection result list panel 922 arranged on the right side. Radio buttons are also associated with each detection result in the detection result list panel 922, and the user can select a detection result by selecting any one of the radio buttons.

[0057] Here, in the present embodiment, the detection result selected from the first detection result list panel 912 is used as the first detection result obtained by image analysis of the first image, and the detection result selected from the second detection result list panel 922 is used as the second detection result obtained by image analysis of the second image. Of course, conversely to this example, the detection result selected from the first detection result list panel 912 may be used as the second detection result, and the detection result selected from the second detection result list panel 922 may be used as the first detection result.

[0058] Then, after detection results are respectively selected from the first detection result list panel 912 and the second detection result list panel 912, when the detection result confirmation button 931 is pressed, the display processing unit 133 causes the screen display to transition from the detection result selection screen 901 to the detection result confirmation screen.

[0059] The detection result confirmation screen 1000 shown in FIG. 10 shows an example of an initial display screen that has transitioned from the detection result selection screen 901. In the following description, it is assumed that the first image described above is an image taken at the point in time closest to the present, and this first image is referred to as the "current image", and the second image taken before the first image is taken is referred to as the "previous image".

[0060] The detection result confirmation screen 1000 is provided with an image display area 1001 and a button area 1009. The button area 1009 is provided with a current image button 1004, a previous image button 1005, a current image heat map button 1006, a previous image heat map button 1007, and an aging change heat map button 1008. In the image display area 1001, an image in which the inspection target area of the structure as described above is extracted and background masking is performed by the mask image 1003 is displayed. As shown in FIG. 10, the mask image 1003 shows an example in which a semi-transparent mask image is used so that the user can recognize which pier of the bridge the inspection target area 10022 corresponds to. In the initial display when the display processing unit 133 transitions to the detection result confirmation screen 1000, the current image (image of the inspection target area with the background masked) is displayed in the image display area 1001.

[0061] Here, on the detection result confirmation screen 1000, when the previous image button 1005 is pressed by the user, the display processing unit 133 displays the previous image (an image of the inspection target area with the background masked) in the image display area 1001. Also, when the current image button 1004 is pressed by the user while the previous image is being displayed in the image display area 1001, the display processing unit 133 displays the current image (an image of the inspection target area with the background masked) in the image display area 1001. That is, the user can instruct the switching of the images displayed in the image display area 1001 by appropriately pressing the current image button 1004 and the previous image button 1005. And the user can confirm whether the extraction of the inspection target area of the structure and the like is appropriately executed by switching between the current image and the previous image and comparing them on the image display area 1001.

[0062] Also, on the detection result confirmation screen 1000, when the aging change heat map button 1008 is pressed by the user, the display processing unit 133 causes the screen display to transition from the detection result confirmation screen 1000 to the overall display heat map screen 1010.

[0063] When transitioning to the overall display heat map screen 1010, the display processing unit 133 switches the aforementioned image display area 1001 to the heat map display area 1011. Note that the button area 1009 is not changed. And the display processing unit 133 displays, within the heat map display area 1011, an image in a state where a grid indicating the divided areas and the aging change heat map 1012 are superimposed after removing the background portion outside the inspection target area. That is, within the heat map display area 1011, an image in a state where the image 870 of the aging change heat map, which is highlighted in a different display mode for each divided area according to the deformation relative amount as described in FIG. 8 above, is superimposed on the image facing the inspection target area is displayed. In the example of FIG. 10, the numerical values representing the deformation relative amount for each divided area shown in the image 870 of FIG. 8 are omitted.

[0064] By viewing the aging heat map 1012 within the heat map display area 1011, the user can check in which divided area the aging change of deformation is large, or conversely, in which area the aging change is small. In the case of the overall display heat map screen 1010 shown in FIG. 10, among the total of 8 divisions, 4 vertical and 4 horizontal, for example, in the vicinity of the four divided areas in the lower right, that is, the divided area surrounded by the dotted line 1018 in the figure, an example where the aging change is large is shown.

[0065] Here, for example, assume that the user designates the area indicated by the dotted line 1018 as the display area to be enlarged, and further presses the aging heat map button 1008. In this case, the display processing unit 133 causes the screen display to transition from the overall display heat map screen 1010 to the enlarged display heat map screen 1020. The enlarged display heat map screen 1020 is provided with a heat map display area 1021 and a button area 1009.

[0066] At this time, the display processing unit 133 displays an image in which a grid indicating divided areas corresponding to the magnification at the time of display and the aging heat map 1022 are superimposed on an enlarged image of the display area within the range specified by the user on the overall display heat map screen 1010. In the example of FIG. 10, since the display area specified by the user on the overall display heat map screen 1010 is a range including the four divided areas in the lower right among the 8 divided areas, the magnification at the time of display is 2 times. That is, within the heat map display area 1021, an image in a state where an image 880 of the aging heat map, which is highlighted in different display modes for each divided area according to the deformation relative amount described in FIG. 8, is superimposed on a 2-fold enlarged image corresponding to the range specified by the user is displayed. Also in this example, the numerical values representing the deformation relative amount for each divided area shown in the image 880 of FIG. 8 are omitted.

[0067] By viewing the aging heatmap 1022 on the enlarged display heatmap screen 1020, the user can check details such as in which divided region the aging change of deformation is large or, conversely, in which region the aging change is small. That is, in the case of the enlarged display heatmap screen 1020 illustrated in FIG. 10, the user can grasp that the vicinity of the three deformations in the upper right of the screen is the part with a large aging change.

[0068] Although not illustrated in FIG. 10, when a further enlarged range is specified for the enlarged display heatmap screen 1020 and the aging change heatmap button 1008 is pressed, the display processing unit 133 causes a transition to an enlarged display heatmap screen with the range further enlarged.

[0069] So far, an example in which the aging change heatmap is displayed has been described. On the other hand, when the current image heatmap button 1006 is pressed by the user on the detection result confirmation screen 1000, the display processing unit 133 causes a transition of the screen display from the detection result confirmation screen 1000 to the current image heatmap screen 1100 shown in FIG. 11(A).

[0070] Even when the transition is made to the current image heatmap screen 1100, the display processing unit 133 switches the above-described image display area 1001 to the heatmap display area 1101. Since each button included in the button area 1109 is the same as each button included in the button area 1009 described above, their descriptions are omitted.

[0071] At this time, the display processing unit 133 displays, within the heat map display area 1101, an image in a state where a grid indicating divided areas and a deformation absolute amount heat map 1121 with different display modes for each divided area according to the absolute deformation amount are superimposed on the image of the inspection target area of the current image. Note that when the display processing unit 133 displays the deformation absolute amount heat map 1121 corresponding to the current image, it may not perform projective transformation, masking with a mask image, etc. on the inspection target area of the structure shown in the current image. In this case, if the inspection target area of the structure is not facing directly at the time of shooting, the inspection target area is displayed in the state at the time of shooting, and the grids representing the respective divided areas are displayed as distorted rectangles. The current image heat map screen 1100 shown in FIG. 11(A) shows an example when projective transformation, masking with a mask image, etc. are not performed. Note that in the example of FIG. 11(A) as well, the numerical values representing the absolute deformation amounts for each divided area are omitted. Of course, the display processing unit 133 may perform projective transformation, masking with a mask image, etc. on the inspection target area. In that case, within the heat map display area 1101, an image in a state where an image 840 of the deformation absolute amount heat map highlighted with different display modes for each divided area according to the absolute deformation amount, as described in FIG. 8 above, is superimposed on the inspection target area of the current image is displayed. Although not shown, in the current image heat map screen 1100 as well, when the user instructs an enlargement of the display area, the display processing unit 133 performs an enlarged display in the same manner as described above.

[0072] By looking at the deformation absolute amount heat map 1121 within the heat map display area 1101, the user can confirm which divided area has a large or small absolute deformation amount in the current image.

[0073] Also, assume that the previous image heat map button 1007 is pressed by the user on the detection result confirmation screen 1000, or the previous image heat map button 1107 is pressed on the current image heat map screen 1100 in FIG. 11. In this case, the display processing unit 133 causes the screen display to transition to the previous image heat map screen 1130 shown in FIG. 11(B).

[0074] At this time, the display processing unit 133 displays, within the heat map display area 1101, a grid indicating divided areas and a deformation absolute amount heat map 1131 with different display modes for each divided area in terms of the deformation absolute amount, superimposed on the image of the inspection target area of the previous image. Note that, also in the case of the previous image heat map screen 1130, the display processing unit 133 may not perform projective transformation or masking with a mask image or the like on the inspection target area of the structure shown in the previous image. The previous image heat map screen 1130 shown in FIG. 11(B) shows an example of the case where projective transformation or masking with a mask image or the like is not performed. In the example of FIG. 11(B), the numerical values representing the deformation absolute amount for each divided area are also omitted. Also, as described above, in the case of the previous image, projective transformation or masking with a mask image or the like may be performed on the inspection target area. In that case, within the heat map display area 1101, an image in a state where an image 841 of a deformation absolute amount heat map, which is highlighted with different display modes for each divided area according to the deformation absolute amount as described in FIG. 8, is superimposed on the inspection target area of the previous image, is displayed. Although illustration is omitted, also in the previous image heat map screen 1130, when the display area is instructed by the user to be enlarged, the display processing unit 133 performs enlargement display in the same manner as described above.

[0075] Thereby, the user can confirm, by looking at the deformation absolute amount heat map 1131 within the heat map display area 1101, which divided area has a large deformation absolute amount or, conversely, a small deformation absolute amount in the previous image. Also, in the previous image heat map screen 1130, when the current image heat map button 1006 is pressed, the display processing unit 133 causes the screen display to transition from the previous image heat map screen 1130 in FIG. 11(B) to the current image heat map screen 1100 in FIG. 11(A). That is, according to the present embodiment, the user can appropriately switch and confirm the deformation absolute amount in the current image and the previous image by pressing the current image heat map button 1106 or the previous image heat map button 1107.

[0076] Also, when the aging change heat map button 1108 is pressed by the user on the current image heat map screen 1100 or the previous image heat map screen 1130, the display processing unit 133 causes the screen display to transition to the overall display heat map screen 1010. That is, according to the present embodiment, it is possible to arbitrarily switch between the display of the absolute amount of change and the aging change (relative amount of change). For this reason, the user can easily check the overview and details by appropriately switching and displaying the absolute amount of change in the current image and the previous image and the relative amount of change (aging change), which is the difference between them.

[0077] <Flowchart showing the flow of information processing according to the present embodiment> FIG. 12 is a flowchart showing the flow of information processing from the user selection of an image to be checked for changes to the visualization of the aging change and the absolute amount of change in the changes in the information processing apparatus 101 according to the present embodiment. It is assumed that the registration of the image and the execution of the change detection process by image analysis have been completed in advance, and the analysis result storage unit 114 already stores a plurality of detection results for which changes are to be confirmed. Also, each step (process) of the flowchart in FIG. 12 will be described as a process performed by the CPU 205 in FIG. 2 that realizes each functional unit in FIG. 1 described above. In the following description, it is assumed that the first detection result is the detection result of the change by image analysis of the first image (the detection result of the current image), and the second detection result is the detection result of the change by image analysis of the second image (the detection result of the previous image).

[0078] As the process of step S1201, the CPU 205 accepts the selection of the first detection result (the detection result of the current image) and the second detection result (the detection result of the previous image) by obtaining the user input to the detection result selection screen 901 in FIG. 9. Next, as the process of step S1202, the CPU 205 extracts the inspection target areas from the first image corresponding to the first detection result and the second image corresponding to the second detection result, respectively. Next, as the process of step S1203, the CPU 205 normalizes and aligns the inspection target areas extracted from the first image and the second image, respectively.

[0079] Next, as the process of step S1204, the CPU 205 divides the images of the inspection target areas of the first image and the second image into a plurality of divided areas in a grid pattern. Next, as the process of step S1205, the CPU 205 quantifies the absolute amount of deformation from the first detection result and the second detection result for each divided area. Next, as the process of step S1206, the CPU 205 calculates the difference between the absolute values of deformation of the first detection result and the second detection result for each divided area, and obtains it as the relative amount of deformation, that is, the secular change of deformation.

[0080] Next, as the process of step S1207, the CPU 205 determines whether the display of the secular change of deformation or the display according to the absolute amount of deformation of the current image or the previous image is specified for the user to confirm the deformation.

[0081] In step S1207, for example, if it is determined that the user presses the secular change heat map button 1108 in FIG. 10 to specify the display of the secular change of deformation, the CPU 205 executes the process of step S1208. That is, as the process of step S1208, the CPU 205 displays each divided area in a different display mode (for example, color-coded) according to the secular change of deformation (relative amount of deformation) for each divided area as described in FIG. 10.

[0082] On the other hand, in step S1207, for example, if it is determined that the user presses the current image heat map button 1106 or the previous image heat map button 1107 in FIG. 11 to specify the display of the absolute amount of deformation, the CPU 205 executes the process of step S1209. For example, when the user presses the current image heatmap button 1106, the CPU 205 executes the process of step S1210. That is, as the process of step S1210, the CPU 205 performs highlighted display with different display modes (for example, color separation) for each divided region according to the absolute amount of deformation of the current image as described in FIG. 11(A). Also for example, when the user presses the previous image heatmap button 1107, the CPU 205 executes the process of step S1211. That is, as the process of step S1211, the CPU 205 performs highlighted display with different display modes (for example, color separation) for each divided region according to the absolute amount of deformation of the previous image as described in FIG. 11(B).

[0083] After step S1208, step S1210, or step S1211, as the process of step S1108, the CPU 205 determines whether an instruction for moving or enlarging the display region to be confirmed in more detail has been input from the user. And when an instruction for moving or enlarging the display region to be confirmed in more detail has been input, in step S1204, the CPU 205 further divides the indicated display region into a grid. Thereafter, the CPU 205 performs the same processing as described above on the display region. On the other hand, when an instruction for moving or enlarging the display region to be confirmed in more detail is not input, for example, when an instruction for ending the process is input, the CPU 205 ends the process of the flowchart in FIG. 12.

[0084] As described above, the information processing apparatus 101 of the present embodiment divides an image according to the magnification at the time of display, and visualizes the secular change of deformation and the absolute amount of deformation for each divided region. Thereby, according to the present embodiment, even when the user confirms the deformation of the structure to be investigated using a high-resolution image, the user can overview and find the important parts where the comparison and confirmation of the deformation should be performed, and the labor of the deformation analysis work can be reduced.

[0085] Also, as a result of checking for deformation as described above, even when it is determined that repair is required and repair is carried out, according to this embodiment, it is possible to check for construction deficiencies after the repair by means of highlighting based on the absolute amount of deformation. Further, by performing highlighting based on aging changes on the structure that has been repaired, it is also possible to check for the aging changes of the repaired part.

[0086] In the above-described embodiment, an example was described in which the display mode for each divided region is determined by citing the absolute amount of deformation or the relative amount of deformation as parameters based on the attributes of the deformation. In addition, as parameters based on the attributes of the deformation, any one of values such as the density of the deformation, the maximum value, the average value, the variance, etc., or a value obtained by combining two or more of them may be used. In that case, the parameter acquisition unit 130 acquires, as parameters based on the attributes of the deformation, any one of those values such as the density of the deformation, the maximum value, the average value, the variance, etc., or a value obtained by combining two or more of them. Then, the display processing unit 133 determines the display mode for each divided region according to those parameters. Also, in the case of this example, changes such as the density, the maximum value, the average value, the variance, etc. of the deformation may be acquired as respective relative amounts, and the display mode for each divided region may be determined according to those relative amounts. Further, the display processing unit 133 can also perform switching of the highlighting according to the density, etc. described above, in addition to switching between the highlighting according to the aging changes and the highlighting according to the absolute amount of deformation, based on an instruction for switching by the user. In addition, it is also possible to appropriately superimpose and display the highlighting according to the aging changes, the highlighting according to the absolute amount of deformation, and the highlighting according to the density, etc. described above.

[0087] <Second Embodiment> In the above-described first embodiment, an example was described in which the inspection target region of the structure to be inspected is extracted from the image stored in the image storage unit 112, and highlighting for each divided region according to the absolute amount of deformation or the relative amount of deformation (aging change) is performed on the image of the inspection target region. In the second embodiment, a method for automatically switching the display method according to the purpose of the user will be described. According to the aging change of deformation for each divided region and the emphasized display of the absolute amount of deformation as described above, the outline of the change and amount of deformation can be overviewed even when the image is reduced, and the user can gradually narrow down the important parts by an enlargement instruction. However, it is considered that if the enlargement continues, the purpose of the user will change from narrowing down the important parts to checking the situation of the deformation itself.

[0088] Therefore, in the case of the second embodiment, the display processing unit 133 adjusts the display mode for the display area at a boundary of a predetermined magnification. As an example, when the magnification at the time of displaying the display area is less than a predetermined first magnification threshold, the display processing unit 133 determines a display mode in which no deformation is displayed and only the emphasized display for each divided region is performed. Also, for example, when the magnification at the time of display is within a range that is equal to or greater than the first magnification threshold and less than the second magnification threshold, the display processing unit 133 causes the deformation to be displayed and the emphasized display to be performed. When the magnification at the time of display is within a range that is equal to or greater than the first magnification threshold and less than the second magnification threshold, the display processing unit 133 may gradually reduce the ratio of the emphasized display as the magnification at the time of display increases, so that the deformation gradually becomes more visible. And, for example, when the magnification at the time of display is equal to or greater than the second magnification threshold, the display processing unit 133 sets a display mode in which only the deformation is displayed without performing the emphasized display.

[0089] In addition, the display processing unit 133 may adjust the display mode for the display area based on any one of the values such as the density, maximum value, average value, variance, etc. of the abnormalities included in the divided area, or a value obtained by combining two or more of them. As an example, when the density of the abnormality is less than a predetermined first threshold value, the display processing unit 133 determines a display mode in which the abnormality is not displayed and only highlighting for each divided area is performed. Also, for example, when the density of the abnormality is within the range from the first threshold value or more to less than the second threshold value, the display processing unit 133 performs abnormality display and highlighting. Note that when the density of the abnormality is within the range from the first threshold value or more to less than the second threshold value, the display processing unit 133 may gradually reduce the ratio of the highlighting as the density of the abnormality increases, and display the abnormality so that it gradually becomes more visible. Also, for example, when the density of the abnormality is the second threshold value or more, the display processing unit 133 sets a display mode in which only the abnormality is displayed without performing highlighting. Or conversely, the display processing unit 133 may perform only abnormality display when the density of the abnormality is less than a predetermined first threshold value, and perform only highlighting when the density of the abnormality is the second threshold value or more. Regarding the maximum value, average value, variance, etc. of the abnormalities included in the divided area, the abnormality display can be changed in the same manner as in the case of the density.

[0090] <Other Embodiments> In the above-described embodiments, an example in which the number of divided areas is 4 vertically and 4 horizontally has been given, but the number of divided areas may be different according to the size of the image or the size of the screen. Also, when the structure to be inspected is a tunnel or the like and the image to be handled is an elongated image, instead of the grid-like area division as described above, for example, strip-like area division may be performed. Also, in the above-described embodiments, a rectangular area has been exemplified as the inspection target area, and an example in which it is equally divided vertically and horizontally and an image subjected to projective transformation is used is given, but the present invention is not limited to this example. For example, it is also possible to divide the entire image into a grid such as an equal rectangle and selectively use a rectangular grid that overlaps the inspection target area.

[0091] In the above-described embodiments, structures such as bridges and tunnels were cited as objects of investigation. However, the objects are not limited to structures, and may be automobiles, airplanes, various machine tools, construction machinery, agricultural machinery, and further, the human body, the body of an animal, plants such as trees, etc. In the above-described embodiments, an example has been given in which a change appearing on the surface of the inspection target area is detected from an image obtained by photographing a subject with a digital camera. However, the change is not limited to that appearing on the surface. The present embodiment is applicable, for example, also to the case of detecting a change occurring inside the structure of the subject based on information obtained by X-ray imaging or an electromagnetic wave radar. Further, for example, imaging of a change occurring inside the human body or the like may be performed by X-ray imaging, CT imaging, MRI imaging, or the like.

[0092] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and causing one or more processors in a computer of the system or apparatus to read and execute the program. Further, it can also be realized by a circuit (for example, ASIC) that realizes one or more functions. The above-described embodiments are merely examples of specific embodiments for carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from its technical idea or its main features.

[0093] The disclosure of the present embodiment includes the following configurations, methods, and programs. (Configuration 1) Detection means for detecting a change that has occurred in the subject from an image obtained by photographing the subject to be inspected; Parameter acquisition means for acquiring a parameter based on an attribute of the change detected from the image; Display processing means for determining a display mode for each divided area according to the parameter for each divided area obtained by dividing a display area set for the image obtained by photographing the subject into a plurality of areas; An information processing apparatus characterized by comprising the above. (Configuration 2) The parameter acquisition means acquires, as the parameter, a change amount of the attribute of the detected change between the attribute of the detected change from a first image obtained by photographing the subject to be inspected and the attribute of the detected change from a second image obtained by photographing the subject at a time different from the first image. The information processing apparatus according to Configuration 1. (Configuration 3) The parameter acquisition means acquires, as the change amount of the attribute of the change, a difference between the absolute amount of the attribute of the detected change from the first image and the absolute amount of the attribute of the detected change from the second image. The information processing apparatus according to Configuration 2. (Configuration 4) It has region matching means for matching the size, position, and shape of the image region of the subject shown in the first image and the image region of the subject shown in the second image. The display processing means displays, by overlapping, an image of the display mode determined for each divided region on the image of the display region of the first image or the second image after the region matching is performed by the region matching means. The information processing apparatus according to Configuration 2 or 3. (Configuration 5) The display processing means masks a background portion excluding the image region of the subject with a mask image. The information processing apparatus according to Configuration 4. (Configuration 6) The parameter acquisition means acquires, as the parameter, the absolute amount of the attribute of the change detected from the image. The information processing apparatus according to any one of Configurations 1 to 5. (Configuration 7) The parameter acquisition means acquires the absolute amount of the attribute of the detected change from a first image obtained by photographing the subject to be inspected as a first parameter, and acquires the absolute amount of the attribute of the detected change from a second image obtained by photographing the subject at a time different from the first image as a second parameter. The display processing means switches and displays, for each divided region of the display region of the first image, an image in the display mode determined according to the first parameter, and, for each divided region of the display region of the second image, an image in the display mode determined according to the second parameter. The information processing apparatus according to Configuration 6 is characterized by this. (Configuration 8) The parameter acquisition means acquires, as the parameter, any value among the density, maximum value, average value, and variance of the detected change state, or a value obtained by combining two or more of them, from the image. The information processing apparatus according to any one of Configurations 1 to 7 is characterized by this. (Configuration 9) The parameter acquisition means uses, as the first parameter, the value of the attribute of the detected change state from a first image obtained by photographing the subject to be inspected, and uses, as the second parameter, the value of the attribute of the detected change state from a second image obtained by photographing the subject at a time different from the first image. The display processing means switches and displays, for each divided region of the display region of the first image, an image in the display mode determined according to the first parameter, and, for each divided region of the display region of the second image, an image in the display mode determined according to the second parameter. The information processing apparatus according to Configuration 8 is characterized by this. (Configuration 10) The display processing means switches and displays an image obtained by overlapping, for each divided region, an image in the display mode determined according to the first parameter on the image of the display region of the first image, and an image obtained by overlapping, for each divided region, an image in the display mode determined according to the second parameter on the image of the display region of the second image. The information processing apparatus according to Configuration 7 or 9 is characterized by this. (Configuration 11) The parameter acquisition means acquires, as the parameter, the state of repair for the change of the subject. The information processing apparatus according to any one of Configurations 1 to 10 is characterized by this. (Configuration 12) The parameter acquisition means The amount of change in the attributes of the deformation detected from a first image of the subject to be inspected and the attributes of the deformation detected from a second image of the subject taken at a time different from the first image, The absolute amount of the attributes of the deformation detected from the image, Any value of the density, maximum value, average value, variance of the deformation detected from the image, or a value obtained by combining two or more of them, The state of repair for the deformation of the subject, Obtain at least two of them as the parameters, The information processing apparatus according to Configuration 1, wherein the display processing means switches and displays an image in the display mode determined according to the at least two parameters. (Configuration 13) The information processing apparatus according to any one of Configurations 1 to 12, wherein the display processing means adjusts the display of the display area based on the magnification when displaying the display area. (Configuration 14) When the magnification is less than a first magnification threshold, the display processing means displays the display area in a display mode determined for each divided area according to the parameter. When the magnification is equal to or greater than the first magnification threshold and less than a second magnification threshold, the display processing means overlays an image in the display mode determined for each divided area according to the parameter on the image of the display area. When the magnification is equal to or greater than the second magnification threshold, the display processing means displays only the image of the display area. The information processing apparatus according to Configuration 13 is characterized in that. (Configuration 15) When the magnification is equal to or greater than the first magnification threshold and less than the second magnification threshold, the display processing means performs a display such that the ratio of the image in the display mode determined for each divided area according to the parameter decreases as the magnification increases. The information processing apparatus according to Configuration 14 is characterized in that. (Configuration 16) The display processing means adjusts the display of the display area based on any one of the density, maximum value, average value, variance of the deformation detected from the image, or a value obtained by combining two or more of them, and is the information processing apparatus according to any one of Configurations 1 to 15. (Configuration 17) When the value is less than the first threshold, the display processing means displays the display area in a display mode determined for each divided area according to the parameter. When the value is equal to or greater than the first threshold and less than the second threshold, the display processing means overlays an image of the display mode determined for each divided area according to the parameter on the image of the display area. When the value is equal to or greater than the second threshold, the display processing means displays only the image of the display area. The information processing apparatus according to Configuration 16 is characterized in that. (Configuration 18) When the value is equal to or greater than the first threshold and less than the second threshold, the display processing means performs a display such that the ratio of the image of the display mode determined for each divided area according to the parameter decreases as the value increases. The information processing apparatus according to Configuration 17 is characterized in that. (Configuration 19) The detection means detects the deformation from an image of an inspection target area estimated by an estimation process from the image obtained by photographing the subject, or an image of an inspection target area designated by a user with respect to the image obtained by photographing the subject. The information processing apparatus according to any one of Configurations 1 to 18 is characterized in that. (Configuration 20) As the display mode for each divided area, the display processing means determines at least any one of a display color, brightness, and pattern. The information processing apparatus according to any one of Configurations 1 to 19 is characterized in that. (Configuration 21) The subject to be inspected is a structure to be investigated. The information processing apparatus according to any one of Configurations 1 to 20 is characterized in that. (Method 1) A detection step of detecting a deformation occurring in the subject from an image obtained by photographing the subject to be inspected, A parameter acquisition step of acquiring a parameter based on the attribute of the deformation detected from the image; A display processing step of determining a display mode for each divided area according to the parameter for each divided area obtained by dividing a display area set for the image of the subject into a plurality of areas; An information processing method characterized by comprising the above. (Program 1) A program for causing a computer to function as the information processing apparatus according to any one of Configurations 1 to 21.

Explanation of Signs

[0094] 101: Information processing apparatus, 111: Image management unit, 112: Image storage unit, 113: Image analysis unit, 114: Analysis result storage unit, 115: User input unit, 121: Region extraction unit, 122: Region alignment unit, 131: Absolute amount acquisition unit, 132: Relative amount acquisition unit, 133: Display processing unit, 202: Mass memory, 203: ROM, 204: RAM, 205: CPU

Claims

1. Detection means for detecting a deformation occurring in the subject from an image of the subject to be inspected; Parameter acquisition means for acquiring a parameter based on the attribute of the deformation detected from the image; Display processing means for determining a display mode for each divided area by dividing a display area set for the image of the subject into a plurality of divided areas according to the parameter for each divided area; An information processing apparatus, characterized by comprising the above.

2. The information processing apparatus according to claim 1, wherein the parameter acquisition means acquires, as the parameter, a change amount of the attribute of the deformation between the attribute of the deformation detected from a first image of the subject to be inspected and the attribute of the deformation detected from a second image of the subject taken at a time different from the first image.

3. The information processing apparatus according to claim 2, wherein the parameter acquisition means acquires, as the change amount of the attribute of the deformation, a difference between an absolute amount of the attribute of the deformation detected from the first image and an absolute amount of the attribute of the deformation detected from the second image.

4. It has area matching means for matching the size, position, and shape of the image area of the subject shown in the first image and the image area of the subject shown in the second image; The information processing apparatus according to claim 2, wherein the display processing means superimposes and displays an image of the display mode determined for each divided area on the image of the display area of the first image or the second image after the area matching is performed by the area matching means.

5. The information processing apparatus according to claim 4, wherein the display processing means masks a background portion excluding the image area of the subject with a mask image.

6. The information processing apparatus according to claim 1, wherein the parameter acquisition means acquires, as the parameter, an absolute amount of the attribute of the change detected from the image.

7. The parameter acquisition means sets, as a first parameter, an absolute amount of the attribute of the change detected from a first image obtained by photographing a subject to be inspected, and sets, as a second parameter, an absolute amount of the attribute of the change detected from a second image obtained by photographing the subject at a time different from that of the first image. The display processing means alternately displays, for each divided region of the display region of the first image, an image of the display mode determined according to the first parameter and, for each divided region of the display region of the second image, an image of the display mode determined according to the second parameter. The information processing apparatus according to claim 6, characterized in that.

8. The information processing apparatus according to claim 1, wherein the parameter acquisition means acquires, as the parameter, any one value of the density, maximum value, average value, and variance of the change detected from the image, or a value obtained by combining two or more of them.

9. The parameter acquisition means sets, as a first parameter, the value of the attribute of the change detected from a first image obtained by photographing a subject to be inspected, and sets, as a second parameter, the value of the attribute of the change detected from a second image obtained by photographing the subject at a time different from that of the first image. The display processing means alternately displays, for each divided region of the display region of the first image, an image of the display mode determined according to the first parameter and, for each divided region of the display region of the second image, an image of the display mode determined according to the second parameter. The information processing apparatus according to claim 8, characterized in that.

10. The display processing means superimposes, on the image of the display area of the first image, the image of the display mode determined for each divided area according to the first parameter, and superimposes, on the image of the display area of the second image, the image of the display mode determined for each divided area according to the second parameter, and performs the switching display. The information processing apparatus according to claim 7 or 9, characterized in that.

11. The parameter acquisition means acquires, as the parameter, the state of repair for the change of the subject. The information processing apparatus according to claim 1, characterized in that.

12. The parameter acquisition means the amount of change in the attribute of the detected change between the attribute of the detected change from the first image of the subject to be inspected and the attribute of the detected change from the second image of the subject taken at a time different from the first image, the absolute amount of the attribute of the detected change from the image, any value of the density, maximum value, average value, variance of the detected change from the image, or a value obtained by combining two or more of them, the state of repair for the change of the subject, acquires at least two of them as the parameter, The display processing means performs switching display of the image of the display mode determined according to the at least two parameters. The information processing apparatus according to claim 1, characterized in that.

13. The display processing means adjusts the display of the display area based on the magnification at the time of displaying the display area. The information processing apparatus according to claim 1, characterized in that.

14. When the magnification is less than the first magnification threshold, the display processing means performs display in a display mode determined for each of the divided regions according to the parameter on the display region; when the magnification is equal to or greater than the first magnification threshold and less than the second magnification threshold, the display processing means performs display by superimposing an image in the display mode determined for each of the divided regions according to the parameter on the image of the display region; and when the magnification is equal to or greater than the second magnification threshold, the display processing means performs display of only the image of the display region. The information processing apparatus according to claim 13, characterized in that.

15. When the magnification is equal to or greater than the first magnification threshold and less than the second magnification threshold, the display processing means performs display such that the ratio of the image in the display mode determined for each of the divided regions according to the parameter decreases as the magnification increases. The information processing apparatus according to claim 14, characterized in that.

16. The display processing means adjusts the display of the display region based on any one value of the density, maximum value, average value, and variance of the change detected from the image, or a value obtained by combining two or more of them. The information processing apparatus according to claim 1, characterized in that.

17. When the value is less than the first threshold, the display processing means performs display in a display mode determined for each of the divided regions according to the parameter on the display region; when the value is equal to or greater than the first threshold and less than the second threshold, the display processing means performs display by superimposing an image in the display mode determined for each of the divided regions according to the parameter on the image of the display region; and when the value is equal to or greater than the second threshold, the display processing means performs display of only the image of the display region. The information processing apparatus according to claim 16, characterized in that.

18. When the value is equal to or greater than the first threshold and less than the second threshold, the display processing means performs display such that the ratio of the image in the display mode determined for each of the divided regions according to the parameter decreases as the value increases. The information processing apparatus according to claim 17, characterized in that.

19. The information processing apparatus according to claim 1, wherein the detection means detects the change from an image of an inspection target area estimated by an estimation process from the image obtained by photographing the subject, or an image of an inspection target area designated by a user with respect to the image obtained by photographing the subject.

20. The information processing apparatus according to claim 1, wherein the display processing means determines at least one of a display color, brightness, and pattern as the display mode for each of the divided areas.

21. The information processing apparatus according to claim 1, wherein the subject to be inspected is a structure to be investigated.

22. A detection step of detecting a change that has occurred in the subject from an image obtained by photographing the subject to be inspected; A parameter acquisition step of acquiring a parameter based on an attribute of the change detected from the image; A display processing step of determining a display mode for each of the divided areas according to the parameter for each of the divided areas obtained by dividing a display area set for the image obtained by photographing the subject into a plurality of areas; An information processing method characterized by comprising:

23. A program for causing a computer to function as an information processing apparatus having: detection means for detecting a change that has occurred in the subject from an image obtained by photographing the subject to be inspected; parameter acquisition means for acquiring a parameter based on an attribute of the change detected from the image; display processing means for determining a display mode for each of the divided areas according to the parameter for each of the divided areas obtained by dividing a display area set for the image obtained by photographing the subject into a plurality of areas. ​

Images