Information processing apparatus and control method thereof

Abstract

To provide an information processing apparatus capable of more appropriately grasping differences between a plurality of images, a control method thereof, and a program.SOLUTION: An information processing apparatus in which a program for executing a state change determination process is stored in a non-volatile memory, is configured to: acquire a first deformation data group indicating a deformation in a target object, which is generated based on a first photographed image captured at a first time point, and a second deformation data group indicating a deformation in the target object, which is generated based on a second photographed image captured at a second time point different from the first time point; correct the position coordinates of the second deformation data group to align the position in the first deformation data group with the position in the second deformation data group; correct the position coordinates of the corrected second deformation data group in a different manner; and determine a common part which is a deformation part common between two pieces of deformation data and a difference part which is a deformation part which is different between two pieces of deformation data based on a comparison between the first deformation data group and the corrected second deformation data group.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present invention relates to a technique for deriving the difference between a plurality of pieces of image data. [Background technology]

[0002] In recent years, image-based inspections have become widely used to determine the soundness of structures from images of the structures. For example, deformed (damaged) parts are extracted from images by manual tracing or image recognition technology, and the soundness is determined based on their location, scale, and quantity. Patent Document 1 discloses a technology for determining changes in deformed parts over time from images of the same structure taken at different times. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-211277 Summary of the Invention [Problem to be solved by the invention]

[0004] However, differences in the shooting conditions can cause misalignment between images taken at different times. Patent Document 1 estimates the relationship between cracks detected in two images, but if the captured areas of the two images differ significantly, misalignment makes comparison difficult. For example, the search range for feature point correspondence becomes significantly wider, increasing computational costs and increasing the number of mismatches, making correct alignment difficult.

[0005] The present invention has been made in view of the above problems, and aims to provide a technique that enables differences between a plurality of images to be more appropriately grasped. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, an information processing device according to the present invention has the following arrangement. A first acquisition means for acquiring first deformation data indicating a deformation in the target object, the first deformation data being generated based on a first photographed image obtained by photographing the target object at a first time point; A second acquisition means for acquiring second deformation data indicating a deformation in the target object, the second deformation data being generated based on a second captured image obtained by photographing the target object at a second time point different from the first time point; A first correction means for correcting position coordinates in the second deformation data by a first correction method in order to align the position in the first deformation data with the position in the second deformation data; A second correction means for further correcting the position coordinates of the second deformation data corrected by the first correction means by a second correction method different from the first correction method; A change determination means for determining a common part, which is a deformation part common to the two deformation data, and a difference part, which is a deformation part that is different between the two deformation data, based on a comparison between the first deformation data and the second deformation data corrected by the second correction means; It has. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a technique that enables differences between a plurality of images to be more appropriately grasped. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a hardware configuration of an information processing device. [Figure 2] FIG. 10 is a diagram illustrating the relationship between a cross-sectional view of a tunnel and an inspection image. [Figure 3] FIG. 1 is a diagram illustrating a situation in which an image of the inner wall of a tunnel is captured. [Figure 4] FIG. 10 illustrates an example of a management table. [Figure 5] FIG. 10 is a diagram illustrating an example of a deformation data table. [Figure 6] FIG. 10 is a diagram showing an example of plotting deformation data. [Figure 7] FIG. 10 is a diagram illustrating an example of a GUI screen. [Figure 8] 10A and 10B are diagrams illustrating examples of display forms of state changes. [Figure 9] 10 is a flowchart of a state change determination process. [Figure 10] 10 is a detailed flowchart of a global positional deviation correction process (S902). [Figure 11] 10A and 10B are diagrams illustrating a method for calculating a global positional deviation amount. [Figure 12] 10A and 10B are diagrams showing examples of inspection images and deformation data tables after correction. [Figure 13] 10 is a detailed flowchart of the local positional deviation correction process (S903). [Figure 14] 10A and 10B are diagrams illustrating a method for calculating a local positional deviation amount. [Figure 15] FIG. 10 is a diagram showing an example of a deformation data table after local positional deviation correction. [Figure 16] 10 is a detailed flowchart of the aging determination process (S904). [Figure 17] 16 is a detailed flowchart of the aging data calculation process (S1603). [Figure 18] FIG. 10 is a diagram illustrating expansion processing. [Figure 19] FIG. 10 is a diagram illustrating an example of the relationship between common parts and different parts. [Figure 20] FIG. 10 is a diagram showing an example of a deformation data table of common parts and different parts. [Figure 21] FIG. 10 is a diagram showing an example of a display screen for the results of calculation of secular changes. [Figure 22] 10A and 10B are diagrams illustrating examples of templates used in the global positional deviation calculation process. [Figure 23] FIG. 10 is a diagram showing an example of a GUI screen in a modified example. [Figure 24] FIG. 10 is a diagram showing an example of a display screen for the result of calculation of secular change in a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0010] (First embodiment) As a first embodiment of the information processing device according to the present invention, an information processing device that detects changes in the state of deformation of a tunnel lining over time (over time) will be described below as an example. More specifically, an example will be described in which the device determines changes in state of deformation, such as progression, shrinkage, disappearance due to repair, etc., creates deformation data that can more accurately reproduce the changes in state of deformation, and displays the changes in state of deformation so that they can be understood from multiple perspectives.

[0011] In the following explanation, "deformation" refers to cracks that appear on the concrete surface due to deterioration of concrete structures (highways, bridges, tunnels, dams, etc.). Also, "cracks" refer to linear damage (expressed by start point, end point, length, and width) that appears on the surface of a structure.

[0012] <Device configuration> 1 is a block diagram showing the hardware configuration of an information processing device 100. The information processing device 100 may be realized by a single computer device or by multiple computers.

[0013] The information processing device 100 includes a control unit 101, a nonvolatile memory 102, a work memory 103, a storage device 104, an input device 105, an output device 106, a network interface 107, and a system bus 108.

[0014] The control unit 101 includes an arithmetic processor such as a CPU or MPU that controls the entire information processing device 100. The non-volatile memory 102 is a ROM that stores programs and parameters executed by the processor of the control unit 101. Here, the program refers to a program for executing a state change determination process, which will be described later. The work memory 103 is a RAM that temporarily stores programs and data supplied from an external device or the like. The storage device 104 is an internal device such as a hard disk or memory card built into the information processing device 100, or an external device such as a hard disk or memory card detachably connected to the information processing device 100. The storage device 104 includes a memory card or hard disk constituted by a semiconductor memory, a magnetic disk, or the like. The storage device 104 also includes a storage medium constituted by a disk drive that reads and writes data from / to optical disks such as DVDs and Blu-ray (registered trademark).

[0015] The input device 105 is an operating member such as a mouse, keyboard, or touch panel that accepts user operations and outputs operation instructions to the control unit 101. The output device 106 is a display device such as a monitor configured with a liquid crystal display (LCD) or an organic EL display, and displays data held by the information processing device 100 or data supplied from an external device. The network interface 107 is communicably connected to a network such as the Internet or a local area network (LAN). The system bus 108 includes an address bus, a data bus, and a control bus that connect the above-mentioned components so that data can be exchanged.

[0016] The nonvolatile memory 102 stores an operating system (OS), which is basic software executed by the control unit 101, and programs that cooperate with the OS to implement applied functions. In this embodiment, the nonvolatile memory 102 also stores a program that enables the information processing device 100 to implement a state change determination process, which will be described later.

[0017] The processing of the information processing device 100 of this embodiment is realized by loading software provided by a program. The program includes a program for utilizing basic functions of the OS installed in the information processing device 100. The OS of the information processing device 100 may include a program for realizing the processing of this embodiment.

[0018] <Inspection image> In the first embodiment, it is assumed that the object to be inspected is the lining surface of a tunnel. Therefore, the deformation data described below is generated from an inspection image of the tunnel lining surface. Below, we will first explain how to acquire the inspection image and the circumstances under which image misalignment occurs.

[0019] FIG. 2 is a diagram illustrating the relationship between a tunnel cross-section and an inspection image. In FIG. 2, a tunnel cross-section 200 (the road surface is on the left side of the figure) and a development 201 of the tunnel's inner wall are shown. The left and right ends of the development 201 are openings 202, which serve as the tunnel entrances and exits, and multiple spans 211 are included between the openings 202 at both ends. A span is a section in which concrete is poured onto the lining surface. Each span 211 includes a lining surface 203, lighting 204, and curbstones 205. Inspection is performed in units of these spans, which are repeated in the direction of the tunnel's extension. In the following explanation, the left-right direction in the development 201 (the direction in which multiple spans are lined up) may be referred to as the horizontal direction, and the up-down direction as the vertical direction.

[0020] Figure 3 is a diagram illustrating the situation in which an image of the tunnel inner wall is captured. Specifically, it is a view of the inside of the tunnel from the tunnel opening 202 (similar to tunnel cross section 200). To capture an inspection image of the lining surface 203 (here, span 211) that is the object of inspection, a vehicle 301 equipped with three line sensor cameras is driven along a route within the tunnel, as shown in Figure 3(a). The three attached line sensor cameras 302 to 304 face in different directions, and boundaries 308 and 309 indicate the boundaries of their capture ranges.

[0021] Figure 3(b) shows an example of the situation when the same tunnel is photographed at a different time (point in time), with the same number of cameras (three), but with a vehicle 310 traveling at a different position than in Figure 3(a). Therefore, in Figures 3(a) and 3(b), the three cameras are photographing the same angle of view, but the boundary positions of the photographing ranges have changed due to differences in the distance to the tunnel wall. Specifically, boundaries 308 and 309 change to boundaries 311 and 312, respectively, and as a result, the photographing range of each camera changes.

[0022] The inspection image is generated by combining images captured by three cameras. In the following explanation, it is assumed that the inspection image is an image of a range equivalent to the span, but it may be configured so that its horizontal length differs from the span. If the capturing range of each camera changes, the amount of adjustment for each image size will also change, resulting in a misalignment between the inspection images obtained in Figure 3(a) and Figure 3(b).

[0023] Furthermore, when multiple inspection areas (spans) are configured horizontally in succession, such as in a tunnel, the horizontal length of the inspection image is adjusted so that the connecting surfaces (boundaries) of adjacent inspection images align. However, in the vertical direction, the inspection image may include an area wider than the inspection area. This can occur due to differences in the methods used to capture and combine the images, or the company responsible for capturing the images. For example, if a non-target area is included in the vertical upper part of the image, the position of the target area will be shifted, resulting in a misalignment between the inspection images taken at two different times (Figures 3(a) and 3(b)). As a result, the position of the abnormality data generated from the inspection images will also be shifted between the two times.

[0024] <Deformation data> In this embodiment, deformation data is used, which represents deformation as vector data. Here, an example of a crack is explained as a deformation. The deformation data is expressed in a format as shown in Figure 5. Each deformation data is generated, for example, by a user tracing a deformation included in a displayed inspection image. It can also be automatically generated by image analysis processing or input by a combination of these. Furthermore, the image analysis processing can be performed using a learning model created by machine learning and deep learning of artificial intelligence (AI).

[0025] Next, the data structure of the inspection target data in this embodiment will be described with reference to FIGS.

[0026] Fig. 4 is a diagram showing an example of a management table. The management table shown in Fig. 4(a) illustrates a first inspection data table 401 showing the relationship between sections, inspection images, and deformation data at a first time point. The management table shown in Fig. 4(b) illustrates a second inspection data table 451 showing the relationship between sections, inspection images, and deformation data at a second time point after the first time point. Here, a set of inspection images and deformation data per section is considered to be one piece of inspection data (i.e., the inspection data includes inspection images and deformation data), and a collection of multiple inspection data corresponding to an entire section is called an inspection data group.

[0027] The first inspection data table 401 includes a section 402, which is information (section number) that identifies the section (span), an inspection image file 403 corresponding to the section, and a deformation data file 404 corresponding to the inspection image. Similarly, the second inspection data table 451 includes a section 452, an inspection image file 453, and a deformation data file 454. The inspection data table is stored as an inspection data file in the storage device 104. The inspection data file is stored, for example, in the folder in which the inspection images are stored (or in any location associated with the inspection images).

[0028] A section represents an inspection unit within a tunnel, and is assigned a consecutive number from the beginning of the tunnel (for example, the uphill side) to the end. An inspection image file is a file that stores images corresponding to a section. A deformation data file is a file that stores a deformation data table that lists deformation data obtained from inspection images of the same section.

[0029] Figure 5 shows examples of deformation data tables. Figure 5(a) illustrates a first deformation data table 501 that lists deformation data acquired from a first inspection image taken at a first time point for a certain section. Figure 5(b) illustrates a second deformation data table 551 that lists deformation data acquired from a second inspection image taken at a second time point for the same section as Figure 5(a).

[0030] In the first deformation data table 501 and the second deformation data table 551, a single crack, which is deformation data, is represented by a continuous polyline. The first deformation data table 501 includes a deformation ID 502 for identifying the deformation, a number of vertices 503, and a vertex coordinate list 504. Similarly, the second deformation data table 551 includes a deformation ID 552, a number of vertices 553, and a vertex coordinate list 554.

[0031] Figure 6 is a diagram showing an example of rendering deformation data. Figure 6(a) shows an example of a superimposed image 600 in which deformation data registered in the first deformation data table 501 is superimposed on the first inspection image. In the superimposed image 600, Ca002 to Ca012 are cracks corresponding to the deformation IDs Ca002 to Ca012 in the first deformation data table 501, respectively.

[0032] 6(b) shows an example of a superimposed image 650 in which the deformation data registered in the second deformation data table 551 is superimposed on the second inspection image. In the superimposed image 650, Cb002 to Cb010 are cracks corresponding to the deformation IDs Cb002 to Cb010 in the second deformation data table 551, respectively.

[0033] <Status change data> 7 is a diagram showing an example of a graphical user interface (GUI) screen 700 displayed by a program that realizes the state change determination process executed in the information processing device 100. The state change determination process will be described later with reference to FIG.

[0034] On the GUI screen 700, a first inspection data input button 701 is a button for selecting a first inspection data group registered in the first inspection data table 401 of FIG. 4(a). When the user operates the button 701, a file selection dialog (not shown) is displayed. When the user selects an inspection data file stored in the storage device 104, information on the inspection image files and deformation data files for all sections described in the selected inspection data file is acquired.

[0035] Similarly, the second data input button 702 is a button for selecting a second inspection data group registered in the second inspection data table 451 of FIG. 4(b). When the user operates the button 702, a file selection dialog (not shown) is displayed. When the user selects an inspection data file stored in the storage device 104, information on the image files and deformation data files for all sections described in the selected inspection data file is acquired.

[0036] The first deformation data display area 711 displays a superimposed image of the inspection data of the section selected as the display target from the inspection data group specified with the first data input button 701. The section to be displayed is selected by operating the section display area 731, which will be described later, and immediately after specifying the inspection data, the first section, which is the initial value, is selected. Note that before inspection data is specified with the first data input button 701, a blank image is displayed.

[0037] Similarly, the second deformation data display area 712 is an area that displays a superimposed screen of the inspection data specified with the second data input button 702. Similarly, immediately after the inspection data is specified (before the section is selected), the first section is displayed. Before the inspection data is specified with the second data input button 702, a blank image is displayed.

[0038] The section display area 731 is an area that displays the entire section of the tunnel, and when a group of inspection data is input using either button 701 or 702, rectangles equal to the number of sections (here, the number of spans) input are displayed. The shape of the area showing each section only needs to be such that the section can be identified, and may be a square or a rectangle that matches the aspect ratio of the inspection image. When a group of inspection data has not been input, the section display area is blank. The section display target area 732 shows the section to be displayed in the first display area 711, the second display area 712, and the result display area 721, and is selected by the user. The section is selected by the user using the mouse (such as by clicking). When the user selects one of the section areas in the section display area, the selected section area is highlighted as the section display target area.

[0039] The result display area 721 is an area that displays status change data, which is the result of the status change determination process described later with reference to FIG. 9, for the first inspection data table 401 and the second inspection data table 451. For example, when the first inspection data and the second inspection data are input, the status change determination process is executed, and an image such as that shown in FIG. 21, which will be described later, is displayed in the result display area 721. A blank image is displayed before and during the status change determination process. When the user has not yet performed a selection operation on the section display target area, the result of the status change determination process corresponding to the first section is displayed. The status change data is data that represents the results of calculating the common parts and differences between deformation data at two different points in time (for example, the first inspection data table 401 and the second inspection data table 451).

[0040] 8 is a diagram showing an example of a display format of a state change, specifically showing the relationship between two pieces of input inspection data and state change data that is a change calculation result.

[0041] The first inspection data 801 at the first time point includes deformation data 811 and deformation data 812, and the second inspection data 802 at the second time point includes deformation data 821 and deformation data 822. Condition change data 803 is an example of the result of calculating the condition change between the first and second deformation data, and is displayed in the result display area 721. The calculation results are displayed based on either the first or second deformation data, but here the first deformation data is used as the basis. Also, it is assumed that the second time point (second inspection data) is later in time than the first time point (first inspection data).

[0042] In the condition change data 803, solid lines 831 and 833 represent deformations determined to be common parts between the first deformation data (reference) and the second deformation data. On the other hand, dotted line 834 and double line 832 represent deformations determined to be different parts between the first deformation data (reference) and the second deformation data. Specifically, dotted line 834 represents a deformation part that exists in the first deformation data (reference) but does not exist in the second deformation data (i.e., a "disappeared" deformation part). Double line 832 represents a deformation part that does not exist in the first deformation data (reference) but exists in the second deformation data (i.e., a newly "appeared" deformation part). In this way, common parts and different parts are displayed in a distinguishable manner. Furthermore, "disappeared" deformation parts and "appeared" deformation parts are displayed in a distinguishable manner. By displaying in this manner, the user can easily understand that the deformation of the double line 832 is an extension of the deformation (crack) of the solid line 831.

[0043] The result output button 751 is a button for outputting the result of the status change determination process as a file. The data and file output as the result of the status change determination process will be described later.

[0044] <Device Operation> FIG. 9 is a flowchart of the status change determination process. The status change determination process is a process in which inspection data groups from two different points in time are input, and aging changes are determined and the results are displayed (for example, FIG. 21, which will be described later). In order to determine aging changes, a positional deviation between the two inspection data groups is detected and corrected. The status change determination process is realized by the control unit 101 of the information processing device 100 shown in FIG. 1 loading a program stored in the nonvolatile memory 102 into the work memory 103, executing it, and controlling each component.

[0045] In S901, the control unit 101 reads out a first inspection data group selected by the user with button 701 and a second inspection data group selected with button 702. In this embodiment, the first inspection data is used as the basis for aging processing and display. That is, the second inspection data is corrected to match the position of the first inspection data.

[0046] In S902, the control unit 101 performs a correction (first correction method) for the global positional deviation between the first inspection data group and the second inspection data group. That is, the control unit 101 aligns the positions in the deformation data included in the first inspection data with the positions in the deformation data included in the second inspection data. Here, the control unit 101 corrects the position coordinates in the deformation data included in the second inspection data. The global positional deviation means, for example, a large positional deviation across the entire inspection target area (span). Details of the process will be described later with reference to FIGS. 10 to 12.

[0047] In S903, the control unit 101 performs correction (second correction method) for local positional deviations between the first inspection data and the second inspection data group corrected in the process of S902. Local positional deviations refer to, for example, small positional deviations within a partial area of ​​the inspection target area (span). Details of this process will be described later with reference to FIGS. 13 to 16.

[0048] In S904, the control unit 101 performs a process of determining aging between two different points in time using the first inspection data and the second inspection data whose position has been corrected in the process of S903. Details of the process will be described later with reference to Figs. 16 to 21.

[0049] In S905, the control unit 101 sets a display target section to be displayed in the display areas 711, 712, and 721 of the GUI screen 700, which is the display unit. For example, the first section is set as the display target section. In S906, the control unit 101 performs display control to perform display corresponding to the display areas 711, 712, and 721 based on the set display target section.

[0050] In S907, the control unit 101 accepts an operation from the user accepted by the input device 105. If an operation described in S908 below has been performed, the process proceeds to S908. In S908, the control unit 101 determines whether the user operation accepted by the input device 105 is "end (of state change determination processing)," "output (of processing results to a file)," or "switch (of the section to be displayed)."

[0051] For example, if an operation to close the window of the GUI screen 700 is performed, the process ends. If the result output button 751 is pressed, the process proceeds to S909. If an operation is performed on the section display area 731 and the section to be displayed is switched, the process proceeds to S910.

[0052] In S909, the control unit 101 outputs a deformation data table, which is the calculation result of the aging process, as a file. The deformation data table is, in other words, deformation change data that shows the aging change of the deformation. The contents of the calculation result data to be output will be described later with reference to FIG. 20. After output, the process returns to S907. In S910, the control unit 101 sets the section specified by the user in the section display area 731 as the section to be displayed, and returns to S906.

[0053] <Comprehensive positional deviation correction processing> 10 is a detailed flowchart of the global positional deviation correction process (S902). In S1001 to S1004, the control unit 101 repeats the process for the first inspection data group and the second inspection data group stored in the recording device 104 in the order of the section numbers written in the inspection data table.

[0054] In S1002, the control unit 101 generates template images for the first inspection image. Fig. 11 is a diagram illustrating a method for calculating a global positional deviation amount. For example, in the first inspection image 600 shown in Fig. 11(a), a horizontal strip-shaped rectangular area surrounded by a dotted line is generated as a template image. In the example of Fig. 11(a), eight template images, images 1101 to 1108, are generated.

[0055] In S1003, the control unit 101 performs template matching (i.e., one-dimensional positional deviation detection) on the second inspection image, with the search range (search direction) limited to the vertical direction only, and calculates the deviation amount for each template. In the example of FIG. 11(b), matching is performed using the template image 1103 within the search range 1151. For example, the origin coordinates of the upper left corners of the first and second inspection images are aligned, and the position in the Y direction (vertical direction) at which the degree of match is maximized is determined using a score calculation method commonly used in existing template matching. The deviation amount is a relative value, which is the difference in coordinates of the correction target (here, the second inspection image) relative to the correction reference (here, the first inspection image). In this case, one-dimensional correction in the Y direction is performed. For example, if the Y coordinate of the top edge of the template 1103 is "200" and the coordinate of the second inspection image at which the degree of match is maximized is "700," the deviation amount is "+500."

[0056] In S1004, the control unit 101 repeats the processes of S1002 to S1003 until all sections (or one or more sections designated in advance by the user) have been processed.

[0057] In S1005, the control unit 101 calculates the amount of deviation for each section from the amount of deviation calculated by template matching in S1003. The amount of deviation for each section is set to the average value calculated by removing outliers from the amounts of deviation calculated for all templates in the section. For example, the average amount of deviation for all templates is calculated, and values ​​whose difference from the average value is greater than a predetermined value are removed as outliers. The average value is then calculated again to set the amount of deviation for the section. An example of the predetermined value is a value twice the standard deviation of the amount of deviation.

[0058] In S1006, the control unit 101 corrects the inspection images and the deformation data table for all sections based on the amount of deviation calculated in S1005. The images are corrected according to the amount of deviation in order to align the upper left origin of the images when performing local correction processing in the subsequent stage.

[0059] If the amount of deviation in the vertical direction (Y direction) of the image is positive (there is a deviation toward the top of the image), the area at the top of the image equal to the amount of deviation is removed. Also, the value of the deviation is subtracted from the y coordinate value in the coordinate list of the deformation data table. Conversely, if the amount of deviation in the vertical direction (y direction) of the image is negative, an area equal to the amount of deviation (pixel value is, for example, a value indicating a blank) is added to the top of the image. Also, the value of the deviation is added to the coordinate value in the coordinate list of the deformation data table.

[0060] Fig. 12 shows an example of a corrected inspection image and a deformation data table. Here, an example is shown in which the amount of deviation is positive. Corrected image 1201 in Fig. 12(a) is an example in which the second inspection image 650 has been corrected. Using the vertical deviation 1211 calculated in S1005, the area indicated by rectangle 1212 is removed so that the Y coordinate value of the origin of the corrected image becomes the deviation 1211.

[0061] The deformation data table 1251 in Figure 12(b) is a data table to which the amount of deviation has been applied to the second deformation data table 551. The letter G is added to the end of the deformation ID 1252 as a character to identify that global position correction has been applied. The Y coordinate of the vertex coordinate list 1254 is the Y coordinate of the vertex coordinate list 554 minus the amount of deviation Gy calculated in S1005.

[0062] <Local positional deviation correction processing> 13 is a detailed flowchart of the local positional deviation correction process (S903). In S1301 to S1306, the control unit 101 repeats the process for the first inspection data group and the second inspection data group stored in the recording device 104 in the order of the section numbers written in the inspection data table.

[0063] In S1302, the control unit 101 creates multiple template images from the first inspection image. The multiple template images are small in size (for example, about the size of a typical deformation) in order to calculate the local positional deviation. FIG. 14 is a diagram illustrating a method for calculating the amount of local positional deviation. For example, as shown in FIG. 14(a), a template image 1401 is created for the first inspection image 600. The template 1401 is smaller in size than the template 1101 shown in FIG. 11(a).

[0064] In S1303, the control unit 101 performs template matching using the template created in S1302, with the second inspection image (corrected image 1201) subjected to the misalignment correction in S1007 as the correction target, and calculates the amount of misalignment for each template. The template matching process in S1303 may be a known template matching process, as in S1003. Unlike S1003, the search range (search direction) in S1303 covers two directions, the vertical and horizontal directions (i.e., two-dimensional misalignment detection). Furthermore, because global misalignment correction processing (S902) has already been performed, the amount of movement required for template search can be expected to be small. Because the search range is handled in terms of two axes, the horizontal and vertical axes, the amount of misalignment in local correction is hereafter handled as a two-dimensional misalignment vector.

[0065] In S1304, the control unit 101 calculates the displacement estimation vector of each vertex coordinate value of the deformation data using the displacement vector in each template calculated in S1303. At this time, the control unit 101 excludes the displacement vectors that are outliers, and calculates the displacement estimation vector from the excluded displacement vector group. As shown in the following mathematical formula (1), the x-coordinate v of the displacement estimation vector v x and the y-coordinate v y is the estimation function f for the input vertex coordinates (x, y). x and f y It can be expressed as:

[0066]

number

[0067] Estimation function f x and f y can be calculated using a known method, and can be calculated using multiple regression analysis with the x and y of the deviation vector as independent variables. x and f y Using this, the corrected vector is calculated as shown in the following formula (2). In other words, since two-dimensional correction is performed here, the estimated displacement vector v is calculated for each coordinate value (X, Y) of the deformation data. x and v y Subtract this and calculate as (X', Y').

[0068]

number

[0069] In S1305, the control unit 101 applies the estimation function calculated in S1304 to the second deformation data table that has undergone the global position deviation correction process (S902) to create a second deformation data table ( FIG. 15 ) that has undergone local position correction. The created table is stored in the recording device 104.

[0070] In S1306, the processes from S1302 to S1305 are repeated until all sections are completed.

[0071] 15 is a diagram showing an example of a deformation data table after local positional deviation correction. Here, the letter C is added to the end of the deformation ID 1502 to identify that local positional correction has been applied. The coordinate list 1504 of the deformation data table lists the corrected coordinate values.

[0072] <Aging Determination Processing> 16 is a detailed flowchart of the aging determination process (S904). In S1601 to S1604, the control unit 101 repeats the process in the order of the section numbers written in the inspection data table stored in the recording device 104.

[0073] In S1602, the control unit 101 specifies the first deformation data as a reference for the aging process. Also, the control unit 101 specifies the position-corrected second deformation data as a comparison target. In S1603, the control unit 101 performs aging data calculation processing using the deformation data specified in S1602. Details of the processing will be described later with reference to FIG. 17.

[0074] In S1604, the processes of S1602 to S1603 are repeated until all sections are completed.

[0075] FIG. 17 is a detailed flowchart of the aging data calculation process (S1603).

[0076] In S1701, the control unit 101 performs an expansion process on the deformation data to be compared. Details of the expansion process will be described later with reference to FIG. 18. In S1702, the control unit 101 determines that the deformation data to be processed that exists within the expansion area of ​​the deformation data to be compared is a common part. In addition, it determines that the deformation data outside the expansion area is a different part. The determination process will be described later with reference to FIG. 19.

[0077] In S1703, the control unit 101 stores the deformation data of the common part and the deformation data of the different part calculated in S1702 as a deformation data table in the recording device 104. The deformation data table of the common part and the different part will be described later with reference to FIG.

[0078] In S1704, the control unit 101 performs an expansion process on the reference deformation data. The expansion process is the same as S1701, except that the deformation data to be expanded is the reference deformation data. In S1705, the control unit 101 determines that the deformation data to be compared that is outside the expansion area of ​​the expanded deformation data is the difference part.

[0079] In S1706, the control unit 101 stores the difference calculated in S1705 in the recording device 104 as a deformation data table.

[0080] Figure 18 is a diagram explaining the expansion process (S1701 and S1704). Figure 18(a) shows an example of an expansion area data table 1801 created by performing expansion process on the deformation data for each deformation ID in the deformation data table specified as the deformation data to be expanded.

[0081] The expansion area data table 1801 includes a deformation ID 1802 and expansion area information 1803. The expansion area information 1803 is vector data that represents the outline of the expansion area created as a result of performing expansion processing on the deformation data, and corresponds to, for example, the vertex coordinate list of the polygon shown in FIG. 18(c).

[0082] Figure 18(b) shows an example of a crack polyline 1804 drawn based on the deformation data of deformation IDCc001 before the expansion process is performed. The thin line 1805 in Figure 18(c) shows an example of the expanded polygonal area created by performing the expansion process on the polyline 1804 in Figure 18(b). The black dots on the polyline 1804 correspond to the vertex coordinates registered in the vertex coordinate list of deformation IDCc001.

[0083] The expansion area 1805 is a polygonal area obtained by translating each line segment constituting the polyline 1804 by a predetermined distance D on both sides of the polyline 1804 (in a direction substantially perpendicular to the polyline 1804). The shape of the expansion area 1805 is not limited to a polygon, but may be any closed area. For example, in FIG. 18(c), the areas surrounding the vertices 1806 and 1807 of the bend portion may be gently sloping. For example, these surrounding areas may be replaced by arcs with a radius equal to the distance D centered on the bend portion of the polyline 1804. In particular, if the bend portion of the polyline 1804 has an acute angle, the distance to each line segment constituting the expansion area will be significantly greater than the predetermined distance D. Therefore, rounding the corners of the expansion area eliminates any missing areas in the expansion area, which is advantageous in the process of determining common and different portions, which will be described later.

[0084] 19 is a diagram illustrating an example of the relationship between the common part and the difference part. Specifically, it illustrates the relationship between the reference deformation data and the expansion area data created from the deformation data that has been subjected to the expansion process.

[0085] In Figure 19(a), polyline 1900 shows a line segment connecting the vertex coordinates of the reference deformation data. Line segments 1901 to 1906 show line segments connecting two adjacent vertices in polyline 1900. Expansion areas 1911 and 1912 show the expansion area data created by the expansion process of S1701. Figure 19(b) shows an example of the display of deformation data when the inside of the expansion area is determined to be a common part and the outside of the expansion area is determined to be a different part.

[0086] The determination of whether the line segments of the deformation data 1900 overlap with the expansion areas 1911 and 1912 uses a collision determination technique that is widely known in the field of computer games, etc. The deformation data 1900 is determined to be two common parts and three different parts, and intermediate points 1931 to 1934 are generated on the boundaries inside and outside the expansion areas on the lines of the deformation data 1900.

[0087] Line segment 1902 is divided into line segments 1921 and 1922 by midpoint 1931. Line segment 1904 is divided into line segments 1923 and 1924 by midpoint 1932. Line segment 1906 is divided into line segments 1925, 1926, and 1927 by midpoints 1933 and 1934.

[0088] The intersection that falls within expansion region 1911 is made up of line segment 1922, line segment 1903, and line segment 1923. The intersection that falls within expansion region 1912 is made up of line segment 1926. The first difference that falls outside the expansion region is made up of line segment 1901 and line segment 1921. The second difference is made up of line segment 1924, line segment 1905, and line segment 1925. The third difference is made up of line segment 1927.

[0089] Figure 20 shows examples of deformation data tables for common parts and different parts. Figure 20(a) illustrates a deformation data table 2001 for common parts (deformations present in both the first and second deformation data) output in S1703. Figure 20(b) illustrates a deformation data table 2051 for different parts (deformations present only in the first deformation data) output in S1703. Figure 20(c) illustrates a deformation data table 2071 for different parts (deformations present only in the corrected second deformation data) output in S1706.

[0090] The deformation data table 2001 includes a deformation ID 2002 that identifies the deformation data of the common part, a reference deformation ID 2003 that identifies the deformation data that serves as the reference before being distinguished between the common part and the different part, a vertex count 2005, and a vertex coordinate list 2006. The reference deformation ID 2003, the vertex count 2005, and the vertex coordinate list 2006 have the same column configuration as the deformation ID 502, the vertex count 503, and the vertex coordinate list 504 in Figure 5(a). In Figure 21, which will be described later, the deformation data in the deformation data table 2001 is displayed with a solid line.

[0091] The deformation data table 2051 includes a deformation ID 2052 that identifies the deformation data of the different parts, a reference deformation ID 2053 that identifies the reference deformation data before it is separated into common parts and different parts, the number of vertices 2054, and a vertex coordinate list 2055. The reference deformation ID 2053, the number of vertices 2054, and the vertex coordinate list 2055 have the same column configuration as the deformation data table in Figure 20(a). In Figure 21, which will be described later, the deformation data in the deformation data table 2051 is displayed with dotted lines.

[0092] The deformation data table 2071 includes a deformation ID 2072 that identifies the deformation data of the different parts, a reference deformation ID 2073 that identifies the deformation data that serves as the reference before being distinguished between the common parts and the different parts, the number of vertices 2074, and a vertex coordinate list 2075. The reference deformation ID 2073, the number of vertices 2074, and the vertex coordinate list 2075 have the same column configuration as the deformation data table in Figure 20(b). In Figure 21, which will be described later, the deformation data in the deformation data table 2071 is displayed with double lines.

[0093] <Displaying the results of the aging assessment process> Fig. 21 is a diagram showing an example of a display screen for the results of the aging calculation. The results of the aging calculation are displayed in the result display area 721 in Fig. 7. A result image 2100 is an image that displays the results of an aging determination after correcting the position of the second deformation data to match the first deformation data. In the result image 2100, each deformation (crack) is shown by either a solid line representing the deformation data of the common part, a dotted line representing the deformation data of the difference part for "disappearance", or a double line representing the deformation data of the difference part for "appearance".

[0094] The deformation data for the common parts corresponds to Cam001 to Cam011 in deformation data table 2001 in Figure 20(a). The deformation data for the difference parts for "disappearance" corresponds to Cax009 and Cax012 in deformation data table 2051 in Figure 20(b). The deformation data for the difference parts for "appearance" corresponds to Cbx001C, Cbx002C, and Cbx006C in deformation data table 2071 in Figure 20(c). Here, the difference parts for "appearance" are aligned with the common parts, and the difference parts for "appearance" that extend from the common deformation (crack) are displayed on the extension of the common deformation.

[0095] As described above, according to the first embodiment, the first deformation data corresponding to the first inspection image and the second deformation data corresponding to the second inspection image are corrected for positional deviation, and then a process for determining changes in the deformation data is performed. In particular, the positional deviation correction is performed in two stages: global positional deviation correction and local positional deviation correction. This enables more appropriate positional deviation correction and makes it possible to accurately derive the commonalities and differences between the first deformation data and the second deformation data.

[0096] Furthermore, because the difference parts of the deformation data can be derived with high accuracy, when the common parts and difference parts are displayed, the user can easily understand the relationship between the difference parts of the deformation data and the common parts of the deformation data. For example, by displaying the difference parts that exist only in the second deformation data aligned with the first deformation data, it becomes possible to intuitively understand whether the difference parts are an extension of the common parts or a separate deformation (such as a new branch).

[0097] Furthermore, when an inspection image is a composite of multiple photographed images, performing local misalignment correction processing makes it possible to appropriately correct misalignment of the deformation data caused by the boundaries or distortion of the photographed images present in the inspection image. Furthermore, performing expansion processing on the deformation data makes it possible to accurately determine that the data is common deformation data even if some misalignment remains or there are differences in the shape of the deformation.

[0098] (Variation) <Global / local positional deviation correction processing> In the first embodiment described above, an example was described in which a template was generated from a first inspection image, a matching process was performed on the second inspection image, and the amount of misalignment was calculated. However, a template may also be generated from a second inspection image, a matching process was performed on the first inspection image, and the amount of misalignment may be calculated. Alternatively, the results of searching the second inspection image using a template generated from the first inspection image and the results of searching the first inspection image using a template generated from the second inspection image may be integrated to calculate the amount of misalignment. This allows the amount of misalignment to be calculated from both directions, thereby improving the accuracy of the amount of misalignment. In either case, the direction of misalignment is calculated as the deviation of the inspection data to be corrected relative to the reference inspection data (inspection image and deformation data).

[0099] In the global positional deviation detection (S1002 and S1003) in the first embodiment, an example has been described in which rectangular images obtained by dividing the inspection image into horizontal strips are used as templates. However, templates of other shapes may also be used. FIG. 22 is a diagram showing another example of a template used in the global positional deviation calculation process. FIG. 22 shows an example in which a rectangular image 2202 circumscribing deformation data 2201 is used as a template. This makes it possible to improve accuracy when the inspection image has few image features (lights, emergency exits, signs, etc.) and many deformations.

[0100] Alternatively, line segments extending in a substantially horizontal direction and having a predetermined length or more may be detected from the inspection image using edge detection or the like, and the detected line segments from each inspection image may be associated based on their similarity, with the vertical distance between the associated line segments being used as the deviation amount. Here, line segments whose vertical height of the circumscribing rectangle or whose angle with respect to the horizontal line is less than a predetermined value may be considered to extend substantially horizontally. Furthermore, the similarity of line segments may be calculated from the difference in the horizontal width of the circumscribing rectangle and the angle between them, and the pair of line segments with the highest similarity is associated. This allows for calculations with a smaller processing load than template matching.

[0101] In addition, the deviation amount can be calculated using the diagonal line of the circumscribed rectangle of the deformation data or the horizontal center line instead of the edge-detected straight line, or both can be used together. By using information on the existence of deformation data, it is possible to calculate the deviation amount even when there are few horizontal image features.

[0102] In S1005, the amount of deviation was calculated for each section (span), but it may also be calculated using the similarity between sections. For example, by using the deviation amounts of all templates for all sections together, it is possible to calculate the amount of deviation even for sections where template matching frequently fails. Furthermore, by calculating the moving average of the deviation amounts of templates within a certain number of adjacent sections, it is possible to calculate the correct amount of deviation even when the shooting vehicle changes its driving position within a long tunnel. Alternatively, if there are sections with different image heights (corresponding to changes in the shape of the tunnel cross section), the images can be grouped accordingly and the amount of deviation calculated for each group, which can also accommodate changes in shape due to tunnel turnout areas or changes in construction method. Note that the difference in shape may be obtained from a prepared design drawing of the structure or from the height of the inspection image.

[0103] <Structures to be inspected> In the first embodiment described above, an example was described in which the lining surface of a tunnel was the inspection target, but structures such as bridge decks can also be inspected. When inspecting bridge decks, inspection images are often generated with the space between beams as one inspection unit. High-resolution images are required to calculate deformation data, so in such cases, multiple captured images are combined to generate the inspection image. In this case, as with tunnel images, areas outside the inspection target are recorded in a direction different from the image connection direction. This can result in differences in the size of individual images, which can cause deviations in the position of deformations.

[0104] Although the above description is based on a pattern in which the inspection target surface is continuous in the horizontal direction, the present invention can also be applied to a structure (or drawing) in which the inspection target surface is continuous in the vertical direction. In this case, it is advisable to limit the search direction when calculating the global positional deviation amount to horizontal only.

[0105] <Status change display format> In the first embodiment described above, an example was described in which the first inspection data was used as the display reference, positional deviation correction was performed on the second inspection data, and aging assessment processing was performed. This is suitable for superimposing condition change data on the first inspection image, which is a past inspection result, and checking for newly appeared abnormality data at a second time point. On the other hand, the reverse configuration may be used when checking for abnormality data that only existed (disappeared) in the first inspection data. That is, the second inspection data may be used as the display reference, positional deviation correction is performed on the first inspection data, and the condition change data may be superimposed and displayed on the second inspection image. Furthermore, a configuration may be used in which two display formats are switchable: a display based on the first inspection data (first embodiment) and a display based on the second inspection data. Furthermore, a configuration may be used in which positional deviation correction is performed based on the first inspection data, and only the display is switchable based on the second inspection data.

[0106] 23 is a diagram showing an example of a GUI screen 2300 in the modified example. In the GUI screen 2300, a display standard selection 2301, which is a UI component for switching between two display modes, is added to the GUI screen 700 of the first embodiment. For example, when the program is started, "First" is selected, which displays the data based on the first inspection data, and the user selects "Second" by operating the input device 105 such as a mouse as necessary. When "Second" is selected, the display in the result display area 721 is updated (switched).

[0107] Fig. 24 is a diagram showing an example of a display screen for the aging calculation results in the modified example. Specifically, it shows an aging result 2400 when "Second" is selected in the display criterion selection 2301. Note that when "First" is selected, the same display as the aging result 2100 in Fig. 21 is shown.

[0108] In Figure 24, deformation data indicated by a solid line and starting with Cbm (such as Cbm001) is deformation data determined to be a common portion between the second deformation data and the position-corrected first deformation data. Deformation data indicated by a double line and starting with Cbx (such as Cbx012) is deformation data of a different portion that is not within the expansion area of ​​the position-corrected first deformation data and exists only in the second deformation data. Deformation data indicated by a dotted line and starting with Cax and ending with C (such as Cax009C) indicates deformation data that exists only in the position-corrected first deformation data.

[0109] By making it possible to switch the display standard, users can check data on changes over time based on the inspection data at the time they want to check. In addition, instead of using the position coordinates of the deformation data as the standard, it is also possible to superimpose the data based on the position coordinates of the inspection image. By displaying not only the deformation data that exists at the target (at that time) to be displayed, but also the deformation data that exists only in the other deformation data after correcting its position, it becomes possible to check for deformations that are not visible in the inspection image that was used as the standard.

[0110] The disclosure of this specification includes the following information processing device, control method, and program. (Item 1) A first acquisition means for acquiring first deformation data indicating a deformation in the target object, the first deformation data being generated based on a first photographed image obtained by photographing the target object at a first time point; A second acquisition means for acquiring second deformation data indicating a deformation in the target object, the second deformation data being generated based on a second captured image obtained by photographing the target object at a second time point different from the first time point; A first correction means for correcting position coordinates in the second deformation data by a first correction method in order to align the position in the first deformation data with the position in the second deformation data; A second correction means for further correcting the position coordinates of the second deformation data corrected by the first correction means by a second correction method different from the first correction method; A change determination means for determining a common part, which is a deformation part common to the two deformation data, and a difference part, which is a deformation part that is different between the two deformation data, based on a comparison between the first deformation data and the second deformation data corrected by the second correction means; An information processing device comprising: (Item 2) the first correction method is one-dimensional correction based on one-dimensional positional deviation detection targeting a first range of the target object; The second correction method is a two-dimensional correction based on two-dimensional positional deviation detection for a second range that is narrower than the first range. 2. The information processing device according to item 1, (Item 3) the first correction method comprises creating a first template image based on the first captured image, and performing template matching between the second captured image and the first template image, thereby detecting the one-dimensional positional deviation; In the second correction method, a second template image smaller in size than the first template image is created based on the first captured image, and template matching is performed using the second template image with the second captured image, thereby detecting the two-dimensional positional deviation. 3. The information processing device according to item 2, (Item 4) the first captured image is one of a plurality of captured images obtained by capturing an image of the target object while moving in a first direction; The first correction means detects the one-dimensional positional deviation in a second direction perpendicular to the first direction. 3. The information processing device according to item 2, (Item 5) The method further includes a storage means for storing the deformation change data representing the common part and the difference part determined by the change determination means. 5. The information processing device according to any one of items 1 to 4. (Item 6) a display control unit that aligns the common portion and the difference portion determined by the change determination unit and displays them on a display unit; The display control means displays the common portion and the different portion in a distinguishable manner. 6. The information processing device according to any one of items 1 to 5, (Item 7) The display control means further displays, with respect to the difference portion, a first difference portion that exists in the first deformation data but does not exist in the second deformation data, and a second difference portion that does not exist in the first deformation data but exists in the second deformation data, in a distinguishable manner. 7. The information processing device according to item 6, (Item 8) The display control means superimposes and displays the common portion and the different portion on the first photographed image based on the position coordinates of the first deformation data. 8. The information processing device according to item 6 or 7, (Item 9) The display control means is configured to be able to switch between a first display that superimposes the common part and the difference part on the first photographed image based on the position coordinates of the first deformation data, and a second display that superimposes the common part and the difference part on the second photographed image based on the position coordinates of the second deformation data. 9. The information processing device according to any one of items 6 to 8, wherein: (Item 10) A control method for an information processing device, comprising: A first acquisition step of acquiring first deformation data indicating a deformation in the target object, the first deformation data being generated based on a first photographed image obtained by photographing the target object at a first time point; A second acquisition step of acquiring second deformation data indicating a deformation in the target object, the second deformation data being generated based on a second captured image obtained by photographing the target object at a second time point different from the first time point; A first correction step of correcting position coordinates in the second deformation data by a first correction method to align the position in the first deformation data with the position in the second deformation data; A second correction process for further correcting the position coordinates of the second deformation data corrected by the first correction process by a second correction method different from the first correction method; A change determination process for determining a common part, which is a deformation part common to the two deformation data, and a difference part, which is a deformation part that is different between the two deformation data, based on a comparison between the first deformation data and the second deformation data corrected by the second correction process; A control method comprising: (Item 11) Item 11. A program for causing a computer to execute the control method according to Item 10.

[0111] (Other Examples) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0112] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0113] 100 Information processing device; 101 Control unit; 102 Non-volatile memory; 103 Work memory; 104 Storage device; 105 Input device; 106 Output device; 107 Network interface; 108 Bus

Claims

1. A first acquisition means for acquiring first deformation data indicating a deformation in the target object, the first deformation data being generated based on a first photographed image obtained by photographing the target object at a first time point; A second acquisition means for acquiring second deformation data indicating a deformation in the target object, the second deformation data being generated based on a second captured image obtained by capturing the target object at a second time point different from the first time point; A first correction means for correcting a position coordinate in the second deformation data by a first correction method in order to align a position in the first deformation data with a position in the second deformation data; A second correction means for further correcting the position coordinates of the second deformation data corrected by the first correction means by a second correction method different from the first correction method; A change determination means for determining a common part, which is a common part between the two deformation data, and a difference part, which is a difference between the two deformation data, based on a comparison between the first deformation data and the second deformation data corrected by the second correction means; 13. An information processing device comprising:

2. the first correction method is one-dimensional correction based on one-dimensional positional deviation detection for a first range of the target object; The second correction method is a two-dimensional correction based on a two-dimensional positional deviation detection targeting a second range narrower than the first range.

2. The information processing apparatus according to claim 1,

3. the first correction method comprises creating a first template image based on the first captured image, and performing template matching between the first template image and the second captured image, thereby detecting the one-dimensional positional deviation; In the second correction method, a second template image having a size smaller than that of the first template image is created based on the first captured image, and template matching is performed between the second captured image and the second template image, thereby detecting the two-dimensional positional deviation.

3. The information processing apparatus according to claim 2.

4. the first captured image is one of a plurality of captured images obtained by capturing an image of the target object while moving in a first direction; The first correction means detects the one-dimensional positional deviation in a second direction perpendicular to the first direction.

3. The information processing apparatus according to claim 2.

5. The method further includes a storage means for storing the deformation change data representing the common part and the difference part determined by the change determination means.

2. The information processing apparatus according to claim 1,

6. a display control unit that aligns the common portion and the difference portion determined by the change determination unit and displays them on a display unit; The display control means displays the common portion and the different portion in a distinguishable manner.

2. The information processing apparatus according to claim 1,

7. The display control means further displays, with respect to the difference portion, a first difference portion that exists in the first deformation data and does not exist in the second deformation data, and a second difference portion that does not exist in the first deformation data and exists in the second deformation data in a distinguishable manner.

7. The information processing apparatus according to claim 6,

8. The display control means displays the common portion and the different portion in a superimposed manner on the first photographed image based on position coordinates of the first deformation data.

7. The information processing apparatus according to claim 6,

9. The display control means is configured to be able to switch between a first display in which the common part and the difference part are superimposed on the first photographed image based on the position coordinates of the first deformation data, and a second display in which the common part and the difference part are superimposed on the second photographed image based on the position coordinates of the second deformation data.

7. The information processing apparatus according to claim 6,

10. A method for controlling an information processing device, comprising: A first acquisition step of acquiring first deformation data indicating a deformation in the target object, the first deformation data being generated based on a first photographed image obtained by photographing the target object at a first time point; A second acquisition step of acquiring second deformation data indicating a deformation in the target object, the second deformation data being generated based on a second photographed image obtained by photographing the target object at a second time point different from the first time point; A first correction step of correcting position coordinates in the second deformation data by a first correction method in order to align a position in the first deformation data with a position in the second deformation data; A second correction process for further correcting the position coordinates of the second deformation data corrected by the first correction process by a second correction method different from the first correction method; A change determination process for determining a common part, which is a common part between the two deformation data, and a difference part, which is a difference between the two deformation data, based on a comparison between the first deformation data and the second deformation data corrected by the second correction process; A control method comprising:

11. A program for causing a computer to execute the control method according to claim 10.

Images