Information processing apparatus, information processing method, storage medium, and computer program product

Abstract

The invention provides an information processing apparatus, an information processing method, a storage medium, and a computer program product. The information processing apparatus includes: an identification unit configured to identify a common portion and a different portion between first deformation data of an object and second deformation data of the object different from the first deformation data; and an acquisition unit configured to acquire, based on the common portion and the different portion, third deformation data having a deformation progress state between the first deformation data and the second deformation data as a legend classification.

Description

Technical Field

[0001] The present disclosure relates to an information processing device, an information processing method, a non-transitory computer-readable storage medium, and a computer program product. Background Art

[0002] A method is proposed for detecting deformations such as cracks from a captured image of a wall of a concrete structure (object) to be inspected. In this case, improved computer performance and detection performance enable automatic detection of deformations from images. A method is also proposed for determining a change in the state of a deformation (such as development or disappearance of a deformation) from images captured at different times.

[0003] International Publication No. 2019 / 021729 describes a technology for performing deformation detection using captured segmented images of the surface of a relatively large structure (object) (such as a concrete structure, etc.), and generating a deformation map by combining the deformation detection results based on combined corresponding points when combining the captured segmented images.

[0004] Japanese Patent Publication No. 2019-211277 describes a technique in which first deformation data generated from a first image is compared with second deformation data generated from a second image to determine a change in a deformation state (such as the length and width of the deformation), wherein the first image and the second image are captured at different times.

[0005] The technology described in International Publication No. 2019 / 021729 and the technology described in Japanese Patent Publication No. 2019-211277 do not take into account the error that may occur due to ambiguity when detecting deformation or determining a change in the state of deformation. Therefore, it is difficult to correctly grasp the state of deformation of an object using the existing technology. Summary of the invention

[0006] The present disclosure provides a technology that can accurately grasp the state of deformation of an object.

[0007] An aspect of an information processing device according to the present disclosure includes a processor and a memory including instructions. When executed by the processor, the instructions cause the processor to identify a common part and a different part between first deformation data of an object and second deformation data of the object different from the first deformation data; and based on the common part and the different part, acquire third deformation data, the third deformation data having a deformation progress state between the first deformation data and the second deformation data as a legend classification.

[0008] Further features of various embodiments will become apparent from the following description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a diagram illustrating an example of a hardware configuration of an information processing device according to an embodiment of the present disclosure.

[0010] Figure 2 is a diagram showing an example of a software configuration of an information processing device according to an embodiment of the present disclosure.

[0011] Figure 3A It is shown Figure 2 An example of a first deformed image is shown in FIG.

[0012] Figure 3B It is shown Figure 2 An example of a second deformed image is shown in FIG.

[0013] Figure 4A It is shown Figure 2 An example of the first deformation data is shown in FIG.

[0014] Figure 4B It is shown Figure 2 An example of the second deformation data is shown in FIG.

[0015] Figure 5A is achieved by visualizing Figure 4B The second deformation data shown in FIG. Figure 4A The graph shown in FIG. 1 is a graph depicting first difference determination data obtained by determining a change in the first deformation data as a difference.

[0016] Figure 5B is achieved by visualizing Figure 4A The first deformation data shown in FIG. Figure 4B The graph shown in FIG. 4 is a graph depicting second difference determination data obtained by determining the variation of the second deformation data as a difference.

[0017] Figure 6 It is shown from Figure 5A The first difference determination data shown in Figure 5B 2 is a diagram showing an example of a deformation map of third deformation data generated by second difference determination data shown in FIG.

[0018] Figure 7 It is shown in Figure 1 FIG. 4 is a diagram showing an example of a deformation map confirmation correction operation display screen displayed on a display output device.

[0019] Fig. 8A It is shown in Figure 1 FIG. 4 is a diagram showing a first display example of a deformation map confirmation correction operation display screen displayed on a display output device.

[0020] Figure 8B It is shown in Figure 1FIG. 4 is a diagram of a second display example of a deformation map confirmation correction operation display screen displayed on a display output device shown in FIG.

[0021] Figure 8C It is shown in Figure 1 FIG. 4 is a diagram of a third display example of a deformation map confirmation correction operation display screen displayed on the display output device shown in FIG.

[0022] Fig.8D It is shown in Figure 1 FIG. 4 is a diagram showing a fourth display example of a deformation map confirmation correction operation display screen displayed on a display output device.

[0023] Fig. 8E It is shown in Figure 1 FIG. 5 is a diagram of a fifth display example of a deformation map confirmation correction operation display screen displayed on a display output device shown in FIG.

[0024] Figure 8F It is shown in Figure 1 FIG. 6 is a diagram showing a sixth display example of a deformation map confirmation correction operation display screen displayed on a display output device.

[0025] FIG. 9A to FIG. 9J It is shown in Figure 1 2 is a diagram showing an example of deformation correspondence candidate patterns that can be displayed in a display area on a deformation map confirmation correction operation display screen displayed on a display output device.

[0026] Fig. 10A It shows storage Figure 4A FIG. 5 is a diagram showing an example of the structure of a first deformation data table of first deformation data.

[0027] Fig. 10B It shows storage Figure 4B FIG. 5 is a diagram showing an example of the structure of a second deformation data table of second deformation data.

[0028] Fig.11A It shows storage Figure 5A 0 is a diagram of an example of a first difference determination table of a data structure of first difference determination data shown in FIG.

[0029] Fig. 11B It shows storage Figure 5B 0 is a diagram of an example of a second difference determination table of the data structure of the second difference determination data shown in FIG.

[0030] Fig.12 is shown by enlarging Figure 4A and Fig. 10A An example of a depiction obtained by showing deformation data is shown in FIG.

[0031] Fig.13Ais shown by enlarging Fig.11A 0 is a diagram showing an example of a drawing obtained by deforming the difference determination data (deformed data) having the ID “4117” in the first difference determination table shown in FIG.

[0032] Fig. 13B is shown by enlarging Fig. 11B 2 is a diagram showing an example of a depiction obtained by deforming the difference determination data (deformed data) having IDs “4218” and “4219” in the second difference determination table shown in FIG.

[0033] Fig.14 is shown by enlarging Figure 6 The deformation diagram shown in FIG. 6 is a diagram of an example of a depiction obtained by combining deformation data indicated by symbols 6115 to 6123.

[0034] Fig.15A and Fig. 15B It shows storage Figure 6 FIG. 5 is a diagram showing an example of a deformation graph data table of a data structure of a deformation graph.

[0035] Fig.16 is a diagram illustrating an embodiment of the present disclosure, and is a diagram illustrating an example of a legend table.

[0036] Fig.17 It is shown Figure 4A The first deformation data shown in the first deformation data recorded in the first deformation and Figure 4B 2 is a diagram showing an example of arrangement of second deformations recorded in second deformation data in the same coordinate system.

[0037] Fig.18 Detailed description is a flowchart illustrating an example of a processing procedure of a deformation graph generation and editing process included in an information processing method performed by an information processing apparatus according to an embodiment of the present disclosure.

[0038] Fig.19 It is shown Fig.18 A flowchart of an example of a detailed processing procedure of the difference determination processing in step S1802.

[0039] Fig. 20 It is shown Fig.18 A flowchart of an example of a detailed processing procedure of the deformation map generation processing in step S1803. DETAILED DESCRIPTION

[0040] Embodiments for implementing the present disclosure (embodiments of the present disclosure) will be described below with reference to the accompanying drawings. Note that some embodiments of the present disclosure are not limited to the embodiments described below. Although multiple features are described in the embodiments of the present disclosure described below, not all of these multiple features are always essential, and these multiple features can be freely combined. In addition, in each of the accompanying drawings, the same symbols are used to mark the same or substantially the same construction elements, and redundant descriptions are appropriately omitted. In addition, in each of the accompanying drawings, some construction elements that are not important for the description are omitted.

[0041] In an embodiment of the present disclosure, for example, a computer device operates as an information processing apparatus. In an embodiment of the present disclosure, a change in the state of deformation of an object detected from an image of the object captured at different times, such as a new deformation, an extension of the deformation, a widening of the deformation, non-detection of the deformation due to an image defect, and disappearance of the deformation due to repair, is determined. In addition, an embodiment of the present disclosure describes an example in which a correct deformation map is generated by displaying the possibility of an error in the deformation map on a display unit so that such an error can be corrected.

[0042] Note that in the embodiments of the present disclosure, "deformation" refers to cracks generated on the surface of an object, for example, due to damage, degradation, or other factors of the object. Examples of objects include concrete structures such as highways, bridges, tunnels, and dams. "Cracks" are linear damages with a start point, an end point, a length, and a width generated on the surface of an object (e.g., a wall of a structure) due to, for example, degradation over time or an earthquake shock.

[0043] Figure 1 1 is a diagram showing an example of a hardware configuration of an information processing apparatus 100 according to an embodiment of the present disclosure. In the present embodiment, as described above, a computer device operates as the information processing apparatus 100. In this case, the processing performed by the information processing apparatus 100 may be implemented by a single computer device, or each function of the processing may be distributed among a plurality of computer devices as needed to implement the processing. In addition, in this case, a plurality of computer devices are connected to each other so that communication can be performed between them.

[0044] The information processing apparatus 100 includes a controller 101 , a nonvolatile memory 102 , a work memory 103 , a storage device 104 , an operation input device 105 , a display output device 106 , a network interface 107 , and a system bus 108 as a hardware configuration.

[0045] The controller 101 includes a processor for arithmetic processing, such as a central processing unit (CPU) or a microprocessing unit (MPU), etc., which controls the entire information processing device 100. The nonvolatile memory 102 is a read-only memory (ROM) that stores programs and parameters executed by the processor of the controller 101. In this case, the program implements the software configuration of the information processing device 100 to be described below ( Figure 2 ), and also causes the processor to execute the flowchart of the information processing method to be described below ( Figures 18 to 20 ) is a processing step shown in FIG. 1 . The working memory 103 is a random access memory (RAM) that temporarily stores programs and data supplied from external devices and other devices. The storage device 104 is an internal device (such as a hard disk or a memory card) built into the information processing apparatus 100, or an external device (such as a hard disk or a memory card) that is detachably connected to the information processing apparatus 100. The storage device 104 includes a memory card, a hard disk, or other devices such as a semiconductor memory or a magnetic disk. The storage device 104 also includes a storage medium composed of a disk drive that is connected to a hard disk such as a DVD or a Blu-ray disk. Data is read / written on an optical disk such as a disk.

[0046] The operation input device 105 includes an operation member (such as a mouse, a keyboard, and a touch panel) that accepts an operation input from a user, and outputs the operation input (instruction) from the user to the controller 101. The display output device 106 is a display device (display unit) such as a liquid crystal display (LCD), an LCD monitor, or an organic electroluminescent (EL) display or monitor, and displays data held by the information processing apparatus 100, data supplied from an external device, and other data. The network interface 107 is communicatively connected to a network (such as the Internet and a local area network (LAN)). The system bus 108 includes an address bus, a data bus, and a control bus that connect the hardware configurations (101 to 107) of the information processing apparatus 100 so that data can be exchanged.

[0047] Note that the nonvolatile memory 102 stores an operating system (OS) as basic software executed by the controller 101, and applications that work in conjunction with the OS to implement application functions. In the present embodiment, the nonvolatile memory 102 also stores applications for the information processing apparatus 100 to implement the deformation map generation process described below.

[0048] The processing performed by the information processing device 100 according to the present embodiment is realized by loading software provided by an application. It is assumed that the application has software using basic functions of the OS installed in the information processing device 100. The OS of the information processing device 100 may have software realizing the processing in the present embodiment.

[0049] Figure 2 is a diagram showing an example of a software configuration of the information processing apparatus 100 according to an embodiment of the present disclosure. The information processing apparatus 100 has, as processing target data, first deformation data 201 and second deformation data 202 of an object and first deformation images 211 and second deformation images 212 of the object. Note that in the embodiment of the present disclosure, the first deformation data 201 and second deformation data 202 of the object are different deformation data. For example, the first deformation data 201 and second deformation data 202 of the object are deformation data of the object acquired at different times. For example, the acquisition time of the second deformation data 202 is later than that of the first deformation data 201. In addition, the first deformation image 211 of the object is an image corresponding to the first deformation data 201, and the second deformation image 212 of the object is an image corresponding to the second deformation data 202. The information processing apparatus 100 includes an input unit 210, a difference determination unit 220, a deformation map generation unit 230, a legend recording unit 240, a fixed common portion detection unit 250, a deformation map editing operation unit 260, a deformation map recording unit 270, and a display data generation unit 280 as software configurations.

[0050] The input unit 210 receives the first deformation data 201 and the second deformation data 202 as processing target data, and the first deformation image 211 and the second deformation image 212 .

[0051] The difference determination unit 220 compares the first deformation data 201 and the second deformation data 202 by associating the first deformation data 201 and the second deformation data 202 with each other based on the position identification of the deformation record stored in each of the first deformation data 201 and the second deformation data 202 of the object input to the input unit 210. The difference determination unit 220 performs a process of determining a common part and a different part between the first deformation data and the second deformation data of the object based on the comparison result to determine the difference between the first deformation data 201 and the second deformation data 202. In this case, the difference determination unit 220 that performs the process for identifying the common part and the different part between the first deformation data and the second deformation data of the object constitutes the "recognition unit" in the embodiment of the present disclosure.

[0052] The deformation map generation unit 230 performs a process of combining common parts and different parts between the first deformation data and the second deformation data of the object obtained as a result of the difference determination performed by the difference determination unit 220 to generate and acquire a deformation map of the third deformation data. For example, the deformation map generation unit 230 combines the difference determination data of the second deformation data 202 with respect to the first deformation data 201 with the difference determination data of the first deformation data 201 with respect to the second deformation data 202 to generate and acquire a deformation map, wherein the difference determination data has been acquired by the difference determination unit 220. In this case, the deformation map generation unit 230 that performs the process of generating and acquiring a deformation map of the third deformation data constitutes the “acquisition unit” in the embodiment of the present disclosure.

[0053] The legend recording unit 240 holds information that defines a recording format of the deformation map of the third deformation data generated by the deformation map generating unit 230 and is referred to when the deformation map generating unit 230 generates the deformation map of the third deformation data.

[0054] The fixed common part detection unit 250 performs processing for detecting depicted items other than visible deformation (common parts not including deformation (fixed common parts)) that exist in both the first deformed image 211 and the second deformed image 212 input to the input unit 210. The fixed common part detection unit 250 that performs processing for detecting common parts not including deformation (fixed common parts) constitutes a “detection unit”.

[0055] The deformation map editing operation unit 260 displays the deformation map of the third deformation data acquired by the deformation map generation unit 230 on the display output device 106 as a display unit via the display data generation unit 280. As a result, the deformation map editing operation unit 260 enables display of potential error portions and provides a function that allows the user to select a correct deformation correspondence relationship between the first deformation data 201 and the second deformation data 202.

[0056] The deformation map recording unit 270 stores the deformation map edited (including correction and change) by the deformation map editing operation unit 260 .

[0057] The display data generating unit 280 performs processing for generating display data based on the deformation map edited (including correction and change) by the deformation map editing operation unit 260 and displays the display data on the display output device 106 as a display unit.

[0058] Figure 3A It is shown Figure 2 An example of a first deformed image 211 is shown in FIG. Figure 3A middle, Figure 2The first deformed image 211 shown in is shown as a first deformed image 3100 . Figure 3B It is shown Figure 2 An example of a second deformed image 212 is shown in FIG. Figure 3B middle, Figure 2 The second deformed image 212 shown in FIG. 1 is shown as a second deformed image 3200. For example, Figure 3A The first deformed image 3100 and Figure 3B The second deformed image 3200 shown in FIG. 3 may be an image obtained by depicting the entire wall surface of a structure (object) or a partial wall surface of the structure. Figure 3A The first deformed image 3100 and Figure 3B The second deformed image 3200 shown in FIG. 1 shows an example of an image obtained by photographing the inner wall surface of a road tunnel as an object. More specifically, Figure 3A The first deformed image 3100 and Figure 3B The second deformed image 3200 shown in FIG. 3 is a partial image divided by span, which is a unit of lining of the inner wall of a road tunnel (ie, an object). Figure 3A Regions 3101 , 3102 , and 3103 of the first deformed image 3100 shown in FIG. 3 schematically illustrate lighting equipment and other items installed on the inner wall lining of a road tunnel (ie, the object). Figure 3A The regions 3101, 3102 and 3103 shown in FIG. Figure 3B The areas 3201, 3202 and 3203 shown in FIG. 1 are identical installation items (ie, fixed common parts). Figure 3A The point 3104 shown in FIG. 310 indicates the origin coordinate (0, 0) of the first deformed image 3100, and Figure 3B The point 3204 shown in indicates the origin coordinate (0, 0) of the second deformed image 3200. In addition, Figure 3A The variations 3110 to 3117 and Figure 3B The deformations 3210 to 3220 shown in FIG. 3 each represent a crack deformation of the inner wall lining of a road tunnel (ie, an object).

[0059] Figure 4A It is shown Figure 2 FIG. 201 is an example of the first deformation data 201 shown in FIG. Figure 4A middle, Figure 2 The first deformed data 201 shown in FIG. 4 is shown as the first deformed data 4100. Specifically, Figure 4A The first deformation data 4100 shown in FIG. 4 is obtained by visualizing only the Figure 3A FIG. 3 is a diagram depicting deformation data of a deformed portion detected in the first deformed image 3100 shown in FIG. Figure 4AIn the first deformation data 4100 shown in FIG. Figure 3A The crack deformations 3110 to 3117 shown in FIG. 3 are recorded as deformation data 4110 to 4117, respectively. In addition, Figure 4A Point 4104 shown in Figure 3A Corresponding to point 3104 shown in .

[0060] Figure 4B It is shown Figure 2 FIG. 2 is a diagram showing an example of the second deformed data 202. Figure 4B middle, Figure 2 The second deformed data 202 shown in FIG. 2 is shown as second deformed data 4200. Specifically, Figure 4B The second deformation data 4200 shown in FIG. 4 is obtained by visualizing only the Figure 3B FIG. 3 is a diagram depicting deformation data of a deformed portion detected in the second deformed image 3200 shown in FIG. Figure 4B In the second deformation data 4200 shown in FIG. Figure 3B The crack deformations 3210 to 3220 shown in FIG. 3 are recorded as deformation data 4210 to 4220, respectively. In addition, Figure 4B Point 4204 shown in Figure 3B Corresponding to point 3204 shown in .

[0061] Fig. 10A is a diagram showing an example of the structure of the first deformation data table 1000, which stores Figure 4A In the first deformation data table 1000, as shown in FIG. Fig. 10A As shown in Figure 4A The deformation data 4110 to 4117 shown in the figure record deformation ID 1001, interval number 1002, interval vertex coordinates 1003, line width 1004, and interval length 1005. The deformation ID 1001 is information that uniquely identifies a single deformation data. The interval number 1002 is information indicating the interval number of the deformation data. The interval vertex coordinates 1003 are information indicating the coordinates of two vertices (i.e., the end points of the interval) for each interval. The line width 1004 is information indicating the line width of the interval. The interval length 1005 is information indicating the length of the interval. Fig. 10A In the first deformation data table 1000 shown in FIG. 1 , the line width 1004 and the interval length 1005 store information related to the actual distance (for example, in millimeters or meters) calculated according to the image resolution. Fig. 10AIn the first deformation data table 1000 shown in FIG. 1 , each deformation data can be input by a user tracing the deformation on an image using, for example, a tablet computer, can be automatically generated by image analysis processing, etc., or can be input using a combination of these methods. Image analysis processing can be performed using a learning model generated by artificial intelligence (AI) machine learning and deep learning. It is also possible to use a learning model generated by machine learning and deep learning to measure or estimate the line width 1004.

[0062] Fig.12 is shown by enlarging Figure 4A and Fig. 10A FIG. 4 is a diagram showing an example of a depiction obtained by using deformation data 4110 shown in FIG. The crack deformation is represented as vector data and is composed of line segments connecting vertices. Fig.12 In FIG. 4 , the deformation data 4110 is composed of 10 vertices and 9 interval line segments connecting these vertices. Fig.12 The vertices shown in Figure 3A Point 3104 in (and Figure 4A The X coordinate and Y coordinate of point 4104 in the image coordinate system are identified with the origin coordinate (0, 0). Fig.12 The X coordinate value of the endpoint 1201 shown in FIG. 1 is X1000_0, and the Y coordinate value is Y1000_0. The X coordinate value of the other endpoint 1202 is X1000_9, and the Y coordinate value is Y1000_9. The above situation of the deformation data 4110 can be similarly applied to Figure 4A and Fig. 10A Other deformation data 4111 to 4117 are shown in FIG.

[0063] Fig. 10B is a diagram showing an example of the structure of the second deformation data table 1010, which stores Figure 4B The second deformation data 4200 shown in FIG. In the second deformation data table 1010, as shown in FIG. Fig. 10B As shown in Figure 4B The deformation data 4210 to 4220 shown in FIG. 1 record a deformation ID 1011 , a section number 1012 , section vertex coordinates 1013 , a line width 1014 , and a section length 1015 . Fig. 10B The deformation ID 1011, the interval number 1012, the interval vertex coordinates 1013, the line width 1014 and the interval length 1015 shown in FIG. Fig. 10A The deformation ID 1001, section number 1002, section vertex coordinates 1003, line width 1004 and section length 1005 shown in FIG. 1 are substantially the same. Fig. 10B The deformation data with the deformation ID 1011 "4210" shown in is composed of (V20+1) vertices and partial intervals corresponding to (V20) interval line segments.

[0064] Figure 5A is achieved by visualizing Figure 4B The second deformation data 4200 shown in FIG. Figure 4A 4 is a graph depicting first difference determination data 5100 obtained by determining the variation of the first deformation data 4100 as a difference. Figure 5B is achieved by visualizing Figure 4A The first deformation data 4100 shown in FIG. Figure 4B The second difference determination data 5200 obtained by determining the change of the second deformation data 4200 shown in FIG. Figure 5A The first difference determination data 5100 and Figure 5B The second difference determination data 5200 is shown in FIG.

[0065] exist Figure 5A The first difference determination data 5100 and Figure 5B In the second difference determination data 5200 shown in FIG. 1 , the result determined based on whether there is a corresponding deformation interval for each partial interval of each deformation data is represented by the width of the drawn solid line. Specifically, in Figure 5A and Figure 5B In the figure, thin lines indicate "no difference" and thick lines indicate "difference". For example, Figure 5A The depicted lines assigned with the symbols 5115, 5116, 5118, 5122, and 5126 are thick lines, and thus are partial intervals where a difference occurs (there is a difference). As a depiction method for determining whether there is a difference, other recognizable display forms such as other line types or other line colors may be used. In addition, Figure 5A The vertices assigned with symbols 5119, 5121, 5123, and 5125 indicate the endpoints of the intervals where the differences occur.

[0066] Fig.11A It shows storage Figure 5A FIG. 1 is a diagram showing an example of a first difference determination table 1100 of a data structure of the first difference determination data 5100. Figure 5A The first difference determination data 5100 shown in FIG. Figure 4A The first deformation data 4100 shown in FIG. 4 is used as a reference, so Fig.11A The first difference determination table 1100 shown in FIG. Figure 4A The deformation ID 1101 of the first deformation data 4100 is shown in FIG. Fig.11AThe first difference determination table 1100 shown in FIG. 1 records, in addition to the deformation ID 1101, the section number 1102, the section vertex coordinates 1103, the line width 1104, the section length 1105, the width change 1106, and the corresponding ID 1107. In this case, the width change 1106 is information indicating the section width change (difference). The corresponding ID 1107 is information indicating the deformation ID of the deformation data determined to be at the same position.

[0067] Fig. 11B It shows storage Figure 5B FIG. 10 is a diagram of an example of a second difference determination table 1110 of the data structure of the second difference determination data 5200 shown in FIG. Figure 5B The second difference determination data 5200 shown in FIG. Figure 4B The second deformation data 4200 shown in FIG. 4 is used as a reference, so Fig. 11B The second difference determination table 1110 shown in FIG. 1 stores Figure 4B The deformation ID 1111 of the second deformation data 4200 is shown in FIG. Fig. 11B The second difference determination table 1110 shown in FIG. 1 records, in addition to the deformation ID 1111 , an interval number 1112 , interval vertex coordinates 1113 , line width 1114 , interval length 1115 , width change 1116 , and a corresponding ID 1117 .

[0068] exist Fig.11A In the first difference determination table 1100, in the case where the interval has the same interval vertex coordinates 1003 as the interval in the first deformation data table 1000, the line width 1104 and the interval length 1105 of the interval are respectively the same as the line width and the interval length of the same interval in the first deformation data table 1000. In addition, in the case where the interval has different interval vertex coordinates, Fig.11A The first difference determination table 1100 shown in FIG. 1 stores the average value of the line width values ​​of adjacent intervals as the line width 1104. Fig.11A In the first difference determination table 1100 shown in FIG. 1107 , in the case where there is no section matching the corresponding deformation data indicated by the corresponding ID 1107 , “no corresponding relationship” is stored in the width change 1106 and the corresponding ID 1107 . Fig.11A In the case where there is an interval that matches the corresponding deformation data indicated by the corresponding ID 1107, when the value obtained by subtracting the value of the line width 1114 from the value of the line width 1104 is a negative value, "-" (minus) is stored; when the value is 0, "0" (zero) is stored; and when the value is positive, "+" (plus) is stored.

[0069] and Fig.11A Similar to the above case of the first difference determination table 1100 shown in FIG. 1 , substantially the same structure is applicable to Fig. 11B The second difference determination table 1110 shown in FIG. Fig. 11B In the second difference determination table 1110 shown in , a change value determined based on a value obtained by subtracting a value of the line width 1104 from a value of the line width 1114 is stored in the width change 1116 .

[0070] Fig.11A The first difference determination table 1100 shown in Fig. 11B The second difference determination table 1110 shown in FIG. 1 stores the difference values ​​represented as Figure 5A and Figure 5B All differences in the depicted data are determined.

[0071] Fig.13A is shown by enlarging Figure 5A The symbols 5120 to 5126 shown in the figure represent an example of a depiction of the difference determination data obtained. Specifically, Fig.13A is shown by enlarging Fig.11A FIG. 1 is a diagram showing an example of a depiction obtained by deforming the difference determination data (deformed data) whose ID 1101 is "4117" in the first difference determination table 1100 shown in FIG. Fig.11A As shown in the first difference determination table 1100 in FIG. 1 , the difference determination data with the deformation ID 1101 being "4117" is composed of 7 vertices and 6 intervals. Fig.13A Vertices P0, P1, P2, and P3 shown in are vertices recorded in the difference determination data (deformation data) whose deformation ID 1101 is "4117", and vertices C1, C2, and C3 are vertices that specify corresponding matching sections. Fig.13A The intervals connected by vertices C1-C2 are shown in Fig.13A The interval connected by vertices C3-P3 shown in is the corresponding matching interval.

[0072] Fig. 13B is shown by enlarging Fig. 11B 2 is a diagram of an example of a depiction obtained by deforming the difference determination data (deformed data) having the IDs 1111 of “4218” and “4219” in the second difference determination table 1110 shown in FIG. Fig. 13B The difference determination data whose deformation ID 1111 is "4218" shown in FIG. 1 is determined to match a portion of the difference determination data whose deformation ID 1111 is "4117", and is therefore drawn with a thin line. Fig. 13B The difference determination data with deformation ID 1111 of "4219" shown in FIG. 1 is composed of four vertices and three intervals. Fig. 13BIn the difference determination data with deformation ID 1111 of "4219" shown in FIG, the section formed by connecting vertices P6-C4 matches the section with deformation ID 4117 and is depicted with a thin line. The two sections formed by connecting vertices C4-P7-P8 are depicted with a thick line as sections with a difference.

[0073] Fig.11A The example shown in FIG. 1 shows that the difference determination data of the deformation ID 1101 being "4110" corresponds to the deformation ID 1011 being "4210" and is stored in Fig. 10B The data in the second deformation data table 1010 is shown in FIG. Fig.11A The example shown in shows that the difference determination data of the deformation ID 1101 being "4110" matches the data of the deformation ID 1011 being "4210" without any change in width because the value in the width change 1106 is "0". Fig.11A The first difference determination table 1100 shown in FIG. Figure 4A The interval data recorded in the first deformation data 4100 shown in FIG. 1 stores the difference determination result, and does not store the interval data recorded only in FIG. Figure 4B Similarly, Fig. 11B The second difference determination table 1110 shown in FIG. Figure 4B The interval data recorded in the second deformation data 4200 shown in FIG. 1 stores the difference determination result, and does not store the interval data recorded only in FIG. Figure 4A The data in the first deformation data 4100 is shown in FIG.

[0074] Figure 6 It is shown from Figure 5A The first difference determination data 5100 and Figure 5B FIG. 6 is a diagram showing an example of a deformation graph (6100) of third deformation data generated by the second difference determination data 5200 shown in FIG. In the deformation graph (6100), the deformation classification legend is not about whether there is a difference, but about whether deformation progress is observed. Figure 6 The deformation diagram (6100) of the third deformation data shown in FIG. 200 has a legend classification indicated in a dotted frame for the state of deformation progress of the first deformation data 201 and the state of deformation progress of the second deformation data 202. Figure 6 In the deformation diagram (6100) shown in FIG. 6 , the deformation determined as “no deformation progress observed” is depicted with a thin solid line, and the deformation determined as “deformation progress observed” is depicted with a thick solid line. Figure 6In the deformation graph (6100) shown in , deformations determined to be changes in the direction of improvement are determined as "differences to be confirmed" and are depicted with dotted lines to distinguish them from "no deformation progress observed" and "deformation progress observed". In this case, the change in the direction of improvement means that the deformation disappears or the width of the deformation decreases. Once a deformation occurs, the deformation does not usually disappear or decrease in width spontaneously. Therefore, deformations determined as "differences to be confirmed" enable the user to determine for each deformation whether the repair work is the correct result of the disappearance of the deformation, or whether a deformation that should have been detected was missed due to poor AI deformation detection. Similar to the display legend of the difference determination data, the depiction method of the deformation legend in the deformation graph (6100) (such as line type or line color, etc.) can be changed to other recognizable display forms. Deformations that require special attention (such as "deformation progress observed" or "differences to be confirmed", etc.) can be displayed in an exaggerated manner.

[0075] Fig.14 is shown by enlarging Figure 6 A diagram of an example of a depiction obtained by combining deformation data indicated by symbols 6115 to 6123 included in the deformation diagram (6100) shown in FIG. Fig.14 The symbols P0 to P8 and C1 to C4 in Fig.13A and Fig. 13B The symbols in correspond to Fig.14 , symbols P0 to P8 are vertices constituting the deformed data to be combined, and symbols C1 to C4 are vertices added to define a partial matching section in the difference determination data.

[0076] Fig.15A and Fig. 15B It shows storage Figure 6 FIG. 15 is a diagram showing an example of a deformation map data table 1500 of a data structure of a deformation map (6100) shown in FIG. The deformation map data table 1500 stores Figure 5A and Figure 5B 1501, interval number 1502, interval vertex coordinates 1503, line width 1504, and interval length 1505 corresponding to the deformation data shown in FIG. 1500. In addition, the deformation diagram data table 1500 also stores a deformation legend 1506 indicating a determination as to whether deformation progress is observed, and a corresponding ID 1507 in the deformation data serving as a comparison source. For example, data whose deformation ID 1501 is 4210 and whose interval number 1502 ranges from 1 to 3 each has "no corresponding relationship" as its corresponding ID 1507, and "deformation progress is observed" as its deformation legend 1506. Accordingly, data whose deformation ID 1501 is 4210 and whose interval number 1502 ranges from 1 to 3 is not included in the deformation data serving as a comparison source, and is not included in the deformation data serving as a comparison source. Figure 6, which corresponds to the symbol 6101 in the comparison data. In contrast, the data whose deformation ID 1501 is 4210 and whose interval number 1502 is 4 has "4110" as its corresponding ID 1507. Figure 6 Symbol 6103 in , symbol 6103 indicates a deformation interval of "no deformation progress observed" which is included in the deformation data serving as the comparison source and has no difference therewith.

[0077] Fig.18 : is a flowchart showing an example of deformation graph generation and editing processing included in the information processing method performed by the information processing device 100 according to the embodiment of the present disclosure.

[0078] In the present embodiment, a first deformed image 211 of an object photographed at different times in the past, first deformed data 201 generated from the first deformed image 211, a second deformed image 212 of the object photographed most recently, and second deformed data 202 generated from the second deformed image 212 are input to the input unit 210. The first deformed data 201 and the second deformed data 202 are data obtained from the first deformed image 211 and the second deformed image 212, respectively, by performing deformation detection by, for example, AI. Then, the difference determination unit 220 identifies common parts and different parts between the first deformed data 201 obtained in the past and the second deformed data 202 obtained most recently later than the acquisition time of the first deformed data 201. Then, the deformation map generation unit 230 combines the common parts and different parts of the first deformed data 201 and the second deformed data 202 of the object identified by the difference determination unit 220 to generate and acquire a deformation map of third deformed data. Thereafter, the deformation map editing operation unit 260 displays the deformation map of the third deformation data acquired by the deformation map generating unit 230 on the display output device 106 as a display unit via the display data generating unit 280 , and edits the deformation map.

[0079] First, in Fig.18 In step S1801, the input unit 210 starts a deformation map generation application (not shown), and inputs the first deformation data 201, the second deformation data 202, the first deformation image 211, and the second deformation image 212 as processing target data.

[0080] Next, in Fig.18In step S1802, the difference determination unit 220 refers to the first deformed image 211 and the second deformed image 212 to perform a difference determination process for identifying a common portion and a different portion between the first deformed data 201 and the second deformed data 202. In this case, the difference determination unit 220 generates, for example, a first difference determination table 1100 and a second difference determination table 1110 in the working memory 103.

[0081] Next, in Fig.18 In step S1803, the deformation map generation unit 230 combines the common part and the different part of the first deformation data 201 and the second deformation data 202 of the object to generate and obtain a deformation map of the third deformation data. In this case, for example, the deformation map generation unit 230 generates in the working memory 103 a deformation map data table 1500 obtained by referring to and combining the first difference determination table 1100 and the second difference determination table 1110 generated in the working memory 103.

[0082] Next, in Fig.18 In step S1804, the deformation diagram editing operation unit 260 displays a user interface for performing editing operations and the deformation diagram data table 1500 generated in the working memory 103 on the display output device 106 via the display data generation unit 280. In this case, the deformation diagram editing operation unit 260 may take the form of displaying a deformation diagram based on the deformation diagram data table 1500 on the display output device 106. Thereafter, the deformation diagram editing operation unit 260 updates the deformation diagram data table 1500 according to the user's confirmation, correction, and selection operation instructions. In addition, according to a save instruction from the user, the deformation diagram recording unit 270 saves the deformation diagram data table 1500 in the storage device 104. According to an edit end instruction from the user, the deformation diagram editing operation unit 260 ends the operation with the deformation diagram. Fig.18 The processing in the flowchart related to the deformation graph generation and editing processing shown in FIG.

[0083] Fig.19 It is shown Fig.18 A flowchart of an example of a detailed processing procedure of the difference determination processing in step S1802.

[0084] First, in Fig.19 In step S1901 , the difference determining unit 220 inputs the data of the first deformed image 211 and the data of the second deformed image 212 stored in the working memory 103 as deformed image data to be processed.

[0085] Next, in Fig.19In step S1902, for example, the fixed common part detection unit 250 detects the position of the fixed common part commonly depicted in the data of the first deformed image 211 and the data of the second deformed image 212. In the present embodiment, as a fixed object used as a fixed common part, a rectangular object other than the deformed area and recorded in the deformed image is regarded as a detection object as a candidate for a fixed object. This is because rectangular features can be seen, for example, in lighting equipment and road signs installed on the inner wall lining of a tunnel as an object. In this rectangle detection, contour lines are extracted from the first deformed image 211 by performing a Hough transform or using a Sobel filter, and a combination of line segments forming a rectangular area is selected. Thereafter, a template matching search is performed on the second deformed image 212 using each selected rectangular area as a pattern, and a matching result whose position is closest to the rectangular area used as a pattern is determined as a fixed object. A certain vertex of the rectangle on the first deformed image 211 and a certain vertex of the rectangle on the second deformed image 212 are set as a first fixed point and a second fixed point, respectively. For example, from Figure 3A Detect rectangular areas 3101, 3102, and 3103 as fixed objects in the first deformed image 3100 (211) shown in FIG. 1 , and set the upper left vertex of the rectangular area 3103 as the first fixed point. Similarly, for example, Figure 3B Rectangular regions 3201, 3202, and 3203 are detected as fixed objects in the second deformed image 3200 (212) shown in FIG. Then, the upper left vertex of the rectangular region 3203 is set as the second fixed point, and the rectangular region 3203 can be estimated as a fixed object located at the same position as the region 3103 of the fixed object.

[0086] Next, in Fig.19 In step S1903, for example, the difference determination unit 220 calculates the relative distance between the first fixed point and the second fixed point set in step S1902 as an offset value. In this case, the offset value is obtained as the difference ΔX between the X coordinate values ​​and the difference ΔY between the Y coordinate values.

[0087] Next, in Fig.19 In step S1904, for example, the difference determination unit 220 corrects the difference by adding the offset value (ΔX, ΔY) calculated in step S1903. Fig. 10A As a result, even if there is a displacement between the imaging range of the first deformed image 211 and the imaging range of the second deformed image 212, the position coordinates on the inner wall lining of the tunnel as the object will match.

[0088] In the difference determination processing performed by the difference determination unit 220, since the difference is obtained by fixing one of two different data as reference data and treating the other data as variable data, it is determined which of the first deformed data 201 and the second deformed data 202 will be used as reference data.

[0089] Next, in Fig.19 In step S1905, the difference determination unit 220 sets the first deformed data 201 as the reference data for the difference determination process. When setting the first deformed data 201, the Fig. 10A The first deformation data table 1000 is shown in FIG.

[0090] Next, in Fig.19 In step S1906, the difference determination unit 220 starts a loop for processing all deformation IDs 1001 recorded in the first deformation data table 1000 set in step S1905.

[0091] Next, in Fig.19 In step S1907 , the difference determining unit 220 starts a loop for processing sections of all section numbers 1002 in the deformation data having the same deformation ID 1001 .

[0092] Next, in Fig.19 In step S1908, the difference determination unit 220 searches whether the interval segment indicated by the interval vertex coordinates 1003 matches the second deformed data 202 on the opposite data side, and obtains the deformation ID of the second deformed data having the matching interval segment. In this case, by performing the coordinate correction processing in step S1904, the approximate coordinates of the deformed data recorded at the same position should match. However, due to the recording error caused by the difference between the deformation recording methods and the detection error generated when the AI ​​detects the deformation, an allowable range is set to determine whether the deformed data is recorded at the same position. In this case, the width of the allowable range is represented by AW. By forming a rectangular interval obtained by widening both sides of the interval segment of the first deformed data 201 by AW in the vertical direction, and determining whether the interval of the second deformed data 202 exists in the rectangular interval, the co-location with an error tolerance can be determined.

[0093] In contrast, a case will be described in which not only is image misalignment caused by the difference in the imaging range, but a single deformed image is generated by combining the separately captured images. In this case, when the imaging conditions (imaging equipment, lens angle of view, and imaging distance to the inner wall lining) change between the capturing time of the first deformed image 211 and the capturing time of the second deformed image 212, local misalignment may occur in the combined image.

[0094] Therefore, as described below Fig.17 As shown in , the allowable range AW when searching for the same deformation between two deformation data can be set larger to increase the photographing rate of the same deformation.

[0095] Fig.17 It is shown Figure 4A The first deformations 4114, 4115 and 4116 recorded in the first deformation data 4100 shown in FIG. Figure 4B 4 is a diagram showing an example of arrangement of second deformations 4216 and 4217 recorded in the second deformation data 4200 in the same coordinate system. Fig.17 The dashed boxes R1, R2, and R3 shown in FIG. 4 are rectangular intervals used during the search for a corresponding deformation for the second deformation 4216, and the length of each arrow is AW. Since the first deformations 4114, 4115, and 4116 are also included in the rectangular dashed boxes R1, R2, and R3 indicating the corresponding allowable range of the second deformation 4216, it is determined that any one of the first deformations 4114, 4115, and 4116 can correspond to the second deformation 4216.

[0096] In this case, possible combinations of candidates for corresponding deformations are generated, and corresponding deformations are determined based on evaluation values ​​obtained from the distance from the reference deformation and the values ​​of the interval length and interval width of the deformation. The combinations of candidates that are not selected are stored in the working memory 103 in the same format as the difference determination table. Fig.17 In the case of, for the combination of candidates for corresponding deformations stored in the working memory 103, there are the following described FIG. 9A to FIG. 9J The 10 patterns are shown.

[0097] Now, back to Fig.19 Description.

[0098] when Fig.19 When the processing in step S1908 is completed, the processing proceeds to step S1909.

[0099] When processing enters Fig.19 In step S1909, the difference determination unit 220 registers the deformation ID of the second deformation data 202 determined to be at the same position as the deformation correspondence relationship data in the correspondence ID 1107 of the first difference determination table 1100. Fig. 10A The reference data of the first deformation data table 1000 shown in FIG. Fig.11A1101, section number 1102, section vertex coordinates 1103, line width 1104, and section length 1105 of the first difference determination table 1100 shown in FIG. Then, the difference determination unit 220 compares the line width 1014 of the matching section of the second deformed data 202 indicated by the corresponding ID with the line width 1004 of the first deformed data 201 as a reference, and registers the width change to Fig.11A In the width variation 1106 of the first difference determination table 1100 shown in FIG. Fig.11A In the width change 1106 in , when the result of subtracting the line width 1014 from the line width 1004 is negative, "-" (minus) is registered; when the result is "0", "0" is registered; and when the result is positive, "+" (plus) is registered. In the case where comparison cannot be made because the corresponding ID is not obtained, "no corresponding relationship" is recorded.

[0100] Next, in Fig.19 In step S1910, the difference determination unit 220 determines whether the processing for all sections corresponding to the same deformation ID has been completed. As a result of this determination, in the case where the processing for all sections corresponding to the same deformation ID has not been completed, an unprocessed section is set, and the processing in and after step S1907 will be performed.

[0101] exist Fig.19 If it is determined in step S1910 that the processing for all sections corresponding to the same deformation ID has been completed, the processing proceeds to step S1911.

[0102] When processing enters Fig.19 In step S1911, the difference determination unit 220 determines whether the processing for all deformation IDs has been completed. In the case where it is determined that the processing for all deformation IDs is not completed, the unprocessed deformation ID is set, and the processing in and after step S1906 will be performed.

[0103] exist Fig.19 If it is determined in step S1911 that processing for all deformation IDs has been completed, the process proceeds to step S1912.

[0104] When processing enters Fig.19 At step S1912, the difference determination unit 220 determines whether the processing of the second deformed data 202 has been completed.

[0105] exist Fig.19 If it is determined in step S1912 that the processing of the second deformed data 202 is not completed (S1912 / No), the processing proceeds to step S1913.

[0106] When processing enters Fig.19In step S1913, the difference determination unit 220 sets the reference data for the difference determination processing to the second deformed data 202. When setting the second deformed data 202, Fig. 10B . Then, in steps S1906 to S1911, the following processing is performed: the second deformed data 202 and the second deformed data table 1010 are used to replace the first deformed data 201 and the first deformed data table 1000, respectively.

[0107] exist Fig.19 In the case where it is determined in step S1912 that the processing of the second deformed data 202 has been completed (S1912 / Yes), a first difference determination table 1100 for the first deformed data 201 is generated in the working memory 103. Similarly, a second difference determination table 1110 for the second deformed data 202 is generated in the working memory 103. Thereafter, Fig.19 The processing in the flowchart shown in is completed.

[0108] Fig. 20 It is shown Fig.18 A flowchart of an example of a detailed processing procedure of the deformation map generation processing in step S1803.

[0109] First, in Fig. 20 In step S2001, the deformation map generation unit 230 sets the second difference determination table 1110 for the second deformation data 202 as difference determination data used as a reference for generating a deformation map. Fig.18 The deformation map generation processing in step S1803 is performed to improve the accuracy of the deformation map of the second deformation data 202 detected from the second deformed image 212 which is the most recently captured image.

[0110] Next, in Fig. 20 In step S2002, the deformation map generation unit 230 starts a loop for processing all deformation IDs 1111 recorded in the second difference determination table 1110 set in step S2001.

[0111] Next, in Fig. 20 In step S2003 , the deformation map generation unit 230 starts a loop for processing sections of all section numbers 1112 in the deformation data having the same deformation ID 1111 .

[0112] Next, in Fig. 20 In step S2004, the deformation map generation unit 230 determines whether there is a corresponding ID 1117 for the section being processed.

[0113] exist Fig. 20If it is determined in step S2004 that the corresponding ID 1117 of the section being processed exists (S2004 / Yes), that is, the deformation ID corresponding to the corresponding ID 1117 of the section being processed is registered, the process proceeds to step S2005.

[0114] When processing enters Fig. 20 In step S2005 , the deformation map generation unit 230 searches and checks the corresponding interval data of the corresponding first deformation ID, and obtains the value of the width change 1106 . Fig.16 1600 is a diagram illustrating an embodiment of the present disclosure, and is a diagram illustrating an example of a legend table 1600 . Fig.16 The legend table 1600 shown in FIG. 1 stores a difference determination table type 1601, a width change 1602, a correspondence relationship 1603, and a determination classification 1604. Regarding the first difference determination table 1100, the deformation map generation unit 230 compares the corresponding ID with the corresponding ID. Fig.16 The legend table 1600 shown in FIG. 1 examines the value of the width variation 1106 to determine Fig.16 1604 of the legend table 1600 shown in FIG. 1604. For example, when the value of the width change 1106 is "0", the determination classification 1604 is determined as "no deformation progress is observed". The deformation map generation unit 230 deletes the data of the corresponding interval of the first difference determination table 1100 in which the determination classification 1604 is determined so as not to be referred to again.

[0115] Next, in Fig. 20 In step S2006, the deformation diagram generation unit 230 registers the section data being processed of the second difference determination table 1110 into Fig.15A and Fig. 15B The deformation diagram data table 1500 is shown in FIG.

[0116] exist Fig. 20 If it is determined in step S2004 that the corresponding ID 1117 of the section being processed does not exist (S2004 / No), that is, the corresponding ID 1117 of the section being processed is "no corresponding relationship", the processing proceeds to step S2007.

[0117] When processing enters Fig. 20 In step S2007, the deformation map generation unit 230 interprets the interval as a new deformation that has been generated and does not exist in the first deformation data 201, and Fig.16 The determined classification 1604 of the legend table 1600 shown in FIG. 1 is determined as "deformation progress is observed". Then, the deformation map generation unit 230 registers the interval data as "deformation progress is observed" in the Fig.15A and Fig. 15B The deformation diagram data table 1500 is shown in FIG.

[0118] exist Fig. 20 The processing in step S2006 has been completed or Fig. 20 If the processing in step S2007 has been completed, the processing proceeds to step S2008.

[0119] When processing enters Fig. 20 In step S2008, the deformation map generation unit 230 determines whether the processing for all sections corresponding to the same deformation ID has been completed. As a result of this determination, if the processing for all sections corresponding to the same deformation ID has not been completed, an unprocessed section is set, and the processing in and after step S2003 will be performed.

[0120] In addition, Fig. 20 If it is determined in step S2008 that the processing for all sections corresponding to the same deformation ID has been completed, the processing proceeds to step S2009.

[0121] When processing enters Fig. 20 In step S2009, the deformation map generation unit 230 determines whether the processing for all deformation IDs has been completed. In the case where it is determined that the processing for all deformation IDs is not completed, the unprocessed deformation ID is set, and the processing in and after step S2002 will be performed.

[0122] exist Fig. 20 If it is determined in step S2009 that the processing for all deformation IDs has been completed, the processing proceeds to step S2010.

[0123] When processing enters Fig. 20 In step S2010, the deformation map generation unit 230 sets the first difference determination table 1100 for the first deformed data 201 as difference determination data used as a reference for generating a deformation map. This is done to verify whether to add the first deformed data 201 not detected in the second deformed data 202.

[0124] Next, in Fig. 20 In step S2011, the deformation map generation unit 230 starts a loop for processing all deformation IDs 1101 recorded in the first difference determination table 1100 set in step S2010.

[0125] Next, in Fig. 20 In step S2012 , the deformation map generation unit 230 starts a loop for processing sections of all section numbers 1102 in the deformation data having the same deformation ID 1101 .

[0126] Next, in Fig. 20In step S2013, the deformation map generation unit 230 determines whether the corresponding ID 1107 of the section being processed exists.

[0127] exist Fig. 20 If it is determined in step S2013 that the corresponding ID 1107 of the section being processed does not exist (S2013 / No), that is, the deformation ID corresponding to the corresponding ID 1107 of the section being processed is "no corresponding relationship", the processing enters step S2014.

[0128] When processing enters Fig. 20 In step S2014, the deformation map generation unit 230 searches and checks the corresponding interval data of the corresponding second deformation ID, and obtains the value of the corresponding joint vertex. Fig.13A In the example of difference determination data having a deformation ID of "4117" shown in Fig.11A In the first difference determination table 1100 shown in FIG. 1 , when the deformation ID 1101 is 4117 and the section number 1102 is 3, the corresponding ID 1107 is "4218". Fig. 11B When searching for data with a deformation ID 1111 of "4218" in the second difference determination table 1110 shown in FIG. 1 , difference determination data with a deformation ID 1111 of 4218, an interval number 1112 of 1, and a corresponding ID 1117 of "4117" is obtained. The interval vertex coordinates 1113 ((X2008_0, Y2008_0) and (X2008_1, Y2008_1)) in the second difference determination table 1110 are Fig. 13B Therefore, the deformation map generation unit 230 will Fig.13A The coordinate values ​​of the vertex C1 (X1017_C1, Y1017_C1) and the coordinate values ​​of the vertex C2 (X1017_C2, Y1017_C2) shown in Fig. 13B The coordinate value of vertex P4 and the coordinate value of vertex P5 are shown in FIG. 2. In addition, the deformation map generation unit 230 corrects other interval vertices P0, P1, P2, C3, and P3 of the same deformation ID "4117" by adding the differences (C1-P4) and (C2-P5).

[0129] Next, in Fig. 20 In step S2015, the deformation map generation unit 230 refers to Fig.16 The legend table 1600 shown in FIG. 16A and determines the determination classification 1604 as "difference to be confirmed" because "no corresponding relationship" is obtained for the first difference determination table 1100. Then, the deformation diagram generation unit 230 registers the interval data as "difference to be confirmed" to Fig.15A and Fig. 15B The deformation diagram data table 1500 is shown in FIG.

[0130] exist Fig. 20 In the case where the processing in step S2015 has been completed, or in the case where Fig. 20 If it is determined in step S2013 that the corresponding ID 1107 of the section being processed exists (S2013 / Yes), the process proceeds to step S2016.

[0131] When processing enters Fig. 20 In step S2016, the deformation map generation unit 230 determines whether the processing for all sections corresponding to the same deformation ID has been completed. As a result of this determination, if the processing for all sections corresponding to the same deformation ID has not been completed, an unprocessed section is set, and the processing in and after step S2012 will be performed.

[0132] exist Fig. 20 If it is determined in step S2016 that the processing for all sections corresponding to the same deformation ID has been completed, the processing proceeds to step S2017.

[0133] When processing enters Fig. 20 In step S2017, the deformation map generation unit 230 determines whether the processing for all deformation IDs has been completed. As a result of this determination, in the case where it is determined that the processing for all deformation IDs is not completed, an unprocessed deformation ID is set, and the processing in and after step S2011 will be performed.

[0134] exist Fig. 20 If it is determined in step S2017 that the processing for all deformation IDs has been completed, a new number is generated in the working memory 103. Figure 6 The deformation diagram (6100) shown in Fig.15A and Fig. 15B The deformation diagram data table 1500 shown in FIG. Fig. 20 The processing in the flowchart shown in is completed.

[0135] Once the deformation map generation unit 230 generates the deformation map of the third deformation data, the user can perform confirmation and correction processing (change processing) on ​​the deformation map. When the application for confirming and correcting the deformation map is started by the application starting unit not shown, the above-mentioned Fig.18 The deformation map confirmation correction processing in step S1804 is performed. Fig.18 The processing in step S1804 will use Figures 7 to 9J The application execution screen shown in FIG.

[0136] Figure 7 It is shown in Figure 1FIG. 7 is a diagram showing an example of a deformation diagram confirmation correction operation display screen 7000 displayed on the display output device 106. Figure 7 On the deformation map confirmation correction operation display screen 7000 shown in FIG. 7 , the deformation map and the deformation image are displayed in the display area 7010. The deformation map and the deformation image are divided by span (i.e., the unit of lining) and displayed in the display area 7010, and the display range can be specified by operating a slider 7013 or a back button 7011 and a forward button 7012.

[0137] exist Figure 7 , the deformation image to be confirmed (third deformation data) is displayed in an enlarged manner in a display area 7021. In a display area 7023, only the first deformation image 211 including the first deformation data 201 is displayed in an enlarged manner. In a display area 7024, only the second deformation image 212 including the second deformation data 202 is displayed in an enlarged manner. In addition, in the display area 7022, an image obtained by superimposing the first deformation data 201 and the second deformation data 202 is displayed in an enlarged manner. In addition, a selection interval number display unit 7031, a deformation total number display unit 7032, a confirmation completion number display unit 7033, and a confirmation recommendation number display unit 7034 are provided to the right side of the display area 7022. Whenever the user operates the switching display button 7041, the deformed image to be superimposed and displayed on the display area 7010 is switched in the order of the first deformed image 211, the second deformed image 212, and the non-displayed image. The display area 7042 displays the current switching state controlled by the switching display button 7041. The confirmation button 7043 is operated by the user to set the state of confirming the correction object deformation. The previous button 7044 is operated by the user to display the previous confirmed correction object deformation. The next button 7045 is operated by the user to display the next confirmed correction object deformation. The save button 7046 is operated by the user to save the deformation map in the storage device 104. The end button 7047 is operated by the user to end the entire deformation map confirmation correction process. The confirm all button 7048 is operated by the user to confirm and set all deformations as objects of recommended confirmation.

[0138] When the user Figure 7 When the confirmation button 7043 is operated on the deformation map confirmation and correction operation display screen 7000 shown in FIG. 7 , the deformation map data table 1500 generated in the working memory 103 is loaded into the application, and the deformation map confirmation and correction process is started.

[0139] Fig. 8A It is shown Figure 1 FIG. 7 is a diagram showing a first display example of a deformation diagram confirmation correction operation display screen 7000 displayed on the display output device 106. Fig. 8AIn, with Figure 7 The same structural elements shown in the drawings will be represented by the same symbols, and their detailed description will be omitted. Figure 7 , when the user operates the confirmation button 7043, the deformation image based on the deformation image data table 1500 loaded in the application is displayed in the display area 7010. In the present embodiment, the superimposed display with the second deformed image 212 is specified as the default value of the display switching, so the deformation image (third deformation data) based on the deformation image data table 1500 is superimposed on the second deformed image 212 and displayed in the display area 7010. In addition, a part of the deformation image that is the object of the recommendation confirmation is displayed in the display area 7021. Fig. 8A The rectangular frame 8001 within the range of the display area 7010 in FIG. 8 indicates an area corresponding to the display area 7021. Assume that the rectangular frame 8001 is formed by adding a predetermined width to the display area 7010 in FIG. Fig.17 The area obtained by combining the top, bottom, left and right sides of the boundary area of ​​the combination of deformation data having deformation correspondence shown in . Note that the rectangular frame 8001 can be the boundary area of ​​a single combination of deformation data having a single deformation correspondence, rather than the boundary area of ​​multiple combinations of deformation data having multiple deformation correspondences. In addition to the partial pattern of the deformation map determined in the deformation map generation process, the deformation data of the deformation map that is the object of recommendation confirmation will also be a part of the deformation map from which other configuration patterns can be selected. Other configuration patterns that are not selected when determining in the deformation map generation process are retained in the working memory 103 as other candidate tables. The "3" displayed in the confirmation recommendation number display unit 7034 will be the total number of the deformation map parts that are the objects of recommendation confirmation.

[0140] exist Fig. 8A In the display area 7022, the deformed parts that are the objects of recommendation for confirmation are displayed, wherein in these deformed parts, the first deformed data 201 and the second deformed data 202 are superimposed on each other so that the corresponding deformed pairs are located at the same position. Note that if the deformed parts are completely superimposed on each other at the same position, it is difficult to confirm the deformation correspondence. Therefore, the display area 7022 displays the deformed parts at positions shifted by a width similar to that of the display line. When the deformation lines are displayed in the display area 7022, the deformation pairs of the same line width are considered to be generated at the same position, which is the deformation correspondence. In the display area 7022, in order to facilitate the distinction between the deformation data, the deformation ID that identifies the deformation data is displayed as a pull-out line display of "4114", "4115", "4116", "4216" and "4217". In this case, if the deformation ID of the first deformation data 201 is displayed on one side of the deformation data and the deformation ID of the second deformation data 202 is displayed on the other side of the deformation data, it is easier to distinguish the first deformation data 201 and the second deformation data 202 from each other. In Fig. 8AIn the example of the deformation correspondence candidate "1 / 10" displayed in the display area 7022 in , there is a deformation correspondence relationship between the deformation IDs "4114" and "4216" drawn with the same line width. Fig. 8A In the example of the deformation correspondence candidate "1 / 10" displayed in the display area 7022 in FIG. 7 , there is a deformation correspondence relationship between the deformation IDs "4115" and "4217" drawn with the same line width. Note that the deformation ID can be erased by inputting a specific key input sequence (not shown) or by using other methods. FIG. 8C to FIG. 8F This is an example of displaying the erased deformation ID. The deformation image displayed in the display area 7021 is a result of combining deformation images that are considered to be the same deformation. Fig. 8A The display area 7022 in the display area 7022 displays three deformation pairs of different coarseness. The "1 / 10" displayed in the lower center of the display area 7022 indicates that there are 10 combination patterns of the corresponding first deformation data 201 and second deformation data 202, and the first of the 10 combination patterns is displayed. The left and right arrows displayed in the display area 7022 are operation icons, and by operating these operation icons, the "previous" or "next" deformation correspondence combination is displayed in the display area 7022.

[0141] exist Fig. 8A In the display area 7023 in FIG. 7021 , only the first deformed data 201 and the first deformed image 211 of the deformed image portion displayed in the display area 7021 are superimposed and displayed. Fig. 8A In display area 7024 in the display area 7021, only the second deformed data 202 and the second deformed image 212 of the deformed image portion displayed in display area 7021 are superimposed and displayed. For example, in display area 7024, the deformation is depicted using the same line width as the deformation displayed in display area 7023, so that the deformation correspondence relationship between the deformation data in display area 7023 and the deformation data in display area 7024 can be determined.

[0142] operate Fig. 8A The right arrow in the display area 7022 in the display area 7022 will display another deformation correspondence combination of the first deformation data 201 and the second deformation data 202, and change to Figure 8B The display screen shown in .

[0143] Figure 8B It is shown Figure 1 FIG. 7 is a diagram showing a second display example of the deformation diagram confirmation correction operation display screen 7000 displayed on the display output device 106. Figure 8B In, with Figure 7 and Fig. 8A The same configuration elements shown in FIG. 1 will be denoted by the same symbols, and a detailed description thereof will be omitted. Figure 8BThe display area 7022 in the display is indicated by the rectangular frame 8001 and Fig. 8A The deformation correspondence relationship combination "2 / 10" of different deformation map parts in the two sets of deformation correspondence remains the same, but Figure 8B The deformation correspondence relationship distinguished by the thickness of the drawing line in the display area 7022 in is different from Fig. 8A The "1 / 10" pattern in the video. Figure 8B Display area 7021 in shows the result of re-determining the difference based on the candidate pattern "2 / 10" and updating the deformation map.

[0144] FIG. 9A to FIG. 9J It is shown in Figure 1 FIG. 7 is a diagram showing an example of a deformation correspondence candidate pattern that can be displayed in the display area 7022 of the deformation map confirmation correction operation display screen 7000 displayed on the display output device 106. Figure 8B In, and use Fig. 8A The situation described above is similar. By the user operating the right arrow operation icon or the left arrow operation icon displayed in the display area 7022, the FIG. 9A to FIG. 9I Therefore, the user can specify the first displayed candidate pattern of the deformation correspondence relationship by operating the confirmation button 7043 used as the deformation correspondence relationship specifying unit. Fig. 9A Is the deformation correspondence "1 / 10" shown in the figure correct? FIG. 9B to FIG. 9I The other deformation correspondences shown in are correct. Figure 9J In the example, since the line widths are all the same, no single deformation can be considered the same as another deformation, and all five deformation data are considered to correspond to deformations that are different from each other. In the case where there is no ideal pattern among the displayed candidate patterns, the user can directly specify the deformation data in the display area 7022 using a pointing device such as a mouse. FIG. 9A to FIG. 9J The candidate patterns shown in are arranged in order of increasing certainty of the correspondence between the deformations. The certainty of the correspondence between the deformations is determined using a reliability value obtained, for example, based on the vertical distance between the deformations, the length of the intervals with corresponding deformation pairs, the similarity of the directions of the intervals with corresponding deformation pairs, and the distance from a fixed point commonly depicted in both the first deformed image and the second deformed image.

[0145] After displaying and examining other candidate patterns, the user can confirm that Fig. 9A When the first deformation correspondence candidate in is the most suitable, the user returns Fig. 8A, and operates confirmation button 7043. Then, the deformation diagram editing operation unit 260 confirms that the candidate pattern "1 / 10" selected in the display area 7022 indicates a correct deformation correspondence combination, updates the data of the deformation diagram part based on the confirmed deformation correspondence, and gives a confirmation. That is, the deformation diagram editing operation unit 260 constitutes a "change unit" that changes the deformation diagram (third deformation data) displayed in the display area 7021, etc. based on the instruction to change the deformation correspondence between the first deformation data 201 and the second deformation data 202 displayed in the display area 7022. When the user operates the next button 7045 for displaying the next object recommended for confirmation, the screen changes to Figure 8C The display screen shown in FIG. 1 displays a second deformation object recommended for confirmation.

[0146] Figure 8C It is shown Figure 1 FIG. 7 is a diagram of a third display example of the deformation diagram confirmation correction operation display screen 7000 displayed on the display output device 106. Figure 8C In, with Figure 7 and FIG. 8A to FIG. 8B The same configuration elements shown in FIG. 1 will be denoted by the same symbols, and a detailed description thereof will be omitted. Figure 8C The display area 7021 in FIG. 70 displays a portion of a deformation diagram of the second object to be recommended for confirmation. Figure 8C A rectangular frame 8002 within the range of the display area 7010 in FIG. 8 indicates an area corresponding to the display area 7021. Figure 8C In the display area 7022, Fig. 8A Similarly, the overlapping deformation part based on the first deformation data 201 and the second deformation data 202 is displayed. Figure 8B A confirmation operation has been performed, so the value in the confirmation completion number display unit 7033 is "1", and the value in the confirmation recommendation number display unit 7034 is "2".

[0147] exist Figure 8C In the display area 7023 in FIG. 7021 , only the first deformed data 201 and the first deformed image 211 of the deformed image portion displayed in the display area 7021 are superimposed and displayed. Figure 8C In the display area 7024 in FIG. 7024 , only the second deformed data 202 and the second deformed image 212 of the deformed image portion displayed in the display area 7021 are superimposed and displayed.

[0148] about Figure 8C The deformation displayed in the display area 7021 in FIG. 2 is a fixed common part detected by the fixed common part detection unit 250 (for example, by Figure 3AThe fixed points of the fixed common parts (indicated by the rectangular areas 3101 to 3103 in FIG. 3 ) exist near the deformation. Therefore, for Figure 8C The display area 7023 and the display area 7024 in the image are cut to include the fixed point 8011 and the fixed point 8021, respectively, and the fixed object as the fixed common part. That is, the fixed common part (including the fixed common part having the fixed point 8011) detected by the fixed common part detection unit 250 and the first deformation data 201 are superimposed and displayed on Figure 8C The fixed common portion (including the fixed common portion having the fixed point 8021) detected by the fixed common portion detection unit 250 and the second deformation data 202 are superimposed and displayed in the display area 7023. Figure 8C By checking the distance from the fixed point, the user can more easily understand that the positions of the first deformation data 201 and the second deformation data 202 forming the deformation graph are the same. Figure 8C In the display area 7023 in FIG. 8 , three values ​​of L, which is the straight-line distance from the fixed point 8011 of the first deformation data 201, DX, which is the distance in the X direction, and DY, which is the distance in the Y direction, are displayed as information 8012 indicating the offset distance. Similarly, in FIG. Figure 8C In the display area 7024 in FIG. 1 , three values ​​of L which is a straight-line distance from the fixed point 8021 of the second deformed data 202, DX which is a distance in the X direction, and DY which is a distance in the Y direction are displayed as information 8022 indicating an offset distance. In this case, the offset distance can be calculated based on the coordinate values ​​of the deformed image data or the actual size of the structure (object) into which the coordinate values ​​are converted.

[0149] exist Figure 8C In the example shown in FIG. 7 , since “1 / 3” is displayed in the display area 7022 , it is clear that there are other deformation correspondence combination candidates. The user can display other deformation correspondence combination candidates in the display area 7022 by operating the left arrow operation icon or the right arrow operation icon displayed in the display area 7022 .

[0150] Fig.8D It is shown Figure 1 FIG. 7 is a diagram showing a fourth display example of the deformation diagram confirmation correction operation display screen 7000 displayed on the display output device 106. Fig.8D In, with Figure 7 and FIG. 8A to FIG. 8C The same structural elements shown in the drawings will be represented by the same symbols, and their detailed description will be omitted. Fig.8D The second deformation correspondence relationship combination candidate of the deformation area of ​​the rectangular frame 8002 within the range of the display area 7010 is displayed. Fig.8DIn the deformation overlap display in the display area 7022 in FIG. 7 , the deformation correspondence combination “2 / 3” also appears to be possible. However, when the offset distance from the fixed point is equal to Figure 8C When compared with the offset distance in Figure 8C The offset distances in are significantly different.

[0151] Fig. 8E It is shown Figure 1 FIG. 7 is a diagram of a fifth display example of the deformation diagram confirmation correction operation display screen 7000 displayed on the display output device 106. Fig. 8E In, with Figure 7 and FIG. 8A to FIG. 8D The same structural elements shown in the drawings will be represented by the same symbols, and their detailed description will be omitted. Fig. 8E The third deformation correspondence relationship combination candidate of the deformation area of ​​the rectangular frame 8002 within the range of the display area 7010 is displayed. Fig. 8E There is no corresponding deformation correspondence relationship for the deformation correspondence relationship combination "3 / 3" in the display area 7022. Therefore, Fig. 8E The display area 7021 in 2000 shows a deformation diagram in which one deformation based on the first deformation data 201 and two deformations based on the second deformation data 202 are mutually different deformations. In the case where there is no corresponding deformation correspondence relationship between the deformations, information indicating the offset distance is not displayed in the display area 7023 or 7024 because the object to be compared is not identified. In the case where it is desired to check the offset distance, it is sufficient to display only the deformation correspondence relationship combination "1 / 3" or "2 / 3".

[0152] when Figure 8C When the offset distance information 8012 and the offset distance information 8022 in FIG. 8 are compared with each other, the values ​​are not exactly the same. However, the values ​​are almost equal to each other and can be regarded as being within the error range of the detected position. Figure 8C The degree of overlap of the deformation correspondences in the display area 7022 also seems to be unsuspected, so the deformation map based on the deformation correspondence combination "1 / 3" can be confirmed.

[0153] Figure 8F It is shown Figure 1 FIG. 7 is a diagram of a sixth display example of the deformation diagram confirmation correction operation display screen 7000 displayed on the display output device 106. Figure 8F In, with Figure 7 and FIG. 8A to FIG. 8E The same structural elements shown in the drawings will be represented by the same symbols, and their detailed description will be omitted. Figure 8F Display area 7021 in shows a portion of the deformation diagram of the third object recommended for confirmation. Figure 8F A rectangular frame 8003 within the range of the display area 7010 in FIG. 8 indicates an area corresponding to the display area 7021. Figure 8F In the display area 7022, Fig. 8A Similarly, the overlapping deformation part based on the first deformation data 201 and the second deformation data 202 is displayed. Figure 8C A confirmation operation has been performed, so the value in the confirmation completion number display unit 7033 is "2", and the value in the confirmation recommendation number display unit 7034 is "1".

[0154] exist Figure 8F In the example, the display area 7022 does not display operation icons of other deformation correspondence combination candidates. This is because there is only one deformation correspondence combination for the first deformation data 201 and the second deformation data 202 that are the sources of the deformation map displayed in the display area 7021.

[0155] Once the user completes Figure 8F By checking the third deformed portion of the recommended confirmation shown in , the user has completed checking all objects of the recommended confirmation. Therefore, when the user operates the save button 7046, the deformed map is saved in the storage device 104. When the user operates the end button 7047, the entire deformed map confirmation correction process ends, and the application ends.

[0156] In the above-mentioned deformation map confirmation correction process, the object deformation parts recommended for confirmation are confirmed one by one, but all object deformation parts currently determined based on the first priority deformation correspondence combination candidates can be confirmed by operating the confirm all button 7048. If the accuracy of the deformation map generation is confirmed to be high enough and no suspicious area is found after reviewing the deformation map, the user can skip the confirmation and generate the deformation map semi-automatically.

[0157] show FIG. 8A to FIG. 8F The display output device 106 of the deformation diagram confirmation correction operation display screen 7000 shown in FIG. 7 constitutes a "display unit" that displays the deformation diagram of the third deformation data and the deformation correspondence relationship between the first deformation data 201 and the second deformation data 202. Specifically, the display output device 106 displays the deformation diagram of the third deformation data in the display area 7021, and displays the deformation correspondence relationship between the first deformation data 201 and the second deformation data 202 in the display area 7022. When the right arrow or the left arrow displayed in the display area 7022 is operated, the display output device 106 displays a plurality of deformation correspondence relationship candidates of the deformation correspondence relationship between the first deformation data 201 and the second deformation data 202 in the display area 7022. For example, the display Figure 8CThe display output device 106 of the deformation diagram confirmation correction operation display screen 7000 shown in FIG. 1 displays the fixed common portion (including the fixed common portion having the fixed point 8011) detected by the fixed common portion detection unit 250 and the first deformation data 201 in a superimposed manner in the display area 7023. In addition, for example, Figure 8C The display output device 106 of the deformation diagram confirmation correction operation display screen 7000 shown in the display area 7024 displays the fixed common portion (including the fixed common portion having the fixed point 8021) detected by the fixed common portion detection unit 250 and the second deformation data 202 in a superimposed manner.

[0158] In the present embodiment, an example has been described in which a fixed object as a fixed common portion used as a reference for position recognition is a rectangular area; however, the present embodiment is not limited to this example. For example, a location where a specific character string co-occurs, an elliptical area, or an area that can be identified by color may be detected, or a pattern of a fixed object may be specified and registered in advance so that the registered object can be detected and searched. Figure 1 The information processing device 100 shown in FIG. 1 adds a graphics processing unit (GPU), and the graphics library can speed up image processing using the GPU.

[0159] Note that the present embodiment is described as an embodiment in which the first deformation data 201 and the second deformation data 202 are expected to be captured, recorded, and detected at different times for an object. This is because the first deformation data 201 has been confirmed, and therefore the certainty of the data is considered to be higher than the uncertainty of the second deformation data 202. However, in the present disclosure, the first deformation data 201 and the second deformation data 202 are not limited to the determination of the difference between the data acquired over time. In the present disclosure, the first deformation data 201 and the second deformation data 202 may also be applied to the determination of the difference between two different deformation data of the same period. In this case, the two different deformation data (i.e., the first deformation data 201 and the second deformation data 202) may be two different deformation data processed from the same image. Processing examples include AI-based detection model difference determination and AI-based detection algorithm difference determination.

[0160] According to the information processing device 100 according to the embodiment of the present disclosure described above, the deformation state of an object can be accurately grasped. In other words, a deformation map recording the deformation state can be accurately and easily generated.

[0161] Other embodiments

[0162] The embodiments of the present invention may also be implemented by providing software (program) for performing the functions of the above-described embodiments to a system or device via a network or various storage media, and a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.

[0163] Although the present disclosure has described exemplary embodiments, it is to be understood that some embodiments are not limited to the disclosed exemplary embodiments.The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. An information processing device, comprising: an identification unit configured to identify a common portion and a different portion between first deformation data of an object and second deformation data of the object different from the first deformation data; as well as An acquisition unit is configured to acquire third deformation data having a deformation progress state between the first deformation data and the second deformation data as a legend classification based on the common portion and the different portion.

2. The information processing device according to claim 1, further comprising: A display control unit is configured to display a plurality of deformation correspondence relationship candidates.

3. The information processing device according to claim 1, further comprising: a detection unit configured to detect a common portion, the common portion existing in both a first deformed image of the object corresponding to the first deformation data and a second deformed image of the object corresponding to the second deformation data, and the common portion does not include deformation; and A display control unit is configured to display the common portion detected by the detection unit, and the first deformation data and the second deformation data in a superimposed manner.

4. The information processing device according to claim 1, wherein: The first deformation data and the second deformation data are deformation data acquired for the object at different times.

5. An information processing method, comprising: identifying common parts and different parts between first deformation data of an object and second deformation data of the object different from the first deformation data; as well as Based on the common portion and the different portion, third deformation data is acquired, the third deformation data having a deformation progress state between the first deformation data and the second deformation data as a legend classification.

6. A non-transitory computer-readable storage medium storing a program that causes a computer to: identifying common parts and different parts between first deformation data of an object and second deformation data of the object different from the first deformation data; and Based on the common portion and the different portion, third deformation data is acquired, the third deformation data having a deformation progress state between the first deformation data and the second deformation data as a legend classification.

7. A computer program product comprising a program which causes a computer to: identifying common parts and different parts between first deformation data of an object and second deformation data of the object different from the first deformation data; and Based on the common portion and the different portion, third deformation data is acquired, the third deformation data having a deformation progress state between the first deformation data and the second deformation data as a legend classification.

Images