Information processor, method for processing information, and program

Abstract

To provide a solution of enabling evaluation of performance of a deformation detection method by an information processing apparatus.SOLUTION: An information processing apparatus includes: acquisition means for acquiring deformation data as data of the same deformation included in an image, the deformation data including first deformation data serving as a reference and second deformation data for comparison with the first deformation data; and evaluation means for comparing the first deformation data with the second deformation data and calculating a score indicating performance of a detection method having detected the second deformation data.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] There are known techniques for detecting cracks and other abnormalities from images of an inspection target such as the wall surface of a structure. For example, Patent Document 1 describes a technique for comparing first abnormality data created from a first image taken at a different time with second abnormality data created from a second image, and determining whether the abnormalities are the same or not by using similarity. A number of different detection methods are known for such techniques. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6749528 Summary of the Invention [Problem to be solved by the invention]

[0004] However, even when images include the same anomaly, the detection results may differ depending on the detection method implemented using a learning model, etc. In such cases, the technology of Patent Document 1 cannot evaluate the performance of the anomaly detection method.

[0005] The present invention has been made in view of the above-mentioned problems, and provides an information processing device, an information processing method, and a program that can evaluate the performance of a method for detecting abnormalities. [Means for solving the problem]

[0006] In order to solve this problem, for example, an information processing device of the present invention has the following arrangement: An acquisition means for acquiring deformation data that is data of the same deformation included in the image, the first deformation data being a reference, and second deformation data for comparison with the first deformation data; An evaluation means for comparing the first deformation data with the second deformation data and calculating a score indicating the performance of the detection method that detected the second deformation data; Equipped with. [Effects of the Invention]

[0007] According to the present invention, the performance of a method for detecting abnormalities can be evaluated. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing a hardware configuration of an information processing apparatus according to an embodiment of the present invention. [Figure 2] FIG. 10 is a diagram showing a deformation data table which is a list of deformation data in this embodiment. [Figure 3] FIG. 10 is a diagram showing an example of displaying deformation based on deformation data in the deformation data table of this embodiment. [Figure 4] FIG. 10 is a diagram of a GUI screen displayed by the performance comparison evaluation process of the present embodiment. [Figure 5] FIG. 10 is a flowchart showing the flow of a performance comparison evaluation process according to the present embodiment. [Figure 6] FIG. 10 is a diagram showing an example of displaying common parts and different parts. [Figure 7] FIG. 10 is a diagram illustrating a deformation data table relating to common parts and different parts. [Figure 8] FIG. 4 is a diagram illustrating a performance evaluation rule table according to the present embodiment. [Figure 9] FIG. 10 is a flowchart showing the flow of score calculation processing according to the present embodiment. [Figure 10] FIG. 10 is a diagram illustrating a score table. [Figure 11] FIG. 10 is a flowchart showing a comparison display process. [Figure 12] FIG. 10 is a diagram illustrating an example of a screen for setting a performance evaluation rule. [Figure 13]FIG. 10 is a diagram showing a modified example of the GUI screen. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0010] In this embodiment, an example will be described in which a computer operates as an information processing device, compares a reference deformation detected from a captured image with deformations detected by one or more detection methods implemented by a learning model or the like, calculates the comparison results as a score, and outputs the score. The score for one deformation may be an index of the accuracy of the detected deformation. The sum of the scores for multiple deformations detected by one detection method may be an index indicating the performance of the detection method for detecting deformations. The format of the output based on the score may be, for example, a display such as an image. The index indicating the performance of the detection method can also be said to be an index indicating the performance of the learning model used to execute the detection method.

[0011] Deformations include cracks that occur on concrete surfaces due to damage, deterioration, or other factors in concrete structures such as highways, bridges, tunnels, and dams. Cracks are linear damage that occur on the walls of structures due to aging, earthquake impacts, etc., and have a starting point, end point, length, and width.

[0012] <Hardware configuration> The hardware configuration of an information processing apparatus according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of an information processing apparatus 100 according to this embodiment.

[0013] The information processing device 100 of this embodiment may be, for example, a computer. The processing of the information processing device 100 of this embodiment may be realized by a single computer, or may be realized by distributing each function among multiple computers as necessary. The multiple computers may be connected to each other so that they can communicate with each other.

[0014] The information processing device 100 includes a control unit 101 , a nonvolatile memory 102 , a work memory 103 , a storage device 104 , an input device 105 , an output device 106 , a communication unit 107 , and a system bus 110 .

[0015] The control unit 101 may be an arithmetic processing unit (also referred to as a processor) that controls the entire information processing device 100. For example, the control unit 101 may be a CPU (Central Processing Unit). Instead of or in addition to the CPU, the control unit 101 may include other processors such as an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), and a QPU (Quantum Processing Unit). Some or all of the functions of the control unit 101 are realized by one or more processors including the CPU reading out programs stored in the storage device 104, expanding the programs into the work memory 103, and executing them. Some or all of the functions of the control unit 101 may be realized by one or more circuits such as an ASIC (Application Specific Integrated Circuit) and a PLD (Programmable Logic Device) including an FPGA (Field Programmable Gate Array).

[0016] The control unit 101 is an example of an acquisition means and an evaluation means. The control unit 101 acquires first deformation data as a reference, which is data on the same deformation contained in an image of a concrete structure or the like, and second deformation data for comparison with the first deformation data. The control unit 101 also compares the first deformation data with the second deformation data and calculates a score indicating the performance of the detection method that detected the second deformation data.

[0017] The nonvolatile memory 102 stores programs and parameters executed by the processor of the control unit 101. The nonvolatile memory 102 is, for example, a ROM (Read Only Memory).

[0018] The work memory 103 temporarily stores programs supplied from external devices, etc., and parameters and data such as images required for executing the programs. The work memory 103 is, for example, a RAM (Random Access Memory).

[0019] The storage device 104 is a nonvolatile large-capacity storage device that stores programs, parameters required for executing the programs, and data such as images. The storage device 104 may be an external device that is detachably attached to the information processing device 100. The storage device 104 may be, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a memory card that is configured from a semiconductor memory, a magnetic disk, or the like. The storage device 104 may also be a storage medium configured from a disk drive that reads and writes data from / to optical disks such as DVDs and Blu-ray (registered trademark) discs.

[0020] The input device 105 is an operating member such as a mouse, keyboard, touch panel, etc. that accepts operation instructions from a user, etc. The input device 105 outputs the accepted operation instructions to the control unit 101.

[0021] The output device 106 is, for example, an image display device such as a liquid crystal display device or an organic EL (Electro Luminescence) display device, etc. The output device 106 displays an image based on data held by the information processing device 100 or data supplied from an external device.

[0022] The communication unit 107 is an interface that connects to a network such as the Internet or a LAN (Local Area Network) so as to be able to send and receive data.

[0023] The system bus 110 includes at least one of an address bus, a data bus, and a control bus that connect the control unit 101, the nonvolatile memory 102, the work memory 103, the storage device 104, the input device 105, and the output device 106 so that data can be exchanged among them.

[0024] Either the nonvolatile memory 102 or the storage device 104 stores an OS (operating system), which is basic software executed by the control unit 101, and applications that work with the OS to realize applied functions. In this embodiment, either the nonvolatile memory 102 or the storage device 104 may store a computer program that enables the information processing device 100 to realize a performance comparison evaluation process, which will be described later.

[0025] The performance comparison and evaluation process executed by the information processing device 100 of this embodiment is realized by loading software (also called a computer program) provided by an application. Note that the application has software for utilizing the basic functions of the OS installed in the information processing device 100. The OS of the information processing device 100 may have software for realizing the process of this embodiment.

[0026] <Deformation Data Table> In this embodiment, the deformation is described as a crack, and data that represents the shape of the deformation as vector data is described as deformation data. For example, one piece of deformation data may be the digitalization of one deformation. Figure 2 is a diagram showing a deformation data table, which is a list of deformation data. Next, the data structure of the deformation data table of this embodiment will be described with reference to Figure 2.

[0027] FIG. 2(a) illustrates a first deformation data table 201 in which reference first deformation data is registered. FIG. 2(b) illustrates a second deformation data table 251 in which second deformation data to be processed for performance comparison is registered. Note that multiple deformation data included in one deformation data table may also be referred to as a deformation data group. One second deformation data table 251 may be a table of a deformation data group detected by one detection method (learning model). The first deformation data table 201 and the second deformation data table 251 contain deformation data of one or more deformations detected from photographed images of areas that at least partially include the same subject (deformation). Furthermore, multiple second deformation data tables 251 to be compared in performance contain deformation data of one or more deformations detected by different detection methods based on photographed images of areas that at least partially include the same subject (deformation). In this embodiment, the plurality of second deformation data tables 251 includes deformation data detected from the same photographed image. Different detection methods refer to, for example, methods of detecting deformations using different learning models.

[0028] The first deformation data table 201 and the second deformation data table 251 each include deformation data, which is information about deformations, such as a deformation ID 202, a deformation ID 252, a maximum width 203, a maximum width 253, a number of vertices 204, a number of vertices 254, a vertex coordinate list 205, and a vertex coordinate list 255. The deformation IDs 202 and 252 are identification information uniquely assigned to each deformation. The maximum widths 203 and 253 are the maximum values ​​of the width (thickness) of the deformation. The number of vertices 204 and 254 and the vertex coordinate lists 205 and 255 are information about the number of vertices and the coordinates of the vertices corresponding to the start and end points of each line segment used to represent the shape of a crack as a polyline consisting of one or more line segments. Each deformation data in the deformation data tables 201 and 251 is entered by a user tracing it on an image using a tablet or the like, automatically generated by image analysis processing or the like, or by a combination of these. In addition, the image analysis process may be performed using a learning model created by machine learning and deep learning of AI (artificial intelligence).

[0029] Fig. 3 is a diagram showing an example of a display of a deformation based on deformation data in a deformation data table. The display of a deformation will be described with reference to Fig. 3. Fig. 3(a) shows an example of a display screen 300 of a deformation drawn based on the first deformation data registered in the first deformation data table 201 shown in Fig. 2(a). On the display screen 300, deformation 301 is drawn based on the deformation data with a deformation ID of Ca001 in the deformation data table 201. Similarly, deformations 302 to 310 correspond to deformations with deformation IDs of Ca002 to Ca010, respectively.

[0030] Figure 3(b) shows an example of a display screen 350 of deformations drawn based on the second deformation data registered in the second deformation data table 251 shown in Figure 2(b). On the display screen 350, deformation 351 is drawn based on the deformation data with the deformation ID Cb001 in the deformation data table 251. Similarly, deformations 352 to 360 correspond to deformations with the deformation IDs Cb002 to Cb010, respectively.

[0031] 4 is a diagram of a GUI screen 400 displayed by the performance comparison evaluation process of this embodiment. GUI is an abbreviation for Graphical User Interface.

[0032] When the user inputs a first deformation data table of the first deformation data that serves as the reference and a second deformation data table of the second deformation data obtained by one or more learning models to be compared, the performance comparison evaluation process described below is initiated, and a GUI screen 400 is displayed as the evaluation results. The user may input multiple pieces of second deformation data for each detection method (learning model). In the following description, multiple pieces of second deformation data detected by one detection method (learning model) are referred to as a second deformation data group.

[0033] Result display areas 401, 402, and 403 are examples of when three deformation data groups are input as the second deformation data group; the number of result display areas may be changed depending on the number of second deformation data groups input. The three second deformation data groups may be deformation data groups detected from the same captured image using different detection methods. Result display areas 401-403 display images of deformations as the detection results of the detection methods in descending order of detection performance, from left to right, based on the performance comparison evaluation results described below. Therefore, the detection method that detected the deformation data displayed on the left has the highest detection performance.

[0034] The result confirmation deformation data designation buttons 411, 412, and 413 display the name of the entered deformation data group (hereinafter also referred to as the input name). The user selects one of the result confirmation deformation data designation buttons 411, 412, and 413 to designate the deformation data group they wish to check. In Figure 4, the result confirmation deformation data designation button 412 is enabled because the user wants to check the deformation data group for the detection method entered as the input name "input B." The input name can be said to be the name of the deformation data group as well as the name of the detection method and learning model.

[0035] The deformation list 421 is a display area in which, when deformation data is specified using one of the result confirmation deformation data specification buttons 411, 412, or 413, the deformation IDs of the deformation data included in the specified deformation data group are listed in order of decreasing performance. Here, a list of deformation data from the deformation data group "inputB" is displayed in the deformation list 421. The deformation ID specification button 422 is a button for specifying the deformation that you want to display for comparison from the deformation list 421. The enlarged deformation display areas 431, 432, and 433 are display areas in which the same deformation included in each deformation data group specified using the deformation ID specification button 422 is displayed at a magnification that is easy for the user to see.

[0036] The performance evaluation rule selection radio button 441 is used to select from among candidates a performance evaluation rule for the performance comparison evaluation process described below. The performance evaluation rules include standard rules and setting rules that can be changed by the user. The background image display radio button 442 is a button for selecting whether or not to display a photographed image of the background wall surface. The legend display area 451 displays a legend to be displayed in the enlarged deformation display areas 431, 432, and 433 so that the types of commonalities and differences between the first deformation that serves as the reference and the second deformation that is the comparison target can be distinguished in the performance comparison evaluation process described below.

[0037] <Performance comparison evaluation process> 5 is a flowchart showing the flow of the performance comparison evaluation process executed in the information processing device 100 of this embodiment. The performance comparison evaluation process executed in the information processing device 100 of this embodiment will be described with reference to FIG.

[0038] The performance comparison evaluation process of Figure 5 is realized by the control unit 101 of the information processing device 100 shown in Figure 1 expanding a program stored in either the non-volatile memory 102 or the storage device 104 into the work memory 103, executing it, and controlling each component.

[0039] In this embodiment, first, a user inputs a first deformation data table (first deformation data group) containing first deformation data as a reference and multiple (here, three) second deformation data tables (second deformation data groups) containing second deformation data to be compared to the information processing device 100. The information processing device 100 compares the input first deformation data group and second deformation data group to calculate common parts and difference parts. The information processing device 100 detects change data of the calculated common parts and difference parts, performs a performance comparison, and displays a GUI screen showing the results. This series of processes will be described as an example of performance comparison evaluation processing.

[0040] In S501, the control unit 101 reads and acquires the first deformation data table 201 that is stored in the storage device 104 and serves as a reference.

[0041] In S502, the control unit 101 acquires one or more second deformation data tables 251 to be compared and evaluated that are stored in the storage device 104. Here, the control unit 101 acquires three second deformation data tables 251 corresponding to inputA to inputC shown in FIG.

[0042] In S503, the control unit 101 repeats the processes from S504 to S511 for each second deformation data table 251 acquired in S502. Here, the control unit 101 repeats the processes from S503 onwards three times, which is the number of second deformation data tables 251.

[0043] In S504, the control unit 101 compares the first deformation data contained in the first deformation data table 201 with the second deformation data contained in the second deformation data table 251 to be processed, and calculates the common parts and difference parts. The calculation method may be a known method, such as the condition change determination method disclosed in Japanese Patent Application Laid-Open No. 2023-83218. In this embodiment, in the condition change determination method used as a reference, difference parts determined to indicate progression of the deformation are treated as false positive difference parts, and difference parts determined to indicate disappearance of the deformation are treated as undetected difference parts. The data structure of the common parts and difference parts calculated by the control unit 101 will be described later using Figures 6 and 7.

[0044] In S505, the control unit 101 repeats the processes from S506 to S510 the number of times equal to the number of performance evaluation rules registered in the performance evaluation rule table stored in the storage device 104. The performance evaluation rules and the performance evaluation rule table will be described later with reference to FIG.

[0045] In S506, the control unit 101 reads the performance evaluation rule from the performance evaluation rule table stored in the storage device 104.

[0046] In S507, the processes from S508 to S509 are repeated the number of times equal to the number of deformations of the common and different parts calculated in S504.

[0047] In S508, the control unit 101 calculates scores for the common and different parts of each deformation based on the performance evaluation rules through the process in Fig. 9, which will be described later, and generates or updates a score table in Fig. 10, which will be described later. The control unit 101 may create a score table for each second deformation data table 251 (i.e., for each detection method and each learning model). The calculated score table includes a score for each deformation ID.

[0048] The control unit 101 proceeds to S5101 after applying all performance evaluation rules to the deformations included in the second deformation data table 251. In other words, the control unit 101 proceeds to S5101 after calculating the scores of all deformations included in the second deformation data table 251.

[0049] In S5101, the control unit 101 calculates the total score of the detection method (i.e., the learning model) corresponding to one second deformation data table 251 by adding up the scores of all deformations corresponding to one second deformation data table 251. The total score can be said to be a numerical representation of the performance of the detection method and the learning model.

[0050] In S512, the control unit 101 performs a comparative display by a comparative display process described in Fig. 11. The control unit 101 also changes the display order of the result display areas 401 to 403 using the total score for each deformation data table (i.e., for each detection method and each learning model) in the score table. For example, the control unit 101 may arrange and display images of deformations in the result display areas 401 to 403 in descending order of the total score.

[0051] Fig. 6 is a diagram showing a display example of the common parts and different parts calculated in S504 of Fig. 5. A display area 600 in Fig. 6 corresponds to any one of the result display areas 401 to 403 in Fig. 4. The five types of legends shown in the legend display area 451 will be described.

[0052] Line segments 611 to 613, each indicated by a single solid line, correspond to a portion of the crack deformation that is common to both the first deformation data and the second deformation data, and where the width values ​​of both deformations are the same. Note that the portion of the deformation may be the entire length (total length) of one deformation.

[0053] Line segments 621 to 622, shown as triple lines, are crack deformation portions that exist in common in the first deformation data and the second deformation data, and correspond to deformation portions where the width value of the second deformation is detected as being thicker than the width value of the first deformation.

[0054] Lines 631 to 635 shown with double lines are crack deformation portions that exist in common in the first deformation data and the second deformation data, and correspond to deformation portions where the width value of the second deformation is detected as being thinner than the width value of the first deformation.

[0055] The dotted line segments 641 to 643 exist only in the second deformation data and correspond to the deformation portions of the cracks that are the difference portions of the false detection.

[0056] The dashed line segments 651 and 652 exist only in the first deformation data and correspond to the crack deformation portions that are undetected difference portions.

[0057] In the example of Figure 6, the five types of abnormalities are displayed with solid lines, double lines, triple lines, dashed lines, and dotted lines to make them identifiable, but the display method is not limited to this. For example, the five types of abnormalities may be distinguished by drawing them in different colors, different thicknesses, or a combination thereof.

[0058] Figure 7 shows deformation data tables relating to common parts and different parts. Specifically, Figure 7(a) illustrates a deformation data table 701 for common parts. Figure 7(b) illustrates a deformation data table 751 for different parts. Figure 7(c) illustrates a correspondence data table 771.

[0059] The common part deformation data table 701 in Figure 7(a) includes a common part deformation ID 702, an original deformation ID 703 that identifies the original second deformation data before it was distinguished into common parts and different parts, a maximum width 704, a number of vertices 705, and a vertex coordinate list 706. The common part deformation ID 702 is information for identifying the common part. Here, one deformation may have multiple common part deformation IDs 702. For example, the deformation IDs 702 "Cbm010" and "Cbm011" in the bottom two rows are linked to the same original deformation ID 703, "Cb010." This is because there is a difference between the common parts "Cbm010" and "Cbm011." In other words, when one second deformation has multiple distant common parts, the deformation IDs 702 of the multiple common parts are linked to one original deformation ID 703 indicating one second deformation data. The original deformation ID 703 is the deformation ID of the second deformation data including the common parts. The maximum width 704 is the maximum width of the common parts. The number of vertices 705 is the number of vertices in the common parts. The vertex coordinate list 706 is a list of the vertex coordinates of the common parts.

[0060] The deformation data table 751 for the difference part in Figure 7(b) includes a deformation ID 752 for the difference part, an original deformation ID 753 that identifies the original second deformation data before it was distinguished into common parts and difference parts, a maximum width 754, a number of vertices 755, and a vertex coordinate list 756. The deformation ID 752 for the difference part is information for identifying the difference part. The original deformation ID 753 is the deformation ID of the second deformation data that includes the difference part. The maximum width 754 is the maximum width of the difference part. The number of vertices 755 is the number of vertices in the difference part. The vertex coordinate list 756 is a list of the vertex coordinates of the difference part.

[0061] The correspondence data table 771 in Figure 7(c) includes a deformation ID 772 of the deformation data of the common part and correspondence information 773. The deformation ID 772 of the deformation data of the common part corresponds to the deformation ID 702 in the common part deformation data table 701 in Figure 7(a). For example, the deformation data in the first row of the correspondence data table 771 for which the common part deformation ID 772 is "Cbm001" corresponds to the deformation data of Cbm001 in the first row of the common part deformation data table 701 in Figure 7(a). The correspondence information 773 includes the deformation ID of the first deformation, which is information indicating which first deformation data the common part deformation ID 772 corresponds to.

[0062] The correspondence information 773 is a set of information that, for each line segment that makes up the polyline of the deformation data of the common portion, includes the deformation ID of the first deformation data that is determined to match, and width increase / decrease information that indicates the increase / decrease in the maximum width of the deformation in the comparison data. The correspondence information 773 is arranged in the order registered in the vertex coordinate list 706 of the deformation data table 701 of the common portion in FIG. 7(a). The correspondence information 773 indicates the correspondence relationship with the deformation ID 202 of the first deformation data table 201 in FIG. 2(a), which is the comparison data for the line segments connecting the vertices. The width increase / decrease information of the correspondence information 773 is obtained by comparing the maximum width value of the second deformation portion identified by the deformation ID of the common portion with the maximum width value of the corresponding first deformation portion. If the maximum width value of the second deformation portion is larger and the second deformation portion is thicker, it is registered as Wide. On the other hand, if the maximum width value of the second deformation portion is smaller and the second deformation portion is thinner than the first deformation portion, the width increase / decrease information in the correspondence information 773 is registered as Narrow. For example, in the case of Cbm001, the maximum width value of the deformation portion of Cbm001 is larger than the maximum width value of the deformation portion of the corresponding Ca001, and the deformation portion of Cbm001 is thicker, so the width increase / decrease information in the correspondence information 773 is registered as Wide. On the other hand, in the case of Cbm002, where the maximum width value is small and the maximum width is thinner, the width increase / decrease information in the correspondence information 773 is registered as Narrow.

[0063] 8 is a diagram illustrating an example of a performance evaluation rule table. The performance evaluation rule table is a table of multiple performance evaluation rules for evaluating and quantifying performance to determine a score. The lower the score, the better the performance, and the higher the score, the lower the performance. The performance evaluation table shown in FIG. 8 may be applied, for example, when the standard rule is selected in the evaluation rule selection radio button 441.

[0064] FIG. 8(a) shows an example of a performance evaluation rule table 801 based on the type of anomaly. The performance evaluation rule table 801 is composed of conditions 802 and scores 803. The score for an undetected item is higher than the score for a false detection. Furthermore, the score for an anomaly determined to be thicker than the score for an anomaly determined to be thinner than the standard is higher. This is because the performance is evaluated as low because an anomaly that requires inspection may not have been detected, or a thick anomaly that requires inspection may be overlooked by being determined to be thin.

[0065] FIG. 8(b) illustrates a performance evaluation rule table 811 based on the length of the deformation of the difference part. The performance evaluation rule table 811 is composed of conditions 812 and scores 813. The condition 812 is the length of the deformation of the difference part that differs from the standard. The longer the length of the deformation that differs from the standard, the more likely it is to be an item requiring inspection, and the score is set higher to evaluate the performance lower. Note that when the width is the same as the standard, the length of the deformation of the difference part that differs from the standard is 0, so the score is 0.

[0066] <Score calculation process> Fig. 9 is a flowchart showing the flow of the score calculation process in S508 of Fig. 5. Next, the score calculation process in S508 for calculating the score will be described with reference to Fig. 9.

[0067] In S901, the control unit 101 reads the performance evaluation rule read in S506.

[0068] In S902, the control unit 101 identifies the type of deformation to be processed and determines a score for each type using the performance evaluation rule table 801. The control unit 101 identifies the type of deformation to be processed using the common part deformation data table 701, the different part deformation data table 751, and the correspondence data table 771. For example, if the control unit 101 determines from the correspondence data table 771 that the width of the deformation to be processed is wider than the standard, it determines the score to be 5 using the performance evaluation rule table 801.

[0069] In S903, the control unit 101 identifies the length of the deformation of the difference portion that differs from the standard, and determines a score based on the length of the difference portion that differs from the standard using the performance evaluation rule table 811. The control unit 101 may calculate the length of the deformation based on the correspondence between the coordinate system of the captured image and the length in real space. For example, if a polyline of the deformation is output in the image coordinate system for an image captured at 1 mm / 1 pixel, the control unit 101 may calculate the length as 10 mm for a 10-pixel length. The control unit 101 may calculate the line length of the deformation by calculating and adding up the Euclidean distances of each line segment between vertices. If the control unit 101 determines that the length of the deformation to be processed is equal to or greater than 1 m and less than 5 m, it determines the score as 7 using the performance evaluation rule table 811.

[0070] In S904, the control unit 101 calculates the score of the deformation by adding up the score for the type of deformation calculated in S902 and the score for the length of the deformation calculated in S903.

[0071] In S905, the control unit 101 registers the score for the deformation ID of the second deformation data to be input as a score in the score table 1001 shown in FIG.

[0072] FIG. 10 is a diagram illustrating a score table 1001 of S905 in FIG. 9. The score table 1001 includes a deformation ID 1002, a score 1003, a common difference ID 1004, a type 1005, and a length 1006. The deformation ID 1002 corresponds to the deformation ID 252 in the second deformation data table 251 in FIG. 2(b). However, as shown in the bottom row, if the type 1005 is undetected, no deformation corresponding to the reference deformation has been detected, and therefore no deformation ID starting with Cb exists. In this case, the deformation ID of the undetected reference first deformation data is registered in the deformation ID 1002 (here, Ca010).

[0073] The score 1003 is the sum of the scores for the deformations of the deformation ID 1002. The lower the score 1003, the better the performance of the detection method, and the higher the value, the lower the performance. For example, the control unit 101 calculates the sum of the scores 1003 of Cb001 and Cb002 as follows: Cb001 score = (thick width) + (false positive) + (length 0.5m) + (length 0.2m) =5+3+5+1=14 Cb002 score = (false positive) + (narrow width) + (length 0.2m) + (length 0.8m) =3+9+1+5=18

[0074] The common difference ID 1004 is either the deformation ID 702 of the common part in Figure 7(a) or the deformation ID 752 of the difference part in Figure 7(b). The type 1005 is information on the deformation type used in the score calculation by type in S902. The length 1006 is information on the length of the deformation used in the score calculation by length in S903.

[0075] <Comparison display processing> Fig. 11 is a flowchart showing the comparative display process. The comparative display process is a process for performing comparative display based on the scores in S512 of Fig. 5. Next, with reference to Fig. 11, the comparative display process for selecting and performing comparative display of deformations based on the score table calculated in S508 will be described.

[0076] In S1101, the control unit 101 acquires information on the input name of the detection method selected by the user. In the case of Figure 4, among the deformation data group from the result confirmation deformation data designation button 411 to the result confirmation deformation data designation button 413, "inputB", which is the input name of the detection method of the result confirmation deformation data designation button 412, has been selected by the user. The information on the input name selected by the user is also information on the learning model selected by the user.

[0077] In S1102, the control unit 101 acquires a score table for the second deformation data group of the input name of the detection method acquired in S1101.

[0078] In S1103, the control unit 101 sorts the score table acquired in S1102 in descending order by score value, and displays a list of the abnormality IDs in the order of the sorted scores in the abnormality list 421. As described above, the lower the score, the higher the detection performance. Therefore, the detection performance of the abnormalities displayed at the top of the abnormality list 421 is low.

[0079] In S1104, the control unit 101 acquires the deformation ID of the second deformation data specified by the deformation ID specification button 422 of the deformation list 421. Here, the control unit 101 acquires "Cb004" as the deformation ID.

[0080] In S1105, the control unit 101 identifies and acquires the deformation ID of a deformation included in another second deformation data group (inputC, inputA in this case) at the same position as the deformation of the deformation ID of the second deformation data specified in S1104. For example, the control unit 101 uses the correspondence data table 771 to identify the deformation ID of the first deformation data that corresponds to the deformation ID of the specified second deformation data. The control unit 101 uses the correspondence data table 771 to identify and acquire the second deformation data of the other second deformation data group that corresponds to the deformation ID of the identified first deformation data.

[0081] In S1106, the control unit 101 displays each deformation side by side. For example, the control unit 101 displays the polyline of the deformation of each deformation ID identified in S1105 in the corresponding result display areas 401 to 403.

[0082] <Example of setting performance evaluation rules> 12 shows a rule setting screen 1201 for setting a performance evaluation rule. The rule setting screen 1201 is a screen for setting a performance evaluation rule when the user selects the performance evaluation rule selection radio button 441 as the setting rule. The rule setting screen 1201 includes a priority type 1202, a priority difference 1203, an undetected width 1204, a length consideration 1205, a minimum common effective length 1206, and a minimum difference effective length 1207.

[0083] The priority type 1202 accepts from the user whether to prioritize differences in thickness of common anomalies or whether to prioritize the presence or absence of differential anomalies including undetected and falsely detected anomalies. The control unit 101 changes the score value in the performance evaluation rule table 801 according to the user's selection of the priority type 1202.

[0084] The priority difference 1203 accepts from the user whether to give a higher score to an undetected difference or to an erroneous detection difference. The control unit 101 changes the score value in the performance evaluation rule table 801 in accordance with the user's selection of the priority difference 1203.

[0085] The undetected width 1204 accepts whether or not to add points to the score when the value of the undetected width is wider than a predetermined value. When the setting of the undetected width 1204 is "Used," the control unit 101 determines the score according to the width of the abnormality that differs from the standard, using the performance evaluation rule table 801 shown in Fig. 8(a). On the other hand, when the setting of the undetected width 1204 is "Not Used," the control unit 101 sets the score of the performance evaluation rule table 801 to "0," or does not use the performance evaluation rule table 801.

[0086] Length consideration 1205 accepts a setting as to whether to change the score depending on the length. When length consideration 1205 is set to "use", the control unit 101 determines a score according to the length of the difference part using a performance evaluation rule table 811 shown in FIG. 8(b). When length consideration 1205 is set to "not use", the control unit 101 sets the score in the performance evaluation rule table 811 shown in FIG. 8(b) to "0", or does not use the performance evaluation rule table 811.

[0087] The minimum common effective length 1206 accepts the setting of the minimum length of the deformation of the common part to be used for calculating the score. The control unit 101 does not use a common deformation of less than the length set in the minimum common effective length 1206 as a target for calculating the score.

[0088] The minimum difference effective length 1207 accepts the setting of the minimum length of the deformation of the difference portion to be used for calculating the score. The control unit 101 does not use the difference deformation of less than the length set in the minimum difference effective length 1207 as a target for calculating the score.

[0089] In this embodiment, a score is calculated by comparing the first deformation data as a reference with the second deformation data as a comparison target. This allows the user to easily evaluate the performance of the detection method that detected the second deformation data based on the score.

[0090] In this embodiment, a score is set for each common part and different part, thereby enabling more accurate performance evaluation.

[0091] In this embodiment, information about the score is displayed, so that the user can easily check the performance of the detection method and the accuracy of the abnormality.

[0092] In this embodiment, a deformation that exists only in either the first deformation data or the second deformation data is classified as an undetected deformation or a false positive deformation and the score is calculated, thereby realizing a more accurate performance evaluation.

[0093] In this embodiment, the sum of the scores of multiple anomalies detected by multiple different detection methods (learning models) is used as an index for evaluating the performance of each detection method, and images of the anomalies detected by each detection method are arranged and displayed according to the sum of the scores. This allows the user to easily recognize the level of performance of each detection method.

[0094] Furthermore, according to this embodiment, it is possible to compare the performance of one or more anomaly detection results detected by other detection methods against the standard anomaly detection results for the same area, and display them in order of performance evaluation from an inspection perspective. Performance comparison is performed on an anomaly data basis, and the performance evaluation results from an inspection perspective allow the user to check items requiring inspection from among many anomalies in order; if the accuracy of anomaly detection is high and there is no difference in the anomaly, it will be lower in the list, eliminating the need for confirmation.

[0095] Furthermore, according to this embodiment, not only is the performance order displayed for each deformation ID included in one deformation data group, but deformations detected by other detection methods are also displayed in comparison with the deformation ID of interest, allowing the user to confirm the differences in performance between different detection methods.

[0096] [Variations] In the present embodiment, the control unit 101 displays the anomalies to be compared side by side, but the present invention is not limited to this. For example, the control unit 101 may display the anomalies side by side. In this way, if the captured image of the object to be detected for anomalies is horizontally long, arranging the images vertically can prevent the display screen from becoming too wide in the horizontal direction. FIG. 13 is a diagram showing a modified example of the GUI screen.

[0097] FIG. 13(a) shows an example in which the control unit 101 switches the display results by displaying layers. The layer display list 1301 lists the input names of the detection methods for the input second deformation data. The layer selection button 1302 selects the deformation data to be displayed in the results table area from the layer display list 1301. This allows the user to compare the performance of deformation detection methods while checking the similarity of shapes and differences in position by switching the display layer or making the display layer transparent, even when the display screen is small.

[0098] FIG. 13(b) shows an example in which the control unit 101 displays deformations side by side, shifting them in a specified direction. The deformation position specification list 1351 lists the input name of the detection method for the input second deformation data and the shift amount for the deformation display position. The deformation shift setting area 1352 is an area for setting the shift amount and shift direction for each deformation data. The control unit 101 displays multiple deformations side by side in the enlarged deformation display area 1353, shifting them based on the shift amount and shift direction set in the deformation shift setting area 1352. For example, when the user selects the “Up 5” button, the control unit 101 shifts deformations other than “input B” (here, the deformation corresponding to deformation Cb004) five units upward from the deformation in “input B” and displays them. The unit of shift amount here may be set appropriately to centimeters (cm). This allows the user to simultaneously compare and confirm the similarity of the shapes of multiple deformations detected by different detection methods.

[0099] In addition, the score table of each deformation data may be output as a list file in CSV format, allowing the performance evaluation results to be confirmed as data.

[0100] In the above embodiment, the control unit 101 extracts the length score for each type from the performance evaluation rule table 811. However, the total score may be calculated based on the extracted score for the total length of all types of deformation to be processed. For example, the control unit 101 may calculate the total score of Cb001 and Cb002 as follows: Cb001 score = (thick width) + (false positive) + (length 0.7m = 0.5m + 0.2m) =5+3+5=13 Cb002 score = (false positive) + (narrow width) + (length 1.0m = 0.2m + 0.8m) =3+9+7=19

[0101] (Other Examples) In the above-described embodiment, the deformation data is assumed to be data that represents deformations detected by a learning model, but the deformation data may be generated by other methods. For example, the deformation data may be generated based on input data that a user writes by hand using a finger or a touch pen on a touch panel on which an image including a deformation is displayed.

[0102] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0103] The disclosure of this specification includes the following information processing device, information processing method, and program. (Item 1) An acquisition means for acquiring deformation data that is data of the same deformation included in the image, the first deformation data being a reference, and second deformation data for comparison with the first deformation data; An evaluation means for comparing the first deformation data with the second deformation data and calculating a score indicating the performance of the detection method that detected the second deformation data; An information processing device comprising: (Item 2) The evaluation means calculates the score using a common part common to the first deformation data and the second deformation data, and a difference part existing in either the first deformation data or the second deformation data. 2. The information processing device according to item 1, (Item 3) The evaluation means displays information about the score. 3. The information processing device according to item 1 or 2, (Item 4) The deformation data includes at least one of identification information of the deformation, the width of the line segment included in the deformation, the number of vertices of the line segment included in the deformation, and the coordinates of the vertices of the line segment included in the deformation. 4. The information processing device according to any one of items 1 to 3, (Item 5) The evaluation means calculates the score based on a comparison between a first width, which is the width of the deformation indicated by the first deformation data, and a second width, which is the width of the deformation indicated by the second deformation data, in the common portion. 3. The information processing device according to item 2, (Item 6) The evaluation means calculates the score so that when the second width is narrower than the first width in the common portion, the performance is lower than when the second width is wider. 6. The information processing device according to item 5, (Item 7) The evaluation means calculates the score based on undetected deformations that exist only in the first deformation data and falsely detected deformations that exist only in the second deformation data in the difference portion. 3. The information processing device according to item 2, (Item 8) The evaluation means calculates the score so as to indicate that the undetected abnormality has a lower performance than the erroneously detected abnormality in the difference portion. 8. The information processing device according to item 7, (Item 9) The evaluation means calculates the score based on the length of a portion where the deformation based on the first deformation data and the deformation based on the second deformation data differ. 9. The information processing device according to any one of items 1 to 8, wherein: (Item 10) The evaluation means displays information about a plurality of deformation data in order of the scores. 10. The information processing device according to any one of items 1 to 9, (Item 11) The acquisition means acquires a plurality of the second deformation data generated by detecting the same deformation by a plurality of different detection methods, The evaluation means calculates a sum of scores for each of the plurality of second deformation data and outputs information based on the sum of scores. 11. The information processing device according to any one of items 1 to 10, (Item 12) The plurality of second deformation data are deformation data detected by a plurality of learning models different from each other. Item 12. The information processing device according to item 11. (Item 13) The evaluation means displays images of the same abnormality detected by the plurality of detection methods side by side. Item 12. The information processing device according to item 11. (Item 14) The evaluation means displays images including a plurality of abnormalities detected by the plurality of detection methods in order of the sum of the scores. Item 12. The information processing device according to item 11. (Item 15) An acquisition process for acquiring first deformation data, which is data of the same deformation included in the image, as a reference, and second deformation data for comparison with the first deformation data; An evaluation process of comparing the first deformation data with the second deformation data and calculating a score indicating the performance of the detection method that detected the second deformation data; An information processing method comprising: (Item 16) A program for causing a computer to function as each means of the information processing device according to any one of items 1 to 14.

[0104] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0105] 100: Information processing device, 101: Control unit, 301-310, 351-360, Deformation.

Claims

1. An acquisition means for acquiring deformation data that is data of the same deformation included in the image, including first deformation data that serves as a reference and second deformation data for comparison with the first deformation data; An evaluation means for comparing the first deformation data with the second deformation data and calculating a score indicating the performance of the detection method that detected the second deformation data; An information processing device comprising:

2. The evaluation means calculates the score using a common part common to the first deformation data and the second deformation data, and a difference part existing in either the first deformation data or the second deformation data.

2. The information processing apparatus according to claim 1, wherein:

3. The evaluation means displays information about the score.

2. The information processing apparatus according to claim 1, wherein:

4. The deformation data includes at least one of identification information of the deformation, the width of the line segment included in the deformation, the number of vertices of the line segment included in the deformation, and the coordinates of the vertices of the line segment included in the deformation.

2. The information processing apparatus according to claim 1, wherein:

5. The evaluation means calculates the score based on a comparison between a first width, which is the width of the deformation indicated by the first deformation data, and a second width, which is the width of the deformation indicated by the second deformation data, in the common portion.

3. The information processing apparatus according to claim 2, wherein:

6. The evaluation means calculates the score so that when the second width is narrower than the first width in the common portion, the performance is lower than when the second width is wider.

6. The information processing apparatus according to claim 5,

7. The evaluation means calculates the score based on undetected deformations that exist only in the first deformation data and falsely detected deformations that exist only in the second deformation data in the difference portion.

3. The information processing apparatus according to claim 2, wherein:

8. The evaluation means calculates the score so as to indicate that the undetected abnormality has a lower performance than the erroneously detected abnormality in the difference portion.

8. The information processing apparatus according to claim 7,

9. The evaluation means calculates the score based on the length of a portion where the deformation based on the first deformation data and the deformation based on the second deformation data differ.

2. The information processing apparatus according to claim 1, wherein:

10. The evaluation means displays information about a plurality of deformation data in order of the scores.

2. The information processing apparatus according to claim 1, wherein:

11. The acquisition means acquires a plurality of the second deformation data generated by detecting the same deformation by a plurality of different detection methods, The evaluation means calculates a sum of scores for each of the plurality of second deformation data and outputs information based on the sum of scores.

2. The information processing apparatus according to claim 1, wherein:

12. The plurality of second deformation data are deformation data detected by a plurality of learning models different from each other.

12. The information processing apparatus according to claim 11,

13. The evaluation means displays images of the same abnormality detected by the plurality of detection methods side by side.

12. The information processing apparatus according to claim 11,

14. The evaluation means displays images including a plurality of abnormalities detected by the plurality of detection methods in order of the sum of the scores.

12. The information processing apparatus according to claim 11,

15. An acquisition process for acquiring first deformation data, which is data of the same deformation included in the image, as a reference, and second deformation data for comparison with the first deformation data; An evaluation process of comparing the first deformation data with the second deformation data and calculating a score indicating the performance of the detection method that detected the second deformation data; An information processing method comprising:

16. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 14.

Images