Information processing apparatus, information processing method, and program

Abstract

To detect a defect requiring correction or editing on an image, and to present it to a user.SOLUTION: An information processing apparatus detects, as a candidate region, a region in which two or more end portions of a predetermined defect are included within a range of a predetermined size from an image obtained by capturing an inspection target, and acquires a degree of difficulty when a user determines whether to perform a predetermined operation on the defect whose end portion is included in the candidate region. Then, the information processing apparatus displays the candidate area based on the difficulty level.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing technique for presenting a deformation of a structure or the like to a user. [Background technology]

[0002] There is a technology in which a computer performs machine learning on images of an object to be inspected, such as the wall surface of a concrete structure, to detect cracks and other defects, or to detect attributes of the defects, such as crack width, through image analysis. In this case, partial degradation of image quality due to shaking or blurring during photography can result in erroneous or undetected detection of defects in parts of the captured image. For this reason, users may check the areas in the image where cracks or other defects have been detected, and then correct or edit the defect data based on the results of this check. Patent Document 1 also discloses a technology in which a captured image is divided into predetermined rectangles, and a crack rate is calculated based on whether cracks are detected within the divided rectangles. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-56303 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology in Patent Document 1 allows users to check within an image where cracks or other abnormalities have occurred. However, to determine whether corrections or editing are necessary for individual abnormalities, users must check the state of each individual abnormality on the image themselves. This is a very burdensome and difficult task for users.

[0005] Therefore, an object of the present invention is to make it possible to present to the user the abnormalities that require correction or editing. [Means for solving the problem]

[0006] The information processing device of the present invention is characterized by having a detection means for detecting, as a candidate area, an area that contains two or more ends of a specified deformation within a range of a specified size from an image of an object to be inspected; an acquisition means for acquiring the difficulty level when a user determines whether or not to perform a specified task for the deformation whose ends are included in the candidate area; and a display means for displaying the candidate area based on the difficulty level. [Effects of the Invention]

[0007] According to the present invention, it is possible to present to the user the portions of the image that are deformed and require correction or editing. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 2 is a block diagram illustrating an example of a hardware configuration of an information processing device. [Figure 2] FIG. 10 is a diagram showing an example of a deformation data table. [Figure 3] FIG. 10 is a diagram showing an example of a display of a crack drawn based on deformation data. [Figure 4] FIG. 4 is a diagram showing an example of a GUI screen for checking a deformation according to the first embodiment. [Figure 5] 10 is a flowchart of a confirmation candidate presentation process. [Figure 6] FIG. 10 is an explanatory diagram of a confirmation candidate area. [Figure 7] FIG. 10 is a diagram illustrating an example of a rule setting screen. [Figure 8] FIG. 10 is a diagram illustrating an example of a confirmation candidate area table. [Figure 9] FIG. 10 is a diagram showing an example of a GUI screen for checking a deformation according to the second embodiment. [Figure 10] FIG. 11 is a diagram showing an example of a GUI screen for checking a deformation according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention, and not all of the combinations of features described in the present embodiments are necessarily essential to the solution of the present invention; multiple features may be combined arbitrarily. The configurations of the embodiments may be modified or changed as appropriate depending on the specifications of the device to which the present invention is applied and various conditions (such as usage conditions and usage environment). Furthermore, in each of the following embodiments, the same or similar components will be designated by the same reference numerals, and redundant explanations will be omitted.

[0010] First Embodiment In this embodiment, an example will be described in which the inspection target is a concrete structure such as a variety of roads, including expressways, bridges, tunnels, and dams, and multiple anomalies are detected from images of the inspection target, and the user corrects or edits these anomalies. Furthermore, in this embodiment, an example will be described in which anomalies detected from images of the inspection target are cracks that occur on the concrete surface due to damage, deterioration, or other factors of the concrete structure. A crack is linear damage that has a start point, an end point, a length, and a width and occurs on the wall surface of a concrete structure due to aging, earthquake impact, or the like.

[0011] In this embodiment, an example of a user's correction or editing (hereinafter referred to as "correction") of a crack detected in an image of an object to be inspected will be described, in which cracks whose ends are close to each other in the image are joined together to form a single crack. For example, if there are cracks whose ends are close to each other in the image, the user determines (confirms) whether the ends of the cracks are actually connected. If the user determines that the cracks should be connected, the user performs a correction to join the close ends of the cracks. However, finding all of the cracks whose ends are close to each other among the multiple cracks present in the image is a burdensome task for the user, and some cracks may be missed. Furthermore, even if cracks whose ends are close to each other are found, it is still difficult for the user to determine whether they should be repaired by joining the ends of the cracks. In particular, when the ends of three or more cracks are close to each other, it becomes very difficult for the user to determine which of the ends of the cracks should be connected and which should not.

[0012] Therefore, in the information processing device of this embodiment, based on the detection results of a predetermined abnormality (crack) obtained from a photographed image of the inspection object, an area containing two or more edges of the predetermined abnormality within a range of a predetermined size is detected as a confirmation candidate area where the user should make a predetermined correction. Furthermore, the information processing device of this embodiment acquires the difficulty (referred to as confirmation difficulty) of determining whether the user should perform a predetermined task for an abnormality whose edge is included in the confirmation candidate area. Then, based on the confirmation difficulty acquired for each confirmation candidate area, the information processing device highlights these confirmation candidate areas. Details of the process of detecting an abnormality from a photographed image of the inspection object, the process of detecting confirmation candidate areas, the process of acquiring the confirmation difficulty, and the highlighting of confirmation candidate areas based on the confirmation difficulty will be described later.

[0013] <Hardware configuration> 1 is a block diagram showing an example of a hardware configuration capable of realizing an information processing device 100 according to this embodiment. In this embodiment, an example is given in which a computer device operates as the information processing device 100. Note that the information processing according to this embodiment may be realized by a single computer device, or may be realized by distributing each function among multiple computer devices as necessary. When each function is distributed among multiple computer devices, the multiple computer devices are connected to each other so that they can communicate with each other.

[0014] The information processing device 100 includes a control unit 101 , a nonvolatile memory 102 , a work memory 103 , a storage device 104 , an input device 105 , an output device 106 , a network interface, and a system bus 110 .

[0015] The control unit 101 includes an arithmetic processor, such as a CPU or MPU, that controls the entire information processing device 100. The non-volatile memory 102 is a ROM that stores programs and parameters executed by the processor of the control unit 101. The programs stored in the non-volatile memory 102 include an operating system (OS), which is basic software executed by the control unit 101, and application programs that work with the OS to implement applied functions. The information processing program according to this embodiment is stored in the non-volatile memory 102 as one of the application programs. The control unit 101 reads and executes the information processing program according to this embodiment from the non-volatile memory 102 to implement a confirmation candidate presentation process, as described below. The application program includes a program for utilizing basic functions of the OS. Alternatively, the OS itself may include the information processing program according to this embodiment. The information processing program according to this embodiment may be stored not only in the non-volatile memory 102 but also in the storage device 104.

[0016] The work memory 103 is a RAM that temporarily stores programs and data supplied from the non-volatile memory 102, the storage device 104, an external device, etc. The storage device 104 is an internal device such as a hard disk or memory card built into the information processing device 100, or an external device such as a hard disk or memory card detachably connected to the information processing device 100. In this embodiment, the storage device 104 also stores a deformation data table in which multiple deformation data detected from captured images of the inspection target are registered. Details of the deformation data table will be described later. The storage device 104 may include a memory card made of semiconductor memory or a hard disk made of a magnetic disk, etc. The storage device 104 may also include a storage medium made of a disk drive that reads and writes data from / to optical disks such as DVDs and Blu-ray Discs (registered trademark).

[0017] The input device 105 includes operating devices such as a mouse, keyboard, and touch panel that accept input operations from the user and outputs operation instructions input by the user to the control unit 101. The output device 106 includes display devices such as a monitor and a display configured with an LCD or organic EL. The control unit 101 generates display data based on data held by the information processing device 100 or data supplied from an external device, and display data for a GUI (Graphical User Interface) screen (described later), and sends the display data to the output device 106. The network interface 107 communicates with a network such as the Internet or a LAN (Local Area Network). The system bus 110 connects the components of the information processing device 100 (such as the control unit 101 to the network interface 107) and enables data to be exchanged between these components. The system bus 110 includes an address bus, a data bus, and a control bus.

[0018] <Deformation Data Table> FIG. 2 shows an example of the data structure of a deformation data table in which multiple deformation data detected from photographed images of an inspection object are registered. In this embodiment, the deformation data table is a table in which deformation data corresponding to multiple deformations detected from photographed images of an inspection object, such as a concrete structure, are registered. The deformation data table is generated by registering deformation data automatically generated from photographed images using image analysis processing, deformation data generated from information obtained by a user tracing deformations on the photographed images, or deformation data generated by combining these. Examples of image analysis processing that automatically generates deformation data from photographed images include processing using a learning model created by machine learning or deep learning using AI (artificial intelligence). A deformation data table is generated for each different inspection object, and the deformation data table generated for each inspection object is stored, for example, in the storage device 104. In this embodiment, the process of detecting deformations from photographed images of the inspection object and the process of registering deformation data corresponding to the multiple detected deformations in the deformation data table are assumed to be performed in advance by the information processing device 100. The process of detecting a deformation from a photographed image of the inspection target and the process of registering the detected plurality of deformation data in the deformation data table may be performed by a computer device or the like other than the information processing device 100.

[0019] The deformation data table 201 shown in Figure 2 is a table in which, for example, crack deformation data is registered, and the crack deformation data includes a deformation ID 202, a maximum width 203, a number of vertices 204, and a vertex coordinate list 205. The deformation ID 202 is identification information uniquely assigned to each crack. The maximum width 203 is the maximum width (thickness) of the crack. The number of vertices 204 is the number of vertices corresponding to the start and end points of each line segment when the shape of the crack is expressed as a polyline consisting of one or more line segments. The vertex coordinate list 205 is information on the coordinates of each vertex corresponding to the start and end points of each line segment when the shape of the crack is expressed as a polyline consisting of one or more line segments. Note that the coordinates of each vertex in the vertex coordinate list 205 are assumed to be coordinates within a coordinate space corresponding to the captured image.

[0020] Figure 3 is a diagram showing an example of a display screen 300 of a crack drawn based on the deformation data registered in the deformation data table 201 shown in Figure 2. On the display screen 300, for example, crack 301 is an example image of a crack drawn based on the deformation data assigned the deformation ID indicated as Ca001 in the deformation data table 201. Similarly, cracks 302 to 310 are images of cracks drawn based on the deformation data assigned the deformation IDs Ca002 to Ca010, respectively.

[0021] FIG. 4(a) is a diagram showing an example of a GUI screen 400 for checking a defect, which is displayed on the output device 106 when the control unit 101 executes a program for the check candidate presentation process. The control unit 101 acquires each piece of deformation data from the deformation data table 201, and displays a GUI screen 400 as shown in FIG. 4(a) on the output device 106 based on the deformation data.

[0022] The GUI screen 400 includes a result display area 401 , a confirmation area list 402 , and a candidate legend display area 403 . The result display area 401 is an area where the confirmation candidate areas are displayed in addition to the crack image drawn based on the deformation data. The confirmation area list 402 is an area where the area ID for each confirmation candidate area is displayed in a list.

[0023] The candidate legend display area 403 is an area where a legend indicating the degree of difficulty of confirmation when a user confirms (judges) whether or not an individual crack deformation requires repair is displayed. In FIG. 4(a), three legends for the degree of difficulty of confirmation are listed: a difficult rank 404, a medium rank 405, and an easy rank 406. In FIG. 4, the candidate legend display area 403 displays the characters "Difficult rank" and a difficult rank-enhanced image representing the difficult rank 404, the characters "Medium rank" and a medium rank-enhanced image representing the medium rank 405, and the characters "Easy rank" and an easy rank-enhanced image representing the easy rank 406. The difficult rank 404 indicates a region with a high degree of difficulty of confirmation, where the user has great difficulty in determining whether or not a crack requires repair. The medium rank 405 indicates a region with a lower degree of difficulty of confirmation when the user himself / herself makes the judgment than the difficult rank but higher than the easy rank. The easy rank 406 indicates a region with a low degree of difficulty of confirmation when the user himself / herself makes the judgment. The control unit 101 calculates a confirmation difficulty score for each confirmation candidate area as described below, and, depending on the confirmation difficulty score, superimposes a translucent emphasis image on each confirmation candidate area in the result display area 401. In Fig. 4, a difficult rank emphasis image representing the difficult rank 404 is superimposed on the confirmation candidate area 411 in the result display area 401, a medium rank emphasis image representing the medium rank 405 is superimposed on the confirmation candidate area 412, and an easy rank emphasis image representing the easy rank 406 is superimposed on the confirmation candidate area 413. Note that, in Fig. 4, a checkerboard pattern and a dot pattern are given as examples of emphasis images representing the difficult, medium, and easy ranks, respectively, but the emphasis images are not limited to these and may be, for example, different colors, different brightness, different marks, etc.

[0024] <Confirmation candidate presentation process> FIG. 5 is a flowchart showing the flow of confirmation candidate presentation processing that is realized by the control unit 101 of this embodiment executing the information processing program according to this embodiment. First, in step S501, the control unit 101 reads out each deformation data from the deformation data table corresponding to the inspection object specified by the user via the input device 105 among the multiple deformation data tables stored in the memory device 104.

[0025] Next, in step S502, the control unit 101 acquires coordinate data of the end of each crack from each deformation data read out in step S501. Furthermore, in step S503, the control unit 101 moves a candidate determination area of ​​a predetermined size (area) within a coordinate space that includes the coordinates of all deformation data, and searches for a candidate determination area that includes the end coordinates of two or more cracks within that area. The control unit 101 then determines a confirmation candidate area based on the candidate determination area that includes the end coordinates of two or more cracks within the coordinate space. If the control unit 101 searches for multiple candidate determination areas that include the end coordinates of two or more cracks within the coordinate space, it determines a confirmation candidate area for each of those candidate determination areas. Note that, in this embodiment, the candidate determination area and the confirmation candidate area are assumed to have the same size and shape, but at least one of the size and shape of the candidate determination area and the confirmation candidate area may differ.

[0026] FIG. 6 is a diagram used to explain the process in step S503 of FIG. 5 in which the control unit 101 determines a confirmation candidate region through a search process using the candidate determination region. The control unit 101 moves the candidate determination area 601 within a coordinate space that includes the coordinates of all the deformation data, searching for a position within the candidate determination area 601 where two or more end coordinates exist. When the control unit 101 finds a candidate determination area 601 where two or more end coordinates exist, it calculates the central coordinates of the end coordinates of the candidate determination area 601 and determines a confirmation candidate area with the central coordinates as the center of the area. Figure 6 shows an example in which an end 611 of a deformation 610, an end 621 of a deformation 620, and an end 631 of a deformation 630 are included within the candidate determination area 601. When the control unit 101 finds a candidate determination area 601 where the three end coordinates of the end 611 to end 631 exist, it determines a confirmation candidate area with the central coordinate 602 of the three end coordinates as the center of the area. Note that Figure 6 shows an example in which a square area having a predetermined size (area) is used as the candidate determination area, but this is not limited thereto. The candidate determination area may be, for example, a circular area with a predetermined radius.

[0027] It should be noted that there may be cases where even more edge coordinates are included within the candidate determination area 601. In such cases, the control unit 101 may calculate center coordinates for each of three adjacent edge coordinates, for example, and determine a confirmation candidate area whose center is the coordinate of the average value of the multiple center coordinates calculated for each of the three adjacent edge coordinates.

[0028] After step S503 described above, the control unit 101 repeats the process from step S504 to step S507. In step S505 of the repeating process, the control unit 101 calculates the verification difficulty of the confirmation candidate area that is the target of the verification difficulty calculation. The control unit 101 acquires crack attributes such as the number of cracks, crack width, and crack direction from the deformation data of cracks whose ends are included in the confirmation candidate area, and calculates the verification difficulty of the confirmation candidate area based on the number and attributes of the cracks. The control unit 101 then assigns a score representing the verification difficulty to the confirmation candidate area.

[0029] For example, the more cracks that contain ends within a confirmation candidate area, the more difficult it is for the user to determine (confirm) whether the cracks need to be repaired or which cracks should be connected. For this reason, the control unit 101 increases the confirmation difficulty for a confirmation candidate area as the number of cracks that contain ends within the confirmation candidate area increases. As an example, the control unit 101 calculates the number of cracks based on the number of deformation IDs of each crack that contains ends within the confirmation candidate area, and calculates the confirmation difficulty by weighting the number. As a weight value according to the number of cracks, a value such as 10 times can be used, for example.

[0030] Furthermore, for example, even if multiple cracks with similar widths (thicknesses) exist within the confirmation candidate area and have edges, it may be difficult for the user to determine (confirm) whether the cracks need repair or which cracks should be connected. For this reason, the control unit 101 may obtain the number of cracks with similar widths among the cracks with edges in the confirmation candidate area and obtain the confirmation difficulty level based on the number of cracks with similar widths. That is, the control unit 101 obtains the confirmation difficulty level using not only the number of cracks with edges in the confirmation candidate area but also the number of cracks with similar widths. In this case, the control unit 101 determines whether the crack widths are similar based on, for example, a ranking according to the size of the crack width. For example, the control unit 101 classifies multiple cracks with edges in the confirmation candidate area into multiple ranks according to the size of their widths. The control unit 101 then classifies cracks whose crack widths fall within the same rank as cracks with a high degree of similarity, while classifying cracks that fall into different ranks as dissimilar cracks with a low degree of similarity.

[0031] The control unit 101 then calculates the degree of difficulty of confirmation by assigning a weight according to the number of cracks with high width similarity in addition to the number of cracks whose ends are included in the confirmation candidate area. A value such as 7 times can be used as the weight value according to the number of cracks with high width similarity. In other words, the control unit 101 acquires the degree of difficulty of confirmation for the confirmation candidate area by assigning a weight according to the number of cracks with high width similarity in addition to the weight value according to the number of cracks described above. In this example, the verification difficulty is calculated based on the number of cracks whose edges are included in the verification candidate area and the number of cracks whose widths are highly similar, but this is not limiting. For example, the control unit 101 may calculate the verification difficulty using only the number of cracks whose widths are highly similar.

[0032] Furthermore, for example, even if there are multiple cracks with similar crack directions for each crack whose end is located within the confirmation candidate area, it may be difficult for the user to determine (confirm) whether the crack needs to be repaired or which cracks should be connected. For this reason, the control unit 101 may acquire the confirmation difficulty level based on the number of cracks with similar directions among the cracks whose end is located within the confirmation candidate area. In this case, the control unit 101 calculates the crack direction based on, for example, an angle calculated from the coordinate values ​​of the start and end points registered in the vertex coordinate list 205 of the deformation data, or the average, median, or mode of the angles of each line segment that constitutes the crack deformation. Furthermore, the control unit 101 determines whether the crack directions are similar based on the angle between the crack directions. For example, the control unit 101 may classify cracks whose angle between the crack directions is less than a threshold as highly similar cracks, and cracks whose angle between the crack directions is greater than or equal to the threshold as dissimilar cracks whose degree of similarity is low. As an example, if 30 degrees is used as the threshold for the angle between crack directions, the control unit 101 determines that the similarity of the crack directions is high if the angle between the crack directions is less than 30 degrees, and determines that the similarity of the crack directions is low and dissimilar if the angle is 30 degrees or more. Alternatively, the control unit 101 may use, for example, the similarity of the angles of the line segments that make up a crack whose ends are included in the confirmation candidate area as the similarity of the crack directions.

[0033] The control unit 101 then calculates the degree of difficulty of confirmation by assigning a weight according to the number of cracks with high directional similarity in addition to the number of cracks whose ends are included in the confirmation candidate area. A value such as 5 times can be used as the weight value according to the number of cracks with high directional similarity. In other words, the control unit 101 acquires the degree of difficulty of confirmation of the confirmation candidate area by assigning a weight according to the number of cracks with high directional similarity in addition to the weight value according to the number of cracks described above.

[0034] In this example, the confirmation difficulty of a confirmation candidate region is calculated based on the number of cracks whose edges are included in the confirmation candidate region and the number of cracks whose direction is similar, but this is not limiting. For example, the control unit 101 may calculate the confirmation difficulty of a confirmation candidate region using only the number of cracks whose direction is similar. Furthermore, for example, the control unit 101 may calculate the confirmation difficulty of a confirmation candidate region by adding the number of cracks whose direction is similar to the number of cracks whose width is similar, as described above, to the number of cracks whose direction is similar. Furthermore, for example, the control unit 101 may calculate the confirmation difficulty of a confirmation candidate region by using the number of cracks whose edge is included in the confirmation candidate region, as well as the number of cracks whose direction is similar and the number of cracks whose width is similar.

[0035] As described above, the control unit 101 of this embodiment increases the difficulty of checking a candidate confirmation area the more cracks with ends within the candidate confirmation area there are, and the greater the similarity in attributes such as width and direction of the cracks with ends within the candidate confirmation area.

[0036] In addition, the control unit 101 can also determine, based on instructions from the user, whether to use the number of cracks whose ends are included in the confirmation candidate area, the similarity in crack width, and the similarity in crack direction when calculating the confirmation difficulty of the confirmation candidate area. Fig. 7 is a diagram showing an example of a rule setting GUI screen 701 that allows the user to set whether to use the number of cracks, the similarity in crack width, and the similarity in crack direction when obtaining the confirmation difficulty of the confirmation candidate area.

[0037] In the GUI screen 701 for setting rules shown in Fig. 7, the number use switch 702 is a toggle switch that the user uses to specify an ON / OFF setting for determining whether or not to use the number of cracks when obtaining the verification difficulty of a verification candidate area. The width similarity use switch 703 is a toggle switch that the user uses to specify an ON / OFF setting for determining whether or not to use the similarity in crack width when obtaining the verification difficulty of a verification candidate area. The direction similarity use switch 704 is a toggle switch that the user uses to specify an ON / OFF setting for determining whether or not to use the similarity in crack direction when obtaining the verification difficulty of a verification candidate area. The control unit 101 determines whether or not to use the number of cracks, the similarity in width, or the similarity in direction to calculate the verification difficulty, depending on the ON / OFF settings of the switches 702 to 704 on the GUI screen 701 for setting rules shown in Fig. 7.

[0038] Returning to the explanation of the flowchart in FIG. In step S506 of the repeating process, the control unit 101 classifies the confirmation candidate area into one of the ranks 404, 405, and 406 illustrated in Fig. 4 based on the confirmation difficulty score assigned to the confirmation candidate area in step S505. The control unit 101 then highlights the confirmation candidate area in the result display area 401 in Fig. 4(a) according to the rank classification. Note that the control unit 101 may highlight only confirmation candidate areas whose confirmation difficulty score is equal to or greater than a predetermined threshold, and not highlight confirmation candidate areas whose score is equal to or less than the threshold.

[0039] The control unit 101 also registers information about each confirmation candidate area acquired through the repeated processing from step S505 to step S507 in a confirmation candidate area table. The confirmation candidate area table is stored in the storage device 104, for example. FIG. 8 is a diagram showing an example of a confirmation candidate area table 801. In the confirmation candidate area table 801, an area ID 802 is identification information uniquely assigned to each confirmation candidate area detected in step S503. Center coordinates 803 are the center coordinates of the confirmation candidate area. Deformation ID 804 is a list of deformation IDs corresponding to crack deformations whose ends are included in the confirmation candidate area. Score 805 is a score indicating the degree of difficulty of confirmation obtained for each confirmation candidate area in step S505.

[0040] Then, when the control unit 101 completes the processes from step S505 to step S507 for all the confirmation candidate areas extracted from the coordinate space in step S503, the control unit 101 advances the process to step S508. In step S508, the control unit 101 determines whether the user has performed an area designation operation to designate any area ID in the confirmation area list 402 in Fig. 4. If the user has performed an area designation operation to designate any area ID, the process of the control unit 101 proceeds to step S509. On the other hand, if an area designation operation has not been performed, the process of the control unit 101 returns to step S508 unless it is determined in step S510, described later, that an instruction to end the confirmation candidate presentation process has been input.

[0041] In step S509, the control unit 101 enlarges and displays the image portion including the confirmation candidate area corresponding to the area ID specified by the user in the result display area 401. Fig. 4(b) shows an example in which the user specifies an area in the confirmation area list 402 that has an area ID of R011, and the image portion including the confirmation candidate area corresponding to that area ID is displayed as an enlarged display image 422.

[0042] Then, in step S510, the control unit 101 determines whether or not the user has input an instruction to end the confirmation candidate presentation process. If the instruction to end the process has been input, the control unit 101 ends the process of the flowchart in Fig. 5, and if the instruction to end the process has not been input, the control unit 101 returns to step S508.

[0043] As described above, the information processing device 100 of this embodiment searches for confirmation candidate areas where crack edges are clustered within a range of a predetermined size based on the crack deformation detection results, and calculates and displays a confirmation difficulty level for each confirmation candidate area. That is, according to this embodiment, by presenting areas on the image where crack deformation is interrupted and where there is a possibility of erroneous detection or undetected cracks as confirmation candidate areas to the user, the user can easily confirm them. Furthermore, according to this embodiment, the confirmation difficulty level is calculated based on the number and attributes of cracks whose edges are included in the confirmation candidate areas, and by highlighting the areas according to the confirmation difficulty level, the user can be presented with areas to be confirmed in coordinate space.

[0044] <Second embodiment> In the first embodiment described above, an example was described in which a confirmation candidate area containing two or more ends of an abnormality is searched for, and the confirmation difficulty level is obtained based on at least one of the number and attributes of the abnormalities containing ends within the confirmation candidate area. In the second embodiment described below, an example is described in which the confirmation candidate area presented (displayed) to the user is adjusted using user attributes as well. Note that the configuration and processing of the information processing device 100 according to the second embodiment are generally similar to those of the above-described embodiment, and therefore illustrations and descriptions of the configuration and processing will be omitted.

[0045] In the second embodiment, the user attribute is an attribute that represents the user's experience level in the confirmation work, such as whether the user is an expert with ample experience in the confirmation work for abnormalities, or a beginner with little experience in the confirmation work for abnormalities. The experience level attribute indicating whether the user is an expert or a beginner may be set by the user himself or another user, or may be set by the control unit 101 based on the number of times or the time the user has performed the confirmation work for abnormalities in the past.

[0046] 9(a) to 9(c) are diagrams showing an example of a GUI screen 400 for checking anomalies according to the second embodiment. In the GUI screens 400 shown in Fig. 9(a) to 9(c), a user attribute switch 901 for setting user attributes is added to the example GUI screen 400 of Fig. 4 described above. The user attribute switch 901 is a toggle switch that allows a user checking anomalies on an image using the information processing device 100 of this embodiment to input whether they are a beginner or an expert.

[0047] 9(a) shows the GUI screen 400 with the user attribute switch 901 set to the beginner side. When the user attribute switch 901 is set to the beginner side, the control unit 101 changes the display order of the area IDs in the confirmation area list 402 to display the area IDs of the confirmation candidate areas in order of the lowest level of confirmation difficulty. This allows a beginner user to check whether or not to correct the anomaly in the confirmation candidate areas in order of the lowest level of confirmation difficulty.

[0048] 9(b) shows the GUI screen 400 when the user attribute switch 901 is set to expert. When the user attribute switch 901 is set to expert, the control unit 101 changes the display order of the area IDs in the confirmation area list 402 to display the area IDs of the confirmation candidate areas in order of increasing difficulty of confirmation. This allows an expert user to check whether or not to correct the anomaly in a confirmation candidate area, giving priority to areas that would be difficult for a beginner to judge.

[0049] In the above-described FIG. 9(a), when the user attribute switch 901 is set to beginner, the area IDs of confirmation candidate areas are displayed in the confirmation area list 402 in order of decreasing confirmation difficulty. However, the area IDs of confirmation candidate areas with high confirmation difficulty may not be displayed. When the user attribute switch 901 is set to beginner, the control unit 101 hides the area IDs of confirmation candidate areas whose confirmation difficulty scores are, for example, equal to or greater than a predetermined threshold. FIG. 9(c) is an example of a GUI screen 400 in which, when the user is a beginner, the area IDs of confirmation candidate areas whose confirmation difficulty scores are equal to or greater than a threshold are not displayed in the confirmation area list 402. That is, the confirmation area list 402 in FIG. 9(c) does not display the area IDs of confirmation candidate areas with high confirmation difficulty scores, such as R038 and R047, which were displayed in the confirmation area list 402 in FIG. 9(b). In addition, the control unit 101 may also hide the legend rank "difficult" 404, which indicates a high confirmation difficulty, in the candidate legend display area 403. This allows a novice user to confirm whether or not to correct the abnormality without being influenced by a confirmation candidate area that is difficult to confirm and that makes it difficult to determine whether or not to correct the abnormality.

[0050] <Third embodiment> In the third embodiment, an example will be described in which, depending on the user's attributes, it is possible to postpone the decision on correction for an abnormality whose edge is included in the confirmation candidate area. Note that, since the configuration and processing of the information processing device 100 according to the third embodiment are generally similar to those of the above-described embodiments, illustration and description of the configuration and processing will be omitted. For example, if the user is a beginner, the user may not be able to determine whether or not to correct the displayed deformation of the confirmation candidate area. For this reason, the information processing device 100 of the third embodiment shares the deformation and the confirmation candidate area with the information processing device of another experienced user, so that the user can entrust the confirmation of the confirmation candidate area to the experienced user.

[0051] 10 shows an example of a GUI screen 400 for confirming a deformation according to the third embodiment, in which a judgment reservation checkbox 1001 has been added to the confirmation area list 402. If a beginner user cannot determine whether or not to correct a crack deformation in a confirmation candidate area, the beginner user can check the judgment reservation checkbox 1001 corresponding to the area ID of the confirmation candidate area. When the judgment reservation checkbox 1001 is checked, the control unit 101 notifies the information processing devices of other users who share the confirmation candidate area and deformation with the information processing device of the beginner user, requesting them to confirm the confirmation candidate area of ​​that area ID. As described above, according to the third embodiment, an inexperienced beginner user can ask another user, such as an expert user, to make a decision on corrections to the abnormality in the confirmation candidate area.

[0052] <Other embodiments> In the above-described embodiments, all processing related to the presentation of defect confirmation candidates is performed on the information processing device 100 used by the user. However, the processing may be shared among other computers, such as in a server-client system. For example, the process of calculating the degree of difficulty of confirmation may be performed on the server side, and the degree of difficulty of confirmation calculated on the server side may be presented (displayed) to the user on the client-side information processing device. Furthermore, the number of client-side information processing devices is not limited to one, and multiple devices may be used. This allows the results of the server's calculation of the degree of difficulty of confirmation to be accessed from the respective information processing devices of multiple client-side users, allowing each user to check for defects in the confirmation candidate areas.

[0053] Furthermore, in the above-described embodiments, cracks have been cited as an example of deformation, but deformation is not limited to cracks. For example, for each type of deformation that occurs on the surface of a structure, such as lifting, peeling, efflorescence, cold joints, peanut brittleness, surface bubbles, sand streaks, and rust stains, areas whose edges are concentrated within a predetermined range may be extracted as candidate areas. Then, for each type of deformation, a difficulty level for the candidate area may be calculated, and each candidate area may be displayed based on the calculated difficulty level.

[0054] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. The above-described embodiments are merely examples of specific embodiments for carrying out the present invention, and the technical scope of the present invention should not be construed as being limited by these embodiments. In other words, the present invention can be carried out in various forms without departing from its technical concept or main features.

[0055] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) A detection means for detecting, as a candidate area, an area that includes two or more edges of a predetermined deformation within a range of a predetermined size from an image of an inspection object; An acquisition means for acquiring a degree of difficulty when a user determines whether or not a predetermined work is to be performed for the deformation in which the end portion is included in the candidate area; a display means for displaying the candidate area based on the degree of difficulty; An information processing device comprising: (Configuration 2) The information processing device described in configuration 1 is characterized in that the acquisition means acquires the difficulty level based on at least one of the number of deformations whose ends are included in the candidate area, the similarity in width of the deformations whose ends are included in the candidate area, and the similarity in direction of the deformations whose ends are included in the candidate area. (Configuration 3) 3. The information processing device according to configuration 2, wherein the acquisition means increases the degree of difficulty as the number of the defects including the edge portions in the candidate area increases. (Configuration 4) 4. The information processing device according to configuration 2 or 3, wherein the acquisition means increases the difficulty level as the degree of similarity of the width of the deformation in which the end portion is included in the candidate area increases. (Configuration 5) The information processing device according to any one of configurations 2 to 4, wherein the acquisition means increases the difficulty level as the similarity of the direction of the deformation in which the end portion is included in the candidate area increases. (Configuration 6) The information processing device described in any one of configurations 2 to 5, characterized in that the acquisition means acquires the direction of the deformation based on the average, median, or mode of the angles of each line segment that constitutes the deformation whose end is included in the candidate area. (Configuration 7) 7. The information processing device according to any one of configurations 1 to 6, wherein the display means highlights the candidate area in accordance with the degree of difficulty. (Configuration 8) 8. The information processing device according to any one of configurations 1 to 7, wherein the display means determines the candidate area to be displayed based on the difficulty level. (Configuration 9) 9. The information processing device according to any one of configurations 1 to 8, wherein the display means changes the display order of the candidate areas based on the difficulty level and an attribute of the user. (Configuration 10) 10. The information processing device according to any one of configurations 1 to 9, wherein the display means controls whether to display or hide the candidate area based on the difficulty level and a user attribute. (Configuration 11) 11. The information processing device according to configuration 9 or 10, further comprising a notification means for issuing a predetermined notification to another information processing device based on the difficulty level and the attribute of the user. (Configuration 12) The information processing device described in configuration 11, characterized in that the notification means sends the specified notification to the other information processing device that shares the deformation and the candidate area to request a decision on whether or not to perform the specified work on the deformation whose end is included in the candidate area. (Configuration 13) 13. The information processing device according to any one of configurations 9 to 12, wherein the user attributes are information set based on the user's experience with the predetermined task. (Configuration 14) The information processing device according to any one of configurations 1 to 13, wherein the detection means detects the candidate area by moving the range of the predetermined size in an image of the inspection object and searching for an area that includes the two or more ends within the range of the predetermined size. (Configuration 15) 15. The information processing apparatus according to configuration 14, wherein the detection means detects, as the candidate area, an area having center coordinates of two or more ends included within the range of the predetermined size. (Configuration 16) The information processing device according to configuration 15, wherein the detection means, when there are multiple central coordinates of two or more ends included within the range of the predetermined size, detects an area centered on the coordinate of the average value of the multiple central coordinates as the candidate area. (Configuration 17) 17. The information processing device according to any one of configurations 1 to 16, wherein the predetermined deformation is a crack in the concrete structure to be inspected. (Method 1) A detection step of detecting, as a candidate area, an area including two or more edges of a predetermined deformation within a range of a predetermined size from an image of the inspection object; An acquisition step of acquiring a degree of difficulty when a user determines whether or not a predetermined work is to be performed for the deformation in which the end portion is included in the candidate area; a display step of displaying the candidate area based on the difficulty level; An information processing method comprising: (Program 1) A program for causing a computer to function as the information processing device according to any one of configurations 1 to 17. [Explanation of symbols]

[0056] 100: Information processing device, 101: Control unit, 102: Nonvolatile memory, 103: Work memory, 104: Storage device, 105: Input device, 106: Output device, 107: Network interface, 108: System bus

Claims

1. A detection means for detecting, as a candidate area, an area that includes two or more edges of a predetermined deformation within a range of a predetermined size from an image of an inspection object; An acquisition means for acquiring a degree of difficulty when a user determines whether or not a predetermined work is to be performed for the deformation in which the end portion is included in the candidate area; a display means for displaying the candidate area based on the degree of difficulty; An information processing device comprising:

2. The information processing device described in claim 1, characterized in that the acquisition means acquires the difficulty level based on at least one of the number of deformations whose ends are included in the candidate area, the similarity in the width of the deformations whose ends are included in the candidate area, and the similarity in the direction of the deformations whose ends are included in the candidate area.

3. The information processing apparatus according to claim 2 , wherein the acquisition means increases the degree of difficulty as the number of the defects including the edge portions in the candidate area increases.

4. The information processing apparatus according to claim 2 , wherein the obtaining means increases the difficulty level as the degree of similarity of the width of the deformation in which the end portion is included in the candidate area increases.

5. The information processing apparatus according to claim 2 , wherein the acquisition means increases the degree of difficulty as the similarity of the direction of the deformation in which the end portion is included in the candidate area increases.

6. The information processing device described in claim 2, characterized in that the acquisition means acquires the direction of the deformation based on the average value, median, or mode of the angles of each line segment that constitutes the deformation whose end is included in the candidate area.

7. 2. The information processing apparatus according to claim 1, wherein the display means highlights the candidate area according to the degree of difficulty.

8. 2. The information processing apparatus according to claim 1, wherein the display means determines the candidate area to be displayed based on the degree of difficulty.

9. 2. The information processing apparatus according to claim 1, wherein the display means changes the display order of the candidate areas based on the degree of difficulty and an attribute of the user.

10. 2. The information processing apparatus according to claim 1, wherein the display means controls whether the candidate area is displayed or not based on the degree of difficulty and an attribute of the user.

11. 10. The information processing apparatus according to claim 9, further comprising a notification means for issuing a predetermined notification to another information processing apparatus based on the difficulty level and the user's attributes.

12. The information processing device described in claim 11, characterized in that the notification means sends the specified notification to the other information processing device that shares the abnormality and the candidate area to request a determination as to whether the specified work should be performed on the abnormality whose end is included in the candidate area.

13. 13. The information processing apparatus according to claim 9, wherein the user attributes are information set based on the user's experience with the predetermined task.

14. The information processing device according to claim 1, characterized in that the detection means detects the candidate area by moving the range of the predetermined size in an image of the object to be inspected and searching for an area that includes the two or more ends within the range of the predetermined size.

15. 15. The information processing apparatus according to claim 14, wherein said detection means detects, as said candidate area, an area having center coordinates of two or more ends included in the range of said predetermined size as its center.

16. The information processing device according to claim 15, characterized in that, when there are multiple center coordinates of two or more ends within the range of the predetermined size, the detection means detects an area centered on the coordinate of the average value of the multiple center coordinates as the candidate area.

17. 2. The information processing apparatus according to claim 1, wherein the predetermined abnormality is a crack in the concrete structure to be inspected.

18. A detection step of detecting, as a candidate area, an area including two or more edges of a predetermined deformation within a range of a predetermined size from an image of the inspection object; An acquisition step of acquiring a degree of difficulty when a user determines whether or not a predetermined work is to be performed for the deformation in which the end portion is included in the candidate area; a display step of displaying the candidate area based on the difficulty level; An information processing method comprising:

19. Computer, A detection means for detecting, as a candidate area, an area that includes two or more edges of a predetermined deformation within a range of a predetermined size from an image of an inspection object; An acquisition means for acquiring a degree of difficulty when a user determines whether or not a predetermined work is to be performed for the deformation in which the end portion is included in the candidate area; a display means for displaying the candidate area based on the degree of difficulty; A program that causes the device to function as an information processing device having the above.

Images