Information processing device, information processing method, and program

Abstract

To reduce the calculation of an erroneous progress level.SOLUTION: An information processing device performs: selecting at least one of first deformations associated with a first image as a reference deformation; and selecting at least one of second deformations associated with a second image photographed in a period different from that of the first image as a correction object deformation. The information processing device performs: correcting the correction object deformation so as to generate a correction candidate; acquiring a matching level which expresses a matching relation between the reference deformation and the correction candidate; and correcting the correction object deformation so as to generate a corrected deformation based on the matching level. Then, the information processing device acquires a progress level which expresses the change of the deformation from the reference deformation based on comparison between the reference deformation and the corrected deformation.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing technique using images of structures and the like. [Background technology]

[0002] When inspecting structures such as bridges and tunnels, it is necessary to understand the extent to which defects such as cracks and exposed rebar in the structure's walls have progressed in order to determine the soundness of the structure's components, in other words, how the defects change over time. In the inspection of infrastructure structures using images, abnormalities are detected from images of the same wall surface of the same structure taken at different times, and the degree of progress of each abnormality is calculated by determining the difference between the detected abnormalities. However, since the shooting conditions such as the shooting position and weather change each time a photograph is taken, even images of the same wall surface of the same structure contain positional deviations between each pixel in the images. Therefore, it is necessary to calculate the degree of progress of abnormalities while taking these deviations into account. Patent Document 1 discloses a method of determining the correspondence between abnormalities while taking the deviations into account. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-20220 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the above-mentioned technology, if the shapes of abnormalities detected from images taken at different times are significantly different, it is not possible to determine the correspondence between the abnormalities, and an incorrect degree of progression may be calculated.

[0005] Therefore, an object of the present invention is to reduce erroneous calculations of the degree of progress. [Means for solving the problem]

[0006] The information processing device of the present invention is characterized by having a selection means for selecting at least one deformation among the first deformations associated with a first image as a reference deformation and selecting at least one deformation among the second deformations associated with a second image taken at a time different from the first image as a deformation to be corrected, a candidate creation means for modifying the deformation to be corrected to create a correction candidate, a consistency acquisition means for acquiring a consistency degree representing the consistency relationship between the reference deformation and the correction candidate, a deformation correction means for correcting the deformation to be corrected based on the consistency degree to create a corrected deformation, and a progress acquisition means for acquiring a progress degree representing the change in deformation from the reference deformation based on a comparison between the reference deformation and the corrected deformation. Effect of the Invention

[0007] According to the present invention, erroneous calculation of the progress degree is reduced. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a diagram illustrating a hardware configuration and a functional configuration of an information processing device. [Diagram 2] FIG. 13 is a diagram for explaining an outline of calculation of the degree of progression of deformation (cracks). [Diagram 3] FIG. 13 is a diagram illustrating the relationship between an image and deformation data. [Figure 4] 4 is a flowchart of information processing according to the first embodiment. [Diagram 5] FIG. 13 is a diagram illustrating an example of selecting a reference deformation and a deformation to be corrected. [Figure 6] FIG. 13 is a diagram illustrating an example of creating a correction candidate for an abnormality. [Figure 7] FIG. 11 is a diagram illustrating an example of calculating a consistency degree. [Figure 8] FIG. 11 is a diagram illustrating an example of selecting a correction candidate. [Figure 9] FIG. 11 is a diagram illustrating an example of a correction process. [Figure 10] FIG. 13 is a diagram illustrating an example of a progress display screen. [Figure 11]FIG. 11 is a diagram illustrating a functional configuration of an information processing device according to a second embodiment. [Figure 12] 10 is a flowchart of information processing according to the second embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of selecting a nearby abnormality. [Figure 14] 13 is a diagram illustrating an example of creating a correction candidate by interpolating an unvisible abnormality portion. FIG. [Figure 15] FIG. 11 is a diagram illustrating a functional configuration of an information processing device according to a third embodiment. [Figure 16] 13 is a flowchart of information processing according to the third embodiment. [Figure 17] 11A and 11B are diagrams illustrating an example of calculating a determination index for determining whether or not to create a correction candidate. [Figure 18] FIG. 13 is a diagram illustrating a functional configuration of an information processing device according to a fourth embodiment. [Figure 19] 13 is a flowchart of information processing according to the fourth embodiment. [Figure 20] FIG. 13 is a diagram illustrating an example of selecting deformation data in group units. [Figure 21] 13A and 13B are diagrams illustrating an example of creating correction candidates in groups. [Figure 22] FIG. 13 is a diagram illustrating an example of calculating a consistency degree for each group. [Diagram 23] 11A and 11B are diagrams illustrating an example of correction processing in units of groups. [Figure 24] 13A to 13C are diagrams illustrating an example of creating a correction candidate in the fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the configurations described in the following embodiments are merely examples, and the scope of the present invention is not limited to these specific configurations. <First embodiment> As a first embodiment, an example of a method for calculating the progress of a change in an inspection target using two images of the same inspection target taken at different times will be described. In particular, in this embodiment, an information processing device for performing so-called infrastructure inspection, such as judging the soundness of a structure such as a bridge, will be described as an example of the inspection target.

[0010] In this embodiment, the terms used in the description are defined as follows. In the example of this embodiment for infrastructure inspection, the "inspection target" refers to a concrete structure or the like. A user described in this embodiment uses the information processing device according to this embodiment for the purpose of inspecting the inspection target for abnormalities such as cracks on its surface based on an image captured of the inspection target. In the case of a concrete structure, "defects" include, for example, cracks, lifting, spalling, efflorescence, exposed rebar, rust, water leakage, dripping, corrosion, damage (missing parts), cold joints, deposits, junk, and the like. The "first image" and the "second image" are images of the same wall surface of the same structure taken at different times. In this embodiment, the first image refers to an image taken earlier than the time when the second image was taken. The "first deformation data" is deformation data obtained from the first image, and the "second deformation data" is deformation data obtained from the second image. The "deformation data" is information resulting from automatic detection of deformation such as cracks on a concrete wall surface, or information recording the results of human input. The "attribute information" is information that indicates the nature (or characteristics) of the deformation, and includes at least one of the following: the extension, area, position, width, coordinates, and connection relationship of the deformation, and is included in the deformation data.

[0011] Before describing the details of the configuration of the information processing device of this embodiment shown in Figures 1(a) and 1(b), an overview of information processing in this embodiment will be described with reference to Figures 2(a) to 2(f). Below, a concrete bridge is taken as an "inspection target" for infrastructure inspection, and an example is given of calculating the progress of cracks on the surface of the bridge, which indicates the aging of the cracks.

[0012] 2(a) and 2(b) are diagrams showing images 201 and 202 of the same wall surface of a bridge taken at different times as an example of an infrastructure structure. The image 201 in FIG. 2(a) is an image taken several years (e.g., 5 years) before the image 202 in FIG. 2(b) was taken. In this embodiment, the image 201 is described as the first image and the image 202 is described as the second image, but the image 202 may be described as the first image and the image 201 may be described as the second image. Cracks may occur on the wall surface of an infrastructure structure due to aging. Therefore, the images 201 and 202 each show a crack. It is assumed that the crack 204 in the image 202 is the result of the crack 203 in the image 201 progressing. The images 201 and 202 also show the same bridge structure 206.

[0013] FIG. 2(c) shows crack data 211 corresponding to cracks on image 201, and FIG. 2(d) shows crack data 212 corresponding to cracks on image 202. The crack data 211 in FIG. 2(c) includes crack data 213, and the crack data 212 in FIG. 2(d) includes crack data 214 and 215. The crack data is data that an inspector inputs the positions and shapes of cracks and deformations such as exposed rebars based on an image and records them in association with the image. As another method of acquiring crack data, for example, a pre-trained model that has been trained in advance by machine learning can be used to detect cracks from an image to acquire crack data. In this example, the crack data 211 is the result input by an inspector, and the crack data 212 is the result of detecting cracks from an image using a machine learning model.

[0014] In order to calculate the progression of cracks due to changes over time, it is necessary to compare cracks taken at different times and determine the differences in crack length, crack width, etc. Here, crack data based on images taken at different times often contains positional and shape deviations due to differences in shooting conditions, detection conditions, etc. Therefore, in order to properly calculate the progression of a crack, it is necessary to determine the correspondence relationship of the cracks while taking these deviations and differences into account. However, when the shapes of the cracks are significantly different, it is difficult to correctly determine the correspondence between them. For example, when comparing the image 202 in FIG. 2(b) with the crack data 212 in FIG. 2(d), the branch 216 on the crack data 212 may be incorrect. In this way, when the branch 216 on the crack data 212 is incorrect, the correspondence between the cracks cannot be correctly determined between the crack data 211 in FIG. 2(c) and the crack data 212 in FIG. 2(d).

[0015] Therefore, the information processing device of this embodiment performs a correction process of the crack data so that the correspondence relationship of the cracks is consistent. The information processing device of this embodiment performs a crack correction process to correct the branch 216 on the crack data 212 according to the shape of the crack data 213. FIG. 2(e) is a diagram showing the corrected crack data 221. Details of the process of correcting the crack data will be described later, but FIG. 2(e) shows an example in which the corrected crack data 222 and 223 are obtained by the crack correction process. FIG. 2(f) is a diagram showing the crack data 213 included in the crack data 211 before correction and the crack data 222 of the corrected crack data 221 superimposed. The information processing device then calculates the progress of the crack based on a comparison between the crack data. For example, the information processing device calculates the difference in the total length of the crack, the difference in the maximum crack width, or the like as the progress of the crack. In this way, in this embodiment, when calculating the degree of progression of a deformation detected from images taken at different times, one deformation is corrected so that it matches the shape of the other deformation, and the correspondence relationship between the deformations can be correctly determined, thereby appropriately calculating the degree of progression of the deformation.

[0016] <Information processing device> FIG. 1(a) is a diagram showing an example of a hardware configuration of an information processing device 100 according to the present embodiment. As shown in FIG. 1(a), the information processing device 100 has a CPU 101, a ROM 102, a RAM 103, a HDD 104, a display unit 105, an operation unit 106, and a communication unit 107. The CPU 101 is a central processing unit (Central Processing Unit) that performs calculations and logical judgments for various processes and controls each component connected to a system bus 108. The ROM (Read Only Memory) 102 is a program memory that stores programs for control by the CPU 101, including information processing procedures according to the present embodiment described later. The RAM (Random Access Memory) 103 is used as a temporary storage area such as the main memory and work area of ​​the CPU 101. The program memory may be realized by loading a program into the RAM 103 from an external storage device connected to the information processing device 100.

[0017] The HDD 104 is a hard disk drive having a hard disk recording medium for storing electronic data and programs according to this embodiment. An external storage device may be used to fulfill a similar role. Here, the external storage device may be realized, for example, by a medium (recording medium) and an external storage drive for realizing access to the medium. Known examples of such media include flexible disks (FDs), CD-ROMs, DVDs, USB memories, MOs, and flash memories. The external storage device may also be a server device connected via a network.

[0018] The display unit 105 includes, for example, a CRT display or a liquid crystal display, and is a device that outputs images, text, and the like on a display screen. The display unit 105 may be an external device connected to the information processing device 100 by wire or wirelessly. The operation unit 106 has a keyboard and a mouse, and accepts various operations by a user. The communication unit 107 performs two-way communication by wire or wirelessly with other information processing devices, communication devices, external storage devices, and the like using known communication technology.

[0019] <Function block diagram> 1(b) is an example of a block diagram showing the functional configuration of the information processing device 100. Each of these functional units is realized by the CPU 101 loading a program stored in the ROM 102 into the RAM 103 and executing processing in accordance with each flowchart described below. The execution results of each processing are stored in the RAM 103. Also, for example, when configuring hardware as an alternative to software processing using the CPU 101, it is sufficient to configure a calculation unit or circuit corresponding to the processing of each functional unit described here.

[0020] The management unit 122 manages images captured at multiple time periods, deformation data corresponding to each image, and structural information related to the structure, which are stored in the storage unit 121. The selection unit 123 selects, from the first deformation data, deformation data to be used as a reference in the correction process described later as reference deformation data, and selects, from the second deformation data, deformation data to be the subject of the correction process as correction target deformation data. In the following description, for the sake of simplicity, the reference deformation data is referred to as the "reference deformation" and the correction target deformation data is referred to as the "correction target deformation". That is, the selection unit 123 performs a deformation selection process such that at least one deformation among the first deformations associated with the first image is selected as the reference deformation, and at least one deformation among the second deformations associated with the second image is selected as the correction target deformation. Details of the deformation selection process in the selection unit 123 will be described later.

[0021] The correction candidate creation unit 124 and the consistency calculation unit 125 perform processing to obtain information that indicates the consistency relationship between the reference deformation and the deformation to be corrected. That is, the correction candidate creation unit 124 performs processing to modify at least a part of the attribute information of the deformation to be corrected to generate multiple correction candidates. Then, the consistency calculation unit 125 performs consistency acquisition processing (consistency calculation processing) to obtain consistency as information that indicates the consistency relationship between the reference deformation and the correction candidates. The correction candidate creation processing in the correction candidate creation unit 124 and the consistency calculation processing in the consistency calculation unit 125 will be described in detail later.

[0022] The correction candidate selection unit 126 and the correction unit 127 perform a deformation correction process to correct the deformation to be corrected based on the information (consistency) indicating the consistency relationship to create a corrected deformation. That is, the correction candidate selection unit 126 performs a process to select one correction candidate from multiple correction candidates based on the calculation result of the consistency. Then, the correction unit 127 corrects at least a part of the attribute information of the deformation to be corrected related to the creation of the correction candidate selected by the correction candidate selection unit 126 to generate corrected deformation data. In the following explanation, the corrected deformation data is referred to as a "corrected deformation". The correction candidate selection process in the correction candidate selection unit 126 and the process of generating the corrected deformation (correction process) in the correction unit 127 will be described in detail later.

[0023] The progress calculation unit 128 performs a progress acquisition process to acquire a progress indicating a change in the deformation from the reference deformation based on a comparison between the reference deformation and the corrected deformation. For example, the progress calculation unit 128 performs a progress calculation process to calculate the difference between at least a part of the attribute information of the reference deformation and the corresponding attribute information of the corrected deformation as the progress. The progress calculation process in the progress calculation unit 128 will be described in detail later.

[0024] <Explanation of the relationship between images and deformation data and structural information> Before describing the configuration of the information processing device of this embodiment, the relationship between images and deformation data, and structural information will be described. In image inspection, it is preferable to manage images of the wall surface of a structure in association with the drawing. FIG. 3(a) is a diagram showing an example in which an image 311 of the wall surface of a bridge, which is an example of an infrastructure structure, is pasted onto a drawing 300 and displayed, for example, on the screen of the display unit 105. Note that in the following explanation, the image and deformation data on the drawing are also examples displayed on the display screen. Drawing 300 has drawing coordinates 301 with point 302 as the origin. Furthermore, the position of image 311 on drawing 300 is defined by the vertex coordinates of the upper left corner of the image. In the example of FIG. 3(a), the vertex coordinates of image 311 are the position (X 312 ,Y 312 ). The image 311 is stored in the storage unit 121 together with the coordinate information. The image used in the image inspection of infrastructure structures as in this embodiment is large in size because it is taken at a high resolution (e.g., 1 mm per pixel) so that minute cracks and the like can be confirmed. For example, the image 311 in FIG. 3(a) is an image of a bridge deck of 20 m×10 m. When the image resolution is 1.0 mm per pixel (1.0 mm / pixel), the image size of the image 311 is 20,000 pixels×10,000 pixels. The image 311 taken at a high resolution shows many (e.g., 1000 or more) cracks, exposed reinforcing bars, and other defects, but since it is difficult to express all the defects on paper, only a portion of the defects are displayed on the paper. In the following description, even in a diagram showing a wide range of images or defect data, only a portion of the defects are displayed.

[0025] The deformation data is information that records the results of automatic detection of deformations such as cracks on the concrete wall surface or the results of human input, as described above. Also, as described above, in this embodiment, the deformation data is managed in association with the drawings. Drawing 300 in Fig. 3(b) shows a state in which deformation data 321 corresponding to image 311 is pasted in the same position as image 311 in Fig. 3(a). It is assumed that the deformation data 321 contains a large amount of deformation data (e.g., 1000 or more), including deformations that are not displayed on the paper. The position on the drawing of each deformation data in the deformation data 321 is defined by the pixel coordinates that make up each deformation data.

[0026] FIG. 3(c) is a diagram showing an example of a deformation data table 331 showing the data structure of deformation data. In the deformation data table 331, ID is identification information of deformation data, and numerical values ​​such as coordinate sequence, numerical value, maximum width, total length / contour length, and area are attribute information in the deformation data. The coordinate sequence is a coordinate that indicates the position of each of a plurality of pixels corresponding to the deformation, and the numerical value sequence is information that indicates the width (deformation width) at each coordinate indicated by the coordinate sequence. For example, the deformation data corresponding to the ID of crack C001 is represented by continuous pixels consisting of n points from (Xc001_1, Yc001_1) to (Xc001_n, Yc001_n). In this way, in this embodiment, the deformation data is represented corresponding to the pixels. Furthermore, among the attribute information of the deformation data in the deformation data table 331, the maximum width is the maximum value of the deformation width, the total length / contour length is the total extension or contour length of the deformation length, and the area is the area value of the deformation. In addition, at least a part of the attribute information of the deformation data may be expressed by vector data such as a polyline or a curve consisting of multiple points. When the attribute information of the deformation data is expressed by vector data, the amount of data is reduced and the expression is simpler. In addition, the deformation data table 331 also describes deformation data other than cracks, and in the example of FIG. 3(c), deformation data of exposed rebars indicated by ID T001 is also described. When exposed rebars are expressed by polylines, the deformation has an area surrounded by the polylines. Note that the attribute information of the deformation data is not limited to the attribute information shown in the deformation data table 331 of FIG. 3(c), and other attribute information may be described.

[0027] The structural information is information about the structure to be inspected, and includes specifications such as the type and basic structure of the structure, various dimensions of the structure, member information, and completion year. Furthermore, the structural information may include information related to maintenance such as the repair year, repair location, and repair method as a repair record. In this embodiment, structural information related to a specific position of the structure, such as member information and repair information, is stored in the storage unit 121 together with position information on the drawing. That is, the position on the drawing of each member and the position on the drawing of the repair location are stored as part of the structural information. Therefore, it is possible to obtain a correspondence relationship between the structural information and the image and the deformation data via the drawing. The structural information is stored in the storage unit 121 together with the image data and the deformation data, and can be acquired by the management unit 122. Note that the information included in the structural information is not limited to the above information, and other information may be stored. In addition, the structural information may hold information limited to each type according to the type of the structure.

[0028] <Flowchart> 4 is a flowchart showing a main flow of information processing related to the information processing device 100 in this embodiment. In the following flowcharts, the letter S added to the beginning of the reference symbol indicates a processing process (step). As described above, in this embodiment, two images, a first image and a second image, taken at different times, are used to calculate the progress of the deformation occurring on the wall surface of the structure. Both the first image and the second image are images taken of the same wall surface of the same structure, and show the deformation occurring on the wall surface. In this embodiment, the first image is taken earlier (for example, five years ago) than the second image. As described above, data obtained by detecting the deformation from the first image is set as the first deformation data, and data obtained by detecting the deformation from the second image is set as the second deformation data. The information processing device 100 of this embodiment accepts an input for starting processing from the operation unit 106 and starts the processing of the flowchart of FIG. 4. Then, when the calculation processing of the progress degree is completed as described later, the information processing device 100 displays display data indicating the progress degree calculation result on the display unit 105 and ends the processing of the flowchart of FIG. 4. Hereinafter, the information processing in the information processing device 100 of this embodiment will be described according to the flowchart of FIG. 4.

[0029] <Selection of Deformation> In S401, the selection unit 123 selects, from the first deformation data, deformation data to be used as a reference for correcting deformation as a reference deformation, and also selects, from the second deformation data, deformation data to be the subject of correction processing as a deformation to be corrected. In this embodiment, as an example for explaining a method for selecting a reference deformation and a deformation to be corrected, a method for selecting one deformation data from the first deformation data as a reference deformation and selecting a deformation to be corrected from the second deformation data based on the coordinate information of the reference deformation will be explained.

[0030] FIG. 5(a) shows a first image 501, and FIG. 5(b) shows first deformation data 502. The first image 501 is an image of a bridge wall surface, and the first deformation data 502 is crack data corresponding to the first image 501. FIG. 5(c) shows a second image 511, and FIG. 5(d) shows second deformation data 512. The second image 511 is an image of the same bridge wall surface taken at a different time (e.g., 5 years later) from the first image 501, and the second deformation data 512 is crack data corresponding to the second image 511. Each image and each deformation data is pasted at the same position on the drawing 500.

[0031] The selection unit 123 first selects one deformation data from the first deformation data as a reference deformation. One method of selecting the reference deformation is to select based on coordinate information on the drawing. For example, when selecting the deformation data located at the top left corner of the drawing as the reference deformation, the selection unit 123 selects the deformation data 503 from the first deformation data 502 as the reference deformation 503. As another method of selecting the reference deformation, the selection unit 123 may select based on attribute information. For example, when the deformation is a crack, the selection unit 123 may select the deformation data with the maximum total length of the crack, or may select the deformation data with the maximum maximum width. As another method of selecting the reference deformation, the selection unit 123 may select the deformation data located closest to a structurally important part of the structure as the reference deformation, or may select the reference deformation based on a user's designated operation.

[0032] Next, the selection unit 123 selects the deformation to be corrected from the second deformation data. One method for selecting the deformation to be corrected is to select deformation data located near the reference deformation as the deformation to be corrected. For example, when selecting the deformation to be corrected corresponding to the reference deformation 503 in FIG. 5(b), the selection unit 123 first obtains a rectangular range 506 surrounding the reference deformation 503. Next, the selection unit 123 superimposes the rectangular range 506 on the second deformation data 512 to obtain a rectangular range 513. Then, the selection unit 123 selects the deformation data of the second deformation data 512, at least a part of which is located within the rectangular range 513, as the deformation to be corrected corresponding to the reference deformation 503.

[0033] At this time, a part of the second deformation data may be divided near the boundary of the rectangular range 513, that is, a part of the deformation data may go outside the rectangular range 513. For example, the deformation data 514 and 515 in FIG. 5(d) are divided near the boundary of the rectangular range 513 (a part of the deformation data goes outside the rectangular range 513). If a part of such divided deformation data is selected as the deformation to be corrected, that is, if only the part inside the rectangular range 513 is selected as the deformation to be corrected, the correction candidate creation process of S402 described later becomes complicated. Therefore, the selection unit 123 selects the deformation data 514, 515, at least a part of which is included within the rectangular range 513, as the deformation to be corrected as it is. Alternatively, the selection unit 123 may expand the rectangular range 513 in the X-axis and Y-axis directions so that the deformation data is not divided, that is, so that a part of the deformation data does not go outside the expanded rectangular range.

[0034] In the above description, a method of selecting one deformation data as the reference deformation was given as a method of selecting the reference deformation, but the selection unit 123 may select multiple deformation data as the reference deformation together. As a method of selecting multiple deformation data as the reference deformation together, there is a method of selecting multiple deformation data based on attribute information of the deformation data. For example, the selection unit 123 selects deformation data 503 and 504 having a close distance between the area centroids of deformation from the first deformation data 502 in FIG. 5(b) together as one reference deformation. The same method as described above can be used to obtain a correction target deformation corresponding to a reference deformation including such multiple deformation data. That is, the selection unit 123 obtains a rectangular range that surrounds all the deformation data summarized as the reference deformation, and performs a process of superimposing the rectangular range on the second deformation data. Then, the selection unit 123 selects the deformation data included in the rectangular range on the second deformation data as a correction target deformation corresponding to the summarized reference deformation.

[0035] <Creating correction candidates> In S402, the correction candidate creating unit 124 creates a correction candidate by modifying the attribute information of the deformation to be corrected. A method of creating a correction candidate in this embodiment will be described with reference to Figs. 6(a) to 6(i). FIG. 6(a) shows a reference deformation 601, and FIG. 6(b) shows two deformations to be corrected 611 and 612 corresponding to the reference deformation 601. One method of creating a correction candidate from a deformation to be corrected is to connect parts of the deformations to be corrected, for example, by connecting end points, and the deformation data after the connection process is set as a correction candidate. As a specific example, the correction candidate creation unit 124 connects end points of the deformations to be corrected 611 and 612 that are close to each other to create a correction candidate 621 as shown in FIG. 6(c). When there are three or more deformations to be corrected, the correction candidate creation unit 124 creates multiple correction candidates by selecting and connecting any two deformations to be corrected. For example, when there are three deformations to be corrected 631, 632, and 633 shown in FIG. 6(d), the correction candidate 641 shown in FIG. 6(e) is obtained by selecting and connecting the deformations to be corrected 631 and 632. In this manner, the correction candidate creating unit 124 can create a plurality of correction candidates with different branch shapes by repeating the operation of selecting and linking two different deformations to be corrected.

[0036] Another method for creating correction candidates is to perform a process of cutting a part of the deformation to be corrected, and to use the deformation data after the cutting process as the correction candidate. Still another method for creating correction candidates is to combine a cutting process and a connection process of the deformation to be corrected, thereby creating a correction candidate having a branching relationship different from the branching of the deformation before cutting (deformation to be corrected).

[0037] Here, an example of creating a correction candidate with a different branch from the two deformations 651 and 652 to be corrected shown in FIG. 6(f), that is, a correction candidate having a different connection relationship from the connection relationship of the deformations to be corrected, will be described. The correction candidate creation unit 124 first cuts the deformations 651 and 652 to be corrected at arbitrary coordinate positions. For example, a method of determining the cutting position is a method of obtaining the shortest distance between the two deformations to be corrected and setting the coordinates of both ends of the shortest distance as the cutting position. Any method can be used to obtain the shortest distance between the deformations, for example, a method of calculating the Euclidean distance between pixels between each deformation and obtaining the minimum value as the shortest distance. FIG. 6(g) is a diagram showing the shortest distance d661 between the deformations 651 and 652 to be corrected. The correction candidate creation unit 124 cuts the deformations 651 and 652 to be corrected, setting the coordinates 671 and 672 of both ends of this shortest distance d661 as the cutting position. That is, the correction candidate creation unit 124 cuts the deformation 651 to be corrected at the coordinate 671, and cuts the deformation 652 to be corrected at the coordinate 671. Fig. 6(h) is a diagram showing deformation data 681 and 682 resulting from cutting the deformation 651 to be corrected, and deformation data 683 and 684 resulting from cutting the deformation 652 to be corrected.

[0038] Next, the correction candidate creation unit 124 selects two pieces of deformation data from among the post-cut deformation data 681, 682, 683, and 684 shown in FIG. 6(h), and connects the end points with close coordinates. At this time, the correction candidate creation unit 124 connects the deformation data so that the branch shape is different from the shape before cutting (FIG. 6(f)). FIG. 6(i) is a diagram showing a correction candidate 685 obtained by selecting the post-cut deformation data 682 and the deformation data 683 and connecting the end point coordinates. In this way, the correction candidate creation unit 124 can create a correction candidate having a branch different from the branch of the deformation to be corrected by cutting and connecting a part of the deformation to be corrected. Note that FIG. 6(i) illustrates one correction candidate, but the correction candidate creation unit 124 can also create multiple correction candidates, such as a correction candidate in which the post-cut deformation data 683 and 681 in FIG. 6(h) are connected, and a correction candidate in which the post-cut deformation data 684 and 682 are connected.

[0039] As another method for determining the cutting position of the deformation to be corrected, the cutting position may be specified by the user. In this case, the information processing device 100 displays the deformation to be corrected and the wall surface image side by side on the display unit 105 and prompts the user to specify the cutting coordinates. Then, the correction candidate creation unit 124 cuts the deformation to be corrected based on the coordinates specified by the user, and creates a correction candidate by connecting the deformations after the cut.

[0040] As another method for correcting the attribute information of the deformation to be corrected and creating a correction candidate, a method of moving the coordinate position of the deformation to be corrected may be used. For example, the correction candidate creation unit 124 can create a correction candidate that takes into account the positional deviation between the reference deformation and the deformation to be corrected by moving the position of the deformation to be corrected in at least one of the X-axis direction and the Y-axis direction. As a method for moving the position of the deformation to be corrected in the X-axis direction, for example, there is a method of calculating the total length of the deformation to be corrected in the X-axis direction and moving it in the positive or negative X-axis direction by a distance of a part of the total length (e.g., 10%). A similar method can be used to move the position of the deformation to be corrected in the Y-axis direction. In addition, the correction candidate creation unit 124 may create a correction candidate by simultaneously changing the shape information and position information of the deformation to be corrected. That is, the correction candidate creation unit 124 connects the end points of the deformations to be corrected, and then moves the coordinate positions in the X-axis direction and the Y-axis direction in the same manner as described above. This makes it possible to create a variety of correction candidates.

[0041] In addition, the deformations to be corrected often include deformations that are highly consistent with the reference deformation from the beginning and do not require correction processing. If correction processing is performed on a deformation to be corrected that is highly consistent with the reference deformation, an appropriate degree of progress may not be obtained. Therefore, the correction candidate creation unit 124 sets the deformation to be corrected itself as one of the correction candidates. Then, in the processing from S403 onwards described below, the consistency with other correction candidates is compared, and if it is determined that the correction candidate for the deformation to be corrected itself has the highest consistency, the correction processing can be skipped. In this way, it is a suitable method to set the deformation to be corrected itself as one of the correction candidates.

[0042] <Calculation of consistency> In S403, the consistency calculation unit 125 performs a process of calculating a consistency degree that indicates a consistency relationship between the reference deformation and the correction candidate. As a method of calculating the consistency degree, there is a method of expanding the shape of each deformation and calculating the degree of overlap between the expanded deformations as the consistency degree. An example of the process of calculating the consistency degree will be described with reference to Fig. 7(a) to Fig. 7(g).

[0043] Fig. 7(a) is a diagram showing a reference deformation 701 selected from the first deformation data. Fig. 7(b) is a diagram showing deformations 711 and 712 to be corrected that correspond to the reference deformation 701 among the second deformation data. Figs. 7(c) to 7(f) are diagrams showing examples of correction candidates 721 to 724 created by the correction candidate creation unit 124 from the deformations 711 and 712 to be corrected. Note that the correction candidate 723 in Fig. 7(e) is a correction candidate for the deformation 711 to be corrected itself, and the correction candidate 724 in Fig. 7(f) is a correction candidate for the deformation 712 to be corrected itself.

[0044] When calculating the consistency, the consistency calculation unit 125 selects one correction candidate from the correction candidates 721 to 724. First, the consistency calculation unit 125 selects the correction candidate 721. Next, the consistency calculation unit 125 performs a process of superimposing the shape of the correction candidate 721 and the reference deformation 701. Here, in order to take into account the position shift and shape shift included between the first deformation data and the second deformation data, the consistency calculation unit 125 expands the shape of the deformation data before superimposing. FIG. 7(g) is a diagram showing a state in which the reference deformation 702 obtained by expanding the reference deformation 710 and the correction candidate 725 obtained by expanding the correction candidate 721, for example, are superimposed. Then, the consistency calculation unit 125 calculates the degree of overlap between the expanded reference deformation 702 and the expanded correction candidate 725 as the consistency C. The equation for calculating the consistency C is shown in Equation (1).

[0045]

number

[0046] Here, parameter P1 in formula (1) indicates the area of ​​the reference deformation 702 after expansion, and parameter P2 indicates the area of ​​the correction candidate 725 after expansion. According to formula (1), if the deformations match, the consistency degree C is 1, and if the deformations do not overlap at all, the consistency degree C is 0. In other words, when the degree of overlap between the deformations is calculated as the consistency degree, the possible range of the consistency degree C is 0 to 1.

[0047] After obtaining one consistency, the consistency calculation unit 125 selects a different correction candidate while keeping the reference deformation selected, and similarly calculates the consistency using formula (1). The consistency calculation unit 125 repeats such consistency calculation processing for each correction candidate, thereby calculating the consistency between the reference deformation and all correction candidates. As another method for calculating the consistency, the consistency calculation unit 125 may calculate the consistency using a feature amount calculated from the deformation data (attribute information of the correction candidate). As an example of the feature amount, the formula for calculating the consistency C using the area center coordinate of the deformation is shown in Equation (2) and Equation (3).

[0048]

number

[0049] Here, the parameter P in equations (2) and (3) 1x ,P 1y is the area center coordinate of the reference deformation (P 1x ,P 1y ) and the parameter P 2x ,P 2y is the area center coordinate of the selected correction candidate (P 2x ,P 2y ) Parameter C c is a fixed value when the area centroid coordinates of the deformations coincide, and any constant greater than 1 is used. From formula (2), the greater the distance between the area centroids of the deformations, the smaller the consistency C becomes. On the other hand, when the area centroids of the deformations coincide, the constant C with the highest consistency is used. c It becomes.

[0050] <Selection of correction candidates> In S404, the correction candidate selection unit 126 performs a process of selecting a correction candidate based on the calculation result of the consistency. An example of selecting a correction candidate using the consistency will be described with reference to FIG. 8. Here, the consistency used in this embodiment is calculated as the degree of overlap (overlap) between the deformations as described above. Therefore, the possible values ​​of the consistency are in the range of 0 to 1, and the closer to 1 the value, the higher the consistency between the deformations.

[0051] Fig. 8(a) shows a reference deformation 801, and Fig. 8(b) shows a deformation to be corrected 802. Fig. 8(c) to Fig. 8(f) show correction candidates 803, 804, 805, and 806 created from the deformation to be corrected 802. Note that CR101, CR102, CR103, and CR104 written next to each deformation of the correction candidates 803, 804, 805, and 806 are deformation IDs for identifying each deformation of the correction candidates. Furthermore, the correction candidate 806 in Fig. 8(f) is an example in which a part of the deformation of the deformation to be corrected 802 shown in Fig. 8(b) itself is set as one of the correction candidates.

[0052] FIG. 8(g) is a diagram showing a consistency list 811 obtained by calculating the consistency between each deformation of the correction candidates 803, 804, 805, and 806 and the reference deformation 801. One method for selecting a correction candidate based on the consistency is to select a correction candidate with the maximum consistency from among multiple consistency. For example, in the case of the consistency list 811, the correction candidate selection unit 126 selects the correction candidate 803 corresponding to the deformation with the maximum consistency (deformation ID: CR101). When there is only one maximum consistency value, as in the consistency list 811, a correction candidate can be uniquely selected using the consistency.

[0053] FIG. 8(h) is a diagram showing an example of another consistency list 812 in which the deformation IDs are CR201 to CR204, and the consistency of the deformation IDs CR203 and CR207 are both 0.75, which are the maximum consistency. When there are multiple deformations with the maximum consistency as in FIG. 8(h), it is not possible to uniquely determine a correction candidate. In this case, it is preferable that the information processing device 100 generates a display image showing multiple correction candidates with the maximum consistency and the reference deformation, displays the display unit 105, and allows the user to determine the correction candidate. Then, when the user determines the desired correction candidate through the display image, the correction candidate selection unit 126 selects the correction candidate.

[0054] In addition, in the consistency calculation result, if the maximum consistency is close to 0 (e.g., 0.1), the correction candidate selected based on the consistency is likely to be inappropriate. Therefore, when the maximum consistency is less than a predetermined threshold (consistency reference value), the information processing device 100 generates a display image showing the maximum consistency, the reference deformation, and the correction candidate, and displays it on the display unit 105. An example of the consistency reference value is 0.2. It is preferable to have the user confirm and decide the correction candidate from the display image. If the user confirms the display image and does not specify a correction candidate (or cancels the selection of the correction candidate), the correction candidate selection unit 126 ends the process of S404 without selecting a correction candidate, and proceeds to S407.

[0055] <Correction of deformation> In S405, the correction unit 127 performs a process of correcting the deformation to be corrected based on the selection result of the correction candidate. A method of correcting the deformation to be corrected in this embodiment will be described with reference to FIG. Fig. 9(a) is a diagram showing deformations 901, 902, and 903 to be corrected, and deformation IDs are written next to each of the deformations 901, 902, and 903 to be corrected. Fig. 9(b) is a diagram showing an example of a correction candidate 911 selected by correcting the deformation to be corrected by the correction candidate selection unit 126. The correction candidate 911 is a correction candidate created by linking the two deformations 901 and 902 to be corrected.

[0056] When correcting the deformation to be corrected, the correction unit 127 first changes the shape of the deformation to be corrected to match the shape of the correction candidate. In this example, the correction unit 127 performs a correction process to connect the shapes of the deformations to be corrected 901 and 902 to match the shape of the correction candidate 911. FIG. 9(c) is a diagram showing a corrected deformation 904 obtained by changing the shapes of the deformations to be corrected 901 and 902 by a correction process that connects the deformations to be corrected 901 and 902. The corrected deformation 904 is generated by a correction process that adds a partial deformation 905 near the center as a new deformation and connects the deformations to be corrected 901 and 902.

[0057] Next, the correction unit 127 obtains attribute information for each pixel of the added partial deformation. One method for obtaining attribute information for partial deformation is to interpolate using attribute information corresponding to neighboring pixels. Specifically, it is assumed that attribute information changes linearly or nonlinearly from one end point position of the partial deformation to the other end point position, and the correction unit 127 uses the attribute information of both end points to approximately interpolate the attribute information for each pixel of the partial deformation. This makes it possible to obtain attribute information for each pixel of the partial deformation.

[0058] As an example, a method for calculating the deformation width of each pixel of the partial deformation 905 will be described. FIG. 9(d) is an enlarged view of the vicinity of the partial deformation 905. Also, assume that the deformation widths of the coordinates 921, 922 at both ends of the partial deformation 905 are 1.0 mm and 2.0 mm. At this time, the correction unit 127 approximates the deformation width of each pixel of the partial deformation 905 to a value between 1.0 mm and 2.0 mm according to the pixel position. Attribute information other than the deformation width can be calculated in a similar manner. Note that single attribute information for each deformation, such as the maximum width and area, can be recalculated and updated for each deformation.

[0059] 9(e) and 9(f) are diagrams showing examples of the deformation data table before and after the update associated with the correction process. The deformation data table 931 in FIG. 9(e) stores three deformation data related to the deformation before the correction, in this example, the deformation data of the deformations 901, 902, and 903 to be corrected in FIG. 9(a). The deformation data table 932 in FIG. 9(f) stores the deformation data after the correction. Compared to the deformation data table 931, the deformation IDs "CR111" and "CR112" are deleted from the deformation data table 932, and a new deformation ID "CR114" is added. In this way, the correction unit 127 performs the update process of the deformation data table, that is, the addition, deletion, correction, etc., in conjunction with the correction process.

[0060] In this embodiment, an example in which the deformation data after correction is newly added in accordance with the correction process has been described, but the correction unit 127 may update the deformation data to be corrected. That is, instead of adding the deformation data of the deformation ID "CR114", the correction unit 127 may update the deformation data of either the deformation ID "CR111" or "CR112" and delete the other deformation data. In addition, the correction unit 127 may store the correction process implementation history, for example, the correction process implementation date, the deformation ID to be corrected, the correction process location coordinates and the numerical value for each coordinate, the maximum width before the correction process, the total length / contour length before the correction process, the area before the correction process, etc., in the deformation data table. By storing the correction process implementation history in the deformation data table, it becomes possible to search for the corrected data and display the attribute information before and after the correction process on the display unit 105 regarding the corrected deformation. In addition, if the correction candidate selection unit 126 selects a correction candidate for the deformation to be corrected itself, there is no need to change the shape or attribute information of the deformation to be corrected in S405.

[0061] <Calculating progress> In S406, the progress calculation unit 128 uses the standard deformation and the corrected deformation to calculate the progress relative to the standard deformation. In this embodiment, the progress calculation unit 128 performs a process of calculating the difference in attribute information of the deformation data as the progress. For example, when the deformation type is a crack, the progress calculation unit 128 calculates the difference in the maximum width and total length. Here, the maximum width and total length of the crack are stored in the storage unit 121 as attribute information. Therefore, the progress calculation unit 128 can calculate the progress relative to the standard deformation by acquiring the maximum width and total length of each of the standard deformation and the corrected deformation, and calculating the difference in the maximum width and the difference in the total length between these deformations. When the deformation type is other than a crack, the progress can be calculated in a similar manner. For example, when the deformation type is exposed rebar, the progress calculation unit 128 can use the difference in area as the progress. The area of ​​the deformation data is stored in the storage unit 121 as attribute information. Therefore, the progress calculation unit 128 can calculate the progress by acquiring the area of ​​the reference deformation and the area of ​​the corrected deformation associated with the reference deformation, and finding the area difference between these deformations.

[0062] <Determining Completion of Progress Calculation Process> In S407, the progress calculation unit 128 performs a process to determine whether the progress calculation process is complete. One method for determining the completion of the progress calculation process is to check whether all deformation data in the first deformation data have been selected as the reference deformation. If the check shows that unselected deformation data exists, the process of the information processing device 100 proceeds to S409, where the next reference deformation and deformation to be corrected are selected, and the process from the creation of correction candidates in S402 onwards is repeated. On the other hand, if the check shows that all deformation data in the first deformation data have been selected as the reference deformation, the process of the information processing device 100 proceeds to S408.

[0063] Here, there may be cases where it is desired to limit the deformation data for which the degree of progress is calculated from among the first deformation data. In that case, it is sufficient to limit the deformation data for which the degree of progress is calculated from among the first deformation data using information such as attribute information of the deformation data and the area range on the image. In this case, in the determination process of S407, the progress calculation unit 128 checks whether or not all of the limited deformation data have been selected as the reference deformation. This allows the progress calculation process to be performed only on the limited deformation data. Note that the user may be prompted to specify the completion determination of the progress calculation process each time. In this case, the information processing device 100 generates display data that prompts the user to determine the completion of the progress calculation process and displays it on the display unit 105. Then, the progress calculation unit 128 determines whether or not to complete the progress calculation process based on the user's specification.

[0064] <Progress display> In S408, the information processing device 100 creates display data for displaying the progress degree calculation result, and performs processing for displaying the result on the display unit 105. FIG. 10 is a diagram showing an example of a progress degree display screen that displays the progress degree calculation result. As shown in FIG. 10, the information processing device 100 creates display data that displays a reference deformation 1001 and a corrected deformation 1011 together with a progress degree 1021 in a window 1000, and displays the data on the display unit 105. Note that the deformation type of the deformation data here is assumed to be crack. Therefore, the progress degree 1021 displays the change amount of the maximum width and total length of the crack as an example of the progress degree. Among the corrected deformation 1011, a partial deformation 1012 displayed by a dotted line represents a portion corrected by the correction process. By displaying the corrected deformation and the reference deformation side by side as in FIG. 10, the user can easily confirm whether the correction process for the deformation to be corrected is appropriate. In addition, it is desirable to display a structure image on the progress degree display screen together with the progress degree calculation result. For example, the progress degree display screen displays an image in which the first image and the reference deformation are superimposed, and an image in which the second image and the corrected deformation are superimposed, together with the progress degree. This makes it easier for the user to check the calculation results of the degree of progression of the deformation while taking into account the texture of the wall surface.

[0065] In addition, it is preferable that the progress display screen can switch between displaying the corrected deformation and other correction candidates not selected by the correction candidate selection unit 126. In this case, the information processing device 100 displays, for example, other correction candidates 1031, 1032, 1033 in the lower part of the window 1000 in Fig. 10 in order of the degree of consistency. Then, in response to the user's selection of a correction candidate, the information processing device 100 displays the selected correction candidate in the display area of ​​the corrected deformation 1011 on the window 1000. This makes it easier for the user to compare the reference deformation with the selected correction candidate.

[0066] In this embodiment, the progress of the other correction candidates 1031, 1032, and 1033 is not calculated. Therefore, it is desirable to be able to additionally calculate the progress of the selected correction candidate. In this case, the progress calculation unit 128 executes a progress calculation process using the reference deformation and the selected correction candidate in response to the user's selection of the correction candidate. Then, the information processing device 100 displays the progress together with the correction candidate. Alternatively, the information processing device 100 may display a progress calculation button on the display screen and execute the progress calculation process when the user issues an instruction such as pressing the progress calculation button. In this way, it is more desirable to additionally calculate the progress of the selected correction candidate with the reference deformation.

[0067] Another example of a display screen for the progress calculation results is a method of displaying the reference deformation and the corrected deformation superimposed on each other. By displaying the reference deformation and the corrected deformation superimposed on each other, the user can easily directly compare the shapes of the deformations. When a structure image is further superimposed on the display screen in which the reference deformation and the corrected deformation are superimposed, it is preferable to be able to select either the first image or the second image.

[0068] In the example of the progress display screen in Fig. 10, the progress calculation result for one reference deformation is displayed, but the progress for multiple reference deformations may be displayed together. That is, in this case, the information processing device 100 displays multiple reference deformations selected by the user and multiple corrected deformations for the selected reference deformations on the progress display screen. Also, when displaying the progress of multiple reference deformations, the progress calculation unit 128 tallies the progress of each reference deformation, and the information processing device 100 displays the tallied result on the display screen. That is, for example, the progress calculation unit 128 tallies the progress of the total length of all reference deformations, and the information processing device 100 displays the tallied total length on the progress display screen.

[0069] <Modification 1 of the First Embodiment> In the first embodiment, a method for calculating the degree of progress using a reference deformation selected from the first deformation data and a corrected deformation obtained by correcting a deformation to be corrected selected from the second deformation data has been described, but an example in which the reference deformation is selected from the second deformation data is also conceivable. That is, the information processing device selects a reference deformation from the second deformation data corresponding to the newer image of two images acquired at different times, while correcting the deformation selected from the first deformation data, and calculates the degree of progress of the deformation using the reference deformation and the corrected deformation. In this way, when the position or shape accuracy of the deformation in the past inspection results is low, it is possible to check how the deformation has progressed while correcting the past inspection results to match the latest inspection results.

[0070] <Modification 2 of the First Embodiment> In the first embodiment, a method for creating correction candidates based on the deformation to be corrected selected from the second deformation data has been described, but an example of creating correction candidates from a second image is also conceivable. Specifically, a plurality of trained models trained in advance by machine learning are prepared, and the information processing device detects deformations from the second image for each model to obtain a plurality of detection results. Then, in each of the plurality of detection results, the information processing device sets deformation data located near the reference deformation as a correction candidate. This makes it possible to create a variety of correction candidates. Note that in the process of detecting deformations for each model from the second image, by obtaining a plurality of detection results using a plurality of detection parameters, it becomes possible to create a larger number of correction candidates.

[0071] According to the first embodiment described above, even if the shapes of deformations such as cracks detected from images taken at different times are different, the correspondence relationship of the deformations can be correctly determined by correcting the deformations, and the progression of the deformations can be appropriately calculated.

[0072] <Second embodiment> In the first embodiment, an example was described in which deformations of the same type are compared to each other, the deformations are corrected, and the degree of progress is calculated. Here, for example, when deterioration of a wall surface progresses, multiple types of deformations may occur at the same position. For example, cracks and efflorescence may occur at the same location on the wall surface. At this time, a part of the shape of the crack may be covered by efflorescence and may not be visible. When a part of the deformation cannot be visually recognized, it becomes difficult to obtain the correspondence relationship between the deformations, and an appropriate degree of progress cannot be calculated. Therefore, the information processing device of the second embodiment performs a correction process to interpolate a part where a part of the deformation is hidden by another deformation and cannot be visually recognized. This makes it possible to calculate the degree of progress taking into account the part that cannot be visually recognized. Hereinafter, the second embodiment will be described, focusing on the difference from the first embodiment.

[0073] The hardware configuration of the information processing device 100 according to the second embodiment is similar to that of the first embodiment shown in FIG. 1(a), and therefore illustration and description are omitted. FIG. 11 is a diagram showing an example of a functional block diagram of the information processing device 100 according to the second embodiment. The functional configuration of FIG. 11 differs from the functional configuration shown in FIG. 1(b) of the first embodiment in that a nearby deformation acquisition unit 1101 is added. The nearby deformation acquisition unit 1101 is a functional unit of the CPU 101, and acquires, as nearby deformations, deformations that exist in the vicinity of the reference deformation and have a type different from the deformation type of the reference deformation among the second deformation data. In the second embodiment, the correction candidate creation unit 124 creates correction candidates by performing correction such as interpolating shape data of the deformation to be corrected using the nearby deformations.

[0074] Fig. 12 is a flowchart showing a main flow of information processing executed by the information processing device 100 according to the second embodiment. In the flowchart of Fig. 12, in steps having the same numbers as those in the flowchart of Fig. 4 described in the first embodiment, the same processing as in the first embodiment is executed. In the case of the second embodiment, after the reference deformation and the deformation to be corrected are selected in S401, the information processing device 100 proceeds to the processing of S1201.

[0075] In S1201, the neighboring deformation acquisition unit 1101 acquires, from among the second deformation data, deformation data that is located in the vicinity of the reference deformation and has a deformation type different from the reference deformation as a neighboring deformation. Details of the processing in the neighboring deformation acquisition unit 1101 will be described later. In the next S1202, the correction candidate creation unit 124 creates a correction candidate by performing a correction such as interpolating the shape of the deformation to be corrected using the neighboring deformations. Details of the processing in the correction candidate creation unit 124 in the case of the second embodiment will be described later. Thereafter, the information processing device 100 proceeds to the consistency calculation processing in S403. The processing from S403 onwards is the same as the processing described in the first embodiment, and therefore the description thereof will be omitted. Note that in the case of the second embodiment, after S409, the processing of the information processing device 100 returns to S1201.

[0076] <Acquisition of nearby deformations> The process of S1201 for acquiring, from the second deformation data, a deformation located near the reference deformation and having a deformation type different from the deformation type of the reference deformation as a neighboring deformation will be described with reference to Fig. 13. In describing this embodiment, the deformation type of the reference deformation is a crack. Fig. 13(a) is a diagram showing first deformation data 1301. It is assumed that a crack 1302 is selected as a reference deformation from the first deformation data 1301. Fig. 13(b) is a diagram showing second deformation data 1311. It is assumed that the second deformation data 1311 is deformation data including cracks 1312, 1313, 1314, 1315, and efflorescence 1316.

[0077] When selecting a nearby deformation from the second deformation data, the nearby deformation acquisition unit 1101 first obtains a rectangular range surrounding the reference deformation. A rectangular range 1303 in Fig. 13(a) is an example of a rectangular range corresponding to a crack 1302 of the reference deformation. Next, the nearby deformation acquisition unit 1101 overlays the rectangular area 1303 on the second deformation data 1311 to obtain a rectangular area 1317 . Then, the nearby deformation acquisition unit 1101 acquires, as the nearby deformation, a deformation that is at least partially included within a rectangular range 1317 on the second deformation data 1311 and has a deformation type different from the deformation type of the reference deformation. In this example, the nearby deformation acquisition unit 1101 acquires efflorescence 1316, which is a deformation other than a crack, as the nearby deformation.

[0078] <Creating correction candidates> The correction candidate creation process in S1202 of the second embodiment will be described with reference to Fig. 14. Fig. 14(a) is a diagram showing a crack 1401 selected as a reference deformation from the first deformation data. Fig. 14(b) is a diagram showing cracks 1411, 1412, and 1413 selected as deformations to be corrected from the second deformation data, and efflorescence 1414 selected as a nearby deformation.

[0079] When creating a correction candidate, the correction candidate creation unit 124 first obtains, as an invisible deformation, a deformation portion of the reference deformation that overlaps with a nearby deformation. Specifically, the correction candidate creation unit 124 overlaps the shape of the nearby deformation with that of the reference deformation, and obtains the portion of the reference deformation that overlaps with the nearby deformation. As a specific example, FIG. 14(c) shows a state in which a crack 1401 and efflorescence 1414 overlap. The correction candidate creation unit 124 obtains, as an invisible deformation, a partial crack 1421 that overlaps with the efflorescence 1414 from the crack 1401 in FIG. 14(c). FIG. 14(d) shows a state in which only the partial crack 1421 is extracted.

[0080] Next, the correction candidate creation unit 124 creates a correction candidate using the invisible deformation and the deformation to be corrected. Here, one method of creating a correction candidate is to connect the end points of the invisible deformation and the end points of the deformation to be corrected. In this method, the correction candidate creation unit 124 connects the end points of any of the cracks 1411, 1412, and 1413 in FIG. 14(b) to at least one end point of the partial crack 1421 in FIG. 14(d). FIG. 14(e) is a diagram showing, as an example, a correction candidate 1422 obtained by performing a correction such as connecting the partial crack 1421 and the cracks 1411 and 1412. It can be seen that the position and shape of this correction candidate 1422 are similar to those of the crack 1401 in FIG. 14(a). In this way, a correction candidate can be created by using the part of the reference deformation that overlaps with the neighboring deformation (invisible deformation) and the deformation to be corrected.

[0081] According to the second embodiment, it is possible to perform a correction process such as comparing deformations of different deformation types and interpolating deformation parts that cannot be visually recognized due to overlapping of deformations. Therefore, according to the second embodiment, it becomes possible to calculate the degree of progression of a deformation taking into account deformation parts that cannot be visually recognized.

[0082] <Third embodiment> In the first embodiment, an example was described in which the consistency between the reference deformation and the correction candidate is calculated, and the deformation to be corrected is corrected based on a comparison of the consistency. Here, the deformations for which the progress is calculated may include deformations that do not require correction processing. If a correction process is performed on a deformation that does not require correction processing, unnecessary processing time is required to create a correction candidate. Therefore, the information processing device of the third embodiment calculates an index for determining whether or not to create a correction candidate before creating the correction candidate, and determines whether or not to create a correction candidate based on a comparison between the calculated index and a predetermined threshold value (set as an index reference value). Then, the information processing device creates a correction candidate only when the calculated index is less than the index reference value. As an index for determining whether or not to create a correction candidate, there is a method of using at least one of a plurality of indexes described below. For example, there is a method of using the consistency before correction between the reference deformation and the deformation to be corrected. Hereinafter, the third embodiment will be described, focusing on the difference from the first embodiment.

[0083] The hardware configuration of the information processing device 100 according to the third embodiment is similar to that of the first embodiment shown in Fig. 1(a), and therefore illustration and description thereof will be omitted. Fig. 15 is a diagram showing an example of a functional block diagram of the information processing device 100 according to the third embodiment. The information processing device 100 according to the third embodiment differs from the configuration of the first embodiment shown in Fig. 1(b) in that an index calculation unit 1501 and a determination unit 1502 are added.

[0084] The index calculation unit 1501 is a functional unit of the CPU 101, and calculates a judgment index for judging whether or not to create a correction candidate by using the reference deformation and the deformation to be corrected. The details of the process in the index calculation unit 1501 will be described later. The determination unit 1502 is a functional unit of the CPU 101, and determines whether or not to create a correction candidate based on the index calculated by the index calculation unit 1501. The process in the determination unit 1502 will be described later in detail.

[0085] Fig. 16 is a flowchart showing a main flow of information processing executed by the information processing device 100 according to the third embodiment. In the flowchart of Fig. 16, in steps having the same numbers as those in the flowchart of Fig. 4 described in the first embodiment, the same processing as in the first embodiment is executed. In the case of the third embodiment, after the reference deformation and the deformation to be corrected are selected in S401, the information processing device 100 proceeds to the processing of S1601.

[0086] In S1601, the index calculation unit 1501 calculates a judgment index for determining whether or not to create a correction candidate by using the reference deformation and the deformation to be corrected. After that, the process of the information processing device 100 proceeds to S1602. In S1602, the determination unit 1502 determines whether the index calculated by the index calculation unit 1501 is less than the index reference value. If the result of the determination is less than the index reference value (Y), the information processing device 100 proceeds to the correction candidate generation process of S402. The processes from S402 onwards are the same as those in the first embodiment, and therefore will not be described.

[0087] On the other hand, if the result of the determination in S1602 is equal to or greater than the index reference value (N), the information processing device 100 proceeds to S406. If the result of the determination in S1602 is that the process proceeds to S406, the correction process in S405 has not been performed and there is no corrected deformation, so the progress calculation unit 128 calculates the progress using the reference deformation and the deformation to be corrected in S406. The details of the deformation to be corrected used in calculating the progress will be described later. In the third embodiment, after S409, the process of the information processing device 100 returns to S402.

[0088] <Calculation of the judgment index for creating correction candidates> The process of S1601 for calculating the determination index for whether or not to create a correction candidate will be described with reference to FIG. FIG. 17(a) shows a reference deformation 1701, and FIG. 7(b) shows deformations 1711, 1712, and 1713 to be corrected. Here, one of the judgment indices is a method of using the pre-correction consistency between the reference deformation and the deformation to be corrected as an index. The calculation method of the pre-correction consistency can be the same as the consistency calculation processing method of S403 in the first embodiment. For example, when the pre-correction consistency calculated by the index calculation unit 1501 as an index is low and is less than the index reference value as a predetermined threshold value for the index, the determination unit 1502 can determine that the similarity of the shapes between the reference deformation and the deformation to be corrected is low, and therefore it is necessary to perform correction processing. In other words, when the pre-correction consistency is less than the index reference value, it can be determined that it is necessary to create a correction candidate from the deformation to be corrected and perform correction processing of the deformation to be corrected based on the correction candidate. FIG. 17(c) is a diagram showing an example of the degree of consistency before correction between a reference deformation 1701 and deformations 1711, 1712, and 1713 to be corrected, in the case where the degree of overlap between the deformations is defined as the degree of consistency before correction.

[0089] Other indices may also be used as a judgment index for whether or not to create correction candidates. For example, there is a method in which the number of deformations to be corrected is used as an index, and a judgment is made using an index reference value for this index (number). Here, if there are multiple deformations to be corrected, it is possible to create correction candidates whose shape is similar to the reference deformation using these deformations. For example, in the example of FIG. 17(b), the number of deformations to be corrected is 3. On the other hand, if the number of deformations to be corrected is too large, the process of creating correction candidates whose shape is similar to the reference deformation becomes enormous. For this reason, if the number of deformations to be corrected calculated as an index by the index calculation unit 1501 is less than the index reference value, the judgment unit 1502 judges that an appropriate number of correction candidates whose shape is similar to the reference deformation can be created without requiring extensive processing, and judges to perform the process of creating correction candidates.

[0090] As another judgment index, a method of using the degree of progress (pre-correction progress) between the reference deformation and the deformation to be corrected and using the index reference value for the pre-correction progress can be considered. For example, if the pre-correction progress becomes a negative value even though the wall surface of the structure has not been repaired, it can be judged that the correspondence between the deformations is highly likely to be incorrect. In this case, the judgment unit 1502 judges that the creation process of the correction candidate will not be performed if the pre-correction progress becomes a value in the negative direction from 0, since the correspondence between the deformations is highly likely to be incorrect. In other words, if the pre-correction progress becomes a negative value even though the deformation has not been repaired, for example, in the case of a crack, it indicates that the crack has become shorter even though it has not been repaired. In this case, the deformation to be corrected is highly likely to be a deformation that should not be compared with the reference deformation. For this reason, it is judged that the creation process of the correction candidate will not be performed so as not to create a correction candidate from the deformation to be corrected. On the other hand, if the pre-correction progress degree is less than the index reference value for a negative value, that is, if the pre-correction progress degree is a positive value with 0 as the index reference value, it is considered that there is no possibility (very low) that the correspondence between the deformations is incorrect. For this reason, the determination unit 1502 determines that a correction candidate creation process should be performed when the pre-correction progress degree is less than the index reference value for a negative value. In other words, in this case, since the deformation is likely to be the subject of comparison with the reference deformation, it is determined that a correction candidate creation process should be performed.

[0091] A method for calculating the pre-correction progress degree will be described with reference to Fig. 17(d) and Fig. 17(e). Fig. 17(d) shows the reference deformation 1721, and Fig. 17(e) shows the correction target deformations 1731 and 1732 corresponding to the reference deformation 1721. When calculating the pre-correction progress degree, the index calculation unit 1501 selects one deformation 1731 from the correction target deformations. As an example of the pre-correction progress degree, the difference in total length can be used. Here, if the total length of the reference deformation 1721 is 50 mm and the total length of the correction target deformation 1731 is 30 mm, the pre-correction progress degree is minus 20 mm. For the remaining correction target deformations 1732, the difference in total length from the reference deformation 1721 can be calculated using the same method.

[0092] As another index, the angle (deformation angle) between the line segment connecting the start point and end point of the reference deformation and the line segment connecting the start point and end point of each deformation to be corrected may be used, and a method of judgment using the index reference value for the deformation angle can also be considered. Here, if the deformation angle is large (i.e., the deformations are different in direction), it can be judged that the correspondence between the deformations is highly likely to be incorrect. The coordinate information of the reference deformation and the deformation to be corrected is stored in the storage unit 121. Therefore, the index calculation unit 1501 can obtain the line segment connecting the start point coordinate and the end point coordinate from the coordinate information of each deformation acquired from the storage unit 121, and calculate the angle between the two lines as the deformation angle, which is used as the index. In this example, if the deformation angle is less than the index reference value (i.e., the difference in the direction of the deformation is small), the judgment unit 1502 judges that the correspondence between the deformations is unlikely to be incorrect and judges to perform the creation process of the correction candidate.

[0093] <Determining whether correction is necessary> As described above, in S1602, the determination unit 1502 determines whether or not to create a correction candidate based on a comparison between the index calculated by the index calculation unit 1501 and the index reference value. Here, the pre-correction consistency is used as the determination index. In this case, the determination unit 1502 determines whether or not to create a correction candidate based on a comparison between the index, which is the pre-correction consistency, and the index reference value. The equation for determining whether or not to create a correction candidate based on the consistency is shown in Equation (4).

[0094] I p c Formula (4)

[0095] Here, the parameter I p is the uncorrected consistency in the pth deformation to be corrected, and this parameter I p The uncorrected consistency calculated in S1601 is input to parameter I c is an arbitrary fixed value that indicates the index reference value, which is the threshold value for judgment. For example, if the judgment criterion is whether the consistency is 50% or more, the parameter I c ​If there are multiple uncorrected consistency levels, the parameter I c The value of remains fixed, and the decision unit 1502 has a parameter I p Another pre-correction consistency is input to and the judgment process of formula (4) is repeated. Then, if the judgment result shows that all pre-correction consistency degrees satisfy formula (4), the information processing device 100 proceeds to the correction candidate creation process of S402. On the other hand, if the judgment unit 1502 judges that formula (4) is not satisfied, the information processing device 100 regards the deformation to be corrected with the maximum pre-correction consistency degree as the corrected deformation, skips the correction candidate creation process, and proceeds to the progress calculation process of S406.

[0096] Even when a judgment index other than the pre-correction consistency is used, the judgment unit 1502 can make a judgment using formula (4). For example, even when a pre-correction progress degree calculated from the reference deformation and the deformation to be corrected is used, it is possible to judge whether or not to create a correction candidate using formula (4). When the pre-correction progress degree is used, the parameter I p The uncorrected progress is input to parameter I c An arbitrary fixed value indicating the judgment threshold is set as the index reference value for parameter I c If multiple uncorrected progress degrees are calculated, the parameter I c The value of remains fixed, and the parameter I p The pre-correction progress degrees are input in order as the progress degree calculation result, and the determination process of formula (4) is repeated. If the determination result shows that all of the pre-correction progress degrees satisfy formula (4), the information processing device 100 proceeds to a correction candidate generation process in S402. On the other hand, if the determination unit 1502 determines that formula (4) is not satisfied, the information processing device 100 proceeds to a progress degree calculation process in S406. However, since the maximum value of the calculated pre-correction progress degrees can be regarded as the progress degree calculation result in S406, the information processing device 100 may proceed directly to the process of S407.

[0097] As described above, according to the third embodiment, an index for determining whether or not to create a correction candidate is calculated using the deformations before correction, and a process for creating a correction candidate is performed only when the calculated index is less than the index reference value. In the third embodiment, the correction candidate calculation and consistency calculation processes can be performed only for deformations that require correction processing, so that an increase in processing time can be suppressed.

[0098] <Fourth embodiment> In the first embodiment, an example of calculating the degree of progress by performing correction processing for each individual deformation has been described. Here, as deterioration of a structure progresses, a large number of deformations (e.g., 1000 or more) occur on the surface of the structure wall. In order to obtain the degree of progress of a large number of deformations occurring on the wall surface, it is necessary to repeat correction processing for each individual deformation, which significantly increases the processing time. Therefore, in the fourth embodiment, an example of correcting a plurality of deformation data collectively will be described. Specifically, an image is divided, deformation data is grouped for each divided area, and the deformation data is corrected in groups. As a method of correcting the deformation data in groups, for example, there is a method of removing deformation data smaller than a certain size as noise. By correcting the deformation data in groups, it is possible to correct a large number of deformation data and calculate the degree of progress while suppressing an increase in processing time. Hereinafter, the fourth embodiment will be described, focusing on the difference from the first embodiment.

[0099] The hardware configuration of the information processing device 100 according to the fourth embodiment is similar to that of the first embodiment shown in Fig. 1(a), and therefore illustration and description thereof will be omitted. Fig. 18 is a diagram showing an example of a functional block diagram of the information processing device 100 according to the fourth embodiment. The storage unit 121 and management unit 122 in Fig. 18 are similar to the configurations shown in Fig. 1(b) of the first embodiment, and therefore description thereof will be omitted.

[0100] The group selection unit 1801 is a function of the CPU 101, which selects a reference deformation group from the first deformation data, and selects a correction target deformation group corresponding to the reference deformation group from the second deformation data. The correction candidate group creation unit 1802 is a functional unit of the CPU 101, and creates a correction candidate group from the deformation group to be corrected. The group consistency calculation unit 1803 is a functional unit of the CPU 101, and performs processing for calculating the consistency for each group based on the correction candidate group and the reference deformation group. The correction candidate group selection unit 1804 is a functional unit of the CPU 101, and performs a process of selecting a correction candidate group based on the calculation result of the consistency degree for each group. The group correction unit 1805 is a functional unit of the CPU 101, and corrects the deformation group to be corrected based on the selection result of the correction candidate group, and obtains a corrected deformation group. The group progress calculation unit 1806 is a functional unit of the CPU 101, and calculates the difference between the reference deformation group and the corrected deformation group as the progress.

[0101] Fig. 19 is a flowchart showing a main flow of information processing executed by the information processing device 100 according to the fourth embodiment. In the flowchart of Fig. 19, in steps having the same numbers as those in the flowchart of Fig. 4 described in the first embodiment, the same processing as in the first embodiment is executed. The processing of S1901 to S1907 in Fig. 19 will be described below.

[0102] <Selection of the reference deformation group and the deformation group to be corrected> FIG. 20 is a diagram for explaining the process of S1901, in which the group selection unit 1801 selects a reference deformation group from the first deformation data and selects a deformation group to be corrected from the second deformation data. FIG. 20(a) shows a first image 2001, and FIG. 20(b) shows first deformation data 2002. FIG. 20(c) shows a second image 2021, and FIG. 20(d) shows second deformation data 2022. In the fourth embodiment, the deformation data is a crack on the surface of a structure, as in the first embodiment. Also, as in the previous embodiment, the first image, the second image, and the deformation data corresponding to each image are all pasted at the same position on the same drawing.

[0103] In S1901, the group selection unit 1801 first selects a reference deformation group from the first deformation data. Here, as a method for selecting a reference deformation group, an example is described in which the first image is divided and the first deformation data is grouped for each divided area. The group selection unit 1801 first divides the first image 2001 in Fig. 20(a) into any fixed size. The lattice-shaped line segments 2011 in the figure are line segments that divide the first image 2001 at equal intervals in the X-axis direction and the Y-axis direction. The group selection unit 1801 divides the first image 2001 at intervals of, for example, 1,024 pixels.

[0104] Next, the group selection unit 1801 selects one of the divided regions. FIG. 20(a) shows an example in which one region 2012 indicated by diagonal lines is selected. Furthermore, the group selection unit 1801 acquires the region size and upper left vertex coordinates 2013 of the region 2012. Then, as shown in FIG. 20(b), the group selection unit 1801 sets a region 2014 having the same position and size as the region 2012 on the first deformation data 2002. Thereafter, the group selection unit 1801 groups the deformation data included in the region 2014 to set it as a reference deformation group 2015.

[0105] Next, the group selection unit 1801 selects a correction target deformation group corresponding to the reference deformation group from the second deformation data. For example, a method of selecting a correction target deformation group is a method of grouping the second deformation data included in the same position and the same range area as the reference deformation group.

[0106] Here, an example of selecting a deformation group to be corrected corresponding to the reference deformation group 2015 in FIG. 20(b) will be described. First, the group selection unit 1801 acquires the area size and the upper left vertex coordinates 2016 of the area 2014. Next, the group selection unit 1801 sets an area 2031 in the same position and the same range as the area 2014 in FIG. 20(b) on the second deformation data 2022 shown in FIG. 20(d). Then, the group selection unit 1801 groups the second deformation data included in the area 2031 to set it as a correction target deformation group 2032. In this way, the group selection unit 1801 selects a correction target deformation group corresponding to the reference deformation group by associating groups of deformation data included in the same position and within the same range between the first deformation data and the second deformation data.

[0107] Here, since the shooting conditions usually change when shooting at different times, even when the same wall surface is shot, there is a positional deviation between each pixel of the first image and the second image. Therefore, there is also a positional deviation between the first deformation data and the second deformation data. The above example is an example in which this positional deviation is ignored and a reference deformation group and a correction target deformation group are simply created. In this embodiment, the purpose is to roughly correct the deformation data on a deformation group basis. Therefore, problems are usually unlikely to occur even if the deformation groups are not strictly matched.

[0108] However, when the positional deviation between the deformation data is large, it is preferable to obtain the correspondence between the deformation groups while taking the positional deviation into consideration. As one method of considering the positional deviation, there is a method of regarding the positional deviation between images as the positional deviation between the deformation groups. In the case of this method, specifically, the group selection unit 1801 divides the first image and the second image into the same fixed size, and obtains the difference in the image feature amount such as the luminance value for each divided area at the same position. Furthermore, the group selection unit 1801 obtains the difference in the image feature amount while shifting the position of one of the divided areas of the first and second images, and obtains the positional deviation amount of the position where the difference is minimum. Then, the group selection unit 1801 moves the position of the deformation group to be corrected by the obtained positional deviation amount. In this way, the group selection unit 1801 can select the deformation group to be corrected while taking the positional deviation into consideration.

[0109] <Creating correction candidate groups> Fig. 21 is a diagram explaining the process of S1902 in which the correction candidate group creation unit 1802 creates a correction candidate group from a correction target deformation group. Fig. 21(a) shows an example of a reference deformation group 2101, and Fig. 21(b) shows a correction target deformation group 2111 corresponding to the reference deformation group 2101. The correction target deformation group 2111 contains many deformations with short lengths compared to the reference deformation group 2101.

[0110] Here, as one method for creating a correction candidate group, there is a method for removing part of the deformation data as noise. For example, deformation data detected from an image using machine learning includes false positives (noise). Since deformation data due to false positives is a factor that causes an error in calculating the degree of progress, a suitable method is to apply a noise removal process to the deformation data and correct multiple deformation data collectively. Figures 21(c) to 21(e) are diagrams showing the results of applying noise removal processes of different intensities to a correction target deformation group 2111 as an example of creating a correction candidate group by noise removal. The correction candidate groups 2121 to 2123 in Figures 21(c) to 21(e) are examples showing the results of applying noise removal processes of different intensities to the correction target deformation group 2111 illustrated in Figure 21(b), arranged in ascending order of noise removal process intensity. Of these three examples, the correction candidate group 2121 in Fig. 21(c) when the strength of the noise removal process is low retains a lot of deformation data included in the correction target deformation group 2111. On the other hand, the correction candidate group 2123 in Fig. 21(e) when the strength of the noise removal process is high has small deformation data removed from the deformation data included in the correction target deformation group 2111, leaving only large deformation data.

[0111] In addition, a method other than noise removal can be used to create the correction candidate group. For example, the correction candidate group creation unit 1802 may create the correction candidate group by using a method of connecting the end points between adjacent deformation data. Here, the deformation data detected from the image using machine learning may become interrupted deformation data due to the influence of image blur. The interrupted deformation data is likely to cause an error in calculating the degree of progress. The correction candidate group creation unit 1802 can reduce the amount of deformation data that is interrupted due to the influence of image blur by connecting the end points between adjacent deformation data, thereby reducing the occurrence of errors in calculating the degree of progress.

[0112] In addition, when creating a correction candidate group, it is desirable to make the deformation group to be corrected itself one of the correction candidate groups. The reason for this is that there are cases where the consistency between the deformation group to be corrected and the reference deformation group is high and correction processing is not required. By making the deformation group to be corrected itself one of the correction candidate groups, the consistency with other correction candidate groups is compared in the processing from S1903 onwards, which will be described later. As a result, if it is determined that the consistency of the correction candidate group of the deformation group to be corrected itself is the highest, it becomes possible to skip the correction processing.

[0113] <Calculation of group consistency> Hereinafter, in the fourth embodiment, the process of S1903 in which the group consistency calculation unit 1803 calculates the group consistency using the reference deformation group and the correction candidate group will be described. Here, there is a method of calculating the group consistency as the degree of agreement of the aggregated data aggregated from the deformation data included in each group. As the aggregated data, for example, the total value of the overlapping degree of the deformation data between the groups is used. The formula for calculating the group consistency C using the total value of the overlapping degree is shown in formula (5).

[0114]

number

[0115] Here, the parameter P 11 is the pixel area of ​​all deformation data in the reference deformation group, and the parameter P 12 is the deformation pixel area of ​​all deformation data in the deformation group to be corrected. The group consistency calculation unit 1803 uses these parameters P 11 ,P 12 When calculating, pixels with multiple overlapping abnormalities within the same group are counted as a single pixel. From formula (5), if the abnormal pixel areas match, the group consistency C is 1, and if the abnormal pixel areas do not overlap at all, the group consistency C is 0. In other words, the range of possible values ​​for group consistency C when using the abnormal pixel area is 0 to 1.

[0116] Note that there is a positional deviation and a shape deviation between the first deformation data and the second deformation data. Therefore, it is desirable to take these deviations into consideration, expand the shape of the deformation data of each group, and then overlap them to obtain the group consistency C. Therefore, after obtaining one group consistency, the group consistency calculation unit 1803 selects a different correction candidate group while keeping the reference deformation group selected, and calculates the group consistency using formula (5). The group consistency calculation unit 1803 can calculate the group consistency between the reference deformation group and all correction candidate groups by repeating such a group consistency calculation process.

[0117] In addition, when calculating the group consistency, other aggregate data than the aggregate data consisting of the sum of the overlapping degrees of deformation data between groups described above may be used. For example, there is a method of using the degree of change in the number of deformation data between groups as another aggregate data. In this example, specifically, the group consistency calculation unit 1803 calculates the number of deformation data in each of the reference deformation group and the correction candidate group, and calculates the inverse of the difference in the number of deformation data to obtain the group consistency. As a result, when the difference in the number of deformation data is large, the group consistency becomes a value close to 0. Note that, when there is no difference in the number of deformation data (0), a constant larger than 1 may be used as the group consistency. As an example of using other aggregate data, the group consistency calculation unit 1803 may use the rate of change in the total length of deformation data between groups as the group consistency. The group consistency calculation unit 1803 can calculate the group consistency by using any one of these various aggregate data or by combining two or more of them. In this way, the group consistency calculation unit 1803 can calculate the group consistency by using the aggregate data related to the deformation group.

[0118] <Selection of correction candidate group> FIG. 22 is a diagram for explaining the process of the correction candidate group selection unit 1804 that selects a correction candidate group based on the calculation result of the group consistency degree. FIG. 22(a) shows the reference deformation group 2201, and FIG. 22(b) shows the correction target deformation group 2211. FIG. 22(c) to FIG. 22(f) show the correction candidate groups 2221, 2222, 2223, and 2224 created from the correction target deformation group 2211. The correction candidate group 2224 in FIG. 22(f) is an example in which the correction target deformation group 2211 itself is one of the correction candidate groups. And FIG. 22(g) shows the group consistency list 2231 between the correction candidate groups 2221, 2222, 2223, and 2224 and the reference deformation group 2201. The group consistency used in this embodiment is the degree of agreement of the overlapping degree of deformation data between groups. Therefore, the range of values ​​that the group consistency can take is 0 to 1, and the closer to 1, the higher the consistency between the reference deformation group and the correction candidate group.

[0119] Here, as one method of selecting a correction candidate group based on the group consistency, there is a method of selecting a correction candidate group with the maximum group consistency. For example, in the case of the group consistency list 2231 of FIG. 22(g), the correction candidate group selection unit 1804 selects the correction candidate group 2222 with the maximum group consistency. In this way, when the maximum value of the group consistency is one, a correction candidate group can be uniquely selected.

[0120] On the other hand, FIG. 22(h) is a diagram showing the group consistency list 2241 when there are multiple maximum group consistency degrees. FIG. 22(f) shows the group consistency list 2241 in the correction candidate groups 2251, 2252, 2253, and 2254 (not shown) created from the correction target deformation group 2211. When multiple group consistency degrees are maximum as in FIG. 22(h), the correction candidate group cannot be uniquely determined. In such a case, the information processing device 100 generates a display image showing multiple correction candidate groups with the maximum group consistency degrees and the reference deformation group, and displays it on the display unit 105. Then, when the user designates a correction candidate group with reference to the display image, the correction candidate group selection unit 1804 selects the correction candidate group designated by the user.

[0121] Also, when the maximum value of the group consistency is close to 0 (e.g., 0.1), the selected correction candidate group is likely to be inappropriate. In this case, when the maximum value of the group consistency is less than a predetermined threshold (a group consistency reference value), the information processing device 100 generates a display image showing the reference anomaly group and the correction candidate group together with the maximum value of the group consistency, and displays it on the display unit 105. Note that an example of the group consistency reference value is 0.2. It is desirable to have the user confirm and decide the correction candidate group from the display image. If the user confirms the correction candidate group through the display image and specifies the correction candidate group, the correction candidate group selection unit 1804 selects the correction candidate group specified by the user.

[0122] In addition, there are cases where selecting the correction candidate group with the maximum group consistency is not optimal. This is because it is assumed that the area of ​​the deformed pixels will increase as the deformed state progresses. In this case, for example, a target group consistency is set according to the assumed tendency of the deformed state to progress, and the correction candidate group selection unit 1804 selects a correction candidate group close to the target group consistency.

[0123] <Correction process for deformation groups> In S1905, the group correction unit 1805 performs a process of correcting the deformation group to be corrected based on the selection result of the correction candidate group. A method of correcting the deformation group to be corrected in this embodiment will be described with reference to Fig. 23. Fig. 23(a) shows a deformation group to be corrected 2301, and Fig. 23(b) shows a correction candidate group 2302. The correction candidate group 2302 is a result of removing deformation data of less than a certain size from the deformation group to be corrected 2301 as noise.

[0124] When correcting the deformation group to be corrected, the group correction unit 1805 corrects the attribute information in the deformation group to be corrected to match the attribute information in the correction candidate group. Specifically, the group correction unit 1805 performs a process of deleting some of the deformation data in the deformation group to be corrected 2301 so that the deformation data in the deformation group to be corrected 2301 matches the deformation data in the correction candidate group 2302. Figure 23(c) is a diagram showing the corrected deformation group 2303 after the correction process has been performed.

[0125] In addition, the group correction unit 1805 performs an update process of the deformation data table in conjunction with the above-mentioned correction process. That is, the group correction unit 1805 performs an update process (e.g., deletion, adding a deletion flag) for the deformation data to be deleted on the deformation data table. The update process of the deformation data table is the same as the process of S405 in the first embodiment, so a description thereof will be omitted.

[0126] <Calculating group progress> In S1906, the group progress calculation unit 1806 performs a process of calculating the progress level (group progress level) for each group using the reference deformation group and the corrected deformation group. In this embodiment, the group progress calculation unit 1806 performs a process of calculating the difference in attribute information of the deformation data between groups as the progress level. For example, when the deformation type is crack, the group progress calculation unit 1806 calculates the difference in the maximum width and total length of the deformation data within the group. The maximum width and total length of the crack are stored in the storage unit 121. Therefore, the group progress calculation unit 1806 can calculate the group progress level for the reference deformation group by acquiring the maximum width and total length of the deformation within each group and calculating the difference in the maximum width and the difference in the total length between the deformation groups. In addition, when the deformation type is other than crack, the group progress level can be calculated in a similar manner. For example, when the deformation type is exposed rebar, the difference in the total area of ​​the deformation data within the group can be used as the group progress level. The area of ​​the deformation data is stored in the storage unit 121 as attribute information. Therefore, the group progress calculation unit 1806 can calculate the group progress by obtaining the total area of ​​deformations in the reference deformation group and the total area of ​​deformations in the corrected deformation group, and finding the difference between them.

[0127] <Modification of the fourth embodiment> In the fourth embodiment, a method of creating a correction candidate group based on a correction target deformation group selected from the second deformation data has been described, but a correction candidate group may be created from a second image. In this case, specifically, a plurality of trained models trained in advance by machine learning are prepared. Then, the information processing device 100 detects a deformation for each model in the second image and obtains a detection result for each model. After that, the information processing device 100 obtains deformation data corresponding to the position and range of the reference deformation group for each of the plurality of detection results, and sets each as a correction candidate group. This makes it possible to create a plurality of correction candidate groups with different detection tendencies. Note that, in the process of detecting a deformation for each model from the second image, the information processing device 100 may obtain a detection result for each detection parameter using a plurality of detection parameters, and create even more correction candidate groups.

[0128] As described above, the information processing device 100 of the fourth embodiment groups a plurality of abnormalities, corrects the abnormalities collectively in units of groups, and calculates the degree of progression of the abnormalities. Therefore, according to the fourth embodiment, even if a large number of abnormalities exist on a wall surface, the multiple abnormalities can be efficiently corrected, and an increase in processing time can be suppressed.

[0129] <Fifth embodiment> In the first embodiment, a method for creating a correction candidate when the deformation data is expressed by pixels has been described, but at least a part of the attribute information of the deformation data may be expressed by vector data such as a polyline composed of multiple points as described above. In the fifth embodiment, an example of creating a correction candidate from a deformation to be corrected when the attribute information of the deformation data is expressed by vector data will be described. When the attribute information of the deformation data is expressed by vector data, the amount of coordinate information is reduced compared to when the attribute information is expressed by pixels, making it possible to reduce the amount of data. Note that the hardware configuration, functional block diagram, and flow chart showing a part of the processing of the information processing device 100 according to the fifth embodiment are similar to those of the first embodiment, and therefore illustration and description thereof will be omitted.

[0130] <Creating correction candidates for polyline data> Using Fig. 24, a process of creating correction candidates when attribute information of deformation data is expressed by vector data will be described. The deformation data in the fifth embodiment is assumed to be cracks on the surface of a structure, as in the first embodiment. In Fig. 24, coordinate points of vector data are shown as black circles, and deformations are expressed by connecting consecutive coordinate points with lines. Fig. 24(a) shows a reference deformation 2401, and Fig. 24(b) shows deformations to be corrected 2402 and 2403. The deformations to be corrected 2402 and 2403 are assumed to be adjacent to each other at coordinate 2411.

[0131] The correction candidate creation unit 124 of the fifth embodiment can use a method similar to the correction candidate creation method of the first embodiment when creating correction candidates from the deformations 2402 and 2403 to be corrected. That is, the correction candidate creation unit 124 cuts a part of the deformation to be corrected, and then connects the deformations to each other, thereby creating deformation data having a branch different from the branch of the shape data (deformation to be corrected) before cutting as a correction candidate. In particular, when the attribute information of the deformation data is expressed by vector data as in this embodiment, the coordinates (branching point) where multiple deformations to be corrected are adjacent can be set as the cutting position.

[0132] Fig. 24(c) shows an example of the result when a correction candidate 2421 is created from the deformations 2402 and 2403 to be corrected, and Fig. 24(d) shows an example of the result when a correction candidate 2422, which has a different branch from the correction candidate 2421, is created from the deformations 2402 and 2403 to be corrected. As shown in Figs. 24(c) and 24(d), when the attribute information of the deformation data is expressed by vector data, a correction candidate can be created by switching the connection relationship of the branch point parts among the coordinate points constituting the vector data.

[0133] According to the information processing device 100 of the fifth embodiment described above, even if the attribute information of the deformation data is expressed as vector data, it is possible to create correction candidates from the deformation to be corrected, and to correct the deformation data and calculate the degree of progress.

[0134] The present invention can also be realized by supplying a program for implementing one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. The above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these. In other words, the present invention can be implemented in various forms without departing from its technical concept or main characteristics. [Explanation of symbols]

[0135] 100: information processing device, 123: selection unit, 124: correction candidate generation unit, 125: consistency calculation unit, 126: correction candidate selection unit, 127: correction unit, 128: progress calculation unit

Claims

1. A selection means for selecting at least one deformation among the first deformations associated with the first image as a reference deformation, and selecting at least one deformation among the second deformations associated with the second image taken at a time different from the first image as a deformation to be corrected; A candidate creation means for creating a correction candidate by correcting the deformation to be corrected; A consistency degree acquisition means for acquiring a consistency degree representing a consistency relationship between the reference deformation and the correction candidate; A deformation correction means for correcting the deformation to be corrected based on the consistency and creating a corrected deformation; A progress acquisition means for acquiring a progress degree representing a change in the deformation from the reference deformation based on a comparison between the reference deformation and the corrected deformation; 13. An information processing device comprising:

2. The information processing device described in claim 1, characterized in that the candidate creation means performs at least one of a cutting process and a connection process on the deformation to be corrected to create the correction candidate having a connection relationship different from the connection relationship of the deformation to be corrected.

3. 3. The information processing apparatus according to claim 1, wherein the candidate creating means creates the correction candidate by changing at least one of a position and a shape of the deformation to be corrected.

4. The information processing device according to any one of claims 1 to 3, characterized in that the consistency acquisition means calculates the consistency based on a degree of overlap when the reference deformation and the correction candidate shapes are expanded and superimposed.

5. The candidate creation means is characterized in that a deformation portion of the reference deformation that overlaps with a portion of the second deformation is treated as an invisible deformation, and a deformation having a shape different from the shape of the deformation to be corrected is created as the correction candidate using the invisible deformation and at least a portion of the deformation to be corrected.

6. 6. The information processing apparatus according to claim 1, further comprising a display unit that displays the degree of progress acquired by the degree of progress acquisition unit.

7. The candidate creation means creates the correction candidate by modifying attribute information of the deformation to be corrected, The deformation correction means corrects attribute information of the deformation to be corrected based on the consistency degree to create the corrected deformation having a corresponding relationship with the reference deformation, The information processing device according to any one of claims 1 to 6, characterized in that the progress acquisition means calculates the progress based on a comparison between attribute information of the reference deformation and attribute information of the corrected deformation.

8. The candidate creation means creates a plurality of the correction candidates by modifying attribute information of the deformation to be corrected, The information processing device described in claim 7, characterized in that the deformation correction means corrects attribute information of the deformation to be corrected related to the creation of one correction candidate selected by a candidate selection means from the multiple correction candidates based on the degree of consistency, and creates the corrected deformation.

9. The information processing device described in claim 7 or 8, characterized in that the consistency acquisition means calculates features of each deformation based on attribute information of the reference deformation and attribute information of the correction candidate, and calculates the consistency based on the features.

10. The deformation correction means changes the shape and the attribute information of a portion of the deformation to be corrected based on the consistency, to create the corrected deformation. Information processing device according to any one of claims 7 to 9.

11. The information processing device according to any one of claims 7 to 10, characterized in that the progress acquisition means calculates a difference between attribute information of the reference deformation and attribute information of the corrected deformation as the progress.

12. The information processing device according to any one of claims 7 to 11, further comprising a determination means for calculating a judgment index based on attribute information of the reference deformation and attribute information of a portion of the deformations to be corrected, and determining whether or not to create the correction candidate based on a comparison of the judgment index with a threshold value.

13. A selection means for selecting a plurality of deformations among the first deformations associated with the first image as a reference deformation group, and selecting a plurality of deformations among the second deformations associated with the second image taken at a different time from the first image as a correction target deformation group; A candidate creating means for creating a correction candidate group by modifying the correction target deformation group; A consistency degree acquisition means for acquiring a group consistency degree representing a consistency relationship between the reference deformation group and the correction candidate group; A deformation correction means for correcting the deformation group to be corrected based on the group consistency degree to create a corrected deformation group; A progress acquisition means for acquiring a group progress degree representing a change from the reference deformation group based on a comparison between the reference deformation group and the corrected deformation group; 13. An information processing device comprising:

14. The candidate creating means creates the correction candidate group by modifying attribute information of the correction target deformation group, The deformation correction means corrects the attribute information of the deformation group to be corrected based on the group consistency degree to create the corrected deformation group having a corresponding relationship with the reference deformation group, The information processing device according to claim 13, characterized in that the progress acquisition means calculates the group progress based on a comparison between attribute information of the reference deformation group and attribute information of the corrected deformation group.

15. The information processing device described in claim 14, characterized in that the consistency acquisition means calculates the group consistency representing the consistency relationship between the reference deformity group and the correction candidate group based on the aggregated data of the attribute information related to the deformities included in the reference deformity group and the aggregated data of the attribute information related to the deformities included in the correction candidate group.

16. The information processing device described in claim 15, characterized in that the consistency acquisition means calculates the group consistency using at least one of the following aggregated data: the degree of consistency between the attribute information of the reference deformity group and the attribute information of the correction candidate group, the degree of change between the number of attribute information of the reference deformity group and the number of attribute information of the correction candidate group, and the degree of consistency of the change rate between the attribute information of the reference deformity group and the attribute information of the correction candidate group.

17. A selection means for selecting at least one of the first deformations associated with the first image as a reference deformation; A candidate creation means for creating at least one deformation as a second deformation from a second image taken at a time different from the first image using a learned model, and further creating a deformation corresponding to the reference deformation among the second deformations as a correction candidate; A consistency degree acquisition means for acquiring a consistency degree representing a consistency relationship between the reference deformation and the correction candidate; A deformation correction means for correcting attribute information of the correction candidate based on the consistency to create a corrected deformation; A progress acquisition means for acquiring a progress degree representing a change in the deformation from the reference deformation based on a comparison between attribute information of the reference deformation and attribute information of the corrected deformation; 13. An information processing device comprising:

18. The attribute information is information that represents the properties of the deformation, and includes at least one of the following information: extension, area, position, width, coordinates, and connection relationship of the deformation. An information processing device as described in any one of claims 7 to 12 and claims 14 to 17.

19. The information processing device according to claim 1 , wherein the deformation includes a crack occurring on a surface of a structure.

20. An information processing method executed by an information processing device, A selection step of selecting at least one deformation among the first deformations associated with the first image as a reference deformation, and selecting at least one deformation among the second deformations associated with the second image taken at a time different from the first image as a deformation to be corrected; A candidate creation step of creating a correction candidate by correcting the deformation to be corrected; A consistency acquisition step of acquiring a consistency degree representing a consistency relationship between the reference deformation and the correction candidate; A deformation correction step of correcting the deformation to be corrected based on the consistency to create a corrected deformation; A progress acquisition step of acquiring a progress degree representing a change in the deformation from the reference deformation based on a comparison between the reference deformation and the corrected deformation; 13. An information processing method comprising:

21. An information processing method executed by an information processing device, A selection step of selecting a plurality of deformations among the first deformations associated with the first image as a reference deformation group, and selecting a plurality of deformations among the second deformations associated with the second image taken at a time different from the first image as a correction target deformation group; A candidate creation step of creating a correction candidate group by modifying the correction target deformation group; A consistency degree acquisition step of acquiring a group consistency degree representing a consistency relationship between the reference deformation group and the correction candidate group; A deformation correction step of correcting the deformation group to be corrected based on the group consistency degree to create a corrected deformation group; A progress acquisition step of acquiring a group progress degree representing a change from the reference deformation group based on a comparison between the reference deformation group and the corrected deformation group; 13. An information processing method comprising:

22. An information processing method executed by an information processing device, A selection step of selecting at least one of the first defects associated with the first image as a reference defect; A candidate creation step of creating at least one deformation as a second deformation from a second image taken at a different time from the first image using a learned model, and further creating a deformation corresponding to the reference deformation among the second deformations as a correction candidate; A consistency acquisition step of acquiring a consistency degree representing a consistency relationship between the reference deformation and the correction candidate; A deformation correction step of correcting attribute information of the correction candidate based on the consistency to create a corrected deformation; A progress degree acquisition step of acquiring a progress degree representing a change in the deformation from the reference deformation based on a comparison between attribute information of the reference deformation and attribute information of the corrected deformation; 13. An information processing method comprising:

23. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 19.

Images