Information processing device, control method for information processing device, and program

Abstract

To efficiently determine the state of changes over time of deformations even when a number of deformations are present in a structure.SOLUTION: An information processing apparatus comprises: determination means that determines a partial area of a first image that is an image of a structure in a first period, and determines a partial area corresponding to the partial area in the first image in a second image that is an image of the structure in a second period after the first period; and generation means that, on the basis of a first deformation included in the partial area of the first image and a second deformation included in the partial area of the second image, generates display data for displaying the degree of progress of the second deformation from the first deformation.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] This invention relates to a technique for determining structural deformation. [Background technology]

[0002] In inspecting structures such as bridges and tunnels, it is necessary to understand how much deformation (age-related changes) has progressed on the structural walls, such as cracks and exposed rebar, in order to determine the structural integrity of the structural members. In particular, when determining structural integrity, the progression of deformation that has a significant impact on the function of the structure, such as wide cracks and widespread exposed rebar, is given importance.

[0003] In image-based inspection of infrastructure structures, deformations are detected from images of the structure's wall surface taken at different times, and the degree of progression of each deformation is calculated by determining the difference between the detected deformations. Patent Document 1 describes treating cracks with similar feature quantities across images taken at different times as the same crack and determining its progression. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2019-20220 [Overview of the project] [Problems that the invention aims to solve]

[0005] Because shooting conditions such as shooting location and weather change with each shot, even images of the same wall surface of the same structure will contain positional shifts in each pixel between images. Therefore, in order to determine the state of deformation over time, it is necessary to determine the correspondence between deformations while taking these shifts into account. However, a great many deformations occur on the walls of structures that have aged over time. Therefore, determining the correspondence between all deformations detected from images taken at different times, while taking into account the shifts between images, requires a great deal of time.

[0006] This invention has been made in view of the above problems, and aims to provide a technology for efficiently determining the state of deterioration over time. [Means for solving the problem]

[0007] To solve this problem, for example, the information processing apparatus of the present invention has the following configuration. That is, A selection means for selecting some of the deformations included in a first image, which is an image of the structure at a first time period, as targets for determining the state of change over time, based on information relating to the deformation of the structure, information relating to the structure, and at least one of two or more pieces of information relating to the deformations, A determination means for determining a deformation corresponding to a deformation selected by the selection means from among a plurality of deformations contained in a second image, which is an image of the structure at a second time different from the first time; A determination means that determines the state of age-related change between the deformation selected by the selection means and the deformation determined by the determination means, based on the deformation information selected by the selection means and the deformation information determined by the determination means, The aforementioned Subregion of the first image And, as stated above In the second image keru , a subregion corresponding to the subregion of the first image and to decide 2nd Decision-making means and Based on the first deformation included in the partial region of the first image and the second deformation included in the partial region of the second image, Deformations included in a subregion Progress and the deformation selected by the selection means A generation means for generating display data for displaying, It is equipped with. [Effects of the Invention]

[0008] According to the present invention, even when numerous deformations exist in a structure, the state of how these deformations change over time can be efficiently determined. [Brief explanation of the drawing]

[0009] [Figure 1]A diagram for explaining the outline of the present invention. [Figure 2] Hardware configuration diagram and functional block diagram. [Figure 3] A diagram for explaining the relationship between variant data and images. [Figure 4] A flowchart showing the processing content of the first embodiment. [Figure 5] A diagram for explaining an example of selecting important variants. [Figure 6] A diagram for explaining an example of setting a search range. [Figure 7] A diagram for explaining an example of calculating a degree of consistency. [Figure 8] A diagram for explaining an example of determining reference variants. <000-0097> [Figure 9] Functional block diagram of the second embodiment. [Figure 10] A flowchart showing the processing content of the second embodiment. [Figure 11] A diagram for explaining the correspondence relationship of variant groups. [Figure 12] A diagram for explaining the progress of variant groups. [Figure 13] Functional block diagram of the third embodiment. [Figure 14] A flowchart showing the processing content of the third embodiment. [Figure 15] A diagram for explaining variant groups of different sizes.

Mode for Carrying Out the Invention

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential for the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0011] [First Embodiment] As a first embodiment, an example of a method for calculating the progression of deformation of an object under inspection using two images taken at different times will be described. In particular, this embodiment will be described using an information processing device for performing so-called infrastructure inspections, such as determining the soundness of structures like bridges, as an example. The terms used in the description of this embodiment are defined as follows.

[0012] In the context of an information processing system used for infrastructure inspection, "inspection target" refers to concrete structures, etc. The user of the information processing system described in this embodiment aims to inspect the surface of the inspection target for any abnormalities such as cracks, based on images captured by the system. "Abnormalities" refer to, for example, cracks, delamination, or spalling of concrete in the case of concrete structures. Other examples include efflorescence, exposed rebar, rust, water leakage, water dripping, corrosion, damage (missing parts), cold joints, deposits, and honeycombing.

[0013] "The first image" and "the second image" are images of the same wall surface of the same structure taken at different times. In this embodiment, the time when the first image was taken is defined as a time several years after the time when the second image was taken.

[0014] The outline of this embodiment will be explained using an example of calculating the progression of cracks, which indicates the state of cracking over time. Figures 1(a) and 1(b) show images 101 and 102 of the wall surface of a bridge, taken at different times, as an example of an infrastructure structure. Image 101 was taken after several years had passed since image 102. Numerous cracks occur on the wall surface of infrastructure structures due to aging. Therefore, many cracks are present in the images 101 and 102. Figure 1(c) shows crack data 111 corresponding to image 101, and Figure 1(d) shows crack data 112 corresponding to image 102. The dashed and solid lines in the crack data 111 and 112 indicate fine cracks and thicker cracks with different widths. Crack data is data recorded by, for example, an inspection engineer who inputs the location and shape of deformations such as cracks and exposed rebar based on the image, and records it in correspondence with the image. Another method for obtaining crack data is to detect cracks from images using a model pre-trained by machine learning. Note that images 101, 102, and crack data 111, 112 include numerous cracks not depicted on the paper.

[0015] To assess the structural integrity of a component, the degree of crack progression on its surface is examined. Calculating the degree of crack progression requires comparing individual cracks from different time periods and determining the differences in crack length, width, etc. This comparison includes positional and morphological discrepancies due to differences in photographic conditions, as well as differences resulting from the progression of the cracks themselves over time. Therefore, determining the correspondence between cracks while considering these discrepancies and differences is necessary to calculate the degree of crack progression.

[0016] Performing this correspondence analysis for all cracks indicated by crack data 111 would be extremely time-consuming. Therefore, the correspondence analysis is limited to cracks that should be given importance in determining structural integrity. For example, from crack data 111, two large cracks 121 and 122 that are highly likely to affect the function of the member are selected. Then, the corresponding cracks for the selected cracks 121 and 122 are found in crack data 112. One method for determining the correspondence between cracks is to determine it based on the degree of overlap when the shapes of the cracks are superimposed. Figure 1(e) shows crack 121 superimposed on crack data 112, with the region 131 enlarged. From Figure 1(e), it can be determined that crack 132 has the highest degree of overlap with crack 121. Subsequently, the degree of crack progression is calculated using cracks 121 and 132. To determine the progression of the crack, for example, the difference between the total length of the crack and the maximum crack width can be calculated.

[0017] As described above, even when numerous cracks occur on the surface of a component, the degree of crack progression can be efficiently calculated by selecting only the cracks that are important in determining the structural integrity.

[0018] <Information Processing Device> Figure 2(a) is a hardware configuration diagram of the information processing device 200 according to this embodiment. As shown in Figure 2(a), the information processing device 200 includes a CPU 201, ROM 202, RAM 203, HDD 204, display unit 205, operation unit 206, and communication unit 207. The CPU 201 is a central processing unit that performs calculations and logical decisions for various processes and controls each component connected to the system bus 208. The ROM (Read-Only Memory) 202 is a program memory that stores programs for control by the CPU 201, including various processing procedures described later. The RAM (Random Access Memory) 203 is used as the main memory, work area, and other temporary storage area of ​​the CPU 201. Program memory can also be realized by loading a program into the RAM 203 from an external storage device connected to the information processing device 200.

[0019] HDD204 is a hard disk for storing electronic data and programs according to this embodiment. An external storage device may be used to perform a similar role. Here, the external storage device can be implemented, for example, by a media (recording medium) and an external storage drive for accessing the media. Examples of such media include flexible disks (FD), CD-ROMs, DVDs, USB memory, MOs, flash memory, etc. The external storage device may also be a server device connected via a network.

[0020] The display unit 205 is, for example, a CRT display or a liquid crystal display, and is a device that outputs an image to the display screen. The display unit 205 may also be an external device connected to the information processing device 200 by wire or wireless. The operation unit 206 has a keyboard and mouse and accepts various operations from the user. The communication unit 207 performs two-way communication by wire or wireless with other information processing devices, communication equipment, external storage devices, etc., using known communication technologies.

[0021] <Functional Block Diagram> Figure 2(b) is an example of a block diagram showing the functional configuration of the information processing device 200. Each of these functional units is realized by the CPU 201 loading the program stored in the ROM 202 into the RAM 203 and executing the processing according to the flowcharts described later. The results of each processing are then stored in the RAM 203. For example, if hardware is to be configured as an alternative to software processing using the CPU 201, then arithmetic units and circuits corresponding to the processing of each functional unit described here should be configured.

[0022] The management unit 222 manages images taken at different times, deformation data corresponding to each image, and structural information related to the structure, which are stored in the storage unit 221 (e.g., HDD 204). The important deformation selection unit 223 selects deformation data to be used for progression calculation as important deformations from the first deformation data. The search range setting unit 224 sets the search range from the second deformation data for deformations corresponding to the important deformations selected by the important deformation selection unit 223. The consistency calculation unit 225 performs a process to calculate the consistency using the deformations in the search range and the important deformations. The reference deformation determination unit 226 determines the deformations corresponding to the important deformations as reference deformations based on the consistency calculation result. The progression calculation unit 227 performs a process to calculate the difference between the important deformations and the reference deformations as the progression.

[0023] <Explanation of the relationship between images and deformation data, and structural information> In describing this embodiment, the relationship between images and deformation data, and structural information will be explained. In image inspection, it is preferable to manage images of the wall surface of a structure in correspondence with its design drawings. Figure 3(a) shows an image 311 of the wall surface of a bridge, as an example of an infrastructure structure, pasted onto a drawing 300. Drawing 300 has drawing coordinates 301 with point 302 as the origin. The position of an image on the drawing is defined by the coordinates of the upper left vertex of the image. For example, the coordinates indicating the position of image 311 on the drawing are the position of vertex 312 (x312, y312). The image is stored in the storage unit 221 along with the coordinate information. In this embodiment, the images used for image inspection of infrastructure structures are taken at high resolution (e.g., 1 mm per pixel) so that fine cracks and other defects can be confirmed. For this reason, the size of the images of structures used for inspection is large. For example, image 311 in Figure 3(a) is an image of the deck slab of a 20 m x 10 m bridge. If the image resolution is 1.0 mm per pixel (1.0 mm / pixel), the image size of image 311 will be 20,000 pixels × 10,000 pixels. Image 311, captured at high resolution, contains numerous (e.g., more than 1000) cracks, exposed rebars, and other deformations. However, it is difficult to represent all deformations on paper, so only a portion of the deformations are shown. In subsequent explanations, even in diagrams showing wide-area images and deformation data, only a portion of the deformations are displayed.

[0024] Deformation data is information that records the results of automatic detection of deformations such as cracks occurring on a concrete wall surface, or the results of human input. For the purposes of describing this embodiment, it is assumed that the deformation data is managed in association with a drawing. Figure 3(b) shows the deformation data 321 corresponding to image 311 pasted onto drawing 300 at the same position as image 311. Deformation data 321 contains a large number of deformations (e.g., more than 1000), including deformations that are not displayed on the paper. The position of each deformation data in deformation data 321 on the drawing is defined by the pixel coordinates that make up the deformation data. Figure 3(c) shows an example of a deformation data table 331 that represents the data structure of the deformation data. In this deformation data table 331, the first and second fields indicate the ID for identifying the deformation and the type of deformation. In addition, the coordinate columns and numerical columns of the third and fourth fields of the deformation data table 331 are attribute values ​​that represent the multiple coordinates that make up the deformation data and the width of the deformation at those coordinates. For example, crack C001 is represented by a series of pixels consisting of n points from (Xc001_1, Yc001_1) to (Xc001_n, Yc001_n). In addition, the 5th to 7th fields in the deformation data table 331 store the maximum width, total length / contour length, and area.

[0025] Thus, in this embodiment, deformation data is represented by pixels. Deformation data may also be represented by vector data such as polylines or curves composed of multiple points. When deformation data is represented by vector data, the data size is reduced and the representation becomes simpler. An example of deformation data other than cracks is exposed rebar with ID T001. When exposed rebar is represented by a polyline, the deformation will have an area enclosed by the polyline. Note that the attribute information of the deformation data is not limited to the attribute information shown in the deformation data table 331, and other attribute information may also be held.

[0026] Structural information refers to information relating to the structure of the structure being inspected, and includes specifications such as the type of structure, basic structure, various dimensions of the structure, member information, and completion year. Furthermore, it may also include maintenance information such as repair year, repair location, and repair method as repair history. In this embodiment, structural information relating to specific locations of the structure, such as member information and repair information, is stored together with the location information on the drawing. That is, the location of each member on the drawing and the location of repair locations on the drawing are stored as part of the structural information. Therefore, the correspondence between structural information, images, and deformation data can be determined via the drawing. Structural information is stored in the storage unit 221 together with images and deformation data and can be retrieved by the management unit 222. Note that the information included in structural information is not limited to the above information, and other information may be held. Also, depending on the type of structure, information limited to each type may be held.

[0027] <Flowchart> Figure 4 is a flowchart showing the main processing flow of the information processing device in this embodiment. Hereafter, each process (step) will be described by adding an S to the beginning of its reference numeral. In this embodiment, first and second images taken at two different times are used to calculate the progression of deformation occurring on the wall surface of a structure. The first image is taken several years (for example, 5 years) after the second image was taken. Both the first and second images are images that capture a large number (e.g., 1,000 or more) of deformations occurring on the wall surface of the structure. Data from which deformations are detected from the first image is used as the first deformation data, and data from which deformations are detected from the second image is used as the second deformation data. In this embodiment, the operation unit 206 receives an input to start processing and starts the processing shown in the flowchart of Figure 4. When the processing of the progression is completed, display data showing the progress calculation result is displayed on the display unit 205, and the processing ends. Hereinafter, the processing of this embodiment will be described according to the flowchart of Figure 4.

[0028] <Selection of Important Changes> In S401, the important deformation selection unit 223 selects the deformations to be used for calculating the progression of the crack from the first deformation data corresponding to the first image as important deformations. One method for selecting important deformations is to select them based on the attribute information of the deformation data. In this embodiment, as an example of explaining the method for calculating the progression of a crack, a method of selecting important deformations from the first deformation data based on the maximum width of the deformation data will be explained.

[0029] Figure 5(a) shows the first image 501 pasted onto coordinate 503 on drawing 500, and Figure 5(b) shows the first deformation data 502 in the first image 501. The size of the first image 501 is 20,000 pixels × 10,000 pixels. The dashed and solid lines in the first deformation data 502 represent cracks with a maximum width of less than 1.0 mm and cracks with a maximum width of 1.0 mm or more, respectively. Numerous deformations exist in the first image 501 and the first deformation data 502 that are not represented on the paper.

[0030] One method for selecting important deformations from the first set of deformation data 502 is to select deformation data with the largest width exceeding a standard value as important deformations to be used for determining the state of change over time. The formula for this determination is shown in equation (1) below. W≦W c …(1) Here, parameter Wc is the maximum width value for each deformation data, and parameter W is the criterion value (or threshold) for determining whether or not a deformation is significant. One method for determining parameter W in this embodiment is to use a uniform criterion value predetermined for the entire structure for each deformation type as the threshold. For example, if the criterion value W for the deformation type being a crack is set to 1.0 mm, then in deformation data 502, solid line deformations 511 and 512, which show cracks of 1.0 mm or more, will be selected as significant deformations. Another method for determining parameter W is to use a criterion value predetermined for each structural information. Figure 5(c) shows an example of a list of different criterion values ​​521 for each structure in a bridge. The correspondence between image / deformation data and the structure of the structure can be determined through drawings. Therefore, the criterion value for selecting significant deformations can be switched according to the structure being inspected. These criterion values ​​may be experimentally determined values, or they may be determined based on a past database of accumulated inspection results. Furthermore, another method for determining parameter W may be to allow the user to specify it directly. When a user specifies a reference value, it is desirable that they be able to specify a range for that reference value. For example, the reference value may be specified on an image-by-image basis or on a partial image basis, or it may be specified for each piece of structural information (e.g., per component, per structure).

[0031] Another method for selecting important changes based on maximum width is to select, for example, the top 10 change data with the largest maximum width as important changes. Fixing the number of important changes to select can suppress the increase in processing time for calculating progress. Alternatively, one could select the top 1% of change data in descending order of maximum width as important changes.

[0032] Up to this point, we have explained an example of selecting significant damage using maximum width, which is one of the attribute pieces of information. However, the attribute pieces of information used are not limited to maximum width. For example, cracks with a length greater than or equal to a standard value can also be selected as significant damage based on the total length, which indicates the crack length. The formula (1) for determining whether or not a crack is significant, and the method for determining the standard value of crack length (parameter W), can be the same as when using maximum width.

[0033] Another method for selecting important deformations is to select composite deformations that combine multiple deformation data. As an example of a composite deformation, Figure 5(d) shows a grid-like crack 531, which is a combination of cracks of the same type. When multiple cracks intersect to form a grid, the concrete wall surface in the area 532 enclosed by the diagonal lines becomes more prone to peeling. As a result, the possibility of concrete falling or deterioration inside the structure due to exposed rebar increases. Therefore, it is desirable to select composite deformations consisting of multiple identical deformations, such as the grid-like crack 531, as important deformations. Any method can be used to identify grid-like cracks. For example, when multiple cracks whose shapes partially overlap are selected in a chain-like manner to form a closed region, the selected multiple cracks can be identified as grid-like cracks. Another type of composite deformation that combines multiple deformation data is a composite deformation that combines deformations of different types. For example, there is a composite deformation 533 shown in Figure 5(e) in which efflorescence occurs near the cracks. Cracks accompanied by precipitates may be progressing into the interior of the structure and have a high impact on the structure's function. Therefore, it is desirable to select deformations such as composite deformation 533 as important deformations. Composite deformations such as cracks accompanied by precipitates can be identified based on the type of deformation, the shape of the deformation, and the relative positional relationship. Another method for selecting important deformations is to use structural information. For example, in a wall surface where reinforcing bars are laid near the surface of the structure, if the concrete of the wall surface delaminates, the reinforcing bars will be exposed, and the deterioration of the structure will progress more easily. Therefore, it is desirable to select deformations near wall surfaces where structural members such as reinforcing bars are laid inside as important deformations. Since information on structural members such as reinforcing bars and member materials is stored in the storage unit 221 as structural information, the relative positional relationship between images and deformation data and the reinforcing bars inside the structure can be determined from drawings.

[0034] <Setting the search range> In S402, the search range setting unit 224 sets a search range for searching for deformations corresponding to important deformations from the second deformation data. The method for setting this search range will be explained with reference to Figure 6.

[0035] Figure 6(a) shows the first deformation data 611, and Figure 6(b) shows the second deformation data 621. Both deformation data are pasted at the same position on drawing 600. Of the first deformation data 611, deformations 612 and 613 shown with solid lines are designated as important deformations by the important deformation selection unit 223.

[0036] One method for setting the search range is to use the coordinate information of important deformations. Specifically, the rectangular areas 614 and 615 surrounding important deformations 612 and 613 on the first deformation data 611 are set as the search range for each important deformation. The size of the search range can be changed to any size as long as it is a rectangular area that encloses the coordinate information of the important deformations. For example, the initial range is set to the rectangular area that encloses the important deformations, and the search range is set to a rectangular area that is extended by an arbitrary amount in the X and Y axis directions on the drawing.

[0037] If the search range is determined using only the coordinate information of the important deformation, a portion of the second deformation data may be fragmented near the boundary of the search range. Figure 6(c) shows an example of fragmentation of a portion of the second deformation data, specifically the search range 632 determined using a portion of the second deformation data 621 (area 631) and the coordinate information of the important deformation 612. In Figure 6(c), the search range 632 fragments deformations 633 and 634 near the boundary. Setting a search range that fragments the second deformation data complicates the consistency calculation process in S403, which will be described later. Therefore, the rectangular range is expanded to include all the coordinates of deformations 633 and 634. Figure 6(d) shows the expanded search range 635. Expanding the search range in this way, without fragmenting the second deformation data, is a preferred method.

[0038] <Calculation of consistency> In S403, the consistency calculation unit 225 performs a process to calculate the consistency using the important deformation and the deformation within the search range. One method for calculating the consistency is to expand the shapes of one deformation within the search range and the important deformation, or one of them, and calculate the degree of overlap between the deformations as the consistency. An example of the consistency calculation process will be explained with reference to Figure 7.

[0039] Figure 7(a) shows the important deformation 701 selected by the important deformation selection unit 223 from the first deformation data. Figure 7(b) shows the deformation data 711 included in the search range corresponding to the important deformation 701 from the second deformation data. In calculating the consistency, first, one deformation 712 is selected from the deformation data 711 as a reference deformation candidate. Next, the deformation 712 and the important deformation 701 are superimposed. At this time, in order to take into account the positional displacement between the first deformation data and the second deformation data, the deformation shapes are expanded before superimposing. Figure 7(c) shows the expanded important deformation 702 and the expanded reference deformation candidate 713 superimposed. Then, the degree of overlap between the important deformation 702 and the reference deformation candidate 713 is calculated as the consistency. This consistency C is calculated, for example, according to the following equation (2). C = {S1Λ S2} / {S1V S2} …(2) In the formula, "Λ" represents logical AND and "V" represents logical OR. Parameter S1 represents the area of ​​the significant deformation 702 after expansion, and parameter S2 represents the area of ​​the reference deformation candidate 713 after expansion. From formula (2), if the deformations coincide, the consistency score C is 1, and if the deformations do not overlap at all, the consistency score C is 0. That is, when calculating the consistency score based on the degree of overlap between deformations, the possible range for the consistency score is 0 to 1.

[0040] After calculating one consistency score using equation (2), the important deformation remains fixed, and another deformation is selected from the deformation data within the search range as the next reference deformation candidate, and the consistency score is calculated. By repeating this consistency score calculation process, the consistency score for all deformation data within the search range is calculated.

[0041] As another method for calculating the degree of integration, feature values calculated from deformation data may be used. As an example, the calculation formulas for calculating the degree of integration C using the area centroid coordinates of the deformation are shown in Formulas (3) and (4). C = 1 / {(S 1x - S 2x ) 2 + (S 1y - S 2y ) 2} 1 / 2 …(3) C = C c …(4) Here, the parameters S 1x , S 1y represent the area centroid coordinates (S 1x , S 1y ) of the important deformation, and the parameters S 2x , S 2y represent the area centroid coordinates (S 2x , S 2y ) of the reference deformation candidate. The parameter C c is a fixed value when the area centroid coordinates of the deformations match, and an arbitrary constant greater than 1 is specified. From Formula (3), the farther the distance between the area centroids of the deformations, the smaller the degree of integration C becomes. Conversely, when the area centroids of the deformations match, the degree of integration is the highest constant C c .

[0042] <Determination of Reference Deformation> In S404, the reference deformation determination unit 226 performs a process of determining a reference deformation corresponding to the important deformation based on the calculation result of the degree of integration. An example of determining the reference deformation using the degree of integration will be described with reference to FIG. 8. Here, the degree of integration used in this embodiment is the degree of integration calculated based on the overlapping degree of the deformations. Therefore, the range of values that the degree of integration can take is 0 to 1, and the closer the value is to 1, the higher the matching relationship between the deformations.

[0043] Figures 8(a) and (b) show the important deformation 801 for which consistency is calculated, and the deformation data 802 within the search range corresponding to the important deformation 801. Near each deformation on the deformation data 802, an ID consisting of a number starting with "CR" is also indicated. Figure 8(c) shows the consistency list 803 calculated for the important deformation 801 and each deformation (reference deformation candidate) on the deformation data 802. One method for determining a reference deformation based on consistency is to determine the deformation with the highest consistency as the reference deformation. For example, in the consistency list 803 in Figure 8(c), the deformation with the highest consistency (ID: CR001) is determined as the reference deformation. In cases where there is only one maximum consistency value, as in the consistency list 803, the reference deformation can be uniquely determined using consistency.

[0044] On the other hand, as shown in Figure 8(d), if there are multiple deformations with the maximum consistency, it is not possible to uniquely determine the reference deformation. In such cases, it is desirable to generate image data showing the multiple deformations with the maximum consistency and the important deformation, display it on the display unit 205, and allow the user to determine the reference deformation.

[0045] In the consistency calculation results, the maximum consistency value may be close to 0 (e.g., 0.1). In this case, while it is possible to uniquely determine the reference deformation based on the consistency value, there is a high possibility that it is not suitable as a reference deformation in the first place. In such cases, if the calculated maximum consistency value is lower than a predetermined standard value (e.g., 0.2), it is desirable to generate image data showing the reference deformation and important deformation along with the maximum consistency value, and display it on the display unit 205 to allow the user to confirm, correct, and select the reference deformation.

[0046] If all calculated consistency scores are 0, or if no reference deformation is specified as a result of user confirmation, it is determined that "there is no reference deformation corresponding to the important deformation." In this case, in the progress calculation process in S405 described later, it is determined that the important deformation itself has progressed. Note that although an example of determining the reference deformation based on consistency scores was explained in S404, it is not limited to this. For example, the deformation included in the second image that has the shortest distance from the coordinate position of the important deformation may be determined as the reference deformation. Alternatively, the deformation may be determined based on the user's selection.

[0047] <Calculation of progress> In S405, the progression calculation unit 227 calculates the progression of a major deformation using the major deformation and the reference deformation. In this embodiment, the difference in attribute values ​​of the deformation data is used to determine the progression. For example, if the deformation type is a crack, the difference in maximum width and total length is calculated. The maximum width and total length of the crack are stored in the storage unit 221. Therefore, by obtaining the maximum width and total length for both the major deformation and the reference deformation and finding the difference between the deformations, the progression of the major deformation can be calculated. The progression can be calculated in the same way even if the deformation type is not a crack. For example, if the deformation type is exposed reinforcement, the difference in area is calculated as the progression. The area of ​​the deformation data is stored as an attribute value in the storage unit 221. Therefore, by obtaining the area of ​​the major deformation and the reference deformation and finding the area difference between the deformations, the progression can be calculated. In S405, a configuration in which the progression is determined as information indicating the state of change in the deformation over time has been illustrated, but the system is not limited to this. For example, in S405, the information processing device 200 may be configured to obtain binary information such as "progressing" or "not progressing" as information indicating the state of aging.

[0048] Then, in S406, the progress calculation unit 227 outputs the calculated progress to the display unit 205, displays the calculated progress, and shows the progress to the user. There are no particular restrictions on the display format of the progress, but it may be a numerical value representing the progress, or the deformation data may be color-coded according to the progress and displayed.

[0049] <Variation of the first embodiment> Up to this point, the first embodiment has described a method for selecting important deformations from first deformation data and calculating the progression by finding reference deformations corresponding to the important deformations from second deformation data. However, an embodiment in which important deformations are selected from second deformation data is also conceivable. That is, important deformations are selected from second deformation data corresponding to past inspection results, reference deformations corresponding to the important deformations are found from first deformation data, and the progression is calculated. This makes it possible, for example, to check how the deformations that were emphasized in the judgment of soundness in past inspection results have progressed.

[0050] According to the first embodiment described above, even if there are many defects such as cracks detected from images taken at different times, the degree of progression can be efficiently calculated by limiting the analysis to the defects that are important in determining the degree of structural integrity.

[0051] [Second Embodiment] In the first embodiment, an example was described in which the progression of some important deformations was determined from among many deformations. When determining the soundness of a structure, it is sometimes necessary to check the overall trend of deformation progression in addition to the progression of some deformations. However, it is difficult to grasp the overall trend of deformation progression from the progression of only some deformations. Therefore, in the second embodiment, an example is described in which the progression of deformation data for each small region is calculated in addition to the progression of some deformations. Specifically, the image is divided, the deformation data is grouped for each divided region, and the progression is calculated for each group. This makes it possible to determine the soundness of a structure or member while checking both the progression of some deformations and the overall progression of deformation. The second embodiment will be described below, focusing on the differences from the first embodiment.

[0052] The hardware configuration of the information processing device 200 according to the second embodiment is the same as the configuration of the first embodiment shown in Figure 2(a), so its description is omitted. Figure 9 is a diagram showing an example of a functional block diagram of the information processing device 200 according to the configuration of the second embodiment. The configuration of the second embodiment is the same as the configuration of Figure 2(b) of the first embodiment, with the addition of a first group setting unit 901 and a second group setting unit 902. The first group setting unit 901 is a functional unit of the CPU 201 that groups a part of the first deformation data to create a first deformation group. The second group setting unit 902 is a functional unit of the CPU 201 that creates a second deformation group corresponding to the first deformation group. In addition, the progress calculation unit 227 performs a process to calculate the progress of some deformations, as well as a process to calculate the progress of deformation groups using the first deformation group and the second deformation group.

[0053] Figure 10 is a flowchart showing a part of the main processing performed by the information processing device 200 according to the second embodiment. In the flowchart of Figure 10, steps numbered the same as those in the flowchart of Figure 4 described in the first embodiment should be understood as performing the same processing as in the first embodiment. In the case of the second embodiment, after calculating the progress of the significant deformation in S405, the process proceeds to S1001.

[0054] In S1001, the first group setting unit 901 performs the process of setting the first deformation group from the first deformation data. In the following S1002, the second group setting unit 902 sets the second deformation group corresponding to the first deformation group. Then, proceeding to S1003, the progress of the deformation group is calculated using the first deformation group and the second deformation group. As the progress of the deformation group, for example, the number of deformations or the total length of deformations within the deformation group is aggregated, and the difference in aggregated data between deformation groups is used as the progress. Then, in S406, the progress calculation unit 227 performs the process of displaying the calculated progress on the display unit 205, and the process ends.

[0055] <Settings for Group 1 and Group 2> Next, referring to Figure 11, the setting of the first deformation group in S1001 and the setting of the second deformation group in S1002 will be explained in order. The deformation data in this second embodiment is cracks on the surface of the structure, as in the first embodiment. The first image, the second image, and the deformation data corresponding to each image are all pasted at the same position on the same drawing. Figure 11(a) shows the first image 1101, and Figure 11(b) shows the first deformation data 1102 showing the deformation contained in the first image 1101. Also, Figure 11(c) shows the second image 1121, and Figure 11(d) shows the second shape data 1122 showing the deformation contained in the second image 1121.

[0056] In the S1001 process, the first group setting unit 901 sets a first deformation group from the first deformation data. An example of setting a first deformation group is described in which the first image is divided and the first deformation data is grouped for each divided region. First, the first group setting unit 901 divides the first image 1101 into arbitrary fixed-size sections. The grid-like line segments 1111 divide the first image 1101 at equal intervals in the X and Y directions, for example, at intervals of 1,024 pixels. Next, the first group setting unit 901 selects one of the divided regions 1112 enclosed by diagonal lines and obtains the coordinates of the top-left vertex 1113 and the size of the region 1112. Then, the first group setting unit 901 sets a diagonal-lined region 1114 on the first deformation data 1102 at the same position and size as region 1112. Subsequently, the deformation data contained in the shaded area 1114 is grouped to form the first deformation group 1115. By repeating the above process for each divided area, the first deformation group can be set up for the entire area. Here, an example of dividing the first image 1101 at equal intervals has been described, but it may also be divided at arbitrary intervals. For example, it may be divided at intervals of 1,024 pixels in the X-axis direction and 512 pixels in the Y-axis direction to create the first deformation group in units of rectangular areas.

[0057] In the subsequent S1002 process, the second group setting unit 902 sets a second deformation group corresponding to the first deformation group. One method for setting a second deformation group is to group the second deformation data that is included in the same location and range as the first deformation group. Here, an example of setting a second deformation group corresponding to the first deformation group 1115 will be explained. First, the second group setting unit 902 obtains the coordinates of the upper left vertex 1116 and the area size of the shaded area 1114. Next, the second group setting unit 902 sets a shaded area 1131 on the second deformation data 1122 in Figure 11(d) that is in the same location and range as the shaded area 1114. Then, it groups the second deformation data included in the shaded area 1131 to form the second deformation group 1132. In this way, a second deformation group can be set by associating groups of deformation data that are in the same location and range with each other.

[0058] Normally, because the shooting conditions differ when images are taken at different times, even when photographing the same wall surface of the same structure, there will be a positional shift between each pixel in the first image and the second image. Therefore, there will also be a positional shift between the first deformation data and the second deformation data. The example above ignores this positional shift and simply creates the first and second deformation groups. The degree of deformation between deformation groups is intended to capture the general trend of progression, so it is not necessary to strictly match the deformation groups. This simple method of creating the first and second deformation groups simplifies the group creation process.

[0059] On the other hand, when determining the correspondence between deformation groups, positional shifts may be taken into consideration. As an example of a method for obtaining positional shifts between images, the outline of a process for obtaining region-specific positional shifts using image features will be explained. First, one divided region is selected on the first image, and a partial image within that region is obtained. Next, the obtained partial image is superimposed on the second image, and the sum of the squares of the brightness value differences of each pixel is calculated. This calculation process is repeated while shifting the superimposition position, and the position where the calculation result is minimized, i.e., the region-specific positional shift, is found. By repeating this process for each divided region of the first image, the positional shifts between images can be obtained on a region-by-region basis throughout the entire image. As an example, Figure 11(e) shows the result of setting the second deformation group from the first deformation group while reflecting the positional shift. Figure 11(e) shows the second deformation group 1141 obtained by correcting the positional shift from the first deformation group 1115. In this way, a second deformation group that takes into account the positional shifts between images can be set.

[0060] Another method for obtaining the positional displacement between deformation groups is to determine it based on the coordinate information of the significant deformations. For example, select at least one significant deformation included in the first deformation group, and use the average of the area centroid coordinates of the significant deformation as the starting point of the positional displacement. Then, use the average of the area centroid coordinates of the reference deformation corresponding to the selected significant deformation as the ending point of the positional displacement. The vector connecting these starting and ending points is taken as the positional displacement of the first deformation group. In this way, by substituting the positional displacement of significant deformations for the positional displacement between deformation groups, a second deformation group that takes positional displacement into account can be efficiently determined.

[0061] <Calculation process for the progression of the deformation group> In S1003, the progression calculation unit 227 calculates the progression between the first and second deformation groups. In this embodiment, the progression is calculated using aggregated data obtained by aggregating the deformation data within each deformation group. For example, if the deformation type is cracking, the number of cracks, crack density, etc., are obtained as aggregated data for both the first and second deformation groups. The progression calculation unit 227 then calculates the difference in the aggregated data as the progression between the deformation groups.

[0062] Figures 12(a) and (b) show examples of aggregated data 1201 and progression 1202 between deformation groups when the deformation type is cracking. Aggregated data 1201 is the result of aggregating the number of cracks, number density, total length, and average maximum width for the first deformation group and the second deformation group, respectively. Of the aggregated data 1201, the number can be obtained by counting the number of deformation data for each deformation group. The number density can be calculated using the number of deformation data, as well as the area size and image resolution of the deformation group. The total length and average maximum width can be calculated by aggregating the attribute values ​​of each deformation data within the deformation group. The progression 1202 can be calculated by finding the difference in aggregated data between deformation groups.

[0063] Figure 12(c) shows, as an example, a visualization screen of the progression calculation results in the second embodiment. Figure 12(c) is an example of displaying the visualization result 1212, which visualizes the progression of important deformations and the progression of deformations for each region, on window 1211. Visualization result 1212 is a visualization of the progression related to the number density of cracks, and is represented by different patterns for each region according to the difference in progression. Visualization result 1212 shows that regions with higher pattern density are regions with higher progression, but it is preferable to represent each region with different brightness or different colors according to the progression. Cracks 1221 and 1222 on Figure 12(c) indicate important deformations, and near the important deformations, information related to the progression, such as progression information 1123, is also displayed near the cracks. In this way, by displaying the progression for each region and the progression of important deformations simultaneously, users can easily check the progression of important deformations and the overall progression trend of the deformations.

[0064] It is desirable that the user be able to switch the content of the progress visualization on the visualization screen. For example, the CPU 201 switches the deformation data to be displayed as progress in the visualization result 1212 according to the user's selection of the type selection item 1213. The CPU 201 switches the display target of the deformation data in the visualization result 1212 (e.g., display only important deformations, display all deformations) according to the user's switch of the display data selection item 1214. The CPU 201 switches the display target of the progress in the visualization result 1212 (e.g., number density, total length, average maximum width) according to the user's switch of the progress display target item 1215. In this way, it is preferable to allow the user to switch the content they want to check. It is also desirable that the progress of important deformations and the progress of each area be checked in detail. For example, an area can be selected using the progress display area selection 1216 or by clicking with the mouse 1224. Detailed progress information is displayed according to the area selection. For example, in Figure 12(c), progress information corresponding to the selected area "A4-2" is displayed in progress result 1217.

[0065] As described above, according to the second embodiment, it is possible to calculate the progression of deformation on a region-by-region basis, in addition to the progression of some deformations. This makes it possible to determine the overall progression of deformation for all deformation data without having to determine the consistency of each piece of data with past deformation data. Furthermore, for important deformation data, it becomes possible to grasp the progression from past deformation data in detail, such as the amount of change in crack width.

[0066] [Third Embodiment] In the second embodiment, an example was described in which the degree of deformation is calculated not only for some parts but also for each region. The deterioration status of a structure varies greatly from part to part due to various factors. Therefore, it is sometimes necessary to check the degree of deformation in detail for some parts. However, when calculating the degree of deformation using a fixed region size, it is necessary to reduce the region size in order to grasp the degree of deformation in detail, which significantly increases the processing time. Therefore, in the third embodiment, an example is described in which the degree of deformation is calculated using different region sizes for each part. By changing the region size for grouping deformations according to the part, it is possible to grasp the degree of deformation in detail for a specific part while suppressing the increase in processing time. The following describes this third embodiment, focusing on the differences from the second embodiment.

[0067] The hardware configuration of the information processing device 200 according to the third embodiment is the same as the configuration of the first embodiment shown in Figure 2(a), so its description is omitted. Figure 13 is a diagram showing an example of a functional block diagram of the information processing device 200 according to the configuration of the third embodiment. This third embodiment is a configuration in which a division size acquisition unit 1301 is added to the configuration shown in the second embodiment (Figure 9). The division size acquisition unit 1301 is a functional unit of the CPU 201 that acquires the size of the range to which a part of the first deformation data is grouped.

[0068] Figure 14 is a flowchart showing a part of the main processing performed by the information processing device 200 according to the third embodiment. In the flowchart of Figure 14, steps numbered the same as those in the flowchart of Figure 10 described in the second embodiment should be understood as performing the same processing as in the second embodiment. In this third embodiment, after calculating the progress of the significant change in S405, the process proceeds to S1401.

[0069] In S1401, the first group setting unit 901 divides the first deformation data and sets the first deformation group based on the division size acquired by the division size acquisition unit 1301. In the following S1002, the second group setting unit 902 sets the second deformation group corresponding to the first deformation group. Then, proceeding to S1003, the progress calculation unit 227 calculates the progress of the deformation group using the first deformation group and the second deformation group. Finally, in S406, the progress calculation unit 227 displays the progress calculation result on the display unit 205 and terminates this process.

[0070] <Setting up the first group with different sizes> Referring to Figure 15, the process of setting the first deformation group with different sizes in S1401 will be explained. In this embodiment, the deformation data is cracks on the surface of the structure, as in the second embodiment.

[0071] In S1401, the first group setting unit 901 divides the first deformation data based on the division size acquired by the division size acquisition unit 1301 and sets a first deformation group for each division area. One method for determining the division size of the first deformation data is to determine the division size based on structural information. Figure 15(a) shows an example of a list of different division sizes 1501 for each part of a bridge. Figures 15(b) and (c) show examples of dividing the first deformation data using the division size list 1501. Figure 15(b) shows the first deformation data 1502 corresponding to the bridge deck image and a grid-like division line 1511. The size of the shaded area 1512 in Figure 15(b) is 512 pixels × 512 pixels. Figure 15(c) shows the first deformation data 1503 corresponding to the bridge pier image and a grid-like division line 1521. The size of the shaded area 1522 in Figure 15(c) is 1,024 pixels × 1,024 pixels. The correspondence between each deformation data and structural information can be determined through the drawing. Therefore, by using structural information, the degree of deformation can be determined in detail for specific parts only. As a method for determining the division size, as shown in the division size list 1501, experimentally determined values ​​may be used, or division sizes specified by the user may be used.

[0072] Another method for determining the division size of the first deformation data is to determine it based on the distribution of deformations. Specifically, the distribution (density) of deformations in the first deformation data is determined in advance. Then, the first deformation data is divided into different sizes according to the distribution of deformations. Figure 15(d) shows an example in which the first deformation data 1531 is divided into different sizes according to the distribution of deformations. In the first deformation data 1531, the division size is small near the center where deformations are dense, and large near the edges where deformations are sparse. By changing the division size based on the distribution of deformations in this way, for example, the degree of deformation can be determined in detail only in areas where deformations are dense.

[0073] Another method for determining the division size of the first deformation data is to determine it based on the importance of the deformation. For example, the division size of the first deformation data can be made smaller near the importance of the deformation and larger at locations further away from the importance of the deformation. By understanding the progression of deformation around the importance of the deformation in detail, it becomes easier to judge the degree of impact of the importance of the deformation on the structural function and the likelihood of future progression of the importance of the deformation.

[0074] Another method for determining the division size of the first deformation data is to repeatedly divide it into multiple different sizes (dividing at multiple division levels) to create multiple first deformation groups with different division sizes. For example, the entire first deformation data can be divided into three division levels (1,024 pixels × 1,024 pixels, 512 pixels × 512 pixels, and 256 × 256 pixels), and a first deformation group can be set at each division level. Then, although the detailed processing is omitted, the degree of deformation is calculated for each divided area. By dividing the first deformation data at multiple division levels in this way, users can easily switch between detailed progress in a narrow range and a general progress over a wide range when checking the degree of deformation. Note that the range of the first deformation data to be divided at multiple division levels may be limited to an arbitrary range for each division level. For example, the first division level (1,024 pixels × 1,024 pixels) divides the entire first deformation data, while the second division level (256 × 256 pixels) divides only the area near the important deformations on the first deformation data. By limiting the range of the first deformation data for each division level, it becomes possible to check the detailed progress of only specific ranges while suppressing the increase in the time required to calculate the progress.

[0075] As described above, according to the third embodiment, it is possible to calculate in detail the degree of deformation of a specific part while suppressing a significant increase in processing time.

[0076] In the above embodiment, a bridge was used as an example of a structure for ease of understanding, but the type of structure is not limited to this, and other structures (for example, buildings, dams, etc.) may also be used.

[0077] (Other examples) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0078] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of symbols]

[0079] 221...Memory Unit, 222...Management Unit, 223...Importance Selection Unit, 224...Search Range Setting Unit, 225...Post-Integration Calculation Unit, 226...Reference Abnormality Determination Unit, 227...Progress Calculation Unit

Claims

1. A selection means for selecting some of the deformations included in a first image, which is an image of a structure at a first time period, as a target for determining the state of change over time, based on information relating to deformation of a structure, information relating to the structure, and at least one of two or more pieces of information relating to deformation, A determination means for determining a deformation corresponding to a deformation selected by the selection means from among a plurality of deformations contained in a second image, which is an image of the structure at a second time different from the first time; A determination means that determines the state of age-related change between the deformation selected by the selection means and the deformation determined by the determination means, based on the deformation information selected by the selection means and the deformation information determined by the determination means, A second determination means for determining a subregion of the first image and a subregion in the second image corresponding to the subregion of the first image, A generation means for generating display data for displaying the degree of progression of the deformation included in the partial region and the deformation selected by the selection means, based on the first deformation included in the partial region of the first image and the second deformation included in the partial region of the second image, An information processing device characterized by comprising:

2. The information processing apparatus according to claim 1, characterized in that the aforementioned partial region is a region obtained by dividing the first image into equally spaced portions.

3. The information processing apparatus according to claim 1, characterized in that the partial region of the first image and the partial region of the second image correspond to the same region of the structure.

4. The information processing apparatus according to claim 1, further comprising a calculation means for calculating the degree of progression of the second deformation included in the partial region of the second image from the first deformation, based on the first deformation and the second deformation.

5. The information processing apparatus according to claim 4, wherein the calculation means calculates the number of cracks or the crack density of the first and second deformations when the type of deformation is a crack.

6. The information processing apparatus according to claim 1, characterized in that the generation means generates the display data for displaying the partial region according to the progress.

7. The information processing apparatus according to claim 1, characterized in that it switches the target of the progress display for the partial area in accordance with user instructions.

8. A selection step of selecting some of the deformations included in a first image which is an image of the structure at a first time period as targets for determining the state of change over time, based on information relating to the deformation of the structure, information relating to the structure, and at least one of two or more pieces of information relating to the deformations, A determination step in which a determination step is made A determination step, based on the information of the deformation selected in the selection step and the information of the deformation determined in the determination step, determines the state of change over time between the deformation selected in the selection step and the deformation determined in the determination step. A second determination step of determining a subregion of the first image and a subregion in the second image that corresponds to the subregion of the first image, A generation step that generates display data for displaying the degree of progression of the deformation included in the partial region and the deformation selected in the selection step, based on the first deformation included in the partial region of the first image and the second deformation included in the partial region of the second image, A control method for an information processing device, characterized by comprising:

9. A program that, when read and executed by a computer, causes the computer to function as one of the means of the information processing device described in any one of claims 1 to 7.

10. The information processing apparatus according to claim 4, characterized in that the generation means generates the display data so as to display the numerical value calculated by the calculation means.

Images