Crack evaluation method, crack evaluation device, and program

Abstract

To provide a crack evaluation method, crack evaluation device, and program that can more easily and accurately match cracks between captured images.SOLUTION: A crack evaluation method includes: (1) a generation step of generating second images containing information about cracks detected from first images of crack detection surfaces captured at different times; (2) an identification step of identifying relative positional deviations between the second images based on positional relationships of multiple cracks within an area containing multiple cracks in the second images; and (3) a detection step of detecting changes in cracks between different times based on differences between cracks located within the same area in the second images after adjustment of the identified relative positional deviations.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to a crack evaluation method, a crack evaluation device, and a program. [Background technology]

[0002] In structures such as tunnels that mainly have concrete walls, cracks can occur due to deterioration over time. To address this issue, one method for efficiently detecting cracks in many structures is to photograph the wall surface of the structure and apply image recognition technology to the photographed image.

[0003] In determining whether or not a structure needs to be repaired, the amount and size of cracks as well as the extent to which the cracks have spread over time are important. The extent of spread can be determined by comparing multiple images taken at different times. Patent Documents 1 and 2 disclose techniques for identifying changes in the shape of cracks by comparing cracks detected in images taken at different times. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-211277 [Patent Document 2] Japanese Patent Application Publication No. 2023-83218 Summary of the Invention [Problem to be solved by the invention]

[0005] However, it is difficult to perfectly match the image capture ranges of image data captured at two different times. In particular, the wall surfaces of structures do not necessarily have distinctive patterns or structures, making it difficult to align them based on these. On the other hand, if you try to align cracks themselves, which may be changing, the accuracy of the alignment is likely to decrease to the extent of the changes. Therefore, when there are multiple cracks, there is the issue that it takes a lot of effort to reliably match the same cracks.

[0006] An object of the present invention is to provide a crack evaluation method, a crack evaluation device, and a program that can more easily and accurately identify corresponding cracks between captured images. [Means for solving the problem]

[0007] In order to achieve the above object, the crack evaluation method of the present invention includes: a generating step of generating second images each including position information of cracks detected from first images of the crack detection target surface obtained by photographing at different times; a specifying step of specifying a relative positional deviation between the second images based on a positional relationship between the plurality of cracks in an area including the plurality of cracks in the second image; a detection step of detecting a change in the crack between the different periods based on a difference between the cracks located within the same range in the second image in which the identified relative positional deviation has been adjusted; The following items were included: This crack evaluation method aligns images based on the relative positions of multiple cracks, making it easy to align images even when inspecting surfaces with no other features than cracks. Furthermore, by including multiple cracks in the alignment target range, the method is less affected by areas where the crack state is changing. Therefore, this crack evaluation method makes it easier and more accurate to match cracks between captured images.

[0008] Also, preferably, the cracks detected from the first image are each represented by one or more vectors; In the generating step, the second images are generated, each defining pixels in the crack range based on the vector, In the specifying step, the relative positional deviation may be specified by comparing the second images. Once the crack has been vectorized, the center line of the crack area is clearly defined. Furthermore, by generating a raster image that defines the crack area around this vector, it is possible to easily eliminate the influence of factors other than the crack itself or minute differences in the crack recognition range when identifying relative positional deviations. On the other hand, by returning the individual vectors to a common image, it becomes easier to compare the relative positions of multiple cracks.

[0009] Also, preferably, In the generating step, a range of a fixed number of pixels in the second image from each pixel position through which the vector passes in a direction perpendicular to the vector may be set as pixels where the crack is present. By specifying a uniform width, alignment is less susceptible to subtle differences in the recognition of crack areas.

[0010] Also, preferably, The number of fixed pixels may be equal to or greater than 2. If the fixed width is too narrow, it becomes difficult to align the pixels, so it is desirable that the fixed width be appropriate.

[0011] Also, preferably, the first image is an image obtained by combining a plurality of captured images, In the specifying step, the second image may be divided into a plurality of ranges, and the relative positional deviation may be specified for each of the ranges. When combining a plurality of captured images, a small positional deviation may be combined with other factors to produce a deviation that cannot be ignored, and the present invention is particularly effective in the case of such images.

[0012] Also, preferably, The second image may be a binary image based on the presence or absence of cracks, which makes it less susceptible to the influence of brightness value distribution of the captured image during alignment.

[0013] Also, preferably, In the second detection step, the extension of the crack range between the different periods may be detected based on the difference. The first crack image I1a and the second crack images I2a and I2a2, which indicate the presence or absence of a crack, can easily and clearly identify and show changes in the shape of the crack, particularly changes in length.

[0014] Also, preferably, Width information about the width of the crack represented by the vector is attached to the vector, In the generating step, the width information of the crack to which each pixel belongs may be assigned to pixels in the second image where the crack exists, based on the width information. In this way, each pixel also holds width information as additional information, which can be referenced after alignment to identify the size and width changes of each crack.

[0015] Also, preferably, In the generating step, pixel value components of a plurality of gradations corresponding to the width of the crack to which each pixel position belongs are set for the second image based on the width information, In the detection step, a change in width of the crack between the different periods is detected based on the difference between the pixel value components. By obtaining a multi-tone image according to the crack width after alignment, it is possible to output an image that allows the user to easily visually recognize changes in crack width.

[0016] In order to achieve the above object, the crack evaluation device of the present invention comprises: A generating means for generating second images each including position information of cracks detected from first images of the crack detection target surface obtained by photographing at different times; An identification means for identifying a relative positional deviation between the second images based on a positional relationship between the plurality of cracks in an area including the plurality of cracks in the second image; a detection means for detecting a change in the crack between the different periods based on a difference between the cracks located within the same range in the second image in which the relative positional deviation has been adjusted; Equipped with. Since alignment is performed based on the positional relationships between multiple cracks in multiple crack images, the crack evaluation device can easily align even when inspecting a surface that has no other features than cracks. Furthermore, by including multiple cracks in the alignment target range, the device is less affected by areas where the crack state is changing. Therefore, this crack evaluation device can more easily and accurately match cracks between captured images.

[0017] In order to achieve the above object, the program of the present invention comprises: Computer, A generating means for generating second images each including position information of cracks detected from first images of the crack detection target surface obtained by photographing at different times; An identification means for identifying a relative positional deviation between the second images based on a positional relationship between the plurality of cracks in an area including the plurality of cracks in the second image; a detection means for detecting a change in the crack between the different periods based on a difference between the cracks located within the same range in the second image in which the relative positional deviation has been adjusted; An identification means for identifying a relative positional deviation between the second images based on a positional relationship between the plurality of cracks in an area including the plurality of cracks in the two images; a second detection means for comparing the cracks located within the same range in the second image in which the identified relative positional deviation has been adjusted, and detecting changes in the cracks between the different periods; Function as. By using such a program, cracks can be easily evaluated using software even on a general-purpose computer. [Effects of the Invention]

[0018] According to the present invention, it is possible to more easily and more accurately align captured images. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 2 is a block diagram showing a functional configuration of the information processing device. [Figure 2] 10A and 10B are diagrams illustrating the identification of a crack position in response to a tracing operation. [Figure 3] FIG. 1 is a diagram illustrating a schematic diagram of a crack region detected using a machine learning model. [Figure 4] 10 shows images of cracks after alignment and the difference between them. [Figure 5] 10A and 10B are diagrams illustrating examples of multi-tone images according to width. [Figure 6] 10 is a flowchart showing a control procedure for crack evaluation processing. [Figure 7] 10A and 10B are diagrams illustrating the setting of crack range pixels in the first crack image. [Figure 8] 10 is a flowchart showing a control procedure for alignment processing called in the crack evaluation processing. DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the functional configuration of an information processing device 1 which is a crack evaluation device according to this embodiment.

[0021] The information processing device 1 may be a normal PC (Personal Computer). The information processing device 1 includes a control unit 11, a RAM (Random Access Memory) 12, a storage unit 13, a communication unit 14, a display unit 15, an operation reception unit 16, and the like.

[0022] The control unit 11 is a processor that controls the overall operation of the information processing device 1. The processor may be a general-purpose CPU (Central Processing Unit). There may be a single CPU or multiple CPUs operating in parallel. The processor may also include a function as a GPU (Graphical Processing Unit), or may have a GPU separate from the CPU.

[0023] The RAM 12 provides a working memory space for the control unit 11 and stores temporary data. The RAM 12 may be a DRAM.

[0024] The storage unit 13 is a non-volatile memory that stores a program 131, setting data, image data 132 related to an evaluation target and evaluation results, etc. The non-volatile memory may be a hard disk drive (HDD) or a flash memory. The storage unit 13 may also include a network drive or a cloud server located on a network. For example, the image data 132 located on a cloud server may be accessible from other information processing devices.

[0025] The communication unit 14 controls communication with external devices. The communication unit 14 has, for example, a network card and transmits and receives data via a LAN, a wireless LAN, or the Internet. The communication unit 14 may also have a connection terminal for a cable such as a USB (Universal Serial Bus) or a serial ATA (SATA). This connection terminal may be connected to a connection terminal of an external device by a cable, allowing data to be transmitted and received directly to and from the external device.

[0026] Display unit 15 has a digital display screen and displays information based on the control of control unit 11. The digital display screen may be, for example, a liquid crystal display screen or an organic EL (Electro-Luminescent) display screen.

[0027] The operation reception unit 16 receives an external input operation from a user or the like, and outputs an electrical signal according to the received operation content to the control unit 11. The operation reception unit 16 may include a pointing device such as a mouse, a keyboard, and the like.

[0028] Note that some or all of the storage unit 13, the display unit 15, and the operation reception unit 16 may be peripheral devices externally attached to the main body of a PC (computer) including at least the control unit 11 and the RAM 12.

[0029] Next, crack evaluation in the information processing device 1 will be described. In this embodiment, the surface of a structure to be detected for cracks, such as the wall surface of a mountain railway tunnel, is photographed, and cracks are detected from the photographed images. Images containing position information of cracks detected from multiple (two) photographed images taken at different times are aligned. Then, differences in the cracks between the two images are extracted to detect changes such as crack expansion, elongation, and deformation. The obtained information on the amount of cracks and the amount of change in cracks may be statistically processed, for example, for each specified distance. The distance range in a mountain railway tunnel may be determined, for example, in kilometers or by the distance from one end of the tunnel.

[0030] As is conventionally known, the tunnel wall surface may be photographed simultaneously in multiple directions along a cross section perpendicular to the tunnel's extension direction using multiple image capture devices mounted on a dedicated measurement vehicle. Repeated image capture at appropriate intervals while the measurement vehicle is traveling allows for efficient image capture of the entire tunnel wall. The surface to be photographed may be illuminated to facilitate crack identification. It is preferable that the image capture ranges of each image capture device overlap. The image capture devices may be digital cameras, and the resulting captured images are raster images with the spatial resolution required for crack detection. If the spatial resolution is too high for crack detection (described below), the raster images may be downsampled. Reducing the resolution may sometimes remove unnecessary noise. The digital camera may also be a line sensor, as long as it can capture images frequently enough for the traveling speed. The image data may be either monochrome or color images of visible light. In the case of monochrome images, the luminance values ​​may be used as is. In the case of color images, only the image of the color component suitable for crack detection may be used subsequently. Alternatively, brightness and the like may be calculated and used by combining the luminance values ​​of multiple colors. Multiple images captured sequentially multiple times by multiple image capture devices are combined to obtain a tunnel unfolded image (first image). This tunnel unfolded image is in a format that allows reversible pixel-by-pixel reading and processing, such as PNG format. When the tunnel unfolded image itself is output as a deliverable such as an inspection report, the image may be embedded in a document file such as PDF in an appropriate format.

[0031] Crack detection in a tunnel unfolded image can be performed manually or mechanically. For example, a technician may trace a portion of the image identified as a crack using a pointing device such as a mouse or a touch pen while viewing the captured image displayed on a display screen. The control unit 11 acquires the crack location by accepting this tracing operation. Alternatively, the technician may specify the crack area by selecting all of the crack pixels using a pointing device or tracing the outer edge of the crack area. For mechanical extraction, crack pixels may be detected using image recognition using a machine learning model, for example. Any known method, such as a convolutional neural network, may be used for image recognition. In this case, the crack is output as a probability distribution of the crack area, and may be binarized based on a reference value, or a center line may be identified as described above. Alternatively, an encoder-decoder structure or a full-layer convolutional network may be used for semantic segmentation to separate the crack area from other areas. Even when a machine learning model is used, the model may be configured to output the center line of the crack pixel as a broken line, curve, etc. Alternatively, a process may be performed to define a line of equal width along the crack separately for the crack pixel output by the machine learning model.

[0032] Alternatively, crack extraction may involve mechanical preliminary extraction, and a person in charge may manually input the crack range or its trace line while referring to the preliminary extraction results. In this case, preliminary extraction may involve, for example, more simply setting a brightness reference value, and determining areas darker than the brightness reference value as candidates for crack pixels. Alternatively, the process may be manual, with a person in charge simply checking the results detected using a machine learning model.

[0033] FIG. 2 is a diagram illustrating how a crack position is identified in response to a tracing operation. In FIG. 2(a), the region of crack C is shown as a schematic representation of changes in brightness values ​​in the captured image. A trace line L is acquired for this crack C through input operations. Based on the acquired trace line L, vectors V1 to V3 are set within a range that allows for linear approximation, as shown in FIG. 2(b). Alternatively, if the person in charge can only input straight lines or broken lines, the trace line L may be a single vector or a combination of multiple vectors. That is, each crack C is represented by a start point and an end point, or a combination of a start point, length, and direction. A long crack with small changes in direction may be represented by a combination of multiple vectors. For a branched crack, vectors extending in the directions branching from the branch points may be defined. In the information processing device 1, vectors V1 to V3 may be set in order along the trace line L from one end of the trace line L acquired as described above. Alternatively, if the crack region is identified using a machine learning model, the vectors V1 to V3 may be set within a range that can be approximated by a straight line along the direction in which the crack region extends.

[0034] At this time, information about the width W1 of the crack C is added to each vector V1 to V3. The width W1 of the crack C may be determined by counting the pixels recognized as being within the crack range in a direction perpendicular to the direction of the vector. A representative value of the counted values, such as the average value, may be assigned to each vector V1 to V3 as the width W1. Alternatively, for a continuous crack C, the person in charge may visually count the width W1 of the part that is thought to be the thickest or the part that is thought to be the average width, and this information may be uniformly added to multiple vectors belonging to this crack as a representative value of the crack. Cracks that branch or intersect may all be included in a continuous crack to the extent that they are connected to each other.

[0035] FIG. 3 is a diagram schematically illustrating a crack area detected using a machine learning model. FIG. 3(a) is an example of a crack detection image I1 at a first time point. FIG. 3(b) is an example of a crack detection image I2 at a second time point different from the first time point. FIG. 3(c) is a diagram explaining the positional deviation between the crack detection image I1 and the crack detection image I2. In reality, the crack detection image I1 and the crack detection image I2 may contain noise or false detections, as described below, but these are not shown here.

[0036] Crack detection image I1 contains multiple cracks C1 indicated by black solid lines within an area surrounded by a dashed line. Crack detection image I2 contains multiple cracks C2 indicated by black solid lines within an area surrounded by a dashed line. Some of the cracks have branched. By calculating the difference between the cracks in crack detection image I1 obtained in this way and the cracks in crack detection image I2, it is believed that changes in the cracks between the two time periods can be obtained.

[0037] However, in the tunnel unfolded image combined as described above, slight relative positional deviations may occur due to differences in the shooting conditions of the individual captured images or the process of combining the captured images. When these deviations accumulate, as shown in Figure 3(c), a deviation of tens to tens of millimeters may occur between crack C1 in crack detection image I1 and crack C2 in crack detection image I2. When comparing crack locations and taking differences, corresponding cracks must be correctly identified between multiple images. Therefore, after cracks are identified as described above, the captured images are aligned to achieve the appropriate positional relationship based on the cracks.

[0038] FIG. 4 shows the first crack image I1a after alignment, the second crack image I2a, the second crack image I2a2 after adjusting the position of the second crack image I2a, and a first difference image Id1 showing the difference between them. Details of the alignment will be described later, but based on the multiple vectors representing the identified cracks, binary raster images are obtained in which the pixel ranges of cracks C1a and C2a are defined. That is, a first crack image I1a (Fig. 4(a)) at the first time and a second crack image I2a (Fig. 4(b)) at the second time are obtained, which are bitmap images indicating the presence or absence of cracks, and the amount of relative positional deviation between them is identified.

[0039] Then, the amount of relative positional deviation is reflected in the second crack image I2a, and an adjusted second crack image I2a2 is obtained. A first difference image Id1 representing changes in the crack is obtained by subtracting the first crack image I1a and the adjusted second crack image I2a2. FIG. 4(c) shows an image in which the crack areas of the first difference image Id1 are overwritten on the adjusted second crack image I2a2 so that they are identifiable. When the image is shown to a user as a detection result, for example, the crack areas included in the first difference image Id1 may be displayed in a color different from the crack areas already detected in the first crack image I1a, such as gray in this case, among the crack areas shown in the adjusted second crack image I2a2. This color coding makes it easy to identify detected cracks, particularly areas where the crack has changed between two periods. If color display is possible, the first difference image Id1 may be displayed in a more noticeable color, such as red. Furthermore, the first crack image I1a and the first difference image Id1 may be displayed overlapping each other in different colors.

[0040] In the first crack image I1a and the second crack image I2a, the crack width W1 is shown as additional information for each pixel of the cracks C1a and C2a, just like in the crack detection images I1 and I2. On the other hand, in the first crack image I1a and the second crack images I2a and I2a2, the cracks C1a and C2a are shown with a predetermined fixed width and do not reflect the actual width W1. This point will also be discussed later in conjunction with a detailed explanation of alignment.

[0041] However, because subtle cracks may be identified during the crack identification process, in practice, even small differences due to differences in contours are likely to be extracted, resulting in a noisy first difference image Id1 (not shown). Therefore, a process for removing noise from this first difference image Id1 may be performed. Known noise removal techniques include morphological transformation. Morphological transformation involves expanding the area detected as the difference range in the first difference image Id1 by a predetermined number of pixels, then shrinking it by a predetermined number of pixels, and then restoring it to its original size. This removes gaps within the area and removes small difference ranges of less than the predetermined number of pixels. In this way, the change portions of the clustered cracks remain at approximately their original size, reducing noise. Furthermore, change-detected pixels in the first difference image Id1 may be thinned. As described above, the fixed widths of the cracks in the first crack image I1a, the second crack image I2a2, and the corresponding first difference image Id1 were somewhat thick. However, when counting the number of pixels whose length has changed in statistical processing, the line width becomes unnecessary, so each crack change portion is converted into a line of one pixel width.

[0042] Changes in a crack include changes in width as well as changes in length (elongation). Changes in width are detected based on the width data attached to each pixel. The width value itself may also be affected by differences in measurement position or measurement errors. Therefore, the pixel value of each pixel is converted into a discrete value with multiple gradations corresponding to the width. The number of gradations corresponding to the crack width is not particularly limited, but may be around four gradations.

[0043] FIG. 5 is a diagram showing an example of a multi-tone image according to width. From the first crack image I1a and the second crack image I2a2, which are binary images containing width data, crack images I1b (FIG. 5(a)) and I2b (FIG. 5(b)) are obtained, respectively, which have uniform line widths and are expressed in five gradations, including four gradations corresponding to the crack width and one gradation representing no crack. Each crack C1b, C2b is displayed with different shading depending on its width. In this case, for example, by consolidating the multiple gradations into the hue (H component) of an HSV format in a color image, both visual and numerical identification becomes easier. However, this is not limited to gradation representation. Figure 5(c) shows the difference between crack image I1b and crack image I2b. The second difference image Id2 indicates the change in width of one of the three cracks shown in crack images I1b and I2b as the crack difference Cd2.

[0044] The second difference image Id2, which is the difference between crack images I1b and I2b, is also subjected to morphological transformation and other processes to reduce noise (not shown), as described above. Pixel positions with non-zero pixel values ​​in the noise-reduced second difference image Id2 (if the width is the representative value, this represents the crack represented by that representative value) indicate that a change has occurred in the crack width.

[0045] On the display, by overlaying the second difference image Id2 and the first difference image Id1 on the first crack image I1a, it is possible to identify all of the cracks on the wall where the length and width of the cracks have changed. Since pixels with non-zero values ​​in the first difference image Id1 also have non-zero values ​​in the second difference image Id2, the first difference image Id1 may be overlaid on the second difference image Id2.

[0046] Furthermore, as described above, the number of pixels with cracks or differences in the first crack image I1a, the first difference image Id1, and the second difference image Id2 are counted at appropriate distances, thereby obtaining the distribution and rate of change status.

[0047] FIG. 6 is a flowchart showing a control procedure for the process of evaluating cracks in a tunnel wall surface executed by the information processing device 1. The control unit 11 acquires unfolded images (first images) of the tunnel at approximately the same position at the evaluation target position, taken at two different times (S1). The control unit 11 detects cracks in each of the unfolded images of the tunnel (S2). As described above, the control unit 11 may accept a manual input operation of the detection position from the operation reception unit 16 or the like, or may perform automatic detection using a machine learning model. These may also be used in combination.

[0048] The control unit 11 sets crack vectors along the obtained cracks. The control unit 11 additionally sets the width of the cracks obtained together with the crack detection results as width information of each crack vector (S3). Among the above, some or all of the processes of steps S1 to S3 may be performed by another information processing device located outside the information processing device 1. In this case, the control unit 11 acquires crack vector data and the shooting position range of the tunnel unfolded image in which the crack vector is set from the other information processing device.

[0049] The control unit 11 determines that pixels passing through each pixel on the set vector and within a fixed width range in a direction perpendicular to the vector are crack pixels. The control unit 11 generates a first crack image I1a and a second crack image I2a (a second image including crack position information) with binary values ​​depending on whether the pixel is a crack or not (S4; generation step, generation means). The crack width value associated with each vector is added to each crack pixel as width information. This value may be the value of a component unrelated to the display of each pixel.

[0050] The control unit 11 performs a position alignment process on the first crack image I1a and the second crack image I2a. The control unit 11 obtains an adjusted second crack image I2a2 in which the second crack image I2a is most closely aligned with the position of the first crack image I1a (S5).

[0051] The control unit 11 calculates the difference between the first crack image I1a and the second crack image I2a2 and generates a first difference image Id1 that shows changes in the crack conditions in the same area of ​​both images (S6).The control unit 11 removes noise from the first difference image Id1 and thins the cracked portions (S7).

[0052] The control unit 11 counts the number of non-zero pixels in the first difference image Id1, that is, the number of pixels where the crack length changes between the two periods, for each region (S8).

[0053] Next, the control unit 11 generates crack images I1b and I2b by converting each pixel value of the first crack image I1a and the second crack image I2a2 into multiple gradation values ​​according to the crack width (S9). The number of gradations may be five, from 0 to 4, as described above. The control unit 11 calculates the difference between the crack images I1b and I2b, which have pixel value components of multiple gradations, to generate a second difference image Id2 (S10). Note that the pixel values ​​do not have to be one-dimensional. Therefore, the multiple gradation values ​​may be represented in one dimension among multiple dimensions. The control unit 11 removes noise from the second difference image Id2 and subdivides the crack width change portion (S11). The control unit 11 also thins the first crack image I1a and / or the adjusted second crack image I2a2.

[0054] The control unit 11 counts the number of non-zero pixels in the second difference image Id2, that is, the number of pixels where the crack width changes between the two periods, for each region (S12).

[0055] The control unit 11 generates and outputs a crack evaluation image in which the crack image I1b or I2b, the first differential image Id1, and the second differential image Id2 are superimposed in that order (S13).Then, the control unit 11 ends the crack evaluation process. The above steps S8, S12, S13, etc. correspond to the detection steps in the crack evaluation method of this embodiment, and correspond to the function of the program 131 and the control unit 11 as the detection means.

[0056] Next, the alignment of the first crack image I1a and the second crack image I2a will be described. As described above, there may be slight misalignment between the multiple images combined into a tunnel unfolded image, which may make it difficult to accurately determine the correspondence between cracks in the images from two different periods. Therefore, in this embodiment, the two images are aligned.

[0057] One factor that makes this registration necessary is the three-dimensional shape of the tunnel. The distance from the imaging device to the tunnel wall can vary non-uniformly and non-linearly depending on the tunnel cross-section, which has a different curvature (such as a horseshoe shape), the curve of the tunnel's extension, etc. When correcting for these and combining the images, minute deviations on the order of millimeters can often remain.

[0058] When aligning the image capture ranges of multiple captured images, the positions of surrounding features are generally used. However, with walls such as concrete, it is often the case that the surrounding characteristic structures are not within the field of view. In such cases, alignment cannot be performed properly. In addition, techniques have been proposed to directly align individual cracks. However, in this case, if changes occur in the cracks between the two periods, alignment will be performed between different shapes, which can easily lead to inaccurate results.

[0059] In this embodiment, alignment is performed using a block (range) containing multiple cracks as a unit, with the cracks being used as characteristic features. If there are many cracks, even if some of them are deformed, the adverse effect on alignment is reduced. Furthermore, by dispersing the position and direction of the deformation for each crack, the effect on overall alignment is reduced. As a result, a decrease in alignment accuracy is suppressed.

[0060] On the other hand, as mentioned above, misalignment is likely to occur locally. Therefore, it is difficult to expect that misalignment can be corrected all at once over a large range that includes many original images. Therefore, alignment is performed by dividing the combined tunnel unfolded image into multiple blocks (ranges) of appropriate size and performing alignment on each block separately.

[0061] Because this alignment is performed on an image block-by-block basis, a raster image is generated that defines the pixels within the crack area based on the previously vectorized crack data. This raster image is the first crack image I1a and the second crack image I2a.

[0062] FIG. 7 is a diagram illustrating the setting of crack range pixels in the first crack image I1a. In the first crack image I1a and the second crack image I2a, pixels within a crack range are defined as those overlapping with the vectors and within a range of a predetermined fixed width W2 extending from each vector in a direction perpendicular to the vector's direction. Other pixels are defined as pixels without cracks (outside the range). The specific pixel values ​​of each pixel are not particularly limited, as long as the presence or absence of a crack can be identified. For example, a pixel without a crack may have a pixel value of "0," and a pixel with a crack may have a pixel value of "1." Alternatively, a pixel with a crack may have a pixel value of "255." If this fixed width W2 (the fixed number of pixels within the range of the fixed width W2 / 2 from the pixel through which the vector passes) is too small (e.g., 1 pixel), even a slight misalignment will remain, making it difficult to match cracks between images. If the fixed width W2 is too large (e.g., 10 pixels or more), different cracks may be merged. The fixed width W2 can also be determined in accordance with the size of each pixel in real space (i.e., the photographed wall surface). Therefore, the fixed width W2 may be determined appropriately between these two.

[0063] As described above, a fixed width W2 is defined in the width direction perpendicular to the vectors V1 to V3, including the range of vectors V1 to V3. Here, the fixed width W2 is three pixels, but is not limited to this. In addition, for each pixel with a crack, identification information for the crack to which it belongs and the width value (representative value) of the crack are added as auxiliary information. As described above, the crack width value is the width W1 previously calculated for the vector, and is set uniformly for all pixels with a crack. The generated first crack image I1a and second crack image I2a may be in the same format as the tunnel unfolded image. In other words, the first crack image I1a and second crack image I2a may be in PNG format.

[0064] In this way, the first crack image I1a and the second crack image I2a contain no information other than the crack position, and the pixel values ​​of the pixels within the crack area do not contain information about the crack width. Therefore, during alignment, no weighting is performed according to the crack width, and the image is not affected by anything other than the crack. Cracks that occupy the same area between images aligned by this process are determined to be corresponding cracks.

[0065] Note that, as described above, when identifying changes in crack width after alignment, the uniform value set as the pixel value of the pixel containing the crack above can be set to a value set in multiple stages according to the crack width W1. In this case, too, the pixel value that is set is completely unrelated to the fixed width W2 of the area identified as a pixel containing a crack in the first crack image I1a and the adjusted second crack image I2a2.

[0066] 8 is a flowchart showing the control procedure of the alignment process called in the crack evaluation process. This alignment process corresponds to the identification step in the crack evaluation method of this embodiment, and corresponds to the function of the program 131 and the control unit 11 as identification means. When the registration process is called, the control unit 11 divides the crack images from the two periods into regions (S21), and then selects unselected corresponding regions (S22).

[0067] The control unit 11 determines whether or not the selected areas each have a detected crack (S23). If it is determined that there is no crack in any of the selected areas (S23; N), the control unit 11 proceeds to step S26.

[0068] If it is determined that each selected region contains a crack (S23; Y), the control unit 11 shifts the second crack image I2a relative to the first crack image I1a within a specified range to calculate the degree of displacement of the crack pixels. The control unit 11 then determines the relative positional displacement amount that minimizes the calculated degree of displacement (S24). Methods for determining the displacement amount that minimizes the displacement of the crack pixels include conventional methods, such as the least-squares method, which determines the position that minimizes the square of the pixel difference between both pixels at each pixel position. Alternatively, the normalized cross-correlation between the two images may be calculated to determine the displacement amount that maximizes the cross-correlation coefficient. Furthermore, phase-only correlation may be used to align two images containing the same components. Based on the above principle, it is desirable for the selected region to contain multiple cracks. Therefore, the size of the divided region may be within the range of two original captured images that can be aligned accurately, and within a range in which at least two cracks are likely to occur closely together. Furthermore, if there is only one crack in a selected area, it may be merged with another unselected area. Even if the area within the range is expanded as much as possible, if there is only one crack, the accuracy of alignment may decrease, but alignment may be performed by matching each crack.

[0069] The control unit 11 generates an adjusted second crack image I2a2 by shifting the second crack image I2a by the calculated shift amount (S25). Then, the processing of the control unit 11 proceeds to step S26.

[0070] In the process of step S26, the control unit 11 determines whether or not all of the regions generated in the process of step S21 have been selected (S26). If it is determined that all of the regions have not been selected (there is an unselected region) (S26; N), the process of the control unit 11 returns to step S22. If it is determined that all of the regions have been selected (S26; Y), the control unit 11 ends the alignment process and returns the process to the crack evaluation process.

[0071] As described above, the crack evaluation method of this embodiment includes the following steps: (1) a generation step of generating a first crack image I1a and a second crack image I2a, each including position information of a crack detected from a tunnel unfolded image of the crack detection target surface captured at different times; (2) a determination step of identifying the relative positional deviation between the first crack image I1a and the second crack image I2a based on the positional relationship of multiple cracks in a range including multiple cracks in these images; (3) a detection step of detecting a change from crack C1a to crack C2a between two different times based on the difference between cracks C1a and C2a located within the same range in the first crack image I1a and the second crack image I2a2, after adjusting the identified relative positional deviation. This crack evaluation method aligns cracks based on their positional relationships, making alignment easy even for inspections of surfaces without other features. Furthermore, including multiple cracks makes it less susceptible to the influence of areas where the crack state is changing. Therefore, this crack evaluation method can more easily and accurately match cracks between captured images.

[0072] Furthermore, cracks detected from the tunnel unfolded image are each represented by one or more vectors. In the generation step, a first crack image I1a and a second crack image I2a are generated, each defining the pixels within the crack range based on these vectors. In the identification step, the relative positional deviation is identified by comparing the first crack image I1a and the second crack image I2a. In this way, by generating and using the first crack image I1a and the second crack image I2a based on the crack once vectorized, the center line of the crack range is clearly defined. Furthermore, by generating a raster image defining the crack range around this vector, it is possible to easily eliminate influences other than the crack itself and influences such as minute differences in the crack recognition range when identifying the relative positional deviation. Meanwhile, by restoring the individual vectors so that they are located within a common image, it is easier to compare the positional relationships of multiple cracks.

[0073] In the generating step, a range of a fixed number of pixels in the first crack image I1a and the second crack image I2a from each pixel position through which the vector passes in the direction perpendicular to the vector may be set as pixels containing cracks. By specifying such a uniform width, alignment is less susceptible to subtle differences in the recognition of crack areas, enabling more accurate alignment.

[0074] The fixed number of pixels may be two or more. If the uniform width is too narrow, the cracks in the two images being compared will not overlap even after alignment, making it difficult to determine which cracks correspond to which. Therefore, by defining the crack area with a certain degree of width relative to the vector position, it is possible to more accurately determine the correspondence between cracks.

[0075] Furthermore, the tunnel unfolded image for detecting cracks may be an image obtained by combining multiple captured images. In such cases, the influence of misalignment between the captured images becomes particularly significant, increasing the benefits of the above-described alignment. Furthermore, in the identification step, the first crack image I1a and the second crack image I2a may each be divided into multiple ranges, and the relative misalignment between them may be identified for each range. Misalignment due to combining captured images may occur locally for each combination. Therefore, by appropriately dividing the alignment range according to the combination between images, alignment can be performed more accurately.

[0076] The first crack image I1a and the second crack image I2a, I2a2 may be binary images that indicate whether or not cracks are present. By performing alignment using images that do not reflect the influence of the brightness value distribution of the captured image, alignment that is less susceptible to the influence of noise and the like is possible.

[0077] In addition, in the detection step, the extension of the crack range between different periods may be detected by calculating the difference. As described above, since a simple crack area based on a vector is imaged, this crack evaluation method makes it possible to easily and clearly identify changes in length.

[0078] Furthermore, width information about the width of the crack represented by the vector may be added to the vector. In the generation step, width information about the crack to which each pixel belongs may be added to pixels in the first crack image I1a and the second crack images I2a and I2a2 that have cracks based on this width information. In this way, the width information may be determined on a crack-by-crack basis. Since crack information for purposes such as repair does not necessarily require the complete crack shape to be obtained, the amount of data and processing can be reduced. Furthermore, by storing the width information in a form that is not used to identify relative positional deviation, this width information can be used as information when detecting changes in the crack state.

[0079] Furthermore, in the generating step, pixel value components of multiple gradations corresponding to the width of the crack to which each pixel position belongs may be set for the first crack image I1a and the second crack image I2a2 based on the width information. In the detecting step, changes in crack width between different periods may be detected based on the difference between the set pixel value components. In this way, after alignment, each pixel value may be set to a multi-gradation value according to the crack width. This makes it possible for the crack evaluation method to easily obtain both changes in crack length and width after accurate alignment.

[0080] The information processing device 1, which is a crack evaluation device of this embodiment, also includes a control unit 11. The control unit 11, as a generating unit, generates a first crack image I1a and a second crack image I2a, each containing positional information of cracks detected from tunnel unfolded images of the crack detection target surface captured at different times. The control unit 11, as an identifying unit, identifies the relative positional deviation between these images based on the positional relationship of multiple cracks in an area containing multiple cracks. The control unit 11, as a detecting unit, detects changes in cracks between different time periods by calculating the difference between cracks located within the same area in the first crack image I1a and the second crack image I2a2, after adjusting the identified relative positional deviation. This information processing device 1 can more easily align images of the target surface captured at two different time periods, more accurately associate cracks, and detect changes. Furthermore, because the relative positional deviation is identified based on multiple cracks, a decrease in alignment accuracy can be suppressed even if the crack area changes between the two time periods.

[0081] Furthermore, by installing and executing the program 131 relating to the crack evaluation method of this embodiment on a computer, it is possible to easily perform accurate crack evaluation even on a computer such as an ordinary PC.

[0082] The present invention is not limited to the above-described embodiment, and various modifications are possible. For example, in the above example, the detected crack range is defined by vectors and then converted back into a raster image representing the positional relationship of the multiple vectors. However, this is not limited to this. The crack may be represented on a new raster image from the beginning using broken lines or curves. Alternatively, the cracks between the two periods may be aligned using multiple vectors. The aligned vectors may be used to obtain an adjusted second crack image I2a2.

[0083] In the above description, only translation is considered in aligning the first crack image I1a and the second crack image I2a, but this is not limiting. Rotation, scale changes, etc. may also be considered. However, because rotation and scale deviations are usually corrected when generating the tunnel unfolded image, they are unlikely to remain until the alignment stage.

[0084] In addition, although the above description has been given of only the difference between the first crack image I1a and the second crack image I2a2 as the information used to compare the two points in time, this is not limited to this. For example, the ratio of the two timings may also be included in the information related to the change.

[0085] Furthermore, although the above describes a case where the crack width increases in the second crack image I2a compared to the first crack image I1a, it is possible that the crack width decreases depending on the surrounding environmental conditions, etc. In this case, the amount of change in the crack width may be calculated and displayed as a signed value. Alternatively, if the crack width decreases, i.e., if the amount of change is a negative value, it may be displayed as no change, i.e., the amount of change is zero.

[0086] In addition, although the above description assumes that the width of a crack is not taken into consideration when aligning, this is not limited to this. By weighting each pixel containing a crack based on the width of the crack and aligning it, the wider the crack, the more likely it is that the more accurately it will be aligned. Alternatively, if a machine learning model is used to obtain the possibility of a crack as a probability value, each pixel that may be included in a crack may be weighted based on the probability value to determine the relative positional deviation.

[0087] Also, variations in crack width may be indicated by lines of different widths instead of or in addition to different colors.

[0088] Furthermore, the multi-valued crack images I1b and I2b corresponding to the crack width are not limited to being generated by applying the width of additional information to the crack pixels of the first crack image I1a, which is a binary image, and the adjusted second crack image I2a2. The original captured images I1 and I2 or a multi-valued image such as a processed grayscale image may be converted to pixel values ​​corresponding to the width after weighting the multi-valued values ​​with the width information. The crack image I2b obtained in this case may be aligned with the crack image I1b by being moved by the same amount as the displacement determined when obtaining the adjusted second crack image I2a2.

[0089] Furthermore, although the wall surface of a mountain railway tunnel has been described as an example, the present invention is not limited to this. Urban tunnels, road tunnels, and other tunnels may be used as long as the wall surface is made of the same or similar material and the problem of the present invention is addressed. Furthermore, the target structure is not limited to tunnels. It may also be various structures such as the wall surface of a bridge made of reinforced concrete or prestressed concrete, or a dam.

[0090] The original images may be taken by a person in charge, who moves around on foot or by other means, with the camera mounted on a tripod at each position, or conversely, the image may be taken by a person on a work vehicle, with the camera scanned in the direction along the cross section.

[0091] In the above description, all the processes are executed by the information processing device 1 alone, but this is not limiting. The processes may be distributed and executed by a plurality of computers.

[0092] In the above description, the computer-readable medium for storing the program 131 related to the crack evaluation control of the present invention has been described as a storage unit 13 including a nonvolatile memory such as an HDD or flash memory, but this is not limited to these. Other computer-readable media may include other nonvolatile memories such as MRAM, and portable recording media such as CD-ROMs and DVD discs. Furthermore, a carrier wave may also be used as a medium for providing the program data related to the present invention via a communication line. In addition, the specific configurations, contents and procedures of the processing operations, etc. shown in the above embodiments can be modified as appropriate without departing from the spirit of the present invention. The scope of the present invention includes the scope of the invention described in the claims and its equivalents. [Explanation of symbols]

[0093] 1. Information processing equipment 11 Control section 12 RAM 13 Storage section 131 Programs 132 Image data 14 Communications Department 15 Display section 16 Operation reception section I1, I2 Crack detection images I1a First crack image I2a, I2a2 Second crack image I1b, I2b crack images L Trace Line V1~V3 vectors W2 Fixed width

Claims

1. A generation step of generating second images each including position information of cracks detected from first images of the crack detection target surface obtained by photographing at different times; a specifying step of specifying a relative positional deviation between the second images based on a positional relationship between the plurality of cracks in an area including the plurality of cracks in the second images; a detection step of detecting a change in the crack between the different periods based on a difference between the cracks located within the same range in the second image in which the identified relative positional deviation has been adjusted; A crack evaluation method including:

2. the cracks detected from the first image are each represented by one or more vectors; In the generating step, the second images are generated, each defining pixels in a crack range based on the vector, In the specifying step, the relative positional deviation is specified by comparing the second image. The crack evaluation method according to claim 1.

3. A crack evaluation method as described in claim 2, wherein in the generation step, a range of a fixed number of pixels in the second image from each pixel position through which the vector passes in a direction perpendicular to the vector is set as pixels where the crack is present.

4. The crack evaluation method according to claim 3 , wherein the fixed number of pixels is two or more.

5. the first image is an image obtained by combining a plurality of captured images, In the specifying step, the second image is divided into a plurality of ranges, and the relative positional deviation is specified for each range. The crack evaluation method according to any one of claims 2 to 4.

6. The crack evaluation method according to any one of claims 2 to 4, wherein the second image is a binary image corresponding to the presence or absence of a crack.

7. The crack evaluation method according to claim 6 , wherein the detection step detects an extension of the crack range between the different periods based on the difference.

8. Width information about the width of the crack represented by the vector is attached to the vector, In the generating step, the width information of the crack to which each pixel belongs is assigned to the pixel in the second image where the crack exists based on the width information. The crack evaluation method according to any one of claims 2 to 4.

9. In the generating step, pixel value components of a plurality of gradations corresponding to the width of the crack to which each pixel position belongs are set for the second image based on the width information, In the detection step, a change in width of the crack between the different periods is detected based on the difference between the pixel value components. The crack evaluation method according to claim 8.

10. A generating means for generating second images each including position information of cracks detected from first images of the crack detection target surface obtained by photographing at different times; an identification means for identifying a relative positional deviation between the second images based on a positional relationship between the plurality of cracks in an area including the plurality of cracks in the second images; a detection means for detecting a change in the crack between the different periods based on a difference between the cracks located within the same range in the second image in which the relative positional deviation has been adjusted; A crack evaluation device comprising:

11. Computer, A generating means for generating second images each including position information of cracks detected from first images of the crack detection target surface obtained by photographing at different times; an identification means for identifying a relative positional deviation between the second images based on a positional relationship between the plurality of cracks in an area including the plurality of cracks in the second images; a detection means for detecting a change in the crack between the different periods based on a difference between the cracks located within the same range in the second image in which the relative positional deviation has been adjusted; A program that functions as a

Images