Crack width measuring device, crack width measuring method, and crack width measuring program
The crack width measuring device addresses the issue of inaccurate crack width measurement by using pixel-level relationships and classification models to calculate precise crack widths at selected positions, enhancing detection accuracy and stability.
Patent Information
- Application Number
- JP2023071143
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-24
- Publication Date
- 2024-11-06
AI Technical Summary
Conventional crack width measurement methods struggle with accuracy when dealing with multiple cracks in a narrow range or when cracks resemble concrete joints, leading to unintended results and inability to measure exact crack width at specific locations.
A crack width measuring device that uses pixel-level crack width relationship information and size correspondence to calculate the actual crack width at arbitrarily selected positions, incorporating features like brightness-pixel relationship, image resolution, and crack classification models to enhance precision.
Enables accurate measurement of crack width at specified positions, improving detection accuracy and stability across varying image conditions.
Smart Images

Figure 2024156565000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a crack width measurement device, a crack width measurement method, and a crack width measurement program that measure crack widths from image data of a structure that includes cracks. [Background technology]
[0002] In structures such as tunnels, it is required to accurately describe the width of cracks and submit them as inspection result reports. Patent Document 1 discloses a method for measuring cracks that determine the width of cracks that occur on the surface of a structure such as a concrete structure. In the crack measurement method described in Patent Document 1, the original image data obtained by photographing the surface of the structure is binarized for each pixel to determine the crack area, and a peripheral area of at least one pixel that goes around the outer edge of the crack area is set. In the crack measurement method described in Patent Document 1, the apparent area of the crack, which is the sum of the brightness values of each pixel in the crack area and the brightness value of the peripheral area, is calculated, and the crack area is calculated from the apparent area. In addition, in the crack measurement method described in Patent Document 1, when photographing the surface of the structure, a crack scale with multiple marks with known lengths is simultaneously photographed, the crack length is calculated from the crack scale photographed in the original image data, and the crack width is calculated by dividing the crack area by the crack length. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2003-214827 A Summary of the Invention [Problem to be solved by the invention]
[0004] However, in the above conventional technology, since the entire image of the structure is processed, there is a problem that if there are multiple cracks in a small area in the original image data, or if there is a concrete joint that closely resembles a crack, an unintended result is obtained. In addition, the crack width of a single crack usually varies depending on the location, but the above conventional technology can only calculate the average crack width, and cannot obtain the accurate crack width at any position. For this reason, a technology that can obtain the crack width at a specified position from an image has been desired.
[0005] The present disclosure has been made in consideration of the above, and aims to obtain a crack width measurement device that can measure crack width at an arbitrarily selected position on an image more accurately than conventional methods. [Means for solving the problem]
[0006] In order to solve the above-mentioned problems and achieve the object, the crack width measurement device according to the present disclosure is a crack width measurement device that measures the actual crack width in a structure from crack image data, which is image data of a structure having cracks, and includes a crack width measurement unit. The crack width measurement unit calculates the total crack width on the crack image data of an area corresponding to a crack that intersects with a line segment connecting two points selected in the crack image data, using pixel crack width relationship information indicating the relationship between the color information value represented by one pixel of the crack image data and the crack width per pixel. In addition, the crack width measurement unit calculates the actual crack width from the total crack width, using size correspondence information indicating the correspondence relationship between the size of the crack image data and the size of the structure. Effect of the Invention
[0007] According to the present disclosure, it is possible to measure the crack width at an arbitrarily selected position on an image more accurately than in the conventional method. [Brief description of the drawings]
[0008] [Figure 1] A block diagram showing an example of the configuration of a crack width measurement device according to embodiment 1. [Diagram 2] A diagram showing an example of an overview of luminance-pixel relationship information. [Diagram 3] A flowchart showing an example of a procedure of a crack width measurement method according to the first embodiment. [Figure 4] FIG. 1 is a diagram showing a schematic example of a procedure for a crack width measurement method according to embodiment 1. [Diagram 5] A block diagram showing an example of the configuration of a computer system that realizes the crack width measurement device according to the first embodiment. [Figure 6] FIG. 13 is a diagram showing a schematic example of the configuration of a crack width measurement device according to embodiment 2. [Figure 7] FIG. 2 is a diagram showing an example of a neural network used by the model generation unit; [Figure 8] FIG. 13 is a schematic diagram showing another example of the configuration of a crack width measurement device according to embodiment 2. [Figure 9] A flowchart showing an example of a procedure of a crack classification trained model generation method in a crack width measurement device according to embodiment 2. [Figure 10] A flowchart showing an example of a procedure of a crack width measurement method according to the second embodiment. [Figure 11] A flowchart showing an example of a procedure of a crack width measurement method according to the second embodiment. [Figure 12] FIG. 10 is a diagram showing an example of a procedure for a crack width measurement method according to embodiment 2. [Figure 13] FIG. 13 is a schematic diagram showing an example of the configuration of a crack width measurement device according to embodiment 3. [Figure 14] A flowchart showing an example of a procedure of a crack width measurement method according to the third embodiment. [Figure 15] FIG. 13 is a diagram showing a schematic example of the configuration of a crack width measurement device according to embodiment 4. [Figure 16] A diagram showing an example of a setting screen and preset conditions [Figure 17] FIG. 13 is a schematic diagram showing an example of the configuration of a crack width measurement device according to embodiment 5. [Figure 18]A flowchart showing an example of the procedure of a brightness-pixel relationship trained model generation method in a crack width measurement device according to embodiment 5. [Figure 19] A flowchart showing an example of a procedure of a crack width measurement method according to a sixth embodiment. [Figure 20] FIG. 13 is a schematic diagram showing an example of the configuration of a crack width measurement device according to embodiment 7. [Figure 21] A flowchart showing an example of a procedure of a crack width learned model generation method in a crack width measurement device according to embodiment 7. [Figure 22] A flowchart showing an example of a procedure of a crack width measurement method according to a seventh embodiment DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Below, a crack width measurement device, a crack width measurement method, and a crack width measurement program relating to embodiments of the present disclosure are described in detail with reference to the drawings.
[0010] Embodiment 1 1 is a block diagram showing an example of the configuration of a crack width measurement device according to embodiment 1. The crack width measurement device 1 is a device that measures the actual crack width in a structure from crack image data, which is image data of a structure having a crack. The crack width measurement device 1 includes a crack image storage unit 11, an image display unit 12, a measurement point selection unit 13, an image resolution information storage unit 14, a brightness-pixel relationship information storage unit 15, a crack width measurement unit 16, and a measurement result display unit 17.
[0011] The crack image storage unit 11 stores crack image data, which is image data of a structure that is the target of crack measurement. Crack image data of a structure photographed by a worker is stored in the crack image storage unit 11. An example of a structure is the inner surface of a tunnel. The crack image data is data composed of multiple pixels. Here, the crack image data is assumed to be monochrome image data.
[0012] The image display unit 12 displays the crack image data stored in the crack image storage unit 11. The crack image data to be displayed is, in one example, crack image data selected by a user of the crack width measurement device 1.
[0013] The measurement point selection unit 13 provides the user with a function to select a position on the crack image data displayed on the image display unit 12 where the crack width is to be measured, and when the user selects a position where the crack width is to be measured, the measurement point selection unit 13 acquires information on the selected position on the crack image data. Since the crack image data is usually rectangular, the selected position can be expressed using a coordinate system in which a specified point on the crack image data is the origin, an axis parallel to one side constituting the outer shape of the crack image data and passing through the origin is the X-axis, and an axis parallel to the other side perpendicular to the side parallel to the X-axis and passing through the origin is the Y-axis. In one example, the user selects two points on the crack image data so that the crack is included between the selected positions.
[0014] The image resolution information storage unit 14 stores image resolution information, which is the actual size of a structure per pixel in the crack image data determined when the structure is photographed. Specifically, the image resolution information is information calculated from the relationship between the actual length of the structure and the length in the crack image data, and the correspondence between the length of one pixel in the crack image data and the actual length of the structure. The length of one pixel in the crack image data can be the length of each side of one pixel in a square shape. The image resolution information corresponds to the size correspondence information.
[0015] The brightness-pixel relation information storage unit 15 stores brightness-pixel relation information, which is information used when calculating the crack width indicated by one pixel included in the crack position of the crack image data based on the brightness value. FIG. 2 is a diagram showing an example of an outline of the brightness-pixel relation information. In FIG. 2, a crack 310 exists on a wall surface 300, and pixels p1-p9 are arranged in the width direction in a part of the crack 310. The crack 310 exists continuously in an actual structure, but the pixels of the crack image data are discrete signals. For this reason, in order to calculate the crack width of the part determined to be the crack 310 in detail, the first embodiment uses the idea of subpixels. What is used at this time is the brightness-pixel relation information 151, which is information indicating the proportion of the crack 310 in one pixel. Specifically, a first brightness value, which is a brightness value that can be determined that all of one pixel is the crack 310, is obtained from the crack image data. In addition, a second brightness value, which is a brightness value that can be determined that all of one pixel does not correspond to a crack, is obtained from the crack image data. When the crack image data is a grayscale image, that is, when the brightness value is expressed in 256 gradations, the closer the brightness value is to 0, the darker the image becomes, and the closer the brightness value is to 255, the whiter the image becomes. In the case of crack image data in which the crack 310 appears blacker than the surroundings, the crack width of a pixel with a first brightness value or less is defined as a crack width of 1 pixel. In other words, it is assumed that all of the pixel is a crack 310. In addition, the crack width of a pixel with a second brightness value or more is defined as a crack width of 0 pixels. In other words, it is assumed that all of the pixel is not a crack 310. Then, the brightness-pixel relationship information 151 is information indicating the relationship between the crack width of one pixel from the brightness value, that is, the crack width per pixel, which value is between a crack width of 0 pixels and a crack width of 1 pixel. In one example, the brightness-pixel relationship information 151 is a function indicating the relationship between the crack width per pixel and the brightness value between the first brightness value and the second brightness value.
[0016] The brightness-pixel relationship information 151 is calculated using crack image data in which the crack 310 and the non-crack portion are segmented. In one example, a plurality of pixels at a position determined to be definitely the crack 310 as a result of the segmentation can be selected, and the average value of the brightness values of the selected plurality of pixels can be set as the first brightness value. Also, a plurality of pixels at a position determined not to correspond to the crack 310 can be selected, and the average value of the brightness values of the selected plurality of pixels can be set as the second brightness value.
[0017] The brightness-pixel relationship information 151 can be the same if the crack image data has the same shooting environment. The shooting environment is an object, a date, etc. In one example, if the shooting environment of the crack image data is the same tunnel on the same date, the same brightness-pixel relationship information 151 can be used. However, if the shooting environment of the crack image data is a different tunnel on a different date, the brightness inside the tunnel will be different, so it is not desirable to use the same brightness-pixel relationship information 151. The brightness-pixel relationship information corresponds to pixel crack width relationship information.
[0018] Returning to FIG. 1, the crack width measuring unit 16 identifies the position of the crack in the area between the two points selected by the measurement point selecting unit 13 in the crack image. In one example, the position of the crack can be identified by binarizing the area between the selected two points. The crack width measuring unit 16 calculates the crack width per pixel from the luminance value for each pixel present at the crack position using the luminance-pixel relationship information. At this time, for each pixel, a crack width per pixel that is greater than a crack width of 0 pixels and less than or equal to a crack width of 1 pixel is calculated. The crack width measuring unit 16 calculates a total crack width by adding up the crack widths per pixel for pixels where the line segment connecting the selected two points passes through the crack position. Then, the crack width measuring unit 16 converts the total crack width on the image into the crack width in the tunnel using the image resolution information.
[0019] In the example of FIG. 2, pixels p1 to p9 are shown, and the luminance values are indicated by the density of hatching of pixels p1-p9. The crack width measuring unit 16 identifies pixels p2 to p7 as the position of the crack 310 by binarization processing. The crack width measuring unit 16 calculates the per-pixel crack width indicating the ratio of the width of the crack 310 per pixel width from the luminance values of each pixel from pixel p2 to pixel p7 using the luminance-pixel relationship information 151. As a result, the per-pixel crack widths of pixels p2, p3, p4, p5, p6, and p7 are 0.1, 0.2, 1.0, 1.0, 0.5, and 0.2, respectively. The crack width measuring unit 16 calculates the total crack width by adding up the per-pixel crack widths of each pixel from pixel p2 to pixel p7 that have been calculated. That is, total crack width = 0.1 + 0.2 + 1.0 + 1.0 + 0.5 + 0.2 = 3.0 pixels. The total crack width is then converted to the actual crack width using the image resolution information, and the actual crack width in the tunnel is calculated.
[0020] When the concept of subpixels is not applied, the pixels in the area determined to be the crack 310 between the two points selected by the user are calculated to have a crack width of 1 pixel. That is, in the example of FIG. 2, the length of all pixels p2-p7 in the area determined to be the crack 310, that is, 6 pixels, which is the length of 6 pixels, is set as the total crack width. On the other hand, in the first embodiment, for each pixel in the area determined to be the crack 310, the crack width per pixel is calculated from the luminance value using the luminance-pixel relationship information 151, and the total crack width is calculated as the sum of the calculated crack widths per pixel. This makes it possible to calculate the crack width more accurately. Note that the luminance value of the crack image can be the lightness of the crack image data.
[0021] Returning to FIG. 1, the measurement result display unit 17 displays the actual crack width in the tunnel, which is a structure, calculated by the crack width measurement unit 16. In one example, the measurement result display unit 17 displays the crack width superimposed on the crack image data. In this case, it is desirable to display the crack width near the line segment connecting the two points selected by the user.
[0022] 1, the processes performed by the measurement point selection unit 13, image display unit 12, and measurement result display unit 17 are processes performed via the operation screen of the crack width measurement device 1, and are processes that display crack image data, measurement results, etc. on the display unit, and acquire information selected by the input unit for the data displayed on the display unit. The processes performed by the crack width measurement unit 16 are internal processes performed within the crack width measurement device 1.
[0023] Next, a crack width measurement method in the crack width measurement device 1 configured as above will be described. FIG. 3 is a flowchart showing an example of the procedure of the crack width measurement method according to the first embodiment. FIG. 4 is a diagram showing a schematic example of the procedure of the crack width measurement method according to the first embodiment. In FIG. 4, processing performed via the operation screen and internal processing are displayed separately. Of these, for processing performed via the operation screen, an image of the contents displayed on the operation screen is shown. It is assumed that the crack image storage unit 11 has pre-stored crack image data obtained by photographing a structure such as a tunnel.
[0024] First, the image display unit 12 reads out and displays the cracked image data 111 from the cracked image storage unit 11 (step S11). The cracked image data 111 to be read out may be selected by the user, or may be selected in a predetermined order such as the shooting date and time.
[0025] The user selects two points from the displayed crack image data 111 so as to include the position where the user wants to measure the crack width. In other words, the user selects two points on the crack image data 111 so as to sandwich the crack 310 where the user wants to measure the crack width. When the user selects two points, the measurement point selection unit 13 selects the positions on the crack image data 111 of the two points selected by the user (step S12).
[0026] The crack width measuring unit 16 identifies the position of the crack 310 from the area connecting the two selected points on the crack image data 111 (step S13). In one example, the crack width measuring unit 16 performs binarization processing on the pixels of the area connecting the two points, segments the area into the crack 310 and the area other than the crack, and identifies the position of the crack 310. Next, the crack width measuring unit 16 obtains the brightness value of each pixel at the position of the identified crack 310 (step S14), and calculates the crack width per pixel from the brightness value for each pixel using the brightness-pixel relationship information 151 (step S15). The crack width measuring unit 16 calculates the total crack width, which is the sum of the crack widths per pixel of each pixel at the position of the identified crack 310 (step S16), and calculates the crack width in the actual structure from the total crack width using the image resolution information 141 (step S17). Then, the measurement result display unit 17 displays the calculated crack width 161 superimposed on the crack image data 111 (step S18). This ends the process.
[0027] Next, the hardware configuration of the crack width measurement device 1 will be described. In the crack width measurement device 1 of the first embodiment, a computer system functions as the crack width measurement device 1 by executing a program, which is a computer program describing the processing in the crack width measurement device 1, on the computer system. FIG. 5 is a block diagram showing an example of the configuration of a computer system that realizes the crack width measurement device according to the first embodiment. As shown in FIG. 5, this computer system includes a control unit 901, an input unit 902, a storage unit 903, a display unit 904, a communication unit 905, and an output unit 906, which are connected via a system bus 907.
[0028] In FIG. 5, the control unit 901 is, in one example, a processor such as a CPU (Central Processing Unit), and executes a program in which the processing in the crack width measurement device 1 of the first embodiment is described. The input unit 902 is, in one example, composed of a keyboard, a mouse, and the like, and is used by a user of the computer system to input various information. The storage unit 903 includes various memories such as a RAM (Random Access Memory), a ROM (Read Only Memory), and a storage device such as a hard disk, and stores the program to be executed by the control unit 901, necessary data obtained in the process of processing, and the like. The storage unit 903 is also used as a temporary storage area for the program. The display unit 904 is composed of a display, a liquid crystal display panel, and the like, and displays various screens to the user of the computer system. In one example, the input unit 902 and the display unit 904 may be configured by a touch panel in which the input unit 902 and the display unit 904 are integrally formed. The communication unit 905 is a receiver and a transmitter that perform communication processing. The output unit 906 is a printer, a speaker, and the like. Note that FIG. 5 is just an example, and the configuration of the computer system is not limited to the example of FIG.
[0029] Here, an example of the operation of the computer system until the program is in an executable state will be described. In the computer system having the above-mentioned configuration, for example, a program is installed in the storage unit 903 from a CD-ROM or DVD-ROM set in a CD (Compact Disc)-ROM drive or DVD (Digital Versatile Disc)-ROM drive (not shown). Then, when the program is executed, the program read from the storage unit 903 is stored in the main storage area of the storage unit 903. In this state, the control unit 901 executes the process as the crack width measurement device 1 of the first embodiment according to the program stored in the storage unit 903.
[0030] In the above explanation, a program describing the processing in the crack width measurement device 1 is provided on a CD-ROM or DVD-ROM as a recording medium, but this is not limited to this. Depending on the configuration of the computer system and the capacity of the program to be provided, for example, a program provided via a transmission medium such as the Internet via the communication unit 905 may be used.
[0031] The measurement point selection unit 13 and the crack width measurement unit 16 shown in FIG. 1 are realized by executing a program stored in the memory unit 903 shown in FIG. 5 by the control unit 901 shown in FIG. 5. The measurement point selection unit 13 and the crack width measurement unit 16 are realized by using the memory unit 903 shown in FIG. 5. The measurement point selection unit 13 is also realized by using the input unit 902 shown in FIG. 5. The image display unit 12 and the measurement result display unit 17 are realized by using the program stored in the memory unit 903 shown in FIG. 5 by using the control unit 901 shown in FIG. 5 to display the results on the display unit 904 shown in FIG. The crack image storage unit 11, the image resolution information storage unit 14, and the brightness-pixel relationship information storage unit 15 are realized by the storage unit 903 shown in FIG. 5.
[0032] The crack width measurement device 1 of the first embodiment includes a crack width measurement unit 16 that calculates the crack width on the crack image data of an area corresponding to a crack that intersects with a line segment connecting two points selected in the crack image data using brightness-pixel relationship information that indicates the relationship between the brightness value represented by one pixel of the crack image data and the crack width per pixel, and calculates the actual crack width from the crack width on the crack image data using image resolution information that indicates the correspondence between the size of the crack image data and the size of the structure. This makes it possible to measure the crack width of a crack sandwiched between two positions arbitrarily selected by a user on the crack image data more accurately than in the past.
[0033] Embodiment 2 In the technology described in Patent Document 1, the position of the crack is identified by binarization using a threshold value, but with this method, the detection accuracy often changes when the image capture conditions, such as lighting, change. For this reason, there was also the problem that the detection accuracy of the crack area was not stable.
[0034] In addition, in the first embodiment, the user selects two points from the crack image data, but there may be cases where there is no crack between the two selected points. For this reason, it is desirable for the user to be able to select two points that sandwich the crack. In the second embodiment, a crack width measurement device is described that can stabilize the detection accuracy of the crack area when calculating the crack width, and make it easier for the user to select two points that sandwich the crack.
[0035] Fig. 6 is a diagram showing a schematic example of the configuration of a crack width measurement device according to embodiment 2. The same components as those in Fig. 1 are given the same reference numerals, and their explanation will be omitted. Only the parts different from Fig. 1 will be explained. The crack width measurement device 1A according to embodiment 2 further includes a learning device 20, a crack classification learned model storage unit 21, and a crack classification unit 22 in addition to the configuration of embodiment 1.
[0036] The learning device 20 is a device that generates a crack classification trained model that is a trained model for classifying cracks in crack image data. The learning device 20 includes a data acquisition unit 201 and a model generation unit 202.
[0037] The data acquisition unit 201 acquires crack image data, which is image data including cracks, and correct answer data indicating the crack positions as learning data. The crack image data and the correct answer data indicating the crack positions are associated with each other. In addition to cracks, the crack image data also includes non-cracks such as tunnel joints. The crack image data can use image data that has been used in the past to measure crack widths.
[0038] The model generation unit 202 learns crack classification image data, which is crack classification image data, based on learning data created based on a combination of the crack image data and the correct answer data indicating the crack position output from the data acquisition unit 201. That is, a crack classification trained model is generated that infers optimal crack classification image data from the crack image data of the crack width measurement device 1A and the correct answer data indicating the crack position. Here, the learning data is data in which the crack image data and the correct answer data indicating the crack position are associated with each other. The crack classification trained model corresponds to the first trained model.
[0039] The learning device 20 is used to learn the crack classification image data of the crack width measurement device 1A, but in one example, it may be a device connected to the crack width measurement device 1A via a network and separate from the crack width measurement device 1A. Alternatively, as shown in Fig. 6, the learning device 20 may be built into the crack width measurement device 1A. Furthermore, the learning device 20 may exist on a cloud server.
[0040] The learning algorithm used by the model generation unit 202 may be a known algorithm such as so-called supervised learning, unsupervised learning, reinforcement learning, etc. As an example, a case where a neural network is applied will be described.
[0041] In one example, the model generation unit 202 learns the crack classification image data by so-called supervised learning according to a neural network model. Here, supervised learning refers to a method of providing a set of data of input and result as a label to the learning device 20, learning the features of the learning data, and inferring the result from the input.
[0042] A neural network is composed of an input layer consisting of multiple neurons, an intermediate layer, which is a hidden layer consisting of multiple neurons, and an output layer consisting of multiple neurons. The intermediate layer may be one layer or two or more layers.
[0043] FIG. 7 is a diagram showing an example of a neural network used by the model generation unit. In the case of a three-layer neural network as shown in FIG. 7, when multiple inputs are input from input layer X1 to input layer X3, the values are multiplied by weights indicated by w11 to w16 and input to intermediate layer Y1 to intermediate layer Y2. When the weights w11 to w16 are not individually distinguished, they are called weight w1. Furthermore, the results from intermediate layer Y1 to intermediate layer Y2 are further multiplied by weights indicated by w21 to w26 and output from output layer Z1 to output layer Z3. When the weights w21 to w26 are not individually distinguished, they are called weight w2. The output results from output layer Z1 to output layer Z3 vary depending on the values of weights w1 and w2.
[0044] In embodiment 2, the neural network learns crack classification image data by so-called supervised learning according to learning data created based on a combination of crack image data acquired by the data acquisition unit 201 and ground truth data indicating the crack position.
[0045] That is, the neural network learns by inputting crack image data into the input layer and adjusting the weights w1 and w2 so that the result output from the output layer approaches the ground truth data indicating the crack position.
[0046] The model generation unit 202 generates and outputs a crack classification trained model by performing the above-mentioned learning.
[0047] Returning to Figure 6, the crack classification trained model memory unit 21 stores the crack classification trained model output from the model generation unit 202.
[0048] The crack classification unit 22 infers and outputs crack classification image data from the crack image data using a crack classification trained model for inferring the position of a crack from the crack image data. That is, by inputting the crack image data acquired from the crack image storage unit 11 to this crack classification trained model, it is possible to output crack classification image data inferred from the crack image data. The crack classification image data in which cracks are classified by the crack classification unit 22 is also called semantic segmented crack image data. In other words, the crack classification image data is semantic segmented image data, and is image data in which cracks and non-crack parts are classified on a pixel-by-pixel basis.
[0049] In this embodiment, it has been described that crack classification image data is output using a crack classification trained model trained by the model generation unit 202 of the crack width measurement device 1A, but it is also possible to obtain a crack classification trained model from an external source, such as another crack width measurement device 1A, and output crack classification image data based on this crack classification trained model.
[0050] The crack classification unit 22 is used to acquire crack classification image data from the crack image data of the crack width measurement device 1A. In the example of Fig. 6, the crack classification unit 22 is built into the crack width measurement device 1A, but in one example, it may be a device connected to the crack width measurement device 1A via a network and separate from the crack width measurement device 1A. Furthermore, the crack classification unit 22 may exist on a cloud server.
[0051] The image display unit 12 displays not only the crack image data in the crack image storage unit 11, but also the crack classification image data output by the crack classification unit 22. The user selects two points in the crack image data or the crack classification image data using the measurement point selection unit 13. By referring to the crack classification image data, the user can easily recognize the position of the crack.
[0052] The measurement point selection unit 13 identifies the positions of two points selected by the user in the crack image data or the crack classification image data.
[0053] The crack width measurement unit 16 identifies the crack position differently from that in embodiment 1. That is, in embodiment 2, the crack width measurement unit 16 identifies the position of the crack boundary between the line segment connecting two points selected by the user using the crack classification image data.
[0054] 6, the processes performed by the measurement point selection unit 13, the image display unit 12, and the measurement result display unit 17 are processes performed via the operation screen of the crack width measurement device 1A, and the processes performed by the crack width measurement unit 16 and the crack classification unit 22 are internal processes performed within the crack width measurement device 1A. Also, the processes performed by the learning device 20 are pre-processing performed in advance.
[0055] FIG. 8 is a diagram showing a schematic diagram of another example of the configuration of a crack width measurement device according to embodiment 2. The same components as those in FIG. 1 and FIG. 6 are given the same reference numerals, and their description will be omitted, and only the different parts will be described. The crack width measurement device 1B shown in FIG. 8 has the same configuration as the crack width measurement device 1 of embodiment 1, and the functions of the image display unit 12, the measurement point selection unit 13, and the crack width measurement unit 16 are the same as those described in FIG. 6. In addition, a learning device 20 and an inference device 25 are provided outside the crack width measurement device 1B. The learning device 20 has the same configuration as that described above.
[0056] The inference device 25 has a data acquisition unit 251, a crack classification learned model storage unit 21, and a crack classification unit 22. The data acquisition unit 251 acquires crack image data. In one example, the data acquisition unit 251 acquires crack image data from the crack image storage unit 11. The functions of the crack classification learned model storage unit 21 and the crack classification unit 22 are similar to those described in FIG. 6.
[0057] In this way, when the inference device 25 is provided outside the crack width measurement device 1B, the crack classification image data is input from the outside to the crack width measurement device 1B. Therefore, the crack classification unit 22 and the crack classification learned model storage unit 21 are not required inside the crack width measurement device 1B. Note that, in FIG. 8, an example is shown in which the inference device 25 generates crack classification image data from the crack image data using the crack classification learned model and inputs the crack classification image data to the crack width measurement device 1B, but crack classification image data created by other methods may be input to the crack width measurement device 1B. In one example, crack classification image data in which the user classifies the crack image data into cracks and non-crack parts on a pixel-by-pixel basis may be input to the crack width measurement device 1B.
[0058] Next, the processing of the thus configured crack width measurement devices 1A, 1B will be described. Below, the learning processing in the learning device 20 will be described, and then the crack width measurement method will be described.
[0059] 9 is a flowchart showing an example of the procedure of a crack classification learned model generation method in the crack width measurement device according to embodiment 2. The data acquisition unit 201 acquires crack image data and ground truth data indicating the crack position (step S31). Note that, although the crack image data and the ground truth data indicating the crack position are acquired simultaneously, it is sufficient that the crack image data and the ground truth data indicating the crack position are input in association with each other, and the crack image data and the ground truth data indicating the crack position may be acquired at different times.
[0060] Next, the model generation unit 202 learns the crack classification image data by so-called supervised learning according to the learning data created based on a combination of the crack image data acquired by the data acquisition unit 201 and ground truth data indicating the crack positions, and generates a crack classification learned model (step S32).
[0061] Thereafter, the crack classification trained model storage unit 21 stores the crack classification trained model generated by the model generation unit 202 (step S33). This ends the crack classification trained model generation method.
[0062] 10 and 11 are flowcharts showing an example of the procedure of the crack width measurement method according to the second embodiment. FIG. 10 and FIG. 11 show the processing procedure of the crack width measurement method in the crack width measurement device 1A shown in FIG. 6. FIG. 12 is a diagram showing a schematic diagram of an example of the procedure of the crack width measurement method according to the second embodiment. In FIG. 12, the processing performed via the operation screen and the internal processing are displayed separately. Of these, for the processing performed via the operation screen, an image of the contents displayed on the operation screen is shown. It is assumed that the crack image storage unit 11 stores crack image data, which is an image of a structure such as a tunnel, in advance.
[0063] First, the image display unit 12 reads out and displays the crack image data 111 from the crack image storage unit 11 (step S51). The crack image data 111 to be acquired may be selected by the user, or may be selected in a predetermined order such as the shooting date and time.
[0064] The crack classification unit 22 acquires the same crack image data 111 as that displayed by the image display unit 12 from the crack image storage unit 11 (step S52). Next, the crack classification unit 22 acquires crack classification image data 221 that is output by inputting the acquired crack image data 111 to the crack classification trained model stored in the crack classification trained model storage unit 21 (step S53). The crack classification image data 221 is image data in which the wall surface 300 and the crack 310 are semantically segmented. The image display unit 12 displays the crack classification image data 221 together with the crack image data 111 (step S54).
[0065] The user selects two points from the displayed crack image data 111 or crack classification image data 221 so as to include the position where the crack width is to be measured. In other words, the user selects two points on the crack image data or crack classification image data so as to sandwich the crack 310 where the crack width is to be measured. Note that the user selects two points so that the line segment connecting the two points is perpendicular to the extension direction of the crack 310. When the user selects two points, the measurement point selection unit 13 acquires the positions of the two points selected by the user on the crack image data or crack classification image data (step S55). Hereinafter, the line segment connecting the two points is referred to as the width calculation reference line.
[0066] The crack width measurement unit 16 obtains the boundary between the crack 310 and the wall surface 300 in the area through which the width calculation reference line passes on the crack classification image data (step S56). After obtaining the boundary between the crack 310 and the wall surface 300, the crack width measurement unit 16 obtains the position of the pixel between the boundaries in the area through which the width calculation reference line passes (step S57). The crack width measurement unit 16 obtains the luminance value at the position of the crack image data 111 corresponding to the position of the obtained pixel (step S58). After that, the crack width measurement unit 16 calculates the crack width per pixel from the luminance value for each pixel using the luminance-pixel relationship information 151 (step S59). The crack width measurement unit 16 calculates the total crack width, which is the sum of the crack widths per pixel of the obtained pixels (step S60), and calculates the crack width in the actual structure from the total crack width using the image resolution information 141 (step S61). Then, the measurement result display unit 17 displays the calculated crack width 161 superimposed on the crack image data 111 (step S62). This ends the process.
[0067] In FIG. 10, the crack classification unit 22 generates the crack classification image data 221 by performing semantic segmentation on the displayed crack image data 111, and the image display unit 12 displays the crack classification image data 221 together with the crack image data 111, and then the user selects two points. However, after the crack image data 111 is displayed in step S51, the user may select two points on the crack image data 111. In this case, the crack classification unit 22 generates partial crack classification image data by performing semantic segmentation on a rectangular area having the two points selected by the user as its diagonal. Then, the process from step S56 onward is performed using this partial crack classification image data.
[0068] In addition, in the case of the crack width measurement device 1B shown in Figure 8, the processing in Figure 10 differs in the following respects. In the case of the crack width measurement device 1B, the processing of steps S52 and S53 is omitted. In addition, in step S54, the image display unit 12 displays the crack classification image data 221 input from outside together with the crack image data 111.
[0069] In the first embodiment, the division of the crack and the wall surface was performed by binarization processing for the area through which the line segment connecting two points selected by the user passes, but with this method, when the image capture conditions such as lighting change, the detection accuracy often changes, and the detection accuracy of the crack area is not stable. However, in the crack width measurement devices 1A and 1B of the second embodiment, the crack width measurement unit 16 sets the width calculation reference line, which is the reference for calculating the crack width, as the line segment connecting the two points selected by the user, and determines the area corresponding to the crack that intersects with the width calculation reference line using crack classification image data, which is image data obtained by segmenting the crack image data into cracks and non-cracks on a pixel-by-pixel basis. This makes it possible to make a judgment that takes into account not only the target area but also the surrounding pixels, and to make a division that distinguishes it from joints and the like, which is difficult to do with conventional image processing. As a result, in addition to the effect of the first embodiment, it has the effect of improving the detection accuracy of the crack area compared to the conventional one.
[0070] Embodiment 3 In the first and second embodiments, it is assumed that the two points selected by the user are perpendicular to the extension direction of the crack. However, it is often difficult for the user to select two points perpendicular to the extension direction of the crack. Therefore, a crack width measurement device is described that performs a correction process so that the two points selected by the user are perpendicular to the extension direction of the crack when they are not perpendicular to the extension direction of the crack.
[0071] Fig. 13 is a diagram showing a schematic example of the configuration of a crack width measurement device according to embodiment 3. Note that the same components as those in Fig. 1 and Fig. 6 are given the same reference numerals, and their explanation is omitted. Only the parts different from Fig. 1 and Fig. 6 are explained.
[0072] The crack width measurement device 1C shown in FIG. 13 further includes a crack direction calculation unit 31 in the crack width measurement device 1A in FIG. 6. The crack direction calculation unit 31 calculates the crack direction, which is the extension direction of the crack that intersects with the line segment connecting the two points selected by the user, using the crack classification image data. Specifically, the crack direction calculation unit 31 calculates the crack direction that intersects with the line segment connecting the two points selected by the user using a coordinate system in which a specified point of the crack classification image data is the origin, an axis that is parallel to one side constituting the outer shape of the crack classification image data and passes through the origin is the X-axis, and an axis that is parallel to the other side perpendicular to the side parallel to the X-axis and passes through the origin is the Y-axis. In one example, the crack direction calculation unit 31 selects a crack that passes through the line segment connecting the selected two points from among the cracks classified by the crack classification unit 22, and calculates an approximation line that has the smallest distance from all pixels constituting the crack. At this time, the approximation line is calculated by the least squares method using the coordinates of all pixels constituting the crack. The crack direction calculation unit 31 determines the slope of the calculated approximate straight line as the crack direction.
[0073] Furthermore, the crack width measurement device 1C according to embodiment 3 has a different function of the crack width measurement unit 16. The crack width measurement unit 16 further has a function of determining whether the line segment passing through the two selected points is perpendicular to the crack direction, and if the line segment is not perpendicular to the crack direction, calculating the crack width by setting the width calculation reference line as a straight line that intersects with the line segment and is perpendicular to the crack direction.
[0074] A specific function of the crack width measurement unit 16 will be described. The crack width measurement unit 16 judges whether the approximation line calculated by the crack direction calculation unit 31 is perpendicular to the line segment passing through the two selected points. If the product of the slope of the approximation line and the slope of the line segment passing through the two selected points is -1, it can be judged that the two are perpendicular, and if the product of the slopes is not -1, it can be judged that the two are not perpendicular. If the two are perpendicular, the crack width measurement unit 16 calculates the crack width obtained by the method calculated in the second embodiment, using the line segment connecting the two selected points as the width calculation reference line. On the other hand, if the two are not perpendicular, the crack width measurement unit 16 calculates one straight line that intersects with the line segment connecting the two selected points and is perpendicular to the approximation line. The one straight line that intersects with the line segment connecting the two selected points and is perpendicular to the approximation line may be a straight line that passes through the intersection of the line segment passing through the two selected points and the approximation line, or may be a straight line that passes through one of the two selected points. The crack width measurement unit 16 calculates the crack width by the method described in the second embodiment, using the calculated straight line as the width calculation reference line.
[0075] Fig. 14 is a flowchart showing an example of the procedure of a crack width measurement method according to embodiment 3. Note that the same processes as those in Fig. 10 are given the same step numbers and their explanations will be omitted.
[0076] After step S55, the crack direction calculation unit 31 calculates an approximation line indicating the extension direction of the crack that intersects with the line segment connecting the two points selected by the user on the crack classification image data (step S71).Then, the crack width measurement unit 16 determines whether the line segment connecting the two selected points is perpendicular to the approximation line (step S72).If it is perpendicular (Yes in step S72), the process proceeds to step S56 in FIG.
[0077] If they are not perpendicular (No in step S72), the crack width measurement unit 16 calculates a straight line that intersects with the line segment connecting the two selected points and is perpendicular to the approximation line (step S73). In one example, the crack width measurement unit 16 calculates a straight line that intersects with the approximation line and passes through the intersection of the line segment connecting the two selected points and the approximation line. Next, the crack width measurement unit 16 sets the calculated straight line as the width calculation reference line (step S74), and the process proceeds to step S56 in FIG. 11.
[0078] In addition, the user may be able to select between the process of calculating the crack width perpendicular to the extension direction of the crack, as described in embodiment 3, and the process of calculating the crack width as the length at which the line segment connecting two points selected by the user intersects with the crack, as described in embodiment 2.
[0079] The crack width measurement device 1C of the third embodiment further includes a crack direction calculation unit 31 that calculates the direction of the crack, and when the line segment connecting the two points selected by the user is not perpendicular to the direction of the crack, the crack width measurement unit 16 calculates a straight line perpendicular to the direction of the crack as a width calculation reference line, finds the boundary between the crack and the wall surface in the area through which the width calculation reference line passes, and calculates the crack width between the boundaries. This makes it possible to calculate a more accurate crack width at a position close to the line segment connecting the two points selected by the user.
[0080] Embodiment 4 In the first to third embodiments, the image resolution information and the brightness-pixel relationship information had to be obtained by the user for each shooting environment of the crack image data. However, when cracks in the same structure are frequently inspected, the crack image data is often shot in the same shooting environment. Also, even if the object is in a different location, the shooting environment may be similar. In such a case, the image resolution information or the brightness-pixel relationship information may be selectable from preset information.
[0081] Figure 15 is a diagram showing a schematic example of the configuration of a crack width measurement device according to embodiment 4. The same components as those in Figures 1 and 6 are given the same reference numerals, their explanations are omitted, and only the parts that differ from Figures 1 and 6 are explained. The crack width measurement device 1D according to embodiment 4 further includes a preset condition storage unit 41 and a condition setting display unit 42 in addition to the configuration of embodiment 2.
[0082] The preset condition storage unit 41 stores preset conditions including predetermined image resolution information and predetermined luminance-pixel relationship information. The predetermined image resolution information and the predetermined luminance-pixel relationship information may be image resolution information and luminance-pixel relationship information in a commonly used shooting environment.
[0083] The number of determined image resolution information and determined luminance-pixel relationship information stored in the preset condition storage unit 41 is not limited to one, and may be multiple. In one example, the image resolution information and luminance-pixel relationship information corresponding to the imaging environment of a structure that is frequently inspected may be stored in the preset condition storage unit 41.
[0084] The condition setting display unit 42 displays a setting screen for setting image resolution information and luminance-pixel relationship information, and accepts settings by the user. In the fourth embodiment, the condition setting display unit 42 displays a setting screen in which the determined image resolution information and determined luminance-pixel relationship information stored in the preset condition storage unit 41 are set as the initial value of the image resolution information and the initial value of the luminance-pixel relationship information, respectively. When a plurality of pieces of image resolution information are stored in the preset condition storage unit 41, the predetermined image resolution information can be set as the initial value. Similarly, when a plurality of pieces of luminance-pixel relationship information are stored in the preset condition storage unit 41, the predetermined luminance-pixel relationship information can be set as the initial value. In the setting screen displayed by the condition setting display unit 42, the image resolution information and the luminance-pixel relationship information can be arbitrarily changed by the user. In addition, the condition setting display unit 42 sets the image resolution information and the luminance-pixel relationship information determined by the user on the setting screen in the image resolution information storage unit 14 and the luminance-pixel relationship information storage unit 15, respectively. As a result, when measuring the crack width, the crack width measurement unit 16 performs the crack width measurement process by referring to the image resolution information stored in the image resolution information memory unit 14 and the brightness-pixel relationship information stored in the brightness-pixel relationship information memory unit 15.
[0085] FIG. 16 is a diagram showing an example of a setting screen and preset conditions. The setting screen 420 has a condition setting display area 421, a setting button 422, and a cancel button 423. In the condition setting display area 421, the determined image resolution information 411 and the determined brightness-pixel relationship information 412 in the preset conditions 410 stored in the preset condition storage unit 41 are set as initial values. Here, it is assumed that only data with a preset name of "AA" and an image resolution of "0.5" mm / pic is set in the determined image resolution information 411 of the preset conditions 410. Since only one data is set in the determined image resolution information 411, this data becomes the initial value preset in the setting screen 420. In addition, two data are set in the determined brightness-pixel relationship information 412 of the preset conditions 410. In this case, the data with the preset name "XX tunnel" set as the initial value becomes the initial value preset in the setting screen 420. Here, the luminance-pixel relation information 412 is information having a plurality of pairs of a luminance value and a pix indicating the crack width per pixel. Also, when it is desired to select other preset data, by pressing the button 421a, the determined luminance-pixel relation information 412 is displayed in a list, and data can be selected from the displayed list.
[0086] The image resolution information and the brightness-pixel relationship information in the condition setting display area 421 can be appropriately edited by the user. In one example, for the image resolution information, any numerical value can be input via the condition setting display unit 42. For the brightness-pixel relationship information, although not shown, multiple pairs of brightness values and crack widths per pixel can be registered using a known method, or data on the brightness-pixel relationship information in which multiple pairs of brightness values and crack widths per pixel are registered can be read into the crack width measurement device 1D.
[0087] The set button 422 is a button for applying the image resolution information and brightness-pixel relationship information displayed in the condition setting display area 421 to the crack width measurement device 1D. When the set button 422 is pressed, the image resolution information set in the condition setting display area 421 is set in the image resolution information storage unit 14, and the brightness-pixel relationship information set in the condition setting display area 421 is set in the brightness-pixel relationship information storage unit 15. The cancel button 423 is a button for canceling the settings on the setting screen.
[0088] 16, if the preset conditions 410 include conditions that match or are similar to the shooting conditions of the crack image data to be measured for crack width, the user selects the image resolution information or brightness-pixel relationship information that matches or is similar to the conditions. By selecting such conditions, it becomes possible to save the user the trouble of searching for or inputting image resolution information or brightness-pixel relationship information.
[0089] Note that the configuration of embodiment 1 or embodiment 3 may be configured to include the preset condition storage unit 41 and the condition setting display unit 42. In the above description, the preset condition storage unit 41 stores the determined image resolution information and the determined luminance-pixel relationship information, but it is sufficient that at least one of the determined image resolution information and the luminance-pixel relationship information is stored.
[0090] The crack width measurement method in the crack width measurement device 1D of embodiment 4 is the same as that described in embodiments 1 to 3, so the description will be omitted.
[0091] The crack width measurement device 1D of the fourth embodiment further includes a preset condition storage unit 41 that stores at least one of the determined image resolution information and the determined luminance-pixel relationship information, and a condition setting display unit 42 that displays a setting screen for image resolution information and luminance-pixel relationship information in which at least one of the determined image resolution relationship information and the determined luminance-pixel relationship information is set as an initial value. If the shooting environment of the crack image data is similar, it may be possible to use the same image resolution information and the same luminance-pixel relationship information. In such a case, the time required for measuring the crack width can be shortened by reusing the image resolution information and the luminance-pixel relationship information that match or are similar to the past, instead of creating the image resolution information and the luminance-pixel relationship information each time, in order to measure the crack width using the crack image data that is the target of the crack width measurement. In other words, it has an effect of being able to omit the trouble of the user creating the image resolution information and the luminance-pixel relationship information.
[0092] Embodiment 5. Fig. 17 is a diagram showing a schematic example of the configuration of a crack width measurement device according to embodiment 5. The same components as those in Fig. 1 and Fig. 6 are given the same reference numerals, and their description will be omitted. Only the parts different from Fig. 1 and Fig. 6 will be described. The crack width measurement device 1E according to embodiment 5 is similar to the crack width measurement device 1A in Fig. 6 except that the luminance-pixel relationship information storage unit 15 is removed, and the crack width measurement device 1E further includes a learning device 50, a luminance-pixel relationship learned model storage unit 51, a luminance-pixel relationship calculation unit 52, and a crack classification image storage unit 53.
[0093] The learning device 50 is a device that generates a luminance-pixel relationship trained model that calculates a crack width per pixel from the luminance value for each pixel in a crack area of crack image data. The learning device 50 includes a data acquisition unit 501 and a model generation unit 502. The luminance-pixel relationship trained model corresponds to the second trained model.
[0094] The data acquisition unit 501 acquires the crack image data and the relationship information between the pixel and the crack width according to the brightness value, which is the correct answer data, as learning data. The crack image data and the relationship information between the pixel and the crack width according to the brightness value are associated with each other. In one example, the relationship information between the pixel and the crack width according to the brightness value is obtained by previously performing a preliminary experiment to obtain brightness-pixel relationship information indicating how many pixels the crack width corresponds to when the width of one pixel is used as a reference, and the brightness value of each pixel of the crack image data is stored in association with the crack image data.
[0095] The model generating unit 502 learns the luminance-pixel relationship information based on the learning data created based on a combination of the crack image data output from the data acquiring unit 501 and the relationship information between pixels according to luminance values and crack widths. That is, a luminance-pixel relationship learned model is generated, which is a learned model that infers optimal luminance-pixel relationship information from the crack image data of the crack width measuring device 1E and the relationship information between pixels according to luminance values and crack widths. Here, the learning data is data in which the crack image data and the relationship information between pixels according to luminance values and crack widths are associated with each other.
[0096] The learning device 50 is used to learn the luminance-pixel relationship information of the crack width measurement device 1E, and in one example, may be a device connected to the crack width measurement device 1E via a network and separate from the crack width measurement device 1E. Alternatively, as shown in Fig. 17, the learning device 50 may be built into the crack width measurement device 1E. Furthermore, the learning device 50 may exist on a cloud server.
[0097] The learning algorithm used by the model generation unit 502 may be a known algorithm such as supervised learning, unsupervised learning, or reinforcement learning. In one example, the brightness-pixel relationship information is learned by supervised learning according to the neural network model described in Fig. 7. That is, in Fig. 7, the neural network learns by inputting crack image data to the input layer and adjusting the weights w1 and w2 so that the result output from the output layer approaches the relationship information between the pixel and the crack width according to the brightness value, and generates a brightness-pixel relationship learned model.
[0098] The luminance-pixel relationship learned model storage unit 51 stores the luminance-pixel relationship learned model output from the model generation unit 502.
[0099] The luminance-pixel relationship calculation unit 52 infers and outputs luminance-pixel relationship information from the crack image data using a luminance-pixel relationship learned model stored in the luminance-pixel relationship learned model storage unit 51 for inferring a crack width per pixel according to the luminance value represented by one pixel from the crack image data. That is, by inputting the crack image data acquired from the crack image storage unit 11 to this luminance-pixel relationship learned model, it is possible to output the luminance-pixel relationship information inferred from the crack image data. The luminance-pixel relationship information is information indicating how much of a crack width per pixel each luminance value of a pixel corresponds to in the crack image data. The luminance-pixel relationship calculation unit 52 corresponds to the relationship information calculation unit.
[0100] In addition, in embodiment 5, it has been described that the brightness-pixel relationship information is output using a brightness-pixel relationship trained model trained by the model generation unit 502 of the crack width measurement device 1E, but it is also possible to obtain a brightness-pixel relationship trained model from an external source, such as another crack width measurement device 1E, and output the brightness-pixel relationship information based on this brightness-pixel relationship trained model.
[0101] Furthermore, the luminance-pixel relationship calculation unit 52 is used to acquire luminance-pixel relationship information from the crack image data of the crack width measurement device 1E. In the example of FIG. 17, the luminance-pixel relationship calculation unit 52 is built into the crack width measurement device 1E, but in one example, it may be connected to the crack width measurement device 1E via a network and may be a device separate from the crack width measurement device 1E. Furthermore, the luminance-pixel relationship calculation unit 52 may exist on a cloud server.
[0102] The crack classification image storage unit 53 stores crack classification image data, which is image data obtained by semantic segmenting the crack image data into cracks and wall surfaces. The crack classification image storage unit 53 outputs the crack classification image data corresponding to the crack image data displayed on the image display unit 12 to the crack width measurement unit 16. In the fifth embodiment, the crack classification image storage unit 53 is configured to store the crack classification image data that has been semantically segmented outside the crack width measurement device 1E, but the crack width measurement device 1E may have a crack classification unit 22 as in the second embodiment, and the crack classification image data that has been semantically segmented by the crack classification unit 22 may be input to the crack width measurement unit 16.
[0103] Next, the processing of the crack width measurement device 1E configured as above will be described. The learning processing in the learning device 50 will be described below.
[0104] 18 is a flowchart showing an example of the procedure of a brightness-pixel relationship learned model generating method in a crack width measurement device according to embodiment 5. The data acquisition unit 501 acquires crack image data and relationship information between pixels and crack widths according to brightness values (step S91). Note that the crack image data and the relationship information between pixels and crack widths according to brightness values are acquired simultaneously, but it is sufficient that the crack image data and the relationship information between pixels and crack widths according to brightness values are input in association with each other, and the crack image data and the relationship information between pixels and crack widths according to brightness values may be acquired at different times.
[0105] Next, the model generation unit 502 learns the brightness-pixel relationship information by so-called supervised learning according to the learning data created based on a combination of the crack image data acquired by the data acquisition unit 501 and the relationship information between pixels according to brightness values and crack widths, and generates a brightness-pixel relationship learned model (step S92).
[0106] After that, the luminance-pixel relationship trained model storage unit 51 stores the luminance-pixel relationship trained model generated by the model generation unit 502 (step S93). This ends the luminance-pixel relationship trained model generation method.
[0107] The crack width measurement method in embodiment 5 is the same as that described in embodiment 2, so its description will be omitted.
[0108] In the fifth embodiment, the crack width measurement device 1E includes a luminance-pixel relationship calculation unit 52 that outputs luminance-pixel relationship information by inputting crack image data into the luminance-pixel relationship learned model. This makes it possible to automatically calculate luminance-pixel relationship information using the luminance-pixel relationship learned model generated by machine learning from the luminance around the crack and past luminance-pixel relationship information that has been accumulated in the past. As a result, it is possible to reduce the effort required by the user to calculate the luminance-pixel relationship information using image data.
[0109] Embodiment 6 In the second embodiment, the crack classification unit 22 performs semantic segmentation of wall surfaces and cracks in the crack image data. When a user selects two points using crack image data rather than crack classification image data, there may be cases where a joint similar to a crack exists between the two points selected by the user, rather than a crack. This section describes a crack width measurement device that can issue a warning in such cases.
[0110] The crack width measurement device according to the sixth embodiment has the same configuration as that of the second embodiment. However, the crack width measurement unit 16 further has a function of determining whether or not a crack exists between two points selected by the user on the crack classification image data. Specifically, the crack width measurement unit 16 further has a function of determining whether or not there exists an area corresponding to a crack that intersects with a line segment connecting the two points selected by the user. In addition, the measurement result display unit 17 further has a function of displaying to the user, when no crack exists between the two selected points, warning information indicating that there is no crack between the two selected points.
[0111] Fig. 19 is a flowchart showing an example of the procedure of a crack width measurement method according to embodiment 6. Note that the same processes as those in Fig. 10 are given the same step numbers and their explanations will be omitted.
[0112] After step S55, the crack width measurement unit 16 determines whether a crack exists in the area through which the line segment connecting the selected two points passes on the crack classification image data (step S111). If a crack exists (Yes in step S111), the process proceeds to step S56 in FIG.
[0113] On the other hand, if no crack exists (No in step S111), the measurement result display unit 17 displays warning information indicating that no crack exists between the selected two points (step S112). This ends the process.
[0114] In the above example, the configuration of embodiment 6 is applied to the configuration of embodiment 2, but the configuration of embodiment 6 may be applied to any one of embodiments 3 to 5.
[0115] In the sixth embodiment, the crack width measurement unit 16 determines whether a crack exists between two points selected by the user on the crack classification image data, and if no crack exists, the measurement result display unit 17 displays warning information indicating that there is no crack between the two selected points. This allows the user to recognize that the part of the crack image data that he or she selected thinking it was a crack is not a crack. In addition, since the crack width of a part that is not a crack is not measured, it is possible to efficiently execute the crack width measurement process.
[0116] Embodiment 7 Figure 20 is a diagram showing a schematic example of the configuration of a crack width measurement device according to embodiment 7. The same components as those in Figure 1 are given the same reference numerals, their explanations are omitted, and only the parts different from Figure 1 are explained. The crack width measurement device 1F according to embodiment 7 is different from the crack width measurement device 1 of embodiment 1 in that it does not include the image resolution information storage unit 14 and the brightness-pixel relationship information storage unit 15, and further includes a learning device 70 and a crack width learned model storage unit 71.
[0117] The learning device 70 is a device that generates a crack width trained model that calculates a crack width in an actual structure from crack image data. The learning device 70 includes a data acquisition unit 701 and a model generation unit 702. The crack width trained model corresponds to the third trained model.
[0118] The data acquisition unit 701 acquires crack image data and past inspection data as learning data. The crack image data and the past inspection data are associated with each other. The past inspection data is data including the crack position and crack width corresponding to the crack image data, and corresponds to the correct answer data. The crack width in the past inspection data may be the crack width on the crack image data, but it is preferable that it is the crack width measured on the actual structure. In other words, it is preferable that the crack width in the past inspection data is not data measured from the crack image data, but data measured using a crack scale on the actual structure. This is because data measured using a crack scale on the actual structure is closer to an accurate value.
[0119] The model generation unit 702 learns the crack width at the selected position based on learning data created based on a combination of the crack image data and past inspection data output from the data acquisition unit 701. That is, a crack width learned model is generated that infers the crack width at the optimal selected position from the crack image data and past inspection data of the crack width measurement device 1F. Here, the learning data is data in which the crack image data and past inspection data are associated with each other.
[0120] The learning device 70 is used to learn the crack width at the selected position of the crack width measurement device 1F, but in one example, it may be a device connected to the crack width measurement device 1F via a network and separate from the crack width measurement device 1F. Alternatively, as shown in Fig. 20, the learning device 70 may be built into the crack width measurement device 1F. Furthermore, the learning device 70 may exist on a cloud server.
[0121] The learning algorithm used by the model generation unit 702 may be a known algorithm such as supervised learning, unsupervised learning, or reinforcement learning. In one example, the crack width at the selected position is learned by supervised learning according to the neural network model described in Fig. 7. That is, in Fig. 7, the neural network learns by inputting crack image data to the input layer and adjusting the weights w1 and w2 so that the results output from the output layer approach past inspection data, and generates a crack width learned model.
[0122] The crack width learned model memory unit 71 stores the crack width learned model output from the model generation unit 702.
[0123] The crack width measurement unit 16 uses a crack width learned model stored in the crack width learned model storage unit 71 for inferring a crack width at a selected position from the crack image data to infer the crack width at the selected position from the crack image data, and infers the crack width at the selected position from the crack image data displayed by the image display unit 12. That is, by inputting the crack image data acquired from the crack image storage unit 11 and the selected position selected by the measurement point selection unit 13 to this crack width learned model, it is possible to output the crack width at the selected position inferred from the crack image data.
[0124] In addition, in embodiment 7, it has been described that the crack width at the selected position is output using a crack width learned model learned by the model generation unit 702 of the crack width measurement device 1F, but it is also possible to obtain a crack width learned model from outside, such as another crack width measurement device 1F, and output the crack width at the selected position based on this crack width learned model.
[0125] Furthermore, the crack width measuring unit 16 is used to obtain the crack width at the selected position from the crack image data of the crack width measuring device 1F. In the example of FIG. 20, the crack width measuring unit 16 is built into the crack width measuring device 1F, but in one example, it may be connected to the crack width measuring device 1F via a network and may be a device separate from the crack width measuring device 1F. Furthermore, the crack width measuring unit 16 may exist on a cloud server.
[0126] Next, the processing of the crack width measurement device 1F configured as above will be described. Below, the learning processing in the learning device 70 will be described, and then the crack width measurement method will be described.
[0127] 21 is a flowchart showing an example of the procedure of a crack width learned model generation method in a crack width measurement device according to embodiment 7. The data acquisition unit 701 acquires crack image data and past inspection data (step S131). The past inspection data includes a crack position and a crack width corresponding to the crack image. Note that, although the crack image data and the past inspection data are acquired simultaneously, it is sufficient that the crack image data and the past inspection data are input in association with each other, and the crack image data and the past inspection data may be acquired at different times.
[0128] Next, the model generation unit 702 learns the crack width at the selected position by so-called supervised learning based on the learning data created based on a combination of the crack image data acquired by the data acquisition unit 701 and past inspection data, and generates a crack width learned model (step S132).
[0129] After that, the crack width trained model storage unit 71 stores the crack width trained model generated by the model generation unit 702 (step S133). This ends the crack width trained model generation method.
[0130] 22 is a flowchart showing an example of the procedure of the crack width measurement method according to embodiment 7. First, the image display unit 12 reads out and displays the crack image data from the crack image storage unit 11 (step S151). The crack image data to be acquired may be selected by the user, or may be selected in a predetermined order such as the shooting date and time.
[0131] The user selects two points from the displayed crack image data so as to include the position where the user wants to measure the crack width. In other words, the user selects two points on the crack image data so as to sandwich the crack where the user wants to measure the crack width. When the user selects two points, the measurement point selection unit 13 selects the positions on the crack image data of the two points selected by the user (step S152).
[0132] The crack width measurement unit 16 acquires the displayed crack image data and the two selected positions on the crack image data (step S153). Next, the crack width measurement unit 16 inputs the crack image data and the two selected positions to the crack width learned model stored in the crack width learned model storage unit 71, and outputs the crack width at the selected positions (step S154). Then, the measurement result display unit 17 displays the crack width calculated by the crack width measurement unit 16 superimposed on the crack image data (step S155). This ends the processing.
[0133] In the crack width measurement device 1F of the seventh embodiment, crack image data and two selected positions are input to the crack width learned model stored in the crack width learned model storage unit 71, and the crack width at the selected positions is output. This makes it possible to measure the crack width simply by selecting two points in the crack image data so that they include the position where the crack width is to be measured.
[0134] In the above-mentioned first to seventh embodiments, the crack image data is monochrome data, but the crack image data may be color data. Even when the crack image data is color data, processing using the luminance value is possible. In this case, the luminance value can be a luminance value indicating luminance. In addition, a hue value, which is a value of hue, and a saturation value, which is a value of saturation, may be used. The luminance value, luminance value, hue value, and saturation value are examples of the value of color information represented by one pixel of the crack image data. In other words, the crack width measuring unit 16 calculates the crack width on the crack image data of the area corresponding to the crack that intersects with the line segment connecting two points selected in the crack image data using pixel crack width relationship information indicating the relationship between the value of color information represented by one pixel of the crack image data and the crack width per pixel, and calculates the actual crack width from the crack width on the crack image data using image resolution information indicating the correspondence relationship between the size of the crack image data and the size of the structure.
[0135] The configurations shown in the above embodiments are merely examples, and may be combined with other known technologies, or the embodiments may be combined with each other. Also, parts of the configurations may be omitted or modified without departing from the spirit of the invention.
[0136] Various aspects of the present disclosure are summarized below as appendices.
[0137] [Appendix 1] A crack width measurement device that measures an actual crack width in a structure from crack image data, which is image data of a structure having a crack, A crack width measuring device characterized by having a crack width measuring unit that calculates a total crack width on the crack image data of an area corresponding to the crack that intersects with a line segment connecting two points selected in the crack image data using pixel crack width relationship information that indicates the relationship between the color information value represented by one pixel of the crack image data and the crack width per pixel, and calculates the actual crack width from the total crack width using size correspondence information that indicates the correspondence relationship between the size of the crack image data and the size of the structure. [Appendix 2] A preset condition storage unit that stores at least one of the determined pixel crack width relationship information and the determined size correspondence information; a condition setting display unit that displays a setting screen for the pixel crack width relationship information and the size correspondence information, in which at least one of the determined pixel crack width relationship information and the determined size correspondence information is set as an initial value; A crack width measurement device as described in appendix 1, further comprising: [Appendix 3] The crack width measuring device described in Appendix 1 or 2 is characterized in that the crack width measuring unit determines the area corresponding to the crack that intersects with the width calculation reference line, which is the basis for calculating the crack width, using the line segment as a width calculation reference line, and using crack classification image data, which is image data obtained by segmenting the crack image data into the crack and other than the crack on a pixel-by-pixel basis. [Appendix 4] The crack width measurement device described in Appendix 3, further comprising a crack classification unit that outputs the crack classification image data from the crack image data using a first trained model for inferring the position of the crack from the crack image data. [Appendix 5] Further comprising a crack direction calculation unit that calculates a crack direction, which is an extension direction of the crack that intersects with the line segment, using the crack classification image data; The crack width measuring device described in Appendix 3 is characterized in that the crack width measuring unit determines whether the line segment is perpendicular to the crack direction, and if the line segment is not perpendicular to the crack direction, the width calculation reference line is set to a straight line that intersects the line segment and is perpendicular to the crack direction. [Appendix 6] The crack width measuring device according to any one of appendices 3 to 5, further comprising a pixel crack width relationship information calculation unit that outputs the pixel crack width relationship information from the crack image data using a second trained model for inferring a crack width per pixel from the crack image data according to the value of color information represented by the pixel. [Appendix 7] A crack image display unit that displays the crack image data; A measurement result display unit that displays the actual crack width calculated by the crack width measurement unit; Further equipped with The crack width measurement unit further has a function of determining whether there is an area corresponding to the crack that intersects with the line segment connecting the two points, A crack width measuring device described in any one of Appendices 3 to 6, characterized in that the measurement result display unit further has a function of displaying warning information indicating that there is no crack between the two points when there is no area corresponding to the crack that intersects with the line connecting the two points. [Appendix 8] An image display unit that displays the crack image data; A measurement result display unit that displays the actual crack width calculated by the crack width measurement unit; Further equipped with The crack width measuring device described in Appendix 1, characterized in that the crack width measuring unit infers the crack width at the selected position from the crack image data displayed by the image display unit using a third learned model for inferring the crack width at the selected position from the crack image data. [Appendix 9] A crack width measuring device described in any one of Appendices 1 to 8, characterized in that the value of the color information represented by the one pixel is at least one selected from the group of brightness value, lightness value, hue value, and saturation value. [Appendix 10] A crack width measurement method for measuring an actual crack width in a structure from crack image data, which is image data of a structure having a crack, using a computer, comprising: A first step of calculating a total crack width on the crack image data of an area corresponding to the crack that intersects with a line segment connecting two points selected in the crack image data using pixel crack width relationship information that indicates the relationship between the color information value represented by one pixel of the crack image data and the crack width per pixel; A second step of calculating the actual crack width from the total crack width using size correspondence information indicating a correspondence relationship between the size of the crack image data and the size of the structure; A crack width measurement method comprising the steps of: [Appendix 11] The crack width measurement method described in Appendix 10, characterized in that in the first step, a width calculation reference line serving as the basis for calculating the crack width is used as the line segment, and an area corresponding to the crack that intersects with the width calculation reference line is determined using crack classification image data, which is image data obtained by segmenting the crack image data on a pixel-by-pixel basis into the crack and other than the crack. [Appendix 12] The crack width measurement method described in Appendix 11, characterized in that in the first step, the crack classification image data is generated from the crack image data using a first trained model for inferring the position of the crack from the crack image data. [Appendix 13] A first step of calculating a total crack width on the crack image data of an area corresponding to the crack that intersects with a line segment connecting two points selected in the crack image data, using pixel crack width relationship information that indicates the relationship between the color information value represented by one pixel of the crack image data, which is image data of a structure having a crack, and the crack width per pixel; A second step of calculating an actual crack width in the structure from the total crack width using size correspondence information indicating a correspondence relationship between the size of the crack image data and the size of the structure; A crack width measurement program characterized by causing a computer to execute the above. [Appendix 14] The crack width measurement program described in Appendix 13, characterized in that in the first step, a width calculation reference line serving as the basis for calculating the crack width is used as the line segment, and an area corresponding to the crack that intersects with the width calculation reference line is determined using crack classification image data, which is image data obtained by segmenting the crack image data into the crack and other than the crack on a pixel-by-pixel basis. [Appendix 15] The crack width measurement program described in Appendix 14, characterized in that in the first step, the crack classification image data is generated from the crack image data using a first trained model for inferring the position of the crack from the crack image data. [Explanation of symbols]
[0138] 1, 1A, 1B, 1C, 1D, 1E, 1F Crack width measurement device, 11 Crack image storage unit, 12 Image display unit, 13 Measurement location selection unit, 14 Image resolution information storage unit, 15 Brightness-pixel relationship information storage unit, 16 Crack width measurement unit, 17 Measurement result display unit, 20, 50, 70 Learning device, 21 Crack classification trained model storage unit, 22 Crack classification unit, 25 Inference device, 31 Crack direction calculation unit, 41 Preset condition storage unit, 42 Condition setting display unit, 51 Brightness-pixel relationship trained model storage unit, 52 Brightness-pixel relationship calculation unit, 53 Crack classification image storage unit, 71 Crack width trained model storage unit, 111 Crack image data, 141, 411 Image resolution information, 151, 412 Brightness-pixel relationship information, 161 Calculated crack width, 201,251,501,701 data acquisition unit, 202,502,702 model generation unit, 221 crack classification image data, 300 wall surface, 310 crack, 410 preset conditions, 420 setting screen, 421 condition setting display area, 421a button, 422 setting button, 423 cancel button, 901 control unit, 902 input unit, 903 memory unit, 904 display unit, 905 communication unit, 906 output unit, 907 system bus, p1-p9 pixels.
Claims
1. A crack width measurement device that measures an actual crack width in a structure from crack image data, which is image data of a structure having a crack, A crack width measuring device characterized by having a crack width measuring unit that calculates a total crack width on the crack image data of an area corresponding to the crack that intersects with a line segment connecting two points selected in the crack image data using pixel crack width relationship information that indicates the relationship between the color information value represented by one pixel of the crack image data and the crack width per pixel, and calculates the actual crack width from the total crack width using size correspondence information that indicates the correspondence relationship between the size of the crack image data and the size of the structure.
2. A preset condition storage unit that stores at least one of the determined pixel crack width relationship information and the determined size correspondence information; a condition setting display unit that displays a setting screen for the pixel crack width relationship information and the size correspondence information, in which at least one of the determined pixel crack width relationship information and the determined size correspondence information is set as an initial value; The crack width measurement device according to claim 1, further comprising:
3. The crack width measuring device described in claim 1, characterized in that the crack width measuring unit determines the area corresponding to the crack that intersects with the width calculation reference line, which is the basis for calculating the crack width, using the line segment as a width calculation reference line, and using crack classification image data, which is image data obtained by segmenting the crack image data into the crack and other than the crack on a pixel-by-pixel basis.
4. The crack width measurement device described in claim 3, further comprising a crack classification unit that outputs the crack classification image data from the crack image data using a first trained model for inferring the position of the crack from the crack image data.
5. Further comprising a crack direction calculation unit that calculates a crack direction, which is an extension direction of the crack that intersects with the line segment, using the crack classification image data; The crack width measuring device described in claim 3, characterized in that the crack width measuring unit determines whether the line segment is perpendicular to the crack direction, and if the line segment is not perpendicular to the crack direction, the width calculation reference line is set to a straight line that intersects the line segment and is perpendicular to the crack direction.
6. The crack width measuring device according to claim 3, further comprising a pixel crack width relationship information calculation unit that outputs the pixel crack width relationship information from the crack image data using a second learned model for inferring a crack width per pixel from the crack image data according to the value of color information represented by the pixel.
7. A crack image display unit that displays the crack image data; A measurement result display unit that displays the actual crack width calculated by the crack width measurement unit; Further equipped with The crack width measurement unit further has a function of determining whether there is an area corresponding to the crack that intersects with the line segment connecting the two points, The crack width measuring device described in claim 3, characterized in that the measurement result display unit further has a function of displaying warning information indicating that there is no crack between the two points when there is no area corresponding to the crack that intersects with the line segment connecting the two points.
8. An image display unit that displays the crack image data; A measurement result display unit that displays the actual crack width calculated by the crack width measurement unit; Further equipped with The crack width measuring device described in claim 1, characterized in that the crack width measuring unit infers the crack width at the selected position from the crack image data displayed by the image display unit using a third learned model for inferring the crack width at the selected position from the crack image data.
9. A crack width measurement device described in any one of claims 1 to 8, characterized in that the value of the color information represented by the one pixel is at least one selected from the group of brightness value, lightness value, hue value and saturation value.
10. A crack width measurement method for measuring an actual crack width in a structure from crack image data, which is image data of a structure having a crack, using a computer, comprising: A first step of calculating a total crack width on the crack image data of an area corresponding to the crack that intersects with a line segment connecting two points selected in the crack image data using pixel crack width relationship information that indicates the relationship between the color information value represented by one pixel of the crack image data and the crack width per pixel; A second step of calculating the actual crack width from the total crack width using size correspondence information indicating a correspondence relationship between the size of the crack image data and the size of the structure; A crack width measurement method comprising the steps of:
11. The crack width measurement method described in claim 10, characterized in that in the first step, a width calculation reference line serving as the basis for calculating the crack width is used as the line segment, and an area corresponding to the crack that intersects with the width calculation reference line is determined using crack classification image data, which is image data obtained by segmenting the crack image data on a pixel-by-pixel basis into the crack and other than the crack.
12. The crack width measurement method described in claim 11, characterized in that in the first step, the crack classification image data is generated from the crack image data using a first trained model for inferring the position of the crack from the crack image data.
13. A first step of calculating a total crack width on the crack image data of an area corresponding to the crack that intersects with a line segment connecting two points selected in the crack image data, using pixel crack width relationship information that indicates the relationship between the color information value represented by one pixel of the crack image data, which is image data of a structure having a crack, and the crack width per pixel; A second step of calculating an actual crack width in the structure from the total crack width using size correspondence information indicating a correspondence relationship between the size of the crack image data and the size of the structure; A crack width measurement program characterized by causing a computer to execute the above.
14. The crack width measurement program described in claim 13, characterized in that in the first step, a width calculation reference line serving as the basis for calculating the crack width is used as the line segment, and an area corresponding to the crack that intersects with the width calculation reference line is determined using crack classification image data, which is image data obtained by segmenting the crack image data on a pixel-by-pixel basis into the crack and other than the crack.
15. The crack width measurement program described in claim 14, characterized in that in the first step, the crack classification image data is generated from the crack image data using a first trained model for inferring the position of the crack from the crack image data.
Citation Information
Patent Citations
Method for measuring craze by image processing
JP2003214827A
