Inspection apparatus, inspection method, and computer-readable storage medium

Abstract

An image is automatically determined whether or not there is a defect in position, size, shape, etc. The inspection device (1) is provided with: an inspection image acquisition unit (103) that acquires an inspection image for determining whether or not an inspection object has an internal defect; and a defect presence / absence determination unit (105) that determines the presence / absence of a defect using a restored image generated by inputting the inspection image into a generation model constructed by machine learning using images of the inspection object without defects as training data, the generation model being configured to generate a new image having the same features as the input image.

Description

TECHNICAL FIELD

[0001] The present application relates to an inspection device or the like which determines the presence or absence of a defect from an image of an inspection object. BACKGROUND

[0002] For the defect inspection of industrial products or the like at the time of shipment or the like, non-destructive inspection has been widely used in the past. For example, in the following Patent Document 1, a method called ultrasonic flaw detection test is used to automatically inspect a defect of a test object. The ultrasonic flaw detection test is a test in which a signal caused by a defective portion of a test object is selected from a measured ultrasonic signal to determine the position of the defective portion. For the technology of Patent Document 1, a signal caused by a defective portion of a test object is detected based on a difference between a waveform obtained by analysis and a waveform obtained by measurement.

[0003] However, the method of Patent Document 1 is based on the waveform of an ultrasonic signal and is difficult to intuitively understand, and therefore, in recent years, ultrasonic images in which waveform data is imaged are also used for defect inspection. In addition, in a radiographic test (RT), it is determined from an image (radiographic photograph) whether or not a test object has a defect.

[0004] PRIOR ART DOCUMENTS

[0005] PATENT DOCUMENT

[0006] Patent Document 1: Japanese Laid-Open Patent Publication "Tokukai 2003-232779" SUMMARY

[0007] (I) PROBLEMS TO BE SOLVED BY THE INVENTION

[0008] However, when a defect whose position, size, shape, or the like is not fixed is inspected, it is difficult to automatically determine the presence or absence of a defect from an image. Therefore, the current situation is that an inspector determines the presence or absence of a defect by visual inspection. Furthermore, visual inspection has problems such as a long operation time, a heavy workload, and a deviation in inspection results due to the proficiency of an inspector.

[0009] An object of one embodiment of the present application is to provide an inspection device or the like which can automatically determine whether or not a defect whose position, size, shape, or the like is not fixed is present in an image.

[0010] (II) SOLUTION TO THE PROBLEM

[0011] To solve the above problems, an inspection device of one embodiment of the present application includes an inspection image acquisition unit which acquires an image for determining the presence or absence of a defect in an internal portion of an inspection target, and a defect presence / absence determination unit which determines whether the inspection target has a defect using a restored image generated by inputting the inspection image to a generation model which is constructed by machine learning using an image of the inspection target having no defect as training data and is capable of generating a new image having the same features as an input image.

[0012] In addition, to solve the above problems, an inspection method of one embodiment of the present application uses an inspection device and includes an inspection image acquisition step of acquiring an image for determining the presence or absence of a defect in an internal portion of an inspection target, and a defect presence / absence determination step of determining whether the inspection target has a defect using a restored image generated by inputting the inspection image to a generation model which is constructed by machine learning using an ultrasonic image of the inspection target having no defect as training data and is capable of generating a new image having the same features as an input image.

[0013] (III) Advantageous Effects

[0014] According to one embodiment of the present application, it is possible to automatically determine whether a defect having a position, size, shape, or the like which is not fixed is present in an image. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 is a block diagram illustrating an example of the structure of main parts of an inspection device of Embodiment 1 of the present application.

[0016] Figure 2 is a diagram illustrating an outline of an inspection system including the above-described inspection device.

[0017] Figure 3 is a diagram illustrating an outline of an inspection performed using the above-described inspection device.

[0018] Figure 4 is a diagram illustrating an example of a distribution of pixel values in a difference image generated on the basis of an ultrasonic image of a defective portion and an example of a distribution of pixel values in a difference image generated on the basis of an ultrasonic image of a non-defective portion.

[0019] Figure 5 is a diagram illustrating an example of generating a heat map from an ultrasonic image and performing threshold processing on the generated heat map.

[0020] Figure 6 is a diagram illustrating a relationship between a position of a defect, an ultrasonic image, and a heat map.

[0021] Figure 7is a diagram illustrating a method of detecting a defect as one defect by synthesizing defects appearing in a plurality of ultrasonic images.

[0022] Figure 8 is a diagram showing an example of an output of a result of inspection.

[0023] Figure 9 is a diagram showing an example of a process of constructing various models used in inspection and determining a threshold value.

[0024] Figure 10 is a diagram showing an example of an inspection method using the inspection apparatus.

[0025] Figure 11 is a flowchart showing an example of a defect type determination process.

[0026] Figure 12 is a block diagram showing an example of a structure of main parts of the inspection apparatus according to Embodiment 2.

[0027] Figure 13 is a diagram illustrating a method of detecting a defect region.

[0028] Figure 14 is a diagram showing an example of a region set according to a defect type.

[0029] Figure 15 is a flowchart showing an example of a defect type determination process performed by the inspection apparatus. DETAILED DESCRIPTION

[0030] (Embodiment 1)

[0031] (Outline of System)

[0032] Based on Figure 2 An outline of an inspection system according to an embodiment of the present application will be described. Figure 2 is a diagram schematically showing an inspection system 100. The inspection system 100 is a system for inspecting whether or not an inspection target object has a defect based on an image of the inspection target object, and includes an inspection apparatus 1 and an ultrasonic flaw detection apparatus 7.

[0033] In the present embodiment, an example in which the inspection system 100 is used to inspect the tube end weld portion of a heat exchanger for defects will be described. Further, the tube end weld portion refers to a portion in which a plurality of metal tubes that constitute the heat exchanger and a metal tube plate that bundles the tubes are welded. In addition, the defect of the tube end weld portion refers to a defect in which a gap is generated inside the tube end weld portion. Further, the tube and the tube plate described above can be made of a non-ferrous metal such as aluminum or can be made of resin. In addition, the inspection system 100 can also be used to inspect whether or not a weld portion (root weld portion) of a tube seat and a tube of a boiler device used in a waste incineration facility or the like has a defect. Of course, the inspection site is not limited to the weld portion, and the inspection object is not limited to the heat exchanger. In addition, an abnormal site detected using an ultrasonic waveform or an ultrasonic image is generally referred to as a "damage" in the field of non-destructive inspection. Such a "damage" is also included in the category of the "defect" described above. In addition, the "defect" described above also includes a defect such as a loss or a crack.

[0034] When inspection is performed, as shown in FIG. 8, the probe coated with the contact medium is inserted from the tube end, the ultrasonic wave is propagated from the inner wall surface side of the tube toward the tube end weld portion through the probe, and the echo of the ultrasonic wave is measured. When a defect in which a gap is generated inside the tube end weld portion occurs, the echo from the gap can be measured, and thus the defect can be detected using the echo. Figure 2

[0035] For example, in the enlarged view of the periphery of the probe shown on the lower left in FIG. 9, the ultrasonic wave indicated by an arrow L3 is propagated to a portion inside the tube end weld portion that does not have a gap. Therefore, the echo of the ultrasonic wave indicated by the arrow L3 is not measured. On the other hand, the ultrasonic wave indicated by an arrow L2 is propagated toward a portion inside the tube end weld portion that has a gap, and thus the echo of the ultrasonic wave reflected at the gap is measured. Figure 2

[0036] In addition, since the ultrasonic wave is also reflected at the peripheral edge portion of the tube end weld portion, the echo of the ultrasonic wave propagated to the peripheral edge portion is also measured. For example, the ultrasonic wave indicated by an arrow L1 is propagated to a position closer to the tube end side than the tube end weld portion, and thus does not reach the tube end weld portion but is reflected at the tube surface on the tube end side of the tube end weld portion. Therefore, the echo from the tube surface is measured by the ultrasonic wave indicated by the arrow L1. In addition, the ultrasonic wave indicated by an arrow L4 is reflected at the tube surface on the tube deep side of the tube end weld portion, and thus the echo is measured.

[0037] ​​Since the pipe end weld portion exists over 360 degrees around the pipe, the probe is rotated by a prescribed angle (for example, 1 degree) and repeated measurements are performed. Also, data indicating the probe measurement results are transmitted to the ultrasonic flaw detection device 7. The probe can be an array probe composed of a plurality of array elements, for example. If it is an array probe, the pipe end weld portion having a width in the extending direction of the pipe is efficiently inspected by being arranged in such a manner that the arrangement direction of the array elements coincides with the extending direction of the pipe. Further, the above-described array probe can be a matrix array probe in which a plurality of array elements are arranged in the longitudinal and lateral directions, respectively.

[0038] The ultrasonic flaw detection device 7 generates an ultrasonic image in which the echoes of the ultrasonic waves propagated toward the pipe and the pipe end weld portion are imaged, using the data indicating the probe measurement results. In Figure 2 An example of the ultrasonic image generated by the ultrasonic flaw detection device 7, that is, an ultrasonic image 111 is shown in FIG. 11. Further, it can also be a structure in which the inspection device 1 generates the ultrasonic image 111. In this case, the ultrasonic flaw detection device 7 transmits the data indicating the probe measurement results to the inspection device 1.

[0039] The measured echo intensity is indicated as a pixel value of each pixel in the ultrasonic image 111. In addition, the image region of the ultrasonic image 111 can be divided into a pipe region arl corresponding to the pipe, a weld region ar2 corresponding to the pipe end weld portion, and a peripheral echo region ar3 and ar4 in which echoes from the periphery of the pipe end weld portion appear.

[0040] As described above, the ultrasonic wave propagated from the probe in the direction indicated by an arrow Ll is reflected on the pipe surface on the pipe end side of the pipe end weld portion. In addition, this ultrasonic wave is also reflected on the inner surface of the pipe and repeatedly occurs. Therefore, repeated echoes al to a4 appear in the peripheral echo region ar3 along the arrow Ll in the ultrasonic image 111. In addition, the ultrasonic wave propagated from the probe in the direction indicated by an arrow L4 is also repeatedly reflected on the outer surface and the inner surface of the pipe. Therefore, repeated echoes a6 to a9 appear in the peripheral echo region ar4 along the arrow L4 in the ultrasonic image 111. These echoes appearing in the peripheral echo regions ar3 and ar4 are also called bottom surface echoes.

[0041] For the ultrasonic wave propagated from the probe in the direction indicated by an arrow L3, since there is no case where it is reflected, no echo appears in the region along the arrow L3 in the ultrasonic image 111. On the other hand, for the ultrasonic wave propagated from the probe in the direction indicated by an arrow L2, since it is reflected in the gap, that is, the defect site in the pipe end weld portion, an echo a5 appears in the region along the arrow L2 in the ultrasonic image 111.

[0042] The inspection device 1 analyzes such an ultrasonic image 111 to inspect whether there is a defect in the pipe end weld portion, the details of which will be described later. In addition, the inspection device 1 also automatically determines the type of the defect when it is determined that there is a defect.

[0043] (Configuration of inspection device)

[0044] Based on Figure 1 The configuration of the inspection device 1 will be described. Figure 1 is a block diagram showing an example of the configuration of the main part of the inspection device 1. As shown in Figure 1 , the inspection device 1 is provided with a control section 10 that collectively controls each section of the inspection device 1, and a storage section 11 that stores various data used by the inspection device 1. In addition, the inspection device 1 is provided with an input section 12 that accepts input operations to the inspection device 1, and an output section 13 that is used for data output from the inspection device 1.

[0045] The control section 10 includes an inspection target region extraction section 101, an inspection image generation section 102, an inspection image acquisition section 103, a recovered image generation section 104, a defect presence / absence determination section 105, a heat map generation section 106, a defect type determination section 107, a comprehensive detection section 108, and a defect length calculation section 109. In addition, the storage section 11 stores an ultrasonic image 111 and inspection result data 112.

[0046] The inspection target region extraction section 101 extracts an inspection target region from the ultrasonic image 111. The inspection target region is a region sandwiched by two peripheral echo regions in which echoes from a peripheral portion of an inspection target site in an inspection target object repeatedly occur, the details of which will be described later. The inspection target region extraction section 101 can extract the inspection target region using an extraction model constructed by machine learning. Furthermore, the inspection target region extraction section 101 generates extraction region information indicating the position and range of the extracted inspection target region in the ultrasonic image 111.

[0047] The inspection image generation section 102 cuts out the portion of the inspection target region indicated by the above-described extraction region information from the ultrasonic image 111, and generates an inspection image for determining whether there is a defect in the inside of the inspection target object.

[0048] The inspection image acquisition section 103 acquires the inspection image. The inspection device 1 is provided with the inspection target region extraction section 101 and the inspection image generation section 102 as described above, and therefore the inspection image acquisition section 103 acquires the inspection image generated by the inspection image generation section 102. Furthermore, the inspection image can be generated by another device. In this case, the inspection image acquisition section 103 acquires the inspection image generated by the other device.

[0049] The restored image generation unit 104 generates a new image having the same characteristics as the input inspection image by inputting the inspection image acquired by the inspection image acquisition unit 103 to a generation model. Hereinafter, the image generated by the restored image generation unit 104 is referred to as a restored image. The generation model used to generate the restored image is also referred to as an autoencoder, and is constructed by machine learning using an image of an inspection target object having no defect as training data, the details of which will be described later. Further, the "characteristics" described above refer to arbitrary information obtained from an image, for example, a distribution state or variance of pixel values in the image, and the like, which are included in the "characteristics" described above.

[0050] The defect presence / absence determination unit 105 determines whether or not the inspection target object has a defect using the restored image generated by the restored image generation unit 104. Specifically, when the variance of the pixel values of each pixel constituting the difference image between the inspection image and the restored image exceeds a predetermined threshold value, the defect presence / absence determination unit 105 determines that there is a defect in the inspection target object. The details of this determination method will be described later.

[0051] The heat map generation unit 106 generates a heat map that represents the pixel values of each pixel constituting the difference image described above in color or gradation. In addition, the heat map generation unit 106 can perform threshold processing on the generated heat map. The details of the heat map and the threshold processing will be described later.

[0052] The defect type determination unit 107 determines the type of the defect appearing in the inspection image determined to have a defect by the defect presence / absence determination unit 105. More specifically, the defect type determination unit 107 determines the defect type based on an output value obtained by inputting the heat map generated by the heat map generation unit 106 to a type determination model. This type determination model is a model constructed by machine learning using a heat map of a difference image generated from an inspection image of an inspection target object having a defect of which the type is known as teacher data.

[0053] When the defect presence / absence determination unit 105 determines that a plurality of ultrasonic images 111 each corresponding to a portion of the inspection target object, i.e., portions adjacent to each other, have defects, the comprehensive detection unit 108 detects the defects appearing in the plurality of ultrasonic images 111 as one defect. The details of the comprehensive detection of defects will be described later.

[0054] The defect length calculation unit 109 calculates the length of the defect that is comprehensively detected by the comprehensive detection unit 108. The method of calculating the length of the defect will be described later.

[0055] As described above, the ultrasonic image 111 is an image obtained by imaging the echo of an ultrasonic wave propagated to the inspection target object, and is generated by the ultrasonic flaw detection device 7.

[0056] The inspection result data 112 is data indicating the result of the defect inspection of the inspection apparatus 1. In the inspection result data 112, recorded is the result of the determination of the presence or absence of a defect by the defect presence / absence determination section 105 with respect to each of the inspection images acquired by the inspection image acquisition section 103. Further, in the inspection result data 112, recorded is the result of the determination of the defect type by the defect type determination section 107 with respect to the inspection image determined to have a defect. Furthermore, in the inspection result data 112, recorded is the defect that has been subjected to the synthesis by the synthesis detection section 108, and also recorded is the length of the defect that has been subjected to the synthesis calculated by the defect length calculation section 109.

[0057] As described above, the inspection apparatus 1 is provided with the inspection image acquisition section 103 that acquires an inspection image used for determining the presence or absence of a defect in the inside of the inspection target. In addition, the inspection apparatus 1 is provided with the defect presence / absence determination section 105 that determines the presence or absence of a defect in the inspection target using the restoration image generated by the restoration image generation section 104. Furthermore, the above-described restoration image is an image generated by inputting the above-described inspection image to a generation model constructed using an ultrasonic image of an inspection target having no defect as training data and capable of generating a new image having the same features as the input image.

[0058] The above-described generation model is constructed using machine learning using an image of an inspection target having no defect as training data. Therefore, in a case where an inspection image of an inspection target having no defect is input to this generation model, there is a high likelihood that a new image having the same features as the inspection image is output as a restoration image.

[0059] On the other hand, in a case where an inspection image of an inspection target having a defect is input to this generation model, even if a defect having an arbitrary shape and size is present at an arbitrary position in the inspection image, there is a high likelihood that the restoration image has features different from those of the inspection image.

[0060] Thus, in the restoration image generated from the inspection image in which a defect is present and the restoration image generated from the inspection image in which no defect is present, there is a difference in whether the inspection image input to the generation model is accurately restored.

[0061] Therefore, according to the above-described structure in which the presence or absence of a defect is determined using a restoration image generated by the above-described generation model, it is possible to automatically determine whether or not a defect having an unfixed position, size, shape, and the like is present using an image of an inspection target.

[0062] (Summary of Inspection)

[0063] A summary of the inspection using the inspection apparatus 1 will be described based on Figure 3 Figure 3 is a diagram summarizing the inspection using the inspection apparatus 1. Further, in​Figure 3 The process after the ultrasonic image 111 generated by the ultrasonic flaw detection device 7 is stored in the storage section 11 of the inspection device 1 is shown in FIG. 12.

[0064] First, the inspection target region extraction section 101 inputs the ultrasonic image 111 into the extraction model and generates extraction region information based on the output value thereof. As described above, the extraction region information indicates a region in the image region of the ultrasonic image 111 that should be extracted as an inspection image. The inspection image generation section 102 cuts out the region indicated by the extraction region information from the ultrasonic image 111 and generates an inspection image 111A. Further, based on the extraction region information, the inspection image generation section 102 generates a region of interest image 111B. Figure 9 The generation method of the extraction model will be described.

[0065] In this way, the inspection image generation section 102 extracts the region indicated by the extraction region information from the ultrasonic image 111 as an inspection target region and generates the inspection image 111A. This inspection target region is a region sandwiched by two peripheral echo regions (the peripheral echo regions ar3 and ar4 in the example) in which echoes from the peripheral portion of the inspection target site in the inspection target object repeatedly occur. Further, the inspection image acquisition section 103 acquires this inspection image 111A. Figure 2 As shown in FIG. 11, in the peripheral portion of the inspection target site in the ultrasonic image 111, a prescribed echo (echoes al to a4 and a6 to a9) caused by the shape or the like of the peripheral portion is repeatedly observed. Therefore, the region in the ultrasonic image 111 corresponding to the inspection target site can be determined from the positions of the peripheral echo regions ar3 and ar4 in which such echoes repeatedly occur. That is, according to the above-described structure, the inspection image 111A can be automatically generated. Further, the case where the prescribed echo appears in the peripheral portion of the inspection target site is not limited to the ultrasonic image 111 of the pipe end welded portion. Therefore, for the structure that extracts the region surrounded by the peripheral echo region as the inspection target region, it is also applicable to other inspections other than the pipe end welded portion.

[0066] As shown in FIG. 11, in the peripheral portion of the inspection target site in the ultrasonic image 111, a prescribed echo (echoes al to a4 and a6 to a9) caused by the shape or the like of the peripheral portion is repeatedly observed. Therefore, the region in the ultrasonic image 111 corresponding to the inspection target site can be determined from the positions of the peripheral echo regions ar3 and ar4 in which such echoes repeatedly occur. That is, according to the above-described structure, the inspection image 111A can be automatically generated. Further, the case where the prescribed echo appears in the peripheral portion of the inspection target site is not limited to the ultrasonic image 111 of the pipe end welded portion. Therefore, for the structure that extracts the region surrounded by the peripheral echo region as the inspection target region, it is also applicable to other inspections other than the pipe end welded portion. Figure 2

[0067] Next, the inspection image acquisition section 103 transmits the acquired inspection image 111A to the restored image generation section 104. The restored image generation section 104 inputs the inspection image 111A generation model and generates a restored image 111B based on the output value thereof. Further, the generation method of the generation model will be described based on the flowchart shown in FIG. 13. Figure 9

[0068] ​​Also, the inspection image acquisition section 103 generates a removed image 111C by removing the peripheral echo region from the inspection image 111A, and generates a removed image (restored) 111D by removing the peripheral echo region from the restored image 111B. Further, the position and size of the peripheral echo region appearing in the inspection image 111A are substantially constant if the inspection object is the same. Therefore, the inspection image acquisition section 103 can remove the peripheral echo region in a prescribed range in the inspection image 111A. In addition, the inspection image acquisition section 103 can analyze the inspection image 111A to detect the peripheral echo region, and remove the peripheral echo region based on the detection result.

[0069] By removing the peripheral echo region in the above-described manner, the defect presence / absence determination section 105 determines the presence / absence of a defect with respect to an image region remaining after the peripheral echo region is removed from the image region of the restored image 111B. Thus, it is possible to prevent the influence of the echo from the peripheral portion, and it is possible to determine the presence / absence of a defect in this way, and to improve the determination accuracy of the presence / absence of a defect.

[0070] Next, the defect presence / absence determination section 105 determines the presence / absence of a defect. Specifically, the defect presence / absence determination section 105 first calculates the difference in units of pixels with respect to the removed image 111C and the removed image (restored) 111D. Next, the defect presence / absence determination section 105 calculates the variance of the calculated difference. Also, the defect presence / absence determination section 105 determines the presence / absence of a defect based on whether the calculated variance value exceeds a prescribed threshold value.

[0071] Here, when it is determined that there is a defect, the defect type is determined based on the difference value of each pixel calculated by the defect presence / absence determination section 105. Further, the difference value of each pixel indicates the difference between the removed image 111C and the removed image (restored) 111D, and therefore these difference values are also referred to as a difference image.

[0072] Further, the timing at which the peripheral echo region is removed is not limited to the above-described example. For example, a difference image of the inspection image 111A and the restored image 111B can be generated, and the peripheral echo region can be removed from the difference image.

[0073] (Variance of difference calculated in units of pixels and presence / absence of defect)

[0074] Based on Figure 4 The relationship between the variance of the difference calculated in units of pixels and the presence / absence of a defect is described. Figure 4 is a graph indicating an example of the distribution of pixel values in a difference image generated based on an ultrasonic image 111 of a defective portion, and an example of the distribution of pixel values in a difference image generated based on an ultrasonic image 111 of a non-defective portion.

[0075] Furthermore, since the removed image 111C and the removed image (restored) 111D are images of the same size, the difference image is also of the same size as these images. The value of each pixel constituting the difference image is equal to the difference between the pixel values ​​of the removed image 111C and the removed image (restored) 111D.

[0076] As shown in the figure, in the pixel values ​​of the differential image generated based on the ultrasonic image 111 of the defect-free area, values ​​in the range of 0 to 20 account for the majority, and values ​​larger than that are almost zero. Therefore, the variance of the pixel values ​​in this differential image is a relatively small value of 20.

[0077] On the other hand, as shown in the figure, in the pixel values ​​of the differential image generated based on the ultrasonic image 111 of the defective area, there are many values ​​in the range of 0 to 20, but as indicated by the box in the figure, there are also several larger values. The pixels corresponding to the echo caused by the defect are those with such larger values. Therefore, the variance of the pixel values ​​in this differential image is a relatively large value of 70.

[0078] Thus, in a differential image of a defective area, the pixel values ​​of pixels in the defective region are larger compared to those in other areas, resulting in a larger variance in pixel values. Conversely, in a differential image of a defect-free area, there may be regions where pixel values ​​are relatively large due to noise or other factors, but the likelihood of extremely large pixel values ​​is low, and the variance in pixel values ​​is relatively small. In other words, when an object being inspected has a defect, the increased variance of pixel values ​​in the differential image is a characteristic phenomenon.

[0079] Therefore, if the defect determination unit 105 determines that there is a defect when the variance of the pixel values ​​of each pixel constituting the differential image exceeds a predetermined threshold, then it is possible to appropriately determine whether there is a defect.

[0080] (Heatmap and threshold processing)

[0081] As mentioned above, heatmaps are used in determining defect types. Here, based on... Figure 5 This section explains the heatmap generated by the heatmap generation unit 106 and the threshold processing performed on the generated heatmap. Figure 5 This diagram illustrates an example of generating a thermal image from an ultrasound image and applying a threshold to the generated thermal image. More specifically, Figure 5 The upper part shows an example of ultrasonic image 111-a of a defective area in the welded part of the pipe end. Figure 5 The lower part represents an example of ultrasonic image 111-b of a defect-free area in the welded part of the pipe end.

[0082] Based on Figure 3As described above, the removed image 111C-a and the removed image (restored) 111D-a are generated from the ultrasonic image 111-a, and a difference image is generated from the removed image 111C-a and the removed image (restored) 111D-a. The heat map generation section 106 generates a heat map which is a map in which each pixel in the difference image is expressed using a color or a shade corresponding to the pixel value thereof.

[0083] Figure 5 A heat map 111E-a from the lower limit value to the upper limit value in which pixel values are expressed using color shades from black to white is shown. As shown by the open arrow in the heat map 111E-a, the region corresponding to the defect (a region in which pixels having large pixel values are concentrated) is a region in which pixels close to white are concentrated. Therefore, in the heat map 111E-a, the region corresponding to the defect is easily visually recognized.

[0084] However, there are regions in which pixel values are made large by noise or the like in the heat map 111E-a. Therefore, it is preferable that the heat map generation section 106 perform threshold processing on the generated heat map, and correct the pixel values of the regions in which pixel values are made large by noise or the like. For example, the heat map generation section 106 can make pixel values below a prescribed threshold value in the heat map 111E-a zero (black). Thereby, a heat map 111F-a in which noise components are removed is generated. According to the heat map 111F-a, the region corresponding to the defect can be more clearly recognized.

[0085] The same is true for the ultrasonic image 111-b of the defect-free portion. From the ultrasonic image 111-b, the removed image 111C-b and the removed image (restored) 111D-b are generated, and a difference image is generated from the removed image 111C-b and the removed image (restored) 111D-b. Also, the heat map generation section 106 generates a heat map 111E-b of the difference image, and performs threshold processing on the heat map 111E-b, and generates a heat map 111F-b. By comparing the heat map 111F-a with the heat map 111F-b, it is possible to clearly determine the presence or absence of a defect. Also, it is possible to clearly determine the position of the defect in the heat map 111F-a.

[0086] (defect type, ultrasonic image, heat map)

[0087] As defects in a pipe end weld, for example, there are known to be initial layer poor penetration, inter-pass poor fusion, undercut, and porosity. The initial layer poor penetration is a defect in which a gap is generated by lack of fusion in the vicinity of a pipe plate. The inter-pass poor fusion is a defect in which a gap is generated by lack of fusion when multiple passes of fusion are performed. The undercut is a defect in which the end portion of a weld bead is recessed in a groove shape. The porosity is a defect in which a spherical cavity is generated in a welded metal.

[0088] The defects are generated at different positions. Therefore, in the ultrasonic image 111, the defect type can be determined from the position at which an echo caused by the defect appears. Likewise, the defect type can be determined from the position of a defect region in a heat map generated based on the ultrasonic image 111, preferably a heat map after threshold processing. Further, as described above, the defect region is a region in which an echo caused by a defect appears, and the pixel value is large compared to other regions.

[0089] Based on Figure 6 The determination of the defect type based on the position of the defect region will be described. Figure 6 is a diagram illustrating the relationship between the position of the defect, the ultrasonic image, and the heat map. In Figure 6 the left end of the first layer indicates a cross section of the pipe end weld portion in which a poor initial layer penetration is generated. Figure 6 The left side in is the pipe end side, and the right side is the pipe deep side. That is, the pipe extends in the left-right direction of Figure 6 . Further, the pipe plate is located on the lower side with respect to the outer surface of the pipe. In addition, in order to know the width of the pipe end weld portion, a scale is brought into contact with the inner wall surface (inner surface) of the pipe.

[0090] In Figure 6 the diagram of the left end of the first layer, the region indicated by the broken line is a pipe plate penetration region generated when welding, and the region of the inverted triangular shape on the left side of the penetration region is a region composed of the weld metal, and the region combining these regions is the pipe end weld portion. The portion indicated by the circle in this pipe end weld portion generates a gap. The gap is near the surface of the pipe, near the pipe deep side end portion of the pipe end weld portion.

[0091] As Figure 6 indicated in the center of the first layer, an echo caused by the gap appears in the ultrasonic image 111-c at the portion in which the gap is present. Further, as Figure 6 indicated in the right end of the first layer, a region corresponding to the above-described gap also appears in the heat map 111F-c generated based on the ultrasonic image 111-c as indicated by the hollow arrow in the diagram.

[0092] In Figure 6 the left end of the second layer indicates a cross section of the pipe end weld portion in which a poor inter-pass penetration is generated. A gap is generated at the portion indicated by the circle. The gap is near the surface of the pipe, near the center portion in the thickness direction of the pipe end weld portion.

[0093] As Figure 6 indicated in the center of the second layer, an echo caused by the gap appears in the ultrasonic image 111-d at the portion in which the gap is present. Further, as Figure 6The region corresponding to the above gap appears in the thermal image 111F-d generated based on the ultrasonic image 111-e as indicated by the hatched arrow in the right end of the third layer as shown in Fig. 11. The region is located on the left side compared with the thermal image 111F-d of the second layer.

[0094] In the left end of the fourth layer of Fig. 11, a cross-sectional surface of the pipe end weld portion in which a blowhole is generated is indicated. A gap is generated at the portion indicated by the circle. The gap is closer to the inside side of the pipe end weld portion than the surface of the pipe, and is located near the center in the width direction of the pipe end weld portion in the left-right direction. Figure 6 As indicated by the center of the fourth layer of Fig. 11, an echo caused by the gap appears in the ultrasonic image 111-f at the portion where the gap exists. Also, as indicated by the right end of the fourth layer of Fig. 11, a region corresponding to the above gap appears in the thermal image 111F-f generated based on the ultrasonic image 111-f as indicated by the hatched arrow in the figure. The region is located near the thermal image 111F-d of the second layer in the left-right direction, and is located more toward the lower side in the up-down direction.

[0095] Figure 6 As indicated by the center of the fourth layer of Fig. 11, an echo caused by the gap appears in the ultrasonic image 111-f at the portion where the gap exists. Also, as indicated by the right end of the fourth layer of Fig. 11, a region corresponding to the above gap appears in the thermal image 111F-f generated based on the ultrasonic image 111-f as indicated by the hatched arrow in the figure. The region is located near the thermal image 111F-d of the second layer in the left-right direction, and is located more toward the lower side in the up-down direction. Figure 6 In the left end of the fourth layer of Fig. 11, a cross-sectional surface of the pipe end weld portion in which a blowhole is generated is indicated. A gap is generated at the portion indicated by the circle. The gap is closer to the inside side of the pipe end weld portion than the surface of the pipe, and is located near the center in the width direction of the pipe end weld portion in the left-right direction.

[0096] Figure 6 As indicated by the center of the fourth layer of Fig. 11, an echo caused by the gap appears in the ultrasonic image 111-f at the portion where the gap exists. Also, as indicated by the right end of the fourth layer of Fig. 11, a region corresponding to the above gap appears in the thermal image 111F-f generated based on the ultrasonic image 111-f as indicated by the hatched arrow in the figure. The region is located near the thermal image 111F-d of the second layer in the left-right direction, and is located more toward the lower side in the up-down direction.

[0097] As indicated by the center of the fourth layer of Fig. 11, an echo caused by the gap appears in the ultrasonic image 111-f at the portion where the gap exists. Also, as indicated by the right end of the fourth layer of Fig. 11, a region corresponding to the above gap appears in the thermal image 111F-f generated based on the ultrasonic image 111-f as indicated by the hatched arrow in the figure. The region is located near the thermal image 111F-d of the second layer in the left-right direction, and is located more toward the lower side in the up-down direction. Figure 6 Figure 6 As indicated by the center of the fourth layer of Fig. 11, an echo caused by the gap appears in the ultrasonic image 111-f at the portion where the gap exists. Also, as indicated by the right end of the fourth layer of Fig. 11, a region corresponding to the above gap appears in the thermal image 111F-f generated based on the ultrasonic image 111-f as indicated by the hatched arrow in the figure. The region is located near the thermal image 111F-d of the second layer in the left-right direction, and is located more toward the lower side in the up-down direction.

[0098] As described above, there is a correlation between the defect type and the appearance of the thermal image 111F. Therefore, a type determination model that determines the defect type from the thermal image 111F can be constructed based on the correlation. Such a type determination model can be constructed by machine learning that uses the thermal image of the differential image generated from the inspection image of the inspection object having a defect of which the type is known as teacher data. Also, the defect type determination section 107 can determine the defect type based on the output value obtained by inputting the thermal image generated by the thermal image generation section 106 to such a determination model.

[0099] ​​​As described above, in a heatmap that uses color or shading to represent the pixel values ​​of each pixel constituting a differential image, the different types of defects reflected in the inspection image based on that differential image can be reflected. Therefore, according to the above structure, the type of defect can be appropriately and automatically determined.

[0100] For example, multiple images 111 generated from ultrasonic images 111 of areas where poor initial penetration depth occurs can be prepared, such as... Figure 7 Heat maps like 111F-c are used as teacher data. From this, a type determination model can be constructed that outputs the probability that the defect type is poor initial penetration. Similarly, if heat maps generated from ultrasonic images 111 of locations producing other types of defects are used as teacher data for machine learning, a type determination model can be constructed that outputs the probability corresponding to each type of defect.

[0101] Therefore, the defect type determination unit 107 can determine the defect type based on the output value obtained by inputting a heatmap into a type determination model. For example, the defect type determination unit 107 can determine that a defect of the type corresponding to the highest probability value among the probability values ​​of various defects output from the type determination model has occurred.

[0102] (Comprehensive analysis of defects)

[0103] The welded portion at the pipe end exists throughout the 360-degree circumference of the pipe. Therefore, as described above, the probe is rotated within the pipe in units of a predetermined angle to generate ultrasonic images 111 of each part of the welded portion at the pipe end, and defects are detected based on each ultrasonic image 111. In this case, a continuous defect may appear across multiple ultrasonic images, and although it is a single defect in substance, it may be detected as multiple defects.

[0104] Therefore, the integrated detection unit 108 integrates the defects reflected in multiple ultrasonic images 111 and detects them as a single defect. More specifically, when the defect presence / absence determination unit 105 determines that there is a defect in each of the multiple ultrasonic images 111 corresponding to a portion of the pipe end weld, i.e., a portion adjacent to each other, the integrated detection unit 108 detects the defect reflected in those multiple ultrasonic images as a single defect. Thus, appropriate detection along the defective entity can be performed.

[0105] based on Figure 7 A comprehensive approach to addressing defects is explained. Figure 7 This diagram illustrates a method for detecting defects by combining defects reflected in multiple ultrasonic images 111 into a single defect. Figure 7 The upper left shows a cross-section of the welded joint between the pipe and its end. Additionally, in... Figure 7A longitudinal cross section of the tube, the tube end weld, and the tube sheet is shown on the lower left.

[0106] In Figure 7 In the example of FIG. 11, a weld defect is generated over a wide range along the outer wall surface of the tube. When the probe is rotated in units of a prescribed angle along the inner wall surface of the tube and the echoes are measured, the echoes from the weld defect are reflected in the measurement results of the range where the weld defect is generated. Thus, as shown on the right side of FIG. 11, echoes caused by the weld defect appear in the ultrasonic images 111g to 111i generated based on the above measurement results. Therefore, in the defect presence / absence determination based on these ultrasonic images 111g to 111i, the defect presence / absence determination section 105 determines that there is a defect. Figure 7

[0107] Here, the ultrasonic images 111g to 111i each correspond to a portion of the tube end weld, i.e., portions adjacent to each other. Therefore, the comprehensive detection section 108 detects the defects reflected in the ultrasonic images 111g to 111i determined by the defect presence / absence determination section 105 as one defect.

[0108] Further, the comprehensive detection section 108 can comprehensively detect the defects in accordance with whether the positions of the defects detected from the ultrasonic images 111g to 111i are the same or similar positions. In addition, as described above, the defects differ in position depending on the type. Therefore, the comprehensive detection section 108 can comprehensively detect the defects in accordance with whether the same type of defect is detected from the ultrasonic images 111g to 111i. According to these configurations, the accuracy of the defect comprehensive detection can be improved.

[0109] In addition, the defect length calculation section 109 calculates the length of the defect that has been comprehensively detected by the above processing. For example, the defect length calculation section 109 can calculate the length of the defect by multiplying the average length of the defects of one ultrasonic image 111 by the number of defects that have been comprehensively detected by the comprehensive detection section 108.

[0110] For example, for the tube end weld formed over 360 degrees around the tube, the probe is moved in units of 1 degree around the central axis of the tube along the inner wall surface of the tube to perform 360 times of echo measurement, and as a result, 360 ultrasonic images 111 are generated. In this case, the length of the defect reflected in one ultrasonic image 111 is approximately (the outer diameter of the tube) x π x 1 / 360. Therefore, when the defects of three ultrasonic images 111g to 111i are comprehensively detected as in Figure 8 , the defect length calculation section 109 can calculate the length of the defect as (the outer diameter of the tube) x π x 3 x 1 / 360. In addition, π is a constant.

[0111] (Example of output of inspection results) ​

[0112] The determination result of whether the inspected object has defects using the inspection device 1 is output via the output unit 13. Here, based on Figure 8 An example of the output of the inspection results is provided. Figure 8 This is a diagram showing an example of the output of the inspection results.

[0113] exist Figure 8 The upper left corner shows defect diagram 300. Defect diagram 300 is a diagram in which line segments 302 representing detected defects are drawn in a circular area 301 representing the welded part at the pipe end as viewed from the pipe end side. According to defect diagram 300, the distribution of defects in the welded part at the pipe end can be easily identified.

[0114] In addition, Figure 2 The upper right corner shows tube sheet diagram 400. Tube sheet diagram 400 schematically illustrates as follows: Figure 8 The diagram shows the condition of a heat exchanger with multiple tubes welded to it, viewed from the tube end side. In tube sheet diagram 400, the inspection results are shown by plotting a graph at the location of each tube, indicating the results of the defect inspection of the tube end weld.

[0115] Specifically, the inspection results are as follows: white circles are drawn at the locations of tubes where no defects were detected, and black circles are drawn at the locations of tubes where damage (defects) was detected. This makes it easy to identify the distribution of defective tube end welds. Additionally, in the tube sheet diagram 400, triangular marks are drawn at the locations of tubes that were not inspected, and square marks are drawn at the locations of tubes not subject to inspection. In this way, various inspection-related information can also be included in the tube sheet diagram 400.

[0116] In addition, Figure 9 The lower side shows the ultrasonic image group 500. The ultrasonic image group 500 contains three ultrasonic images (501-503). Ultrasonic image 501 is obtained by a fan-shaped scan at the tube end, ultrasonic image 502 is obtained by a linear scan, and ultrasonic image 503 is obtained by a fan-shaped scan at the tube depth.

[0117] Furthermore, linear scanning is a scan in the inspection direction perpendicular to the central axis of the tube. The aforementioned ultrasonic image 111 was also obtained through linear scanning. Pipe-end sector scanning is a scan in which ultrasonic waves propagate in an inspection direction inclined from the central axis of the tube towards the deep side of the tube. Additionally, deep-side sector scanning is a scan in which ultrasonic waves propagate in an inspection direction inclined from the central axis of the tube towards the pipe end.

[0118] In these ultrasonic images, the reflected echoes corresponding to the detected defects are marked. By displaying these marked ultrasonic images as inspection results, the location of defects can be easily identified.

[0119] Here, the ultrasonic images 501 to 503 are all images obtained by scanning the same position on the tube end weld portion, but since the flaw detection directions are different, the defect manifestation is also different. Therefore, the inspection device 1 determines the presence or absence of a defect for a plurality of ultrasonic images 111 in which the flaw detection directions are different, and when a defect is determined in any flaw detection direction, the final determination result can be set to a defect even if a defect is not determined in the other flaw detection directions. Thus, it is possible to reduce the probability of a defect being missed. In addition, the inspection device 1 can determine the presence or absence of a defect with respect to a synthesized image obtained by synthesizing an ultrasonic image obtained by linear scanning and an ultrasonic image obtained by fan scanning.

[0120] Further, the inspection device 1 can output all of the defect map 300, the tube sheet map 400, and the ultrasonic image group 500 as the inspection result, or can output only a part of them. In addition, the inspection device 1 can output information indicating the determination result of the defect type or the like as the inspection result. Of course, these are merely examples, and the inspection device 1 can output the determination result in any manner in which a person can recognize the content thereof.

[0121] (Flow of processing performed before inspection)

[0122] Before performing defect inspection with the inspection device 1, various models used in the inspection and threshold values need to be prepared. Here, the flow of processing of constructing various models used in the inspection and determining threshold values will be described based on FIG. 10. Figure 9 Figure 2 is a diagram illustrating an example of processing of constructing various models used in the inspection and determining threshold values. Further, these processes can be performed in the inspection device 1 or in another computer.

[0123] In S1, an extraction model is constructed. The construction of the extraction model is performed by machine learning using teacher data in which extraction region information is associated with the ultrasonic image 111 as correct data. The extraction region information can be, for example, information generated based on input of a region that should be extracted by an operator while the ultrasonic image 111 is displayed on a display device.

[0124] The extraction model can be constructed by any learning model suitable for extracting a region from an image. For example, the extraction model can be constructed by YOLO (You Only Look Once) or the like, which is excellent in extraction accuracy and processing speed.

[0125] ​The region to be extracted can be a region including the pipe end weld portion as the inspection target portion. Further, the region to be extracted preferably also includes at least a portion of the region in which the echo from the peripheral portion exists. This is because, when there is no defect in the inspection target portion, it is possible that a feature point capable of machine learning is not observed in the portion in the ultrasonic image 111, and in this case, it is difficult to construct an extraction model. For example, in the ultrasonic image 111 shown in FIG. 10, as long as a region including a portion of the echoes al, a2, a6, and a7 is the region to be extracted. Thereby, an extraction model that extracts a region including the pipe end weld portion and also including the echo from the peripheral portion can be constructed. Figure 2

[0126] In S1, when the extraction model constructed by machine learning that takes a region in which the echo from the peripheral portion exists as correct data is constructed, the inspection image generation portion 102 extracts the inspection target region using the extraction model. Since the echo from the peripheral portion of the inspection target portion has a feature capable of machine learning as shown in FIG. 10, the structure enables automatic and high-precision extraction of the inspection target portion. Figure 6

[0127] The generation model is constructed in S2. The generation model is constructed by machine learning that takes the ultrasonic image 111 of the inspection target object having no defect, and more specifically, an image of the inspection target region extracted from the ultrasonic image 111 using the extraction model constructed in S1, as training data. As described above, the generation model can be an autoencoder. Further, the generation model can be a model obtained by improving or changing the autoencoder. For example, a variational autoencoder or the like can be applied as the generation model.

[0128] Further, when the extraction model constructed by machine learning that takes a region in which the echo from the peripheral portion exists as correct data is constructed in S1, the region in which the echo from the peripheral portion exists is also included in the training data for constructing the generation model. In the ultrasonic image 111 of the inspection target object having no defect, the echo is not included in the inspection target region, although a feature point that should be subjected to machine learning is lacking, but by using training data including the region in which the echo from the peripheral portion exists, it is possible to construct a suitable generation model.

[0129] ​​In S3, a test image is input to the generation model constructed in S2 to construct a restoration image. In the test image, there are included: an image generated by extracting the inspection target region from the ultrasonic image 111 of the inspection object without defects using the extraction model constructed in S1; and an image generated by extracting the inspection target region from the ultrasonic image 111 of the inspection object with defects. In addition, the test images generated from the ultrasonic image 111 of the inspection object with defects are classified according to the defect type thereof.

[0130] In S4, a threshold for defect presence / absence determination is determined. Specifically, the difference between the test image and the restoration image generated in S3 is calculated on a pixel-by-pixel basis, and the variance of the difference is calculated. Further, the threshold is determined in such a manner that the values of the variances calculated for the plurality of test images generated from the ultrasonic image 111 of the inspection object without defects and the values of the variances calculated for the plurality of test images generated from the ultrasonic image 111 of the inspection object with defects can be distinguished.

[0131] In S5, a class determination model is constructed using the heat maps respectively generated from the test images of the respective defects as teacher data. Since the features corresponding to the defect types appear in the heat maps as described above, the class determination model can be constructed by machine learning using the heat maps as the teacher data. Figure 10

[0132] The class determination model can be constructed using any learning model suitable for image classification. For example, the class determination model can be constructed using a convolutional neural network or the like that is excellent in classification accuracy of images.

[0133] (Flow of processing in inspection)

[0134] The flow of processing in inspection will be described based on Figure 10 Figure 11 is a diagram showing an example of an inspection method using the inspection apparatus 1. Further, it is described below that the ultrasonic image 111 obtained by imagezing the echoes from the pipe end weld and the peripheral portion thereof obtained by measurement while rotating the probe is stored in the storage 11.

[0135] In S11, the inspection target region extraction unit 101 acquires one of the ultrasonic images 111 stored in the storage 11, inputs the one ultrasonic image 111 to the extraction model, and generates extraction region information based on the output value thereof. Further, the inspection image generation unit 102 extracts the region indicated by the above extraction region information from the ultrasonic image 111 to generate the inspection image 111A.

[0136] ​​In S12 (inspection image acquisition step), the inspection image acquisition section 103 acquires the inspection image 111A generated through Sll. Also, the inspection image acquisition section 103 duplicates the acquired inspection image 111A, and sends one of them to the restored image generation section 104, and the other to the defect presence / absence determination section 105. Further, the inspection image acquisition section 103 can send the removed image 111C, which is the inspection image 111A from which the peripheral echo region is removed, to the defect presence / absence determination section 105.

[0137] In S13, the restored image generation section 104 inputs the inspection image 111A acquired through S12 to the generation model, and generates the restored image 111B based on the output value thereof. Also, when configured so that the inspection image acquisition section 103 sends the removed image 111C to the defect presence / absence determination section 105, it is preferable that the restored image generation section 104 generate the removed image (restored) 111D from the generated restored image 111B, and send it to the defect presence / absence determination section 105. Further, it can also be configured so that the removed image 111C and the removed image (restored) 111D are generated by the defect presence / absence determination section 105.

[0138] In S14, the defect presence / absence determination section 105 calculates the difference of each pixel with respect to the inspection image 111A generated through S12 and the restored image 111B generated through S13. Further, in the case where the removed image 111C and the removed image (restored) 111D are generated, the defect presence / absence determination section 105 calculates the difference of these images.

[0139] In S15, the defect presence / absence determination section 105 calculates the variance of the difference calculated through S14. Also, in S16 (defect presence / absence determination step), the defect presence / absence determination section 105 determines the presence / absence of a defect based on the variance value calculated through S15. Specifically, the defect presence / absence determination section 105 determines that there is a defect when the variance value exceeds a prescribed threshold value, and determines that there is no defect when the variance value is equal to or less than the prescribed threshold value.

[0140] When it is determined that there is a defect in S16 (Yes in S16), the processing proceeds to S17. In S17, the defect presence / absence determination section 105 stores the determination result of S16, i.e., that the inspection target has a defect, in the inspection result data 112. Thereafter, the processing proceeds to S18, and the defect type determination processing is performed. Further, the details of the defect type determination processing will be described later based on the flowchart of FIG. 6. Figure 7

[0141] On the other hand, when it is determined that there is no defect in S16 (No in S16), the processing proceeds to S19. In S19, the defect presence / absence determination section 105 stores the determination result of S16, i.e., that the inspection target has no defect, in the inspection result data 112. Thereafter, the processing proceeds to S20.​

[0142] In S20, it is determined whether the inspection target area extraction unit 101 has completed processing of all the ultrasound images 111 that are the inspection target. Here, if it is determined that there are unprocessed ultrasound images 111 (no in S20), the process returns to S11, the inspection target area extraction unit 101 reads the unprocessed ultrasound images 111 from the storage unit 11, and generates extraction area information. On the other hand, if it is determined that there are no unprocessed ultrasound images 111 (yes in S20), the process proceeds to S21.

[0143] In step S21, the comprehensive inspection unit 108 synthesizes the defects detected by the defect presence / absence determination unit 105. Furthermore, the comprehensive inspection unit 108 records the synthesis results in the inspection result data 112. The defect synthesis method is based on... Figure 10 As explained above, this will not be repeated here. Furthermore, if there are no defects that require synthesis, then S21 and S22 will not be processed, and the process will end. Figure 10 The processing.

[0144] In step S22, the defect length calculation unit 109 calculates the length of the defects aggregated by the comprehensive inspection unit 108. For example, the defect length calculation unit 109 can calculate the defect length by combining the average length of defects in an ultrasonic image 111 with the number of defects aggregated by the comprehensive inspection unit 108. The defect length calculation unit 109 then records the calculation result in the inspection result data 112. This concludes the process. Figure 11 The processing.

[0145] (Defect type identification and handling process)

[0146] based on Figure 10 For Figure 11 The process of defect type determination and processing in S18 is explained. Figure 10 This is a flowchart illustrating an example of defect type determination processing. In S31, the heatmap generation unit 106 uses... Figure 5 The difference value calculated in S14 generates a heatmap. Furthermore, in S32, the heatmap generation unit 106 performs thresholding processing on the heatmap generated in S31. The thresholding processing is based on... Figure 12 As explained above, it will not be repeated here.

[0147] In S33, the defect type determination unit 107 uses a type determination model to determine the defect type. Specifically, the defect type determination unit 107 inputs a heatmap that has undergone threshold processing in S32 into the type determination model and determines the defect type based on its output value. For example, if the type determination model outputs a value representing the probability of belonging to that type for each defect type, the defect type determination unit 107 determines the type with the largest value as that defect type.

[0148] In S34, the defect type determination section 107 records the determination result of S33 in the inspection result data 112. Thereby, the defect type determination processing is ended.

[0149] (Embodiment 2)

[0150] Another embodiment of the present application will be described below. Furthermore, for the convenience of explanation, the same reference numerals are attached to the components having the same function as those explained in the above embodiment, and the explanation thereof will not be repeated. In the present embodiment, an example in which the defect type is determined by a method different from that of the above embodiment will be explained.

[0151] (Structure of inspection device)

[0152] Based on Figure 12 The structure of the inspection device 2 of the present embodiment will be explained. Figure 1 is a block diagram showing an example of the structure of the main part of the inspection device 2. The inspection device 2 differs from the inspection device 1 shown in Figure 13 in that the control section 10 is changed to a control section 20. In the control section 20, the thermal map generation section 106 and the defect type determination section 107 included in the control section 10 are not included, and instead, a defect region detection section 201 and a defect type determination section 202 are included.

[0153] The defect region detection section 201 detects a region constituted by a plurality of pixels having a pixel value of a threshold value or more in the differential image as a defect region. Also, the defect type determination section 202 determines the defect type of the defect region according to the position at which the above defect region is detected in the image region of the differential image. The detection of the defect region and the determination of the defect type based on the position of the defect region will be explained in detail below.

[0154] (Detection of defect region)

[0155] Based on Figure 13 The method of detecting the defect region by the defect region detection section 201 will be explained. Figure 13 is a diagram showing the method of detecting the defect region. Furthermore, in Figure 13 , an example of detecting the defect region using a thermal map is shown, but as explained below, the generation of the thermal map is not essential.

[0156] In Figure 13 , a thermal map 111E generated from the ultrasonic image 111 of the inspection target object having a defect, and a thermal map 111F to which threshold processing has been applied to the thermal map 111E are shown. Also, in Figure 13 , an enlarged view of the upper left end portion of the thermal map 111F is shown, in which the pixel value of each pixel of the thermal map 111F is written.

[0157] In defect area detection, firstly, the defect area detection unit 201 detects the pixel with the largest pixel value in the heat map 111F. Figure 11 In this example, since the largest pixel value is 104, this pixel is detected. Next, the defect area detection unit 201 detects pixels adjacent to the detected pixel whose values ​​are within a specified threshold (and...). Figure 6 The threshold used in the thresholding process of S32 is a larger threshold than the threshold used for pixels above that.

[0158] The defect region detection unit 201 repeats this process until no adjacent pixels with pixel values ​​above a certain threshold are detected. Therefore, the defect region detection unit 201 can detect a continuous region consisting of pixels with pixel values ​​above a predetermined threshold as a defect region. Furthermore, the defect region detection unit 201 can also detect a rectangular region ar5 containing the defect regions detected as described above as a defect region.

[0159] The above processing can be performed as long as there is a difference image, which represents the difference values ​​of each pixel in the inspection image 111A and the restored image 111B. That is, by repeatedly performing the process of detecting the pixel with the largest pixel value in the difference image and detecting pixels with pixel values ​​above a predetermined threshold adjacent to that pixel, defect areas can be detected. Therefore, as described above, it is not necessary to generate heatmap 111E or heatmap 111F for detecting defect areas.

[0160] As described above, the defect region detection unit 201 detects a region in the differential image consisting of multiple pixels with pixel values ​​above a threshold as a defect region. In the differential image, the pixel values ​​of the pixels in the defect region are larger than the pixel values ​​of the pixels in other regions; therefore, based on this structure, suitable defect regions can be automatically detected.

[0161] (Location-based defect type determination)

[0162] Based on Figure 14 As explained, among the defects in the weld, there are known to be various types, such as poor initial penetration and poor fusion between weld beads. These different types of defects will appear as differences in location in the ultrasonic image. Based on this, the defect type determination unit 202 determines the defect type of the defect area according to the location of the detected defect area in the image area of ​​the differential image. Thus, the defect type can be determined automatically.

[0163] For example, if regions corresponding to various types of defects are pre-defined in the differential image, the defect type determination unit 202 can determine the defect type based on which region the defect region detected by the defect region detection unit 201 is contained in.

[0164] Figure 14 is an example of a region set in accordance with a defect type. In Figure 14 the example, a region AR1 corresponding to an undercut is set in the upper left corner of the heat map 111F, a region AR2 corresponding to a poor inter-bead fusion is set in the center of the upper end, and a region AR3 corresponding to a poor first layer penetration is set in the upper right corner. In addition, a region AR4 corresponding to a pore is set in a position slightly above the center. For such regions, it is sufficient to set them in advance by analyzing the difference image, the heat map, and the like based on the inspection image of each defect site. In Figure 14 the example, since a defect region indicated by the hollow arrow is detected in the region AR3, the defect type determination unit 202 determines that the defect is a defect due to a poor first layer penetration.

[0165] In Figure 14 the example, a part of the region AR4 corresponding to a pore overlaps a part of the regions AR1 to AR3. It is possible to set a region for determining a defect type to partially overlap other regions.

[0166] In this case, when a defect region is detected in a region in which a plurality of regions overlap, the defect type determination unit 202 can determine all types corresponding to these respective regions as a determination result of a defect type. For example, when a defect region is detected in the overlapping region of the regions AR1 and AR4, the defect type determination unit 202 can output both the undercut and the pore as a determination result.

[0167] In addition, the defect type determination unit 202 can limit the determination result of a defect type in accordance with whether a condition unique to each defect type is satisfied. For example, if a defect has a characteristic in shape, a condition related to shape can be set, and if a defect has a characteristic in size, a condition related to size can be set.

[0168] Taking a specific example, a pore is a defect that generates a spherical hollow, and its diameter is generally 2 mm or less. Therefore, in a case where one ultrasonic image 111 covers a range of 1 mm or so in width of the inspection target, one pore is contained in 2 to 3 ultrasonic images 111 or so. Therefore, in a case where a defect is continuously detected in a plurality of ultrasonic images 111 corresponding to adjacent parts of the inspection target, if the number of ultrasonic images 111 is 3 or less, the defect is likely to be a pore. On the other hand, if the number of ultrasonic images 111 in which a defect is continuously detected is 4 or more, the defect is highly likely not to be a pore.

[0169] Therefore, when a defective region is detected in the overlapping region of the region AR4 and other regions, the defect type determination section 202 can determine the defect type as a pore, with the number of ultrasonic images 111 in which a defect is continuously detected being a threshold value (for example, 3) or less as a condition.

[0170] For example, in the case where a defective region is detected in the overlapping region of the region AR4 and the region AR2, the defect type determination section 202 can determine the defect type as a pore when the number of ultrasonic images 111 in which a defect is continuously detected is a threshold value or less, and determine the defect type as a lack of fusion between welds if the threshold value is exceeded. Figure 10

[0171] In addition, as described above, since a pore is spherical, in the case where one pore is detected across a plurality of ultrasonic images 111, the peak of the echo caused by the pore in each ultrasonic image 111 tends to be different. Such a difference in the peak is expressed as a difference in pixel value in the ultrasonic image 111. For example, assume that one pore is detected across three ultrasonic images 111. In this case, if the peak of the echo caused by the pore in the central ultrasonic image 111 among the three ultrasonic images 111 is set to 50%, the peak of the echo caused by the pore in the ultrasonic images 111 before and after it is a value of 30% lower than that.

[0172] Therefore, when a defective region is detected in the overlapping region of the region AR4 and other regions, the defect type determination section 202 can determine the defect type as a pore, with the presence of a difference in the pixel value of the defective region in each ultrasonic image 111 in which a defect is continuously detected being a condition. For example, the defect type determination section 202 can calculate the average of the pixel values of each pixel included in the defective region of each ultrasonic image 111, and determine that there is a difference when the difference in the average is a threshold value or more.

[0173] (Flow of defect type determination processing)

[0174] The defect inspection of the inspection device 2 is performed in the same manner as the inspection device 1 according to the flow of the flowchart shown in FIG. 17. However, the content of the defect type determination processing performed by S18 is different. Here, the flow of the defect type determination processing performed by the inspection device 2 in S18 will be described based on FIG. 18. Figure 15 Figure 10 Figure 15 is a flowchart showing an example of the flow of the defect type determination processing performed by the inspection device 2. Figure 10 is a flowchart showing an example of the flow of the defect type determination processing performed by the inspection device 2.

[0175] In S41, the defective region detection section 201 detects a defective region in the overlapping region of the region AR4 and the region AR2 by the same method as in S31 of FIG. 17. Figure 13 ​​​The difference value calculated in S14 is subjected to threshold processing. Also, in S42, the defect region detection section 201 detects a defect region from the difference value after the threshold processing. The method of detecting the defect region is as explained based on Figure 14 The explanation is not repeated here.

[0176] In S43, the defect type determination section 202 determines the defect type from the position of the defect region determined in S42. For example, the defect type determination section 202 can determine the defect type from which one of the regions AR1 to AR4 the defect region detected in S42 is included in. Figure 10

[0177] In S44, the defect type determination section 202 records the determination result of S43 in the inspection result data 112. Thereby, the defect type determination processing is ended.

[0178] (Application Examples)

[0179] In each of the above-described embodiments, the example of determining the presence or absence of a defect using an ultrasonic image is explained, but the inspection devices 1 and 2 can also be applied to determining the presence or absence of a defect from other images. For example, in a radiographic test (RT), the inspection devices 1 and 2 can also be applied to the inspection of determining the presence or absence of a defect of an inspection target. In this case, instead of a radiographic image, a defect image is detected from image data obtained using an electronic device such as an imaging plate.

[0180] (Modified Examples)

[0181] The execution subject of each of the processes explained in each of the above-described embodiments can be appropriately changed. For example, other information processing devices can be made to execute ​ in the flowchart, S11 (generation of an inspection image), S13 (generation of a restored image), S18 (defect type determination), S21

[0182] (composition of defects), and S22 (calculation of a defect length). Also, in this case, the other information processing devices can be one or a plurality. In this way, the functions of the inspection devices 1 and 2 can be realized by various system structures. Also, in the case of constructing a system including a plurality of information processing devices, a part of the information processing devices can be configured on the cloud. That is, the functions of the inspection device 1 can also be realized by one or a plurality of information processing devices that perform information processing on the line.

[0183] (Software-based Implementation Examples)

[0184] ​The control modules of the inspection apparatuses 1 and 2 (particularly, the respective units included in the control units 10 and 20) can be implemented by logic circuits (hardware) formed in integrated circuits (IC chips) and the like, or can be implemented by software.

[0185] In the latter case, the inspection apparatuses 1 and 2 are provided with a computer that executes a command of a software, i.e., an inspection program, that implements each function. The computer is provided with one or more processors, and is provided with a storage medium that is readable by the computer and stores the inspection program. In the computer, the processor reads the inspection program from the storage medium and executes the program, thereby achieving the objects of the present application. As the processor, a CPU (Central Processing Unit) can be used, for example. In addition to the processor such as a CPU, the inspection apparatuses 1 and 2 can be provided with a GPU (Graphics Processing Unit). By using the GPU, it is possible to perform calculations using the various models described above and the like at high speed. As the storage medium, a "non-transitory tangible medium" can be used, such as a magnetic tape, an optical disk, a card, a semiconductor memory, a programmable logic circuit, and the like, in addition to a ROM (Read Only Memory) and the like. In addition, a RAM (Random Access Memory) or the like that expands the program can be provided. In addition, the program can be provided to the computer via any transmission medium (communication network, electric wave, and the like) that can transmit the program. Furthermore, one embodiment of the present application can be implemented in a manner in which a data signal in which the program is embodied by being electronically transmitted is embedded in a carrier wave.

[0186] The present application is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims, and embodiments obtained by appropriately combining the technical solutions disclosed in the respective embodiments are also included in the technical scope of the present application.

[0187] Explanation of Reference Numerals

[0188] 1 - inspection apparatus; 102 - inspection image generation unit; 103 - inspection image acquisition unit; 105 - defect presence / absence determination unit; 106 - heat map generation unit; 107 - defect type determination unit; 108 - comprehensive detection unit; 111 - ultrasonic wave image; 2 - inspection apparatus; 201 - defect region detection unit; 202 - defect type determination unit.

Claims

1. An inspection apparatus comprising: an inspection image acquisition section that acquires an image for determining whether an inspection target has a defect, that is, an inspection image; a defect presence / absence determination section that determines whether the inspection target has a defect using a restored image generated by inputting the inspection image into a generation model that is constructed by machine learning using images of the inspection target without a defect as training data and is capable of generating a new image having the same features as an input image; and an inspection image generation section that extracts, from an ultrasonic image in which an echo of an ultrasonic wave propagating toward the inspection target is imaged, a region sandwiched by two peripheral echo regions in which an echo from a peripheral portion of an inspection target site in the inspection target repeatedly appears, as an inspection target region, and generates the inspection image, the inspection image acquisition section acquiring the inspection image generated by the inspection image generation section. 2.An inspection apparatus comprising: an inspection image acquisition section that acquires an image for determining whether an inspection target has a defect, that is, an inspection image; and a defect presence / absence determination section that determines whether the inspection target has a defect using a restored image generated by inputting the inspection image into a generation model that is constructed by machine learning using images of the inspection target without a defect as training data and is capable of generating a new image having the same features as an input image, the defect presence / absence determination section determining that the inspection target has a defect when a variance of pixel values of each pixel constituting a difference image between the inspection image and the restored image exceeds a prescribed threshold value. 3.The inspection apparatus according to claim 1, wherein the inspection image generation section extracts the inspection target region based on an output value obtained by inputting the ultrasonic image into an inspection target region extraction model that is a model constructed by machine learning using, as correct data, a region including the inspection target site and at least a part of a region in which an echo from a peripheral portion of the inspection target site is reflected. 4.The inspection apparatus according to claim 3, wherein the defect presence / absence determination section determines whether the inspection target has a defect with respect to an image region remaining after a region in which an echo from a peripheral portion of the inspection target site is reflected is removed from the image region of the restored image. 5.The inspection apparatus according to claim 2, comprising a defect region detection section that detects a region in which a plurality of pixels having a pixel value of a threshold value or more are present in the difference image as a defect region. 6.The inspection apparatus according to claim 5, wherein the defect is a defect of a welded site of the inspection target, and the inspection apparatus comprises a defect type determination section that determines a defect type of the defect region based on a position in which the defect region is detected in the image region of the difference image. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ 7. The inspection apparatus according to any one of claims 1 to 6, characterized by a comprehensive defect detection unit that detects a defect as one defect when the defect presence / absence determination unit determines that a plurality of ultrasonic images each corresponding to a portion of the inspection object, i.e., portions adjacent to each other, are defective.

8. The inspection device of claim 2, wherein, provided with: a heat map generation unit that generates a heat map that represents pixel values of each pixel that constitutes the difference image in color or gradation; and a defect type determination unit that determines a defect type based on an output value obtained by inputting the heat map generated by the heat map generation unit to a type determination model that is a model constructed by machine learning that uses a heat map of a difference image generated from an inspection image of an inspection object having a defect of which the type is known as teacher data.

9. An inspection method that uses an inspection apparatus and includes: an inspection image acquisition step of acquiring an image for determining the presence or absence of a defect in an internal portion of an inspection object, i.e., an inspection image; a defect presence / absence determination step of determining whether the inspection object is defective using a restored image generated by inputting the inspection image to a generation model constructed by machine learning that uses an ultrasonic image of an inspection object having no defect as training data and capable of generating a new image having the same features as an input image; and an inspection image generation step of extracting, from an ultrasonic image in which an echo of an ultrasonic wave propagated to the inspection object is imaged, a region sandwiched by two peripheral echo regions in which an echo from a peripheral portion of an inspection site in the inspection object repeatedly appears, as an inspection target region, and generating the inspection image, in the inspection image acquisition step, the inspection image generated in the inspection image generation step is acquired.

10. An inspection method that uses an inspection apparatus and includes: an inspection image acquisition step of acquiring an image for determining the presence or absence of a defect in an internal portion of an inspection object, i.e., an inspection image; and a defect presence / absence determination step of determining whether the inspection object is defective using a restored image generated by inputting the inspection image to a generation model constructed by machine learning that uses an ultrasonic image of an inspection object having no defect as training data and capable of generating a new image having the same features as an input image, in the defect presence / absence determination step, when a variance of pixel values of each pixel that constitutes a difference image between the inspection image and the restored image exceeds a prescribed threshold value, it is determined that there is a defect in the inspection object.

11. A computer-readable storage medium that stores an inspection program for causing a computer to function as the inspection apparatus according to claim 1, for causing the computer to function as the inspection image acquisition unit, the defect presence / absence determination unit, and the inspection image generation unit.

Images