Defect detection method, defect detection device, and defect detection program

Abstract

This defect detection method is executed by a computer to: acquire a first inference result output by a first detection model in response to receiving input of a target image, the first detection model being used for detecting a first type of defect; acquire a second inference result output by a second detection model in response to receiving input of the target image, the second detection model being used for detecting a second type of defect; and output a detection result indicating an integrated defect or a series of defects detected from the target image on the basis of the first inference result and the second inference result.

Description

Defect detection method, defect detection device, and defect detection program

[0001] The technology disclosed herein relates to a defect detection method, a defect detection device, and a defect detection program.

[0002] Patent Document 1 (JP 2021-170298 A) describes "a road defect detection device comprising: a detection unit that detects pixel areas containing road defects from an image of the road; and a calculation unit that calculates a road defect rate for each section using the area of ​​the detected pixel areas, wherein the detection unit detects the pixel areas for each type of defect including white cracks, and the calculation unit calculates the defect rate using the total area of ​​the union of the pixel areas detected by the detection unit for each type."

[0003] Patent Document 2 (JP 2022-139796 A) describes an "assistance device that detects an abnormality occurring in an object to be inspected from a captured image of the object and assists in a judgment test that determines the type of the detected abnormality, the assistance device comprising: a classification processing unit that calculates feature amounts of the abnormality occurring in the object from the captured image, classifies the calculated feature amounts into a plurality of classes, and selects one or more classes to be processed, the type of which is to be determined using a learning and discrimination unit; and a learning and discrimination unit that includes, for each type of abnormality, a learning and discrimination unit that has learned discrimination criteria for determining whether the abnormality to be determined is of the type that is determined as a correct answer, based on set data that combines an elemental image that induces the abnormality and an abnormality type that corresponds to the elemental image as a correct answer, or a type other than the type that is determined as a correct answer, and the learning and discrimination unit discriminates the type of abnormality for the class to be processed selected by the classification processing unit, based on individual discrimination results of the learning and discrimination units that have learned discrimination criteria for each type of abnormality."

[0004] Patent Document 3 (JP 2021-110629 A) describes an inspection device comprising: a defect candidate extraction unit that extracts one or more defect candidates based on predetermined feature quantities from an image of an object to be inspected; a judgment area extraction unit that extracts a judgment area from the image containing at least a portion of the extracted defect candidate; and a judgment unit that judges the presence or absence of a defect using a learning model constructed by machine learning for the extracted judgment area.

[0005] Patent Document 4 (JP 2023-16347 A) describes an "abnormality inspection system including an acquisition unit that acquires image data of a component so that the same location of the component is included in different areas of multiple consecutive image data, and a determination unit that detects the presence or absence of an abnormality in the multiple consecutive image data, and determines that the component is abnormal if an abnormality is detected in all of the multiple image data."

[0006] Defects vary in shape, size, and other characteristics depending on the mechanism by which they occur. When detecting such defects, it is possible to use a dedicated detection model for each type of typical defect. However, dedicated detection models are specialized for detecting typical defects, and there is a problem in that the detection accuracy tends to be low for atypical defects that deviate from typical examples.

[0007] Here, Patent Documents 1 to 4 describe the use of multiple models in technologies for detecting defects, etc. In particular, Patent Document 1 describes taking the union of pixel areas detected by multiple trained models trained for each type of defect and calculating the defect rate using the sum of the areas of the union. However, in Patent Document 1, multiple models detect multiple different defects, and the respective detection areas are merely merged after the fact for convenience. Thus, Patent Document 1 does not involve multiple models jointly inferring a single defect or a series of defects. Furthermore, Patent Document 2 describes discriminating the type of anomaly using a learning classifier trained with discrimination criteria for each type of anomaly. However, Patent Document 2 states that if the discrimination result of the first learning classifier discriminates a first anomaly and the discrimination result of the second learning classifier discriminates a second anomaly, visual confirmation is required. Thus, Patent Document 2 states that if multiple models detect a single anomaly, the anomaly is not subject to discrimination. Furthermore, Patent Document 3 describes a method for determining the presence or absence of a defect in a first judgment region using a first learning model constructed by machine learning of a learning image including a first defect image, and for determining the presence or absence of a defect in a second judgment region using a second learning model constructed by machine learning of a learning image including a second defect image smaller than the first defect image. However, in Patent Document 3, the regions for determining the presence or absence of a defect are different to begin with, so the multiple learning models do not jointly infer a single defect. Furthermore, Patent Document 4 describes a method for inputting imaging data into a second learning model different from the first learning model when a visual abnormality is determined based on the judgment result of the first learning model, and determining whether the input imaging data is a false abnormality. However, in Patent Document 4, an abnormality is determined in stages, so multiple learning models do not jointly infer a single defect in a single stage. As such, Patent Documents 1 to 4 cannot combine inference results from multiple models for a single defect, and therefore still have difficulty dealing with atypical defects, etc.

[0008] Therefore, the technology disclosed herein aims to provide a method, an apparatus, and a program that can obtain detection results that can also handle atypical defects, even when a model for detecting typical defects is used.

[0009] In order to achieve the above-mentioned object, the defect detection method according to the disclosed technology includes a computer acquiring a first inference result output by a first detection model in response to inputting a target image into the first detection model for detecting a first type of defect, acquiring a second inference result output by a second detection model in response to inputting the target image into a second detection model for detecting a second type of defect, and outputting a detection result indicating a single defect or a series of defects detected from the target image based on the first inference result and the second inference result.

[0010] The computer may use the first inference result and the second inference result to extract at least one of a first detection area detected as a first type of defect and a second detection area detected as a second type of defect, and output information corresponding to the extraction result as a detection result. The computer may determine whether to extract the second inference result as a second detection area detected as a second type of defect based on the first inference result, or may determine whether to extract the first inference result as a first detection area detected as a first type of defect based on the second inference result. The computer may output at least one of the first detection area and the second detection area as a detection result. The computer may determine the type of defect detected from the target image based on at least one of the first detection area and the second detection area. The computer may output the type of defect as a detection result. The first type of defect and the second type of defect may have different feature quantities that can be calculated from the shape of the defect. One of the first type of defect and the second type of defect may be a circular defect, and the other of the first type of defect and the second type of defect may be a linear defect.

[0011] A defect detection device according to the disclosed technology includes a processor, which obtains a first inference result output by a first detection model in response to inputting a target image into a first detection model for detecting a first type of defect, obtains a second inference result output by a second detection model in response to inputting the target image into a second detection model for detecting a second type of defect, and outputs a detection result indicating a single defect or a series of defects detected from the target image based on the first inference result and the second inference result.

[0012] A defect detection program relating to the technology of the present disclosure causes a computer to obtain a first inference result output by a first detection model in response to inputting a target image into a first detection model for detecting a first type of defect, obtain a second inference result output by a second detection model in response to inputting a target image into a second detection model for detecting a second type of defect, and output a detection result indicating a single defect or a series of defects detected from the target image based on the first inference result and the second inference result.

[0013] According to the technology of the present disclosure, even when a model for detecting typical defects is used, it is possible to obtain detection results that can also handle atypical defects.

[0014] FIG. 1 is a diagram showing an example of a schematic configuration of a defect detection system 1 according to the present embodiment. FIG. 2 is a diagram showing an example of a hardware configuration of a defect detection device 100 according to the present embodiment. FIG. 3 is a diagram showing an example of a functional configuration of the defect detection device 100 according to the present embodiment. FIG. 4 is a diagram showing an example of a first inference result acquired by the defect detection device 100 according to the present embodiment. FIG. 5 is a diagram showing an example of a second inference result acquired by the defect detection device 100 according to the present embodiment. FIG. 6 is a diagram showing an example of a defect detection flow executed by the defect detection device 100 according to the present embodiment. FIG. 7 is a diagram showing an example of a detection result output by the defect detection device 100 according to the present embodiment. FIG. 8 is a diagram schematically showing an effect of the defect detection device 100 according to the present embodiment. FIG. 9 is a diagram showing an example of a functional configuration of the defect detection device 100 according to a modified example of the present embodiment. FIG. 10 is a diagram showing an example of a defect detection flow executed by the defect detection device 100 according to the modified example of the present embodiment. FIG. 11 is a diagram showing an example of a detection result output by the defect detection device 100 according to the modified example of the present embodiment.

[0015] An example of an embodiment of the present disclosure will be described below with reference to the drawings. In each drawing, the same or equivalent components and parts are designated by the same reference numerals. Furthermore, the dimensional proportions in the drawings are exaggerated for the sake of explanation and may differ from the actual proportions.

[0016] 1 is a diagram showing an example of a schematic configuration of a defect detection system 1 according to this embodiment. This diagram shows an example in which the defect detection system 1 detects defects in a butt joint formed by groove welding. However, the present invention is not limited to this. The defect detection system 1 may also detect various defects, such as defects in other welded joints, such as fillet welds, plug welds, or slot welds, such as T-joints, cross joints, corner joints, back joints, lap joints, or edge joints, and defects in castings formed by casting.

[0017] Defects in welded joints are known as welding defects, and there are various types of welding defects, such as blowholes (also called gas holes), porosity, poor penetration, slag inclusions, and cracks. Similarly, defects in castings are known as casting defects, and there are various types of casting defects, such as shrinkage cavities, blowholes, cracks, poor molten metal running, cold shuts, and ripples. These defects have unique geometric features. The defect detection system 1 may detect various defects that may have different geometric features.

[0018] The defect detection system 1 includes, as an imaging system, an object 10 and a radiation source 20. The object 10 is an object to be imaged. As an example, the object 10 may be a welded structure including a butt joint formed by groove welding.

[0019] The radiation source 20 is a source that emits radiation. Any type of radiation may be used as the radiation, such as α-rays, β-rays, neutron rays, γ-rays, or X-rays. From here on, an example in which X-rays are used as the radiation will be described.

[0020] In such an imaging system, X-ray imaging is performed by irradiating X-rays from a radiation source 20 onto an object 10 and visualizing the X-rays that have passed through the object 10. This allows an object image i to be obtained. Such an object image i is widely used as a useful image in non-destructive testing (NDT).

[0021] However, manually analyzing defects from the target image i obtained by such X-ray photography is sometimes undesirable because it is a complicated and time-consuming process and the analysis results vary depending on the level of proficiency. Therefore, in the defect detection system 1, an AI (Artificial Intelligence) model that has been trained by machine learning is used to detect defects when analyzing the target image i.

[0022] The defect detection system 1 includes an analysis system including a first detection model 50P, a second detection model 50Q, and a defect detection device 100. The first detection model 50P and the second detection model 50Q are collectively referred to as the "detection model 50" unless there is a need to distinguish between them. This figure illustrates an example in which the detection model 50 is provided outside the defect detection device 100. Generally, machine-learned AI models can be large in capacity. Therefore, it is preferable that the detection model 50 be stored outside the defect detection device 100, for example, on the cloud. However, this is not a limitation. To improve security and reduce communication time and costs, at least one of the first detection model 50P and the second detection model 50Q may be provided inside the defect detection device 100. While this figure illustrates an example in which there are two detection models 50, there may be three or more detection models 50.

[0023] The detection model 50 is a model for detecting defects. The detection model 50 may be any of various AI models, such as, for example, Fully Convolutional Networks (FCN), SegNet, Feature Pyramid Networks (FPN), Region-Convolutional Neural Network (R-CNN), and Recurrent Neural Network (RNN).

[0024] The detection model 50 may perform Assisted Defect Recognition (ADR) by segmentation. Examples of segmentation include semantic segmentation, instance segmentation, and panoptic segmentation, but semantic segmentation is used here. That is, the detection model 50 may infer whether each pixel is a "defect" (i.e., "non-defect") in response to the input of the target image i.

[0025] To build such a detection model 50, it is necessary to learn many typical defect images so that features similar to the learned defect images can be extracted from new images. However, as described above, defects vary in shape, size, and the like depending on the mechanism of their occurrence. For example, blowholes and porosity have circular shapes, while poor penetration, slag inclusions, and cracks have linear shapes.

[0026] The JIS (Japanese Industrial Standards) specifies that blowholes and porosity are classified as Type 1, poor penetration and slag inclusions as Type 2, and cracks as Type 3. However, because cracks, like poor penetration and slag inclusions, have characteristics that are close to linear, cracks will be classified here as the same type as poor penetration and slag inclusions.

[0027] It is difficult to detect various defects having such mutually different characteristics using a single model. Therefore, the defect detection system 1 uses at least two independent detection models 50, a first detection model 50P and a second detection model 50Q. Note that the first detection model 50P and the second detection model 50Q do not have to be completely independent models, and may be models in which at least a part of the input layer or the intermediate layer overlaps.

[0028] The first detection model 50P is a model for detecting a first type of defect. The first type of defect may be, for example, a defect of a first shape, here, a circular defect. Therefore, the first detection model 50P may be machine-learned using defect images of defects such as blowholes and porosity as learning data. When a target image i is input, the first detection model 50P may infer, for each pixel, whether the defect has characteristics similar to those of the learned defect image, i.e., whether the defect is circular or not. The first detection model 50P may then output a first inference result that can distinguish pixels inferred as "defects" from pixels inferred as "non-defects." In this case, as an example, the first detection model 50P may output, as the first inference result, an image in which pixels inferred as "defects" are marked with a first color or a first pattern.

[0029] The second detection model 50Q is a model for detecting a second type of defect different from the first type of defect. The second type of defect may be, for example, a second shape, e.g., a linear defect. Therefore, the second detection model 50Q may be machine-learned using defect images of defects such as poor penetration, slag inclusion, and cracks as learning data. When the target image i is input, the second detection model 50Q may infer, for each pixel, whether a defect has characteristics similar to the learned defect image, i.e., whether the defect is a linear defect. The second detection model 50Q may then output a second inference result that distinguishes pixels inferred as "defects" from pixels inferred as "non-defects." In this case, for example, the second detection model 50Q may output, as the second inference result, an image in which pixels inferred as "defects" are marked with a second color or a second pattern.

[0030] In this manner, one of the first type defect and the second type defect may be a circular defect, and the other of the first type defect and the second type defect may be a linear defect. Here, a common operational standard is to consider defects whose major axis length to minor axis length ratio is greater than 3 as type 2 defects, and defects whose major axis length to minor axis length ratio is 3 or less as type 1 defects. Therefore, the first type defect and the second type defect may be defects whose major axis length to minor axis length ratio is different from each other, such as a circular defect and a linear defect. However, this is not limited thereto. The first type defect and the second type defect may be any defects whose feature values ​​that can be calculated from the defect shapes are different from each other.

[0031] However, defects that actually occur in welded joints, castings, etc. cannot always be clearly classified as type 1 defects or type 2 defects. For example, defects with shapes intermediate between circular and linear shapes exist. For example, such atypical defects that fall on the border between type 1 defects and type 2 defects do not fall within either of the typical examples learned to construct the first detection model 50P and the second detection model 50Q. As a result, the detection accuracy of both the first detection model 50P and the second detection model 50Q tends to be low, which poses a problem of making it unclear which of the first detection model 50P and the second detection model 50Q should be responsible for detection.

[0032] Therefore, the defect detection device 100 acquires a first inference result output by a first detection model 50P for detecting a first type of defect and a second inference result output by a second detection model 50Q for detecting a second type of defect, and outputs a detection result based on both the first inference result and the second inference result. This allows the two detection models 50 to be used to deal with typical defects, respectively, and also allows the device to deal with atypical defects by complementarily utilizing and integrating the inference results of the two detection models 50. This will be described in detail.

[0033] 2 is a diagram showing an example of the hardware configuration of the defect detection apparatus 100 according to this embodiment. The defect detection apparatus 100 includes a processor 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a storage 104, a communication interface 105, and a user interface 106. These components are connected to each other via a bus 109 so as to be able to communicate with each other.

[0034] The processor 101 executes various programs and controls each component. The ROM 102 stores various programs and various data. The RAM 103 temporarily stores programs or data as a working area. The storage 104 is configured with an HDD (Hard Disk Drive) or an SSD (Solid State Drive) and stores various programs including the operating system and various data.

[0035] The communication interface 105 is an interface that allows the defect detection apparatus 100 to communicate with other apparatuses. The user interface 106 is an input / output interface that allows the defect detection apparatus 100 to exchange information with a user. The user interface 106 may include a touch panel, a keyboard, a microphone, etc. as input devices, and may include a monitor, a speaker, etc. as output devices.

[0036] 3 is a diagram showing an example of the functional configuration of the defect detection device 100 according to this embodiment. The defect detection device 100 according to this embodiment includes a first inference result acquisition unit 110, a second inference result acquisition unit 120, an extraction unit 130, and an output unit 140. These functional configurations are realized by the processor 101 reading out a defect detection program from the ROM 102 or the storage 104 and expanding the program in the RAM 103. The same applies to other functional configurations described below.

[0037] The first inference result acquisition unit 110 acquires the first inference result output by the first detection model 50P in response to inputting a target image i into the first detection model 50P for detecting a first type of defect.

[0038] The second inference result acquisition unit 120 acquires the second inference result output by the second detection model 50Q in response to inputting the target image i into the second detection model 50Q for detecting a second type of defect.

[0039] The extraction unit 130 uses the first inference result and the second inference result to extract at least one of a first detection area detected as a first type of defect and a second detection area detected as a second type of defect.

[0040] The output unit 140 outputs a detection result indicating a single defect or a series of defects detected from the target image based on the first inference result and the second inference result.

[0041] 4 is a diagram showing an example of a first inference result acquired by the defect detection device 100 according to this embodiment. As an example of the first inference result, the diagram shows an image in which a plurality of pixel groups p1 to p6, each of which is a continuous group of pixels inferred as "defects" by the first detection model 50P for circular defects, are marked with a horizontal line pattern.

[0042] 5 is a diagram showing an example of a second inference result acquired by the defect detection device 100 according to this embodiment. As an example of the second inference result, the diagram shows an image in which a plurality of pixel groups q1 to q6, each of which is a continuous group of pixels inferred as a "defect" by the second detection model 50Q for linear defects, are marked with a horizontal line pattern.

[0043] The defect detection device 100 according to this embodiment acquires, for example, the first inference result and the second inference result, and outputs a detection result based on both the first inference result and the second inference result. This will be described in detail using a flow chart.

[0044] 6 is a diagram showing an example of a defect detection flow executed by the defect detection device 100 according to this embodiment. This flow is executed by the processor 101 reading out a defect detection program from the ROM 102 or the storage 104 and loading it into the RAM 103. The same applies to other flows described later.

[0045] In step S210, the processor 101, as the first inference result acquisition unit 110, acquires a first inference result output by the first detection model 50P in response to inputting the target image i to the first detection model 50P for detecting a first type of defect. For example, the processor 101 may acquire the target image i from an imaging system via a network and input the acquired target image i to the first detection model 50P. In response to this, the first detection model 50P may output an image such as that shown in FIG. 4. The processor 101 may acquire, for example, the image output by the first detection model 50P in this manner as the first inference result.

[0046] In step S220, the processor 101, as the second inference result acquisition unit 120, acquires a second inference result output by the second detection model 50Q in response to inputting the target image i to the second detection model 50Q for detecting a second type of defect. For example, the processor 101 may input to the second detection model 50Q the same target image i as that input to the first detection model 50P in step S210. In response to this, the second detection model 50Q may output an image such as that shown in FIG. 5 . The processor 101 may acquire, for example, the image output by the second detection model 50Q in this manner as the second inference result.

[0047] In step S230, the processor 101, functioning as the extraction unit 130, extracts a first detection region Rp detected as a first type of defect. Here, it is assumed that the inference result obtained by the first detection model 50P trained to detect circular defects has a relatively high degree of certainty. In this case, the processor 101 may uniformly extract, for example, the first inference result obtained in step S210 as the first detection region Rp. In other words, all of the pixel groups p1 to p6 may be adopted as the first detection region Rp.

[0048] In step S240, the processor 101, functioning as the extraction unit 130, extracts a second detection region Rq detected as a second type of defect. Here, it is assumed that the inference result obtained by the second detection model 50Q trained to detect a linear defect has a relatively low probability. In this case, the processor 101 may determine, for example, based on the first inference result obtained in step S210, whether to extract the second inference result obtained in step S220 as the second detection region Rq.

[0049] In this case, the processor 101 can make a determination using, for example, the following algorithms A1 to A3. <Algorithm A1> If the inference results from different detection models 50 at least partially overlap or adjoin, both are adopted as detection areas. <Algorithm A2> For two regions among the inference results from at least one of the detection models 50, whose major axes form an angle within a reference angle (i.e., which can be considered to be on a straight line), if the distance between the two regions is equal to or less than the length of the longer major axis of the two regions, both are adopted as detection areas. <Algorithm A3> In cases other than algorithm A1 or A2, if the feature amount calculated from the inference result from a specific detection model 50 is equal to or less than a reference value, the region is not adopted as a detection area.

[0050] As an example, the pixel group q1 is adjacent to the pixel group p1 and is within a predetermined distance from the pixel group p1, so the processor 101 determines to extract the pixel group q1 as the second detection region Rq in accordance with the algorithm A1.

[0051] Pixel groups q2 and q3 do not overlap or adjoin any of pixel groups p1 to p6. Furthermore, the angle formed by the major axes of pixel groups q2 and q3 is greater than a predetermined reference angle, and they cannot be considered to be on a straight line. Furthermore, the ratio of the length of the major axis to the length of the minor axis of pixel groups q2 and q3 is 3 or less. Therefore, in accordance with algorithm A3, processor 101 determines not to extract pixel groups q2 and q3 as the second detection region Rq.

[0052] The pixel group q4 at least partially overlaps with the pixel group p2 and the pixel group p3, and therefore the processor 101 determines to extract the pixel group q4 as the second detection region Rq in accordance with the algorithm A1.

[0053] Pixel group q5 does not overlap or adjoin any of pixel groups p1 to p6. However, the angle formed by the major axes of pixel group q5 and pixel group q6 is within a predetermined reference angle, and they can be considered to be on a straight line. The distance between pixel group q5 and pixel group q6 is equal to or less than the length of the major axis of pixel group q6. Therefore, in accordance with algorithm A2, processor 101 determines to extract pixel groups q5 and q6 as the second detection region Rq.

[0054] It should be noted that "JIS Z 3104-1995, Appendix 4" stipulates that "When flaws exist in a straight line and the distance between them is equal to or less than the length of the larger flaw, the measurement including the distance between the flaws shall be the flaw length of the group of flaws," and the criteria for Algorithm A2 are based on this provision.

[0055] In this way, the processor 101 extracts the pixel groups q1, q4, q5, and q6 from among the pixel groups q1 to q6 as the second detection region Rq, but does not extract the pixel groups q2 and q3 as the second detection region Rq. In this way, for example, the processor 101 may not uniformly extract the second inference results as the second detection region Rq, but may determine whether or not to extract them as the second detection region Rq based on the first inference result.

[0056] In the above description, an example has been given in which the processor 101 uniformly extracts the first inference result as the first detection region Rp and determines whether to extract the second inference result as the second detection region Rq based on the first inference result. However, this is not limiting. The processor 101 may also determine whether to extract the first inference result as the first detection region Rp based on the second inference result and uniformly extract the second inference result as the second detection region Rq. Alternatively, the processor 101 may determine whether to extract the first inference result as the first detection region Rp based on the second inference result and determine whether to extract the second inference result as the second detection region Rq based on the first inference result. In this way, the processor 101 may determine whether to extract the second inference result as the second detection region Rq detected as the second type of defect based on the first inference result, or may determine whether to extract the first inference result as the first detection region Rp detected as the first type of defect based on the second inference result. In this way, for example, the processor 101 can use the first inference result and the second inference result to extract at least one of the first detection region Rp detected as the first type of defect and the second detection region Rq detected as the second type of defect.

[0057] In step S250, the processor 101 outputs, as the output unit 140, a detection result indicating a single defect or a series of defects detected from the target image based on the first inference result and the second inference result. For example, the processor 101 may output, as the detection result, information corresponding to the results extracted in at least one of step S230 and step S240. In this case, the processor 101 may output, as the detection result, at least one of the first detection region Rp extracted in step S230 and the second detection region Rq extracted in step S240. Note that a single defect or a series of defects refers to defects that can be regarded as a single entity, and in this case, refers to defects that can be regarded as a single linear defect or a circular defect.

[0058] FIG. 7 is a diagram showing an example of a detection result output by the defect detection device 100 according to this embodiment. This diagram illustrates an example in which the defect detection device 100 outputs both a first detection region Rp and a second detection region Rq as detection results. It should be noted here that the first detection region Rp includes all of the pixel groups p1 to p6 inferred by the first detection model 50P, whereas the second detection region Rq includes only a portion of the pixel groups q1 to q6 inferred by the second detection model 50Q, namely, q1, q4, q5, and q6. In other words, the second detection region Rq is based not only on the second inference result, but also on both the first and second inference results. For example, the defect detection device 100 outputs a detection result based on both the first and second inference results in this manner.

[0059] When the second detection model 50Q outputs a detection result by assuming that all of the pixel groups inferred to be "defects" are linear defects, a model with a high overdetection rate cannot be used as the second detection model 50Q. There is generally a trade-off between the overdetection rate and the correct detection rate. That is, suppressing the overdetection rate also reduces the correct detection rate. In contrast, the defect detection device 100 can delete a portion of the pixel groups inferred to be "defects" by the second detection model 50Q after the fact without extracting them as the second detection region Rq. This allows a model with a high overdetection rate to be used as the second detection model 50Q. That is, when constructing the second detection model 50Q, learning can be performed to maximize the correct detection rate.

[0060] More specifically, for example, pixel groups q2, q3, and q4 are thicker defects than pixel groups q1, q5, and q6. If the second detection model 50Q learns thin defects so as not to overdetect such thick defects, the second detection model 50Q will not infer that pixel groups q2 and q3, nor pixel group q4, are "defects." However, pixel group q4 is an area that should be detected as a linear defect. In this case, the correct detection rate of the second detection model 50Q will decrease.

[0061] Therefore, the defect detection device 100 according to this embodiment subsequently deletes some of the pixel groups that the second detection model 50Q inferred to be defects. This allows the second detection model 50Q to use a model with a high overdetection rate that also infers pixel groups q2, q3, and q4 as "defects," while extracting pixel group q4, which is a region that should be detected as a linear defect, as a second detection region Rq, and subsequently deleting pixel groups q2 and q3, which should not be detected as a linear defect, without extracting them as the second detection region Rq. The defect detection device 100 utilizes the inference results of the first detection model 50P to determine whether to extract these pixel groups.

[0062] As an example, a thick linear defect may have a circular substructure at its end. Therefore, pixel groups p2 and p3 overlap the end of pixel group q4. That is, a thick linear defect may be inferred as a "defect" by the second detection model 50Q, and its end may also be inferred as a "defect" by the first detection model 50P. Also, a linear defect with uneven thickness may have a circular substructure in its middle. Therefore, pixel group p5 overlaps the middle of pixel group q6. That is, a linear defect with uneven thickness may be inferred as a "defect" by the second detection model 50Q, and its middle may also be inferred as a "defect" by the first detection model 50P.

[0063] In view of these characteristics, the defect detection device 100 according to this embodiment extracts, as a second detection region Rq, the pixel group q4 that at least partially overlaps with the pixel groups p2 and p3 among the pixel groups q2, q3, and q4, while deleting the pixel groups q2 and q3 that do not overlap with or are adjacent to any of the pixel groups p1 to p6 without extracting them as the second detection region Rq. In this way, for example, the defect detection device 100 can determine whether or not to extract the inference result by one detection model 50 as a detection region based on the inference result by the other detection model 50.

[0064] Conventionally, dedicated detection models have been specialized for detecting typical defects, and have had the problem of prone to low detection accuracy for atypical defects that deviate from typical examples. In contrast, the defect detection device 100 according to the present embodiment acquires a first inference result output by a first detection model 50P for detecting a first type of defect and a second inference result output by a second detection model 50Q for detecting a second type of defect, and outputs a detection result based on both the first inference result and the second inference result. As a result, the two detection models 50 are used to deal with each typical defect, and the inference results of the two detection models 50 are complementarily utilized and integrated to deal with atypical defects.

[0065] FIG. 8 is a diagram schematically illustrating the effect of the defect detection device 100 according to this embodiment. (i) indicates a thin line defect, (ii) a thick line defect, (iii) an elliptical circular defect, and (iv) a perfect circular defect. The first detection model 50P excels at detecting defect (iv), but becomes less effective as the defect approaches defect (i). On the other hand, the second detection model 50Q excels at detecting defect (i), but becomes less effective as the defect approaches defect (iv). In this case, conventionally, typical defects such as (i) and (iv) could be detected based on the inference results of the first detection model 50P or the second detection model 50Q, but detection accuracy was likely to be low for atypical defects such as (ii) and (iii). In contrast, the defect detection device 100 according to this embodiment detects atypical defects such as (ii) and (iii) based on two inference results from two detection models 50. This allows each detection model 50 to complementarily cover areas where detection is weak, thereby enabling detection results that also correspond to atypical defects to be output.

[0066] Up to this point, the case where the defect detection device 100 outputs at least one of the first detection region Rp and the second detection region Rq as the detection result has been described as an example, but the present invention is not limited to this. The defect detection device 100 can also determine the type of defect based on at least one of the first detection region Rp and the second detection region Rq and use this as the detection result.

[0067] 9 is a diagram showing an example of the functional configuration of a defect detection device 100 according to a modified example of this embodiment. The defect detection device 100 according to this modified example further includes a discrimination unit 135. The discrimination unit 135 discriminates the type of defect detected from the target image i based on at least one of the first detection region Rp and the second detection region Rq. The defect detection device 100 according to the modified example of this embodiment then outputs the discriminated type of defect as a detection result.

[0068] 10 is a diagram showing an example of a defect detection flow executed by the defect detection apparatus 100 according to the modified example of this embodiment. This flow further includes step S245.

[0069] In step S245, the processor 101, functioning as the discrimination unit 135, discriminates the type of defect detected in the target image i based on at least one of the first detection region Rp and the second detection region Rq. For example, the processor 101 may discriminate the type of defect using the following algorithm B based on at least one of the first detection region Rp extracted in step S230 and the second detection region Rq extracted in step S240. <Algorithm B1> In the first detection region Rp and the second detection region Rq, at least partially overlapping or adjacent regions are considered as a single region, and the two are integrated. <Algorithm B2> If the feature value of the region integrated by algorithm B1 is greater than a reference value, the defect is discriminated as a second type of defect. <Algorithm B3> The target to which the judgment of algorithm A2 is applied is discriminated as a second type of defect. <Algorithm B4> In cases other than algorithm B2 or B3, the first detection region Rp is discriminated as a first type of defect, and the second detection region Rq is discriminated as a second type of defect.

[0070] As an example, pixel group p1 is adjacent to pixel group q1, with the distance thereto being within a predetermined range. Therefore, processor 101 integrates pixel groups p1 and q1 as a series of defects according to algorithm B1. Furthermore, the ratio of the length of the major axis to the length of the minor axis of the integrated pixel groups p1 and q1 is greater than 3. Therefore, processor 101 determines pixel groups p1 and q1 as a series of linear defects according to algorithm B2.

[0071] Pixel groups p2 and p3 at least partially overlap with pixel group q4. Therefore, processor 101 integrates pixel groups p2, p3, and q4 as a single defect in accordance with algorithm B1. Furthermore, the ratio of the length of the major axis to the length of the minor axis of the integrated pixel groups p2, p3, and q4 is greater than 3. Therefore, processor 101 determines pixel groups p2, p3, and q4 as a single linear defect in accordance with algorithm B2.

[0072] Pixel group p4 does not overlap or adjoin any other pixel groups. Furthermore, the ratio of the length of the major axis to the length of the minor axis of pixel group p4 is less than or equal to 3, and pixel group p4 is not subject to the judgment of algorithm A2. Therefore, processor 101 determines pixel group p4 as a single circular defect according to algorithm B4.

[0073] Pixel groups p5 and p6 overlap with pixel group q6. Therefore, processor 101 integrates pixel groups p5, p6, and q6 as a single defect. Furthermore, pixel groups q5 and q6 are subject to the determination of algorithm A2. Therefore, processor 101 determines pixel groups p5, p6, q5, and q6 as a series of linear defects in accordance with algorithm B3.

[0074] In step S250, the processor 101 can output, for example, the type of defect determined in this manner as a detection result.

[0075] 11 is a diagram showing an example of the detection result output by the defect detection device 100 according to a modified example of this embodiment. In this figure, white marks indicate circular defects. Also, black marks indicate linear defects. The processor 101 can output the type of defect in this manner, for example.

[0076] At this time, the processor 101 may output the number of defects detected for each type of defect, such as "one circular defect was detected" or "three linear defects were detected." The processor 101 may also output the center coordinates and estimated size of each defect, such as "coordinates: (x, y), size: 0 mm x △ mm."

[0077] Referring to this figure, pixel group p1 was originally detected as a circular defect in the first detection region Rp, but was ultimately determined to be a linear defect. Here, pixel group p1 and pixel group q1 appear disconnected due to an insufficient X-ray dose during imaging, and it is highly likely that they were actually a single, continuous defect. Even if this were not the case, linear defects are more likely to trigger cracks in the component and are therefore more serious. Therefore, considering the impact of defects on the product, they should be treated as a large, continuous defect. Therefore, the defect detection device 100 takes this impact on the product into consideration and ultimately determines pixel group p1 as a linear defect, even though it was originally detected as a circular defect.

[0078] In this way, the defect detection device 100 according to the modified example of this embodiment determines the type of defect detected from the target image i based on at least one of the first detection region Rp and the second detection region Rq. As a result, the defect detection device 100 according to the modified example of this embodiment can use the type of defect determined based on both the first detection region Rp and the second detection region Rq as the final determination result.

[0079] In the above-described embodiments, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0080] Furthermore, the operations of the processors in the above embodiments may be performed not only by a single processor but also by multiple processors located at physically separate locations working together. Furthermore, the order of the operations of the processors is not limited to the order described in the above embodiments, and may be changed as appropriate.

[0081] Furthermore, the defect detection device 100 according to the above embodiment has been described as being configured by a single device as an example, but may be configured by a plurality of devices.

[0082] The processes performed by the defect detection device 100 according to the above-described embodiment may be performed by software, by hardware, or a combination of both. The processes performed by each unit of the defect detection device 100 may be stored as a program on a storage medium and distributed.

[0083] Furthermore, the present disclosure is not limited to the above, and it goes without saying that various modifications can be made without departing from the spirit of the present disclosure.

[0084] The disclosure of Japanese Patent Application No. 2023-158496, filed on September 22, 2023, is incorporated herein by reference in its entirety. In addition, all documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually indicated to be incorporated by reference.

[0085] The following supplementary notes are further disclosed with respect to the above embodiments. (Supplementary Note 1) A defect detection method, in which a computer acquires a first inference result output by a first detection model for detecting a first type of defect in response to inputting a target image into the first detection model, acquires a second inference result output by a second detection model for detecting a second type of defect in response to inputting the target image into a second detection model for detecting a second type of defect, and outputs a detection result indicating a single defect or a series of defects detected from the target image based on the first inference result and the second inference result. (Supplementary Note 2) The defect detection method according to Supplementary Note 1, in which the computer uses the first inference result and the second inference result to extract at least one of a first detection region detected as the first type of defect and a second detection region detected as the second type of defect, and outputs information corresponding to the extracted result as the detection result. (Supplementary Note 3) The defect detection method according to Supplementary Note 1 or 2, wherein the computer determines, based on the first inference result, whether to extract the second inference result as a second detection area detected as the second type of defect, or determines, based on the second inference result, whether to extract the first inference result as a first detection area detected as the first type of defect. (Supplementary Note 4) The defect detection method according to Supplementary Note 2 or 3, wherein the computer outputs at least one of the first detection area and the second detection area as the detection result. (Supplementary Note 5) The defect detection method according to any one of Supplementary Notes 2 to 4, wherein the computer determines the type of defect detected from the target image based on at least one of the first detection area and the second detection area. (Supplementary Note 6) The defect detection method according to Supplementary Note 5, wherein the computer outputs the type of defect as the detection result. (Supplementary Note 7) The defect detection method according to any one of Supplementary Notes 1 to 6, wherein the first type defect and the second type defect have different feature amounts that can be calculated from the shapes of the defects.(Supplementary Note 8) The defect detection method according to Supplementary Note 7, wherein one of the first type defect and the second type defect is a circular defect, and the other of the first type defect and the second type defect is a linear defect. (Supplementary Note 9) A defect detection device comprising: a processor, the processor acquiring a first inference result output by a first detection model in response to inputting a target image into the first detection model for detecting a first type defect, acquiring a second inference result output by the second detection model in response to inputting the target image into a second detection model for detecting a second type defect, and outputting a detection result indicating a single defect or a series of defects detected from the target image based on the first inference result and the second inference result. (Supplementary Note 10) A defect detection program that causes a computer to: obtain a first inference result output by a first detection model in response to inputting a target image into the first detection model for detecting a first type of defect; obtain a second inference result output by the second detection model in response to inputting the target image into a second detection model for detecting a second type of defect; and output a detection result indicating a single defect or a series of defects detected from the target image based on the first inference result and the second inference result.

Claims

1. A defect detection method, comprising: a computer obtaining a first inference result output by a first detection model in response to inputting a target image into the first detection model for detecting a first type of defect; obtaining a second inference result output by the second detection model in response to inputting the target image into a second detection model for detecting a second type of defect; and outputting a detection result indicating a defect or a series of defects detected in the target image based on the first inference result and the second inference result.

2. The defect detection method described in claim 1, wherein the computer uses the first inference result and the second inference result to extract at least one of a first detection area detected as the first type of defect and a second detection area detected as the second type of defect, and outputs information corresponding to the extracted result as the detection result.

3. The defect detection method described in claim 1, wherein the computer determines whether or not to extract the second inference result as a second detection area detected as the second type of defect based on the first inference result, or determines whether or not to extract the first inference result as a first detection area detected as the first type of defect based on the second inference result.

4. A defect detection method according to claim 2 or 3, wherein the computer outputs at least one of the first detection area and the second detection area as the detection result.

5. A defect detection method described in any one of claims 2 to 4, wherein the computer determines the type of defect detected from the target image based on at least one of the first detection area and the second detection area.

6. The defect detection method according to claim 5, wherein the computer outputs the type of the defect as the detection result.

7. A defect detection method according to any one of claims 1 to 6, wherein the first type of defect and the second type of defect have mutually different feature amounts that can be calculated from the shape of the defect.

8. The defect detection method according to claim 7, wherein one of the first type defect and the second type defect is a circular defect, and the other of the first type defect and the second type defect is a line defect.

9. A defect detection device comprising a processor that: obtains a first inference result output by a first detection model in response to inputting a target image into a first detection model for detecting a first type of defect; obtains a second inference result output by the second detection model in response to inputting the target image into a second detection model for detecting a second type of defect; and outputs a detection result indicating a defect or a series of defects detected in the target image based on the first inference result and the second inference result.

10. A defect detection program which causes a computer to: obtain a first inference result output by a first detection model in response to inputting a target image into a first detection model for detecting a first type of defect; obtain a second inference result output by the second detection model in response to inputting the target image into a second detection model for detecting a second type of defect; and output a detection result indicating a defect or a series of defects detected in the target image based on the first inference result and the second inference result.

Images