Flaw determination method

Abstract

To provide a flaw determination method capable of flaw determination with a training model that is close to the feeling of on-site flaw determination operation.SOLUTION: A flaw determination method includes classifying a surface flaw in an analyte M, calculating a flaw determination ratio through visual inspection conducted by operators 4a-4i for the surface flaws obtained through the classification, performing machine learning of an AI model 31 with training data of a surface flaw image applied with an annotation at the same ratio as the flaw determination ratio, and determining a flaw determination threshold of the machine learning so that an output result of the machine learning becomes a flaw detection ratio closest to the flaw determination ratio through visual inspection conducted by the operators 4a-4i.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a defect detection method, and more particularly to a defect detection method using machine learning. [Background technology]

[0002] For example, Patent Document 1 discloses a defect inspection system that uses an AI model, in which a second learning model is further provided to avoid overdetection of defects (flaws) by the first learning model and increase the accuracy of defect judgment, and the second learning model is made to re-judge the judgment results of the first learning model, and the re-judgement results are fed back to the first learning model as training data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-170299 Summary of the Invention [Problem to be solved by the invention]

[0004] However, because annotations such as defect location and defect class assigned to training data are left to the discretion of the assigner, if the inference results of a model trained by one assigner differ from the opinions of the majority, there is a possibility that the defect judgment sensitivity of the learning model will be significantly biased from the overall opinion. In particular, in workplaces such as intermediate inspection of steel billets, where judgments vary greatly between workers, the above-mentioned annotation method poses a problem of being significantly deviated from the sense of the defect judgment work.

[0005] The present invention solves such problems and aims to provide a method that enables defect determination using a learning model that is close to the average sense of a defect determination worker on-site. [Means for solving the problem]

[0006] In order to achieve the above object, in the first invention, surface defects of an object (M) are classified by defect type, the defect determination ratio by visual inspection by workers (4a to 4i) for each classified surface defect is calculated, a machine learning model (31) is trained using training data of surface defect images annotated at the same ratio as the defect determination ratio, and a defect determination threshold for the machine learning is determined so as to obtain a defect detection rate closest to the defect determination ratio of the workers (4a to 4i). The defect determination ratio here refers to the ratio of defects that each worker considers to be defects or not.

[0007] According to the first invention, the machine learning defect determination threshold is determined so as to obtain a defect determination rate that is closest to the defect determination rate of an operator, thereby enabling defect determination using a learning model that is close to the intuition of on-site defect determination work without causing underdetection or overdetection.

[0008] In the second invention, the surface defects are classified by defect type including their size.

[0009] In the third invention, the machine learning is performed using an object detection AI model (31), and under the condition that the detection rate of the object detection AI model (31) is 100% when the worker's defect determination rate is 100%, a reliability score threshold is selected that minimizes the error in the detection rate of the object detection AI model (31) relative to the worker's defect determination rate, and this is determined as the defect determination threshold of the object detection AI model (31).

[0010] The symbols in parentheses above indicate, for reference, the correspondence with specific means described in the embodiments to be described later. [Effects of the Invention]

[0011] As described above, the defect determination method of the present invention makes it possible to determine defects using a learning model that is close to the feel of on-site defect determination work. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing the configuration of an apparatus for carrying out the method of the present invention. [Figure 2] FIG. 10 is a diagram showing a flaw determination screen. [Figure 3] FIG. 10 is a diagram showing a surface image in which a flaw portion is enclosed by a square region. [Figure 4] This is a graph showing the change in the defect detection rate of trained AI models with different confidence score thresholds relative to the defect judgment rate of workers. [Figure 5] This is a graph showing the change in the defect detection rate of a trained AI model in which the optimal threshold value of the reliability score is used as the defect determination threshold, relative to the defect determination rate of an operator. DETAILED DESCRIPTION OF THE INVENTION

[0013] The embodiments described below are merely examples, and various design improvements made by those skilled in the art without departing from the gist of the present invention are also included in the scope of the present invention.

[0014] As shown in Fig. 1, in order to detect surface defects on a horizontally elongated steel material M as an object to be inspected, multiple cameras 1A to 1C are provided above the steel material M to capture a surface image 2a over the entire length of the steel material M, and the surface image 2a is input into a processing device 3 having a built-in computer that implements an AI model 31. The surface image 2a is then processed by a trained AI model (in this embodiment, the object detection model YOLO) 31 in the processing device 3, and a defect determination screen 2b is output in which the position is identified by being surrounded by a rectangular bounding box as shown in Fig. 2 and the defect types I to VIII (large), (medium), and (small), which will be described later, are labeled.

[0015] The training data for learning (machine learning) of the AI ​​model 31 is prepared as follows: A plurality of surface images 2a (for example, 250 images) are acquired, and each defect in the surface image 2a is judged by a plurality of workers 4a to 4i (for example, nine workers).

[0016] That is, as shown in Fig. 3, the defect image portion included in the surface image 2a is enclosed in a square area, and each defect image enclosed in each square area is assigned a defect number (No. 1, No. 2, No. 3, ...), and the workers 4a to 4i are asked to judge whether it is a defect (x) or not (o), and if it is a defect, the type of defect. An example is shown in Table 1. A defect judgment ratio is calculated for each defect assigned defect number 1 to n from the judgment results of the workers 4a to 4i. At this time, the type of defect for each defect is also confirmed.

[0017] [Table 1] There are 24 types of defect types, for example, eight major categories I to VIII, each with three minor categories (large, medium, and small).Then, the judgment results of the above workers 4a to 4i are reflected in each of the 24 defect types, and the defect judgment ratios of the workers 4a to 4i are calculated for each of the defect types I to VIII (large, medium, and small), as shown in Table 2.After that, for each defect type, annotations are added at the same ratio (rate) as the defect judgment ratio to create training data, and the AI ​​model 31 is trained.

[0018] [Table 2]

[0019] Thereafter, the evaluation data is provided to the trained AI model 31, and for each target defect for which the defect determination rate by the workers 4a to 4i is known, the corresponding defect detection rate of the trained AI model 31 is checked. The defect detection rate is calculated using the following formula (1). Defect detection rate = Number of defects detected by the AI ​​model for target defects / Number of target defects…(1)

[0020] Figure 4 is a graph showing the defect detection rate of workers 4a to 4i on the horizontal axis and the defect detection rate of trained AI model 31 on the vertical axis. Line X in the figure is a reference line where the defect detection rate of trained AI model 31 is the same as the defect detection rate of workers 4a to 4i. The bar graph in Figure 4 also shows the number of target defects for each defect detection rate. Table 3 shows an example of the number of people who determined there to be a defect, the defect detection rate at that time, and the number of target defects.

[0021] [Table 3]

[0022] In Figure 4, there are multiple detection lines showing the defect detection rate of the trained AI model 31 relative to the defect determination ratios of the workers 4a to 4i because different thresholds for the reliability score Cs are set for the AI ​​model 31. For example, the detection line La in Figure 4 is for the AI ​​model 31 with the threshold set to 0.05, and the detection line Lb is for the AI ​​model 31 with the threshold set to 0.6.

[0023] In this embodiment, since the object detection model YOLO is used as the AI ​​model 31 as described above, the reliability score Cs is calculated by the following formula (2). Cs=Pr(Classi|Object)·Pr(Obeject)·IoU…(2)

[0024] In equation (2), Pr(Classi|Object) indicates the predicted probability of the class (defect type), and Pr(Obeject)·IoU indicates the extent to which the bounding box contains objects (defects). In other words, the confidence score Cs indicates the degree to which the bounding box contains objects (defects) of which class (defect type), i.e., the probability of detection and identification (0 to 1).

[0025] Therefore, among the detection lines shown in Fig. 4, under the condition that the defect detection rate of the AI ​​model 31 is 100% when the defect judgment rate by the workers is 100%, the detection line Lc with the smallest error relative to the reference line X is selected (Fig. 5). The defect detection line Lc is obtained by the AI ​​model 31 with the threshold value of the reliability score Cs set to 0.15, and this 0.15 is determined as the defect judgment threshold value of the AI ​​model 31 for final defect detection.

[0026] When automatically setting the threshold value of the reliability score Cs, the threshold value of the reliability score Cs that maximizes the coefficient of determination (R2) of the AI ​​model 31 is selected under the condition that the defect detection rate of the AI ​​model 31 is 100% when the defect judgment rate of the worker is 100%, and this is determined as the final defect judgment threshold value of the AI ​​model 31 for defect detection. [Explanation of symbols]

[0027] 1A, 1B, 1C...camera, 2a...surface image, 2b...defect determination screen, 3...processing device, 31...AI model, M...steel material (subject).

Claims

1. A defect detection method that classifies surface defects on a test specimen by defect type, calculates the defect determination rate by an operator's visual inspection for each classified surface defect, trains a machine learning model using training data of surface defect images annotated at the same rate as the defect determination rate, and determines a defect determination threshold for the machine learning so as to achieve a defect detection rate closest to the operator's defect determination rate.

2. The flaw determination method according to claim 1, wherein the surface flaws are classified by the flaw type including the size of the flaw.

3. 3. The defect determination method according to claim 1 or 2, wherein the machine learning is performed using an object detection AI model, and under conditions where the detection rate of the object detection AI model is 100% when the defect determination rate of the worker is 100%, a reliability score threshold is selected that minimizes the error in the detection rate of the object detection AI model relative to the worker's defect determination rate, and this is determined as the defect determination threshold of the object detection AI model.

Images