Training data generation device, training data generation method, program, defect inspection device, and recording medium

Abstract

Provided are a training data generation device, a training data generation method, a program, and a defect inspection device which are hardly influenced by the size of an object to be detected. The training data generation device comprises a processor, wherein the processor extracts, as an area-of-interest, an area equal to or greater than 2 px including a defective pixel to be detected with respect to an image to be inspected including a subject to be inspected, and extracts, as training data, a patch image of a patch size including the area-of-interest.

Description

[[Technical Field]]

[0001] The present invention relates to a teacher data generation apparatus, a teacher data generation method, a program 、 and a defect inspection apparatus and recording media . [[Background Art]]

[0002] In non-destructive inspection of industrial products using X-ray transmission images and the like, defects such as air bubbles, foreign matter, and cracks occurring in products are detected by visual observation of images obtained by transmission photography of the products to be inspected, and non-defective products or defective products are determined.

[0003] In recent years, when determining whether a product is non-defective or defective, the use of a learning model trained by machine learning for the determination has been studied. When performing machine learning, teacher data is extracted from inspection target images, and a learning model is trained by machine learning using the extracted teacher data.

[0004] In such machine learning, for example, a technique of adjusting the number of data for each type to be detected in order to improve recognition accuracy is applied (Patent Document 1). [[Prior Art Documents]] [[Patent Documents]]

[0005] [[Patent Document 1]] Japanese Unexamined Patent Application Publication No. 2021-047826 [[Summary of the Invention]] [[Problem to be Solved by the Invention]]

[0006] By the way, when extracting teacher data from an inspection target image, if teacher data is extracted starting from each pixel that is a defect to be detected, the larger the size of the defect, the greater the number of teacher data that will be extracted. Even for defects belonging to the same type, imbalance occurs in the teacher data, and learning for micro defects is relatively delayed, which causes the detection rate not to improve.

[0007] This invention has been made in view of these circumstances, and provides a training data generation device, a training data generation method, and a program that are less affected by the size of the object being inspected. 、 Defect inspection device and recording media The purpose is to provide. [Means for solving the problem]

[0008] In a training data generation device equipped with a processor according to the first embodiment, the processor extracts a region of 2 pixels or more containing defective pixels to be detected from an image of an object to be inspected, which includes the object to be inspected, as a region of interest, and extracts a patch image of patch size including the region of interest as training data.

[0009] In the training data generation apparatus of the second embodiment, the region of interest is a pixel region acquired according to a specific rule, and as a result of determining whether or not each pixel region contains a defective pixel corresponding to a defect, it is determined that the region contains a defective pixel to be detected.

[0010] In the training data generation device of the third embodiment, the specific rule is to scan the image to be inspected in a random or exhaustive order to obtain a pixel range of a fixed size.

[0011] In the training data generation device of the fourth embodiment, a specific rule is used in which the range of the object to be detected is limited in advance to a part of the image to be inspected, and within that range, a fixed-size pixel range is obtained by scanning in a random or exhaustive order.

[0012] In the fifth embodiment of the training data generation apparatus, the processor extracts patch images of a patch size, which include pixels to be distinguished from the detection target, from the image to be inspected, as training data.

[0013] In the training data generation apparatus of the sixth embodiment, the region of interest is determined by identifying a detection target that includes the pixel to be detected, and acquiring a certain number of pixel regions that include the pixel to be detected.

[0014] In the training data generation device of the seventh embodiment, the patch size is a size determined by the input of the machine learning device that uses the training data to create a learning model.

[0015] In the eighth embodiment of the training data generation apparatus, the size of the region of interest is determined based on the size of the defect to be detected or the distance between defects.

[0016] In the training data generation apparatus of the ninth embodiment, the processor determines whether or not each region of interest contains defective pixels based on ground truth data derived from the image to be inspected.

[0017] In the training data generation device of the tenth embodiment, the distance between adjacent regions of interest is less than or equal to the patch size.

[0018] In the training data generation apparatus of the 11th embodiment, the processor determines whether or not each region of interest contains a defective pixel, making the determination for each type of defect.

[0019] In the training data generation apparatus of the twelfth embodiment, the processor adjusts the number of training data for each type of defect.

[0020] In the training data generation device of the 13th embodiment, the image to be examined is an X-ray transmission image.

[0021] In the training data generation apparatus of the 14th embodiment, the image to be inspected is an X-ray transmission image of a cast metal part, a forged metal part, or a welded metal part.

[0022] A training data generation method performed by a training data generation device equipped with a processor according to the 15th embodiment includes the steps of: extracting a region of 2 pixels or more containing defective pixels to be detected from an image of an object to be inspected, which includes an object to be inspected, as a region of interest; and extracting a patch image of patch size including the region of interest as training data.

[0023] In a program for executing a training data generation method by a training data generation apparatus comprising the processor according to the sixteenth aspect, causing execution of the steps of: extracting, from an inspection object image including an inspection object, a region of 2 pixels or more including defective pixels to be detected as a region of interest; and extracting a patch image of a patch size that includes the region of interest as training data.

[0024] A defect inspection apparatus according to the seventeenth aspect comprises: the above-described training data generation apparatus; an inspection object image acquisition unit that acquires an inspection object image; and a defect inspection unit that performs learning using training data created by the training data generation apparatus and detects defects in the inspection object image.

Effects of the Invention

[0025] According to the training data generation apparatus, training data generation method, and program 、 and defect inspection apparatus and recording media of the present invention, it is possible to generate training data that is less susceptible to the size of a detection target, and suppress a decrease in the accuracy of the defect detection rate.

Brief Description of Drawings

[0026] [Figure 1] Figure 1 is a block diagram showing a defect inspection system. [Figure 2] Figure 2 is a block diagram showing an example of the hardware configuration of the training data generation apparatus. [Figure 3] Figure 3 is a block diagram showing processing functions implemented by a CPU. [Figure 4] Figure 4 is a block diagram showing an example of a machine learning apparatus. [Figure 5] Figure 5 is a flowchart showing a training data generation method using the training data generation apparatus according to the first embodiment. [Figure 6] Figure 6 is a diagram explaining an image acquisition step. [Figure 7] Figure 7 is a diagram explaining a region of interest extraction step. [Figure 8] Figure 8 is a diagram explaining a defective pixel determination step. [Figure 9] Figure 9 illustrates the training data extraction step. [Figure 10] Figure 10 illustrates a modified example of the first embodiment. [Figure 11] Figure 11 illustrates the region extraction step, defective pixel determination step, and training data extraction step of the second embodiment. [Figure 12] Figure 12 illustrates the region extraction step, defective pixel determination step, and training data extraction step of the third embodiment. [Figure 13] Figure 13 illustrates the defect pixel detection step when a pixel contains multiple types of defects. [Figure 14] Figure 14 illustrates the training data extraction step in the case of Figure 13. [Figure 15] Figure 15 is a block diagram showing an example configuration of a defect inspection device. [Figure 16] Figure 16 is a block diagram showing an example of the configuration of the imaging system. [Figure 17] Figure 17 is a conceptual diagram showing the average sizes of Gas, FMLD, and FMMD. [Modes for carrying out the invention]

[0027] Hereinafter, embodiments of the training data generation apparatus, training data generation method, program, and defect inspection apparatus according to the present invention will be described with reference to the attached drawings.

[0028] [Configuration of the defect inspection system] Figure 1 is a block diagram of the defect inspection system.

[0029] As shown in Figure 1, the defect inspection system 10 performs defect inspection on an object to be inspected, such as an industrial product (hereinafter also referred to as the object being inspected), using machine learning, and comprises a training data generation device 100 and a defect inspection device 200.

[0030] The training data generation device 100 receives training image P10, which is an image of the object to be inspected, and creates training data to be used for training the machine learning device 140 (see Figure 4) based on the training image P10. Here, the training image P10 is an image containing a sample of the defect to be detected, and is an image obtained by imaging an industrial product, etc., that is the same as or similar to the object to be inspected OBJ (see Figure 16) to be inspected by the defect inspection device 200. The training image P10 may be an image of an object to be inspected in which defects have naturally occurred during use, or it may be an image of an object to be inspected in which defects have been artificially added. As the training image P10, for example, one provided as a sample by the designer or operator of the facility, etc., to which the object to be inspected is installed can be used. The training image P10 is, for example, an X-ray transmission image. Furthermore, this X-ray transmission image is, for example, an X-ray transmission image of a cast metal part, a forged metal part, or a welded metal part.

[0031] The learning model trained by the machine learning device 140 is used by the defect inspection device 200. The learning model can be incorporated into the defect inspection device 200. The defect inspection device 200 can utilize the incorporated learning model. In addition, the learning model can be incorporated into a server on the network connected to the defect inspection device 200. The defect inspection device 200 can utilize the learning model on the server via the network. The defect inspection device 200 can utilize a learning model trained for each industrial product being inspected.

[0032] <Configuration of the training data generation device> Next, the training data generation device 100 according to the embodiment will be described with reference to Figure 2. Figure 2 is a block diagram showing an example of the hardware configuration of the training data generation device 100 according to the present invention.

[0033] As shown in Figure 2, a computer or workstation can be used as the training data generation device 100. In this example, the training data generation device 100 mainly consists of an input / output interface 112, a storage unit 116, an operation unit 118, a CPU (Central Processing Unit) 120, a RAM (Random Access Memory) 122, a ROM (Read Only Memory) 124, and a display control unit 126. A display device 130, which constitutes a display, is connected to the training data generation device 100, and under the command of the CPU 120, the display control unit 126 controls the display on the display device 130.

[0034] The input / output interface 112 can input various types of data (information) to the training data generation device 100. For example, data stored in the storage unit 116 can be input via the input / output interface 112. Furthermore, the input / output interface 112 can communicate with external devices via a network. As a method for sending and receiving data between the training data generation device 100 and external devices, wired communication or wireless communication (e.g., LAN (Local Area Network), WAN (Wide Area Network), Internet connection, etc.) can be used.

[0035] The CPU (processor) 120 reads various programs, including the training data generation program of the embodiment, stored in the memory unit 116 or ROM 124, and loads them into the RAM 122 to perform calculations, thereby providing overall control of each unit. The CPU 120 also reads programs stored in the memory unit 116 or ROM 124, uses the RAM 122 to perform calculations, and carries out various processes of the training data generation device 100. The CPU 120 receives operation input from the operator via the operation unit 118 and transmits control signals corresponding to this operation input to each unit of the training data generation device 100 to control the operation of each unit.

[0036] The memory unit 116 is a memory composed of a hard disk drive, an eMMC (embedded Multi Media Card), an SSD (Solid State Drive), or other flash memory. The memory unit 116 stores data and programs that operate the training data generation device 100, such as the operating system and a program that executes the training data generation method. The memory unit 116 also stores information used in the embodiments described below.

[0037] The control unit 118 includes a keyboard and mouse, and the user can have the training data generation device 100 perform the necessary processing via these devices. By using a touch panel type device, the display device 130 may also function as the control unit.

[0038] The display device 130 is, for example, a device such as a liquid crystal display, and can display various types of information.

[0039] Figure 3 is a diagram showing an example of a block diagram illustrating the processing functions implemented by CPU 120.

[0040] The CPU 120 includes an image acquisition unit 151, a region extraction unit 152, a defective pixel detection unit 153, a training data extraction unit 154, and the like. The image acquisition unit 151, region extraction unit 152, defective pixel detection unit 153, training data extraction unit 154, etc. are all parts of the CPU 120, and the CPU 120 executes the processing of each of these units.

[0041] The image acquisition unit 151 executes the process of acquiring the training image P10 via the input / output interface 112. The method by which the image acquisition unit 151 acquires the training image P10 is, for example, via communication over a network. Alternatively, a USB (Universal Serial Bus) cable, Bluetooth (registered trademark), infrared communication, etc., may be used. Alternatively, the training image P10 may be stored in a storage medium (e.g., a memory card) that can be attached to and read by the teacher data generation device 100, and the image acquisition unit 151 may acquire the training image P10 via this storage medium. Furthermore, the training image P10 can be stored in the storage unit 116 of the teacher data generation device 100, and the image acquisition unit 151 can acquire the training image P10 from the storage unit 116 via the input / output interface 112.

[0042] The region extraction unit 152 performs a process to extract multiple regions from the training image P10 acquired by the image acquisition unit 151. A region is a pixel region of 2 pixels or more. In this process, the region may be of a fixed size. For example, the region extraction unit 152 sets a virtual frame of a size corresponding to the region of interest described later, and extracts multiple regions by scanning this frame over the training image P10 and cutting out the image within the frame (a part of the training image P10). The region extraction unit 152 can arbitrarily set the slide width for scanning the frame and the scanning order, and these arbitrary settings can be made into a specific rule. For example, it is preferable that the slide width of the frame is less than or equal to the patch size described later, and as a result, it is preferable that the distance between multiple adjacent regions is less than or equal to the patch size.

[0043] The defective pixel determination unit 153 determines whether each of the multiple regions extracted by the region extraction unit 152 contains at least some defective pixels corresponding to a defect. The defective pixel determination unit 153 extracts regions that contain at least some defective pixels as regions of interest. That is, regions of interest are pixel regions acquired by the region extraction unit 152 according to a specific rule, and which the defective pixel determination unit 153 determines to contain defective pixels to be detected for each pixel region. The defective pixel determination unit 153 acquires information for extracting regions that contain at least some defective pixels as regions of interest. For example, when the defective pixel determination unit 153 determines whether a region contains defective pixels corresponding to a defect, it is preferable to acquire ground truth data corresponding to the training image P10. The ground truth data includes information such as the position information at the pixel level (1px), whether or not it is a region with defects, and the type of defect, corresponding to the entire image (all pixels) of the training image P10.

[0044] The training data extraction unit 154, for example, refers to ground truth data and extracts patch images of a patch size containing defective pixels from the region of interest as training data. The training data extraction unit 154 extracts the patch images of a patch size extracted from the region of interest and the ground truth data as a set of training data. The ground truth data is, for example, a mask image in which the regions of defects present in the patch size are filled in.

[0045] Figure 4 is a block diagram showing an example of a machine learning device 140 that utilizes training data.

[0046] The machine learning device 140 shown in Figure 4 consists of a learning model (a convolutional neural network (CNN) 142, which is one type of learning model), a loss value calculation unit 144, and a parameter control unit 146.

[0047] This machine learning device 140 uses the training data generated by the training data generation device 100 to train the CNN 142. The training data generated by the training data generation device 100 is stored, for example, in memory 148.

[0048] CNN142 is the part that infers the regions of defects captured in an input image when the input image is a patch image of a patch size extracted from the region of interest. It has a multi-layer structure and holds multiple weight parameters. The weight parameters are the filter coefficients of a filter called a kernel, which is used in the convolution operation in the convolutional layer. Here, the patch size of the image extracted from the region of interest matches the patch size (e.g., m × n:m, where n is an integer) determined by the input used in the machine learning device 140 of the neural network.

[0049] CNN142 can change from an untrained model to a trained model by updating its weight parameters from their initial values ​​to optimal values.

[0050] This CNN142 comprises an input layer 142A, an intermediate layer 142B having multiple sets of convolutional and pooling layers, and an output layer 142C, with each layer having a structure in which multiple "nodes" are connected by "edges".

[0051] The input layer 142A receives the training images as input images. The training images are the patch-sized patch images from the training data (pairs of patch-sized patch images and ground truth data) stored in memory 148.

[0052] The intermediate layer 142B has multiple sets of convolutional layers and pooling layers, and is the part that extracts features from the image input from the input layer 142A. The convolutional layer filters (performs a convolution operation using filters) the nodes that are close to the previous layer and obtains a "feature map". The pooling layer reduces the feature map output from the convolutional layer to create a new feature map. The "convolutional layer" is responsible for feature extraction such as edge extraction from the image, and the "pooling layer" is responsible for providing robustness so that the extracted features are not affected by translation, etc.

[0053] Furthermore, the intermediate layer 142B may include not only a set of convolutional layers and pooling layers, but also consecutive convolutional layers, activation processes using activation functions, and normalization layers.

[0054] The output layer 142C is the part that outputs a feature map showing the features extracted by the hidden layer 142B. In the trained CNN52, the output layer 142C also outputs inference results, such as region classification (segmentation) of defects in the input image at the pixel level or in units of several pixels grouped together.

[0055] The coefficients and offset values ​​of the filters applied to each convolutional layer of the CNN142 before training are set to arbitrary initial values.

[0056] The loss value calculation unit 144, which functions as a learning control unit, and the parameter control unit 146, specifically the loss value calculation unit 144, compare the feature map output from the output layer 142C of the CNN 142 with the mask image (ground truth data read from memory 148), which is the ground truth data for the input image (patch image of patch size), and calculate the error between the two (the loss value, which is the value of the loss function). Possible methods for calculating the loss value include, for example, softmax cross-entropy and sigmoid.

[0057] The parameter control unit 146 adjusts the weight parameters of the CNN 142 using backpropagation based on the loss value calculated by the loss value calculation unit 144. In backpropagation, the error is backpropagated sequentially from the last layer, and stochastic gradient descent is performed at each layer, repeating the parameter update until the error converges.

[0058] This weight parameter adjustment process is repeated until the difference between the output of CNN142 and the ground truth data becomes small.

[0059] The machine learning device 140 repeatedly performs machine learning using the training data stored in memory 148, so that the CNN 142 becomes a trained model.

[0060] Furthermore, R-CNN (Regions with Convolutional Neural Networks) can be applied as CNN142. In R-CNN, bounding boxes of varying sizes are slid within the captured image to detect the region of the bounding box where defects occur. Then, defects are detected by evaluating only the image portion within the bounding box (extracting CNN features). In addition, Fast R-CNN, Faster R-CNN, Mask R-CNN, etc. can be used instead of R-CNN.

[0061] The inference results of the trained model constructed in this manner are used in the defect inspection device 200, as shown in Figure 1.

[0062] <First Embodiment> Next, the method for generating training data in the first embodiment using the training data generation device 100 will be described with reference to the flowchart shown in Figure 5.

[0063] As shown in Figure 5, the method for generating training data includes, as an example, the steps of: acquiring an image containing the object to be inspected (step S1); extracting multiple regions (step S2); determining whether or not the extracted regions contain defective pixels corresponding to defects (step S3); and extracting patch images of a patch size containing defective pixels as training data (step S4).

[0064] In step S1, an image containing the object to be inspected is acquired. The image acquisition unit 151 of the CPU 120 acquires an image containing the object to be inspected (training image P10) (step S1).

[0065] Figure 6 is a diagram illustrating the image acquisition step (step S1). In this embodiment, as shown in Figure 6, the training image P10 contains multiple defects D, one of which is a larger defect D1, and the others are defects D2 which are smaller in size than defect D1. As previously described, the training image P10 is, for example, an image of an object under inspection in which defects have spontaneously occurred during use.

[0066] In step S2, multiple regions are extracted from the training image P10. The region extraction unit 152 of the CPU 120 extracts multiple regions (pixel regions of 2 pixels or more) from the training image P10 (step S2).

[0067] Figure 7 illustrates the region extraction step (step S2). As shown at 0700A in Figure 7, the region extraction unit 152 sets a virtual frame F of a size corresponding to region AR in order to extract region AR. In this embodiment, region AR is a pixel area of ​​2px or more and has a fixed size, and the frame F corresponding to region AR also has a fixed size. This virtual frame F is scanned over the training image P10, and the area enclosed by frame F is extracted as region AR.

[0068] As shown in 0700B, by scanning frame F over the training image P10 in the direction indicated by the arrow, the area enclosed by frame F is extracted from the entire area of ​​the training image P10 as multiple AR regions. Frame F is scanned with a predetermined slide width. For example, in 0700B, frame F is scanned with a slide width of frame F (1x frame F).

[0069] As shown in 0700C, the region extraction unit 152 scans the frame F over the entire area of ​​the training image P10 and extracts region AR. In 0700C, for example, 32 region ARs are extracted.

[0070] In this example, when acquiring region AR from the training image P10, the frame F is scanned starting from the left edge of the first row and moving towards the right edge to complete the acquisition of region AR for the first row. Next, the frame is scanned from the left edge to the right edge of the second row to complete the acquisition of region AR for the second row. The same scanning is repeated until the final row is reached. In the final row, the frame is scanned from the left edge to the right to complete the acquisition of region AR for the final row. Therefore, this particular rule scans the training image P10 in an exhaustive order and acquires region AR, which is a fixed-size pixel range, for the entire area of ​​the training image P10. Note that the slide width for scanning the frame F can be set arbitrarily.

[0071] The specific rule is not limited to this example, but involves scanning the training image P10 in a random order and obtaining a region AR, which is a fixed-size pixel range, for the entire area of ​​the training image P10. Unlike an exhaustive order, the random order scans the frame F in no particular order and obtains a region AR, which is a fixed-size pixel range, for the entire area of ​​the training image P10.

[0072] Furthermore, under certain rules, regional AR may be acquired not only for the entire area of ​​training image P10, but also for only a portion of the training image P10.

[0073] Whether in a random or exhaustive order, reducing the distance between adjacent region ARs increases the number of region ARs extracted from the training image P10. As will be described later, since the region IR of interest containing defective pixels is extracted from the region ARs, increasing the number of region ARs increases the number of region IRs of interest that are extracted. The distance between adjacent region ARs may also be determined considering the number of region ARs.

[0074] In step S3, it is determined whether each region AR contains a defective pixel corresponding to a defect. The defective pixel determination unit 153 of the CPU 120 determines whether a region AR contains a defective pixel corresponding to defect D (including defects D1 and D2) (step S3). Among the regions AR, the region AR that the defective pixel determination unit 153 determines to contain a defective pixel is designated as the region of interest IR, and in step S3, the region of interest IR is extracted.

[0075] Figure 8 illustrates the defective pixel detection step (step S3). The defective pixel detection unit 153 acquires information from the reference ground truth data GT (correct data) to determine the presence or absence of defective pixels. As shown in 0800A, for the entire image (all pixels) of the training image P10, information such as location information, whether or not it is a region of defect D (defects D1, D2), and the type of defect is acquired on a pixel (px) basis. As shown in 0800B, the ground truth data GT defines the correct answers for each pixel in the training image P10 of 0800A, such as whether there is a defect, background (no defect), and the type of defect. The definition of the correct answers can be predetermined and changed by the user.

[0076] The following is an example of a definition. In ground truth data (GT), defects (including scratches) are distinguished and labeled. The distinction between "scratch" and "defect" may be made in advance by using a "scratch" label or a "defect" label. Also, if "scratch" is defined as a "defect" with a weaker degree, the distinction between "scratch" and "defect" may be made later (after reviewing the detection results) based on size, shape, and distribution.

[0077] For example, a location that has the same cause as a gas defect but falls under the category of "defect" because its size is below a certain standard will be labeled as a "gas defect." Therefore, the learning model will also detect it as a "gas defect."

[0078] Types of defects include: (1) Porosity (a collection of tiny bubbles), (2) Gas Hole, (3) FMMD: Foreign Material More Dense, (4) FMLD: Foreign Material Less Dense, (5) Excess Metal, (6) Crack, (7) Saw Cut, (8) Surface Damage (other scratches such as chips on the surface).

[0079] Furthermore, defects specific to weld metal include (1) spatter, (2) undercuts, (3) slag inclusion, and (4) poor penetration.

[0080] The defective pixel determination unit 153 determines whether or not a defective pixel is included in a plurality of regions AR acquired in step S2 shown in 0800C, based on the ground truth data GT. In step S3, the defective pixel determination unit 153 determines whether or not a defective pixel corresponding to a defect is included in each pixel region AR, which is a pixel region acquired according to a specific rule, and extracts the regions determined to contain the defective pixel to be detected (defect D1 or D2) as regions of interest IR. In 0800C, four regions of interest IR containing a part of defect D1 are extracted, and four regions of interest IR containing a part of defect D2 are extracted. In step S4, for the regions of interest IR where the defective pixel Dp exists, Defective pixel Dp The patch image IM, which includes the patch size, is extracted as training data TD. The training data extraction unit 154 of the CPU 120 includes the defective pixel Dp. For IR in areas of interest , Defective pixel Dp The patch image IM containing the patch size is extracted as the training data TD.

[0081] Since patch images IM containing the region of interest IR are extracted as training data TD, it is preferable that the distance between adjacent regions of interest IR is less than or equal to the patch size. This ensures that training data can be reliably extracted from training images P10.

[0082] The size of the area of ​​interest (IR) is preferably determined based on the size of the defects or the distance between them. By setting the area of ​​interest (IR) to an appropriate size, it is possible to avoid having different defects included in a single area of ​​interest (IR). Furthermore, it is possible to correct the imbalance in the number of defects extracted between defect D1 and defect D2.

[0083] The size of the defects is, for example, between 100 μm and 15 mm. In this case, if the image resolution is 100 μm / px, the size of the IR region of interest is 192 px, and the patch size, as described later, can be 128 px.

[0084] The distance between defects also depends on the defect size. For example, defects of a few pixels (= several hundred micrometers) can be densely packed at small intervals, so the minimum distance is 200 micrometers or more. Also, medium to large defects of a few millimeters or more (= for example, 20 pixels or more) occur relatively infrequently. Therefore, the distance between defects is 1 to 2 times the defect length (defect size) (approximately 40 pixels) or more.

[0085] Based on these values, the size of the IR of the region of interest can be determined. However, these values ​​are just examples and are not limited to these values.

[0086] Figure 9 illustrates the training data extraction step (step S4). As shown in 0900A, four regions of interest IR containing the large defect pixel Dp of defect D1 are extracted from the training image P10. From the four regions of interest IR, patch images IM with a patch size (m × n) containing the defect pixel Dp are extracted, and pairs of these four patch images IM with four ground truth data CDs corresponding to them are extracted as training data TD.

[0087] Similarly, as shown in 0900B, four regions of interest IR containing the defective pixel Dp of defect D2 are extracted from the training image P10. From the four regions of interest IR, patch images IM with a patch size (m × n) containing the defective pixel Dp are extracted, and pairs of these four patch images IM with four ground truth data CDs corresponding to them are extracted as training data TD. In Figure 9, the same number (4) of training data TDs are extracted regardless of the size of the defect.

[0088] As previously mentioned, patch images IM of patch size are obtained from multiple areas of interest IR of 2px or more, and training data TD is used. and extract Therefore, it is possible to mitigate the imbalance in which the number of training data points TD increases in proportion to the size of the defect D (resulting in more learning opportunities). Regions containing minute defects are extracted as training data points TD to the same extent as regions containing larger defects.

[0089] A modification of the first embodiment will now be described. In the modification of the first embodiment, in the training data extraction step (step S4), in addition to extracting a patch image IM of a patch size containing defective pixels as training data TD1, a patch image IM of a patch size containing pixels that should be distinguished from the defects D to be detected is extracted as training data TD2.

[0090] Figure 10 illustrates the training data extraction step (step S4) of a modified example. As shown in Figure 10, four regions of interest IRs containing the large defect pixel Dp (not shown) of defect D1 are extracted from the training image P10. From the four regions of interest IRs for defect D1, patch images IM of patch size containing the defect pixel Dp are extracted, and pairs of these four patch images IM with four ground truth data CDs corresponding to these patch images are extracted as training data TD1.

[0091] Similarly, four region IRs of interest containing the defective pixel Dp of defect D2 are extracted from the training image P10. From the four region IRs of interest for defect D1, patch images IM of patch size containing the defective pixel Dp are extracted for each, and pairs of these four patch images IM with four ground truth data CDs are extracted as training data TD1.

[0092] Furthermore, four patch images IM, which include pixels that should be distinguished from the defect D to be detected, i.e., pixels that do not contain defect D, are extracted from the training image P10 as training data TD2. In Figure 10, pairs of four ground truth data CDs corresponding to the four patch images IM from the four regions AR in the lower right of the training image P10 that do not contain defect D are extracted as training data TD2. This allows the so-called "normal regions" in the training image P10 that do not contain defect D to be used as training data.

[0093] <Second Embodiment> Next, the extraction of the region of interest IR and the extraction of training data TD in the second embodiment will be described. Figure 11 is a diagram illustrating the region extraction step (step S2), the defective pixel determination step (step S3), and the training data extraction step (step S4) in another embodiment. In the extraction of the region of interest IR in Figure 11, one inspection target containing the pixel to be detected, in this case defect D1 or defect D2, is identified as one inspection target, a certain number of pixel regions containing the pixel to be detected are acquired, and the region of interest IR is determined.

[0094] Since a certain number of region IRs of interest are extracted from one test target, and training data TDs are extracted from the region IRs of interest, a certain number of training data TDs containing the detection target can be extracted from one test target. As a result, training data TDs can be extracted equally for regions containing minute defects D2 and regions containing other larger defects D1.

[0095] In 1100A, similar to 0700A in Figure 7, the region extraction unit 152 sets a virtual frame F to extract region AR from the training image P10, and extracts the region enclosed by frame F as region AR.

[0096] As shown in 1100B, in extracting the region of interest IR, for example, defect D1 is divided into four sections by vertical and horizontal lines passing through the centroid coordinates of defect D1, so as to include the pixel area of ​​defect D1 that is the target of detection, and four region of interest IRs are extracted. Similarly, defect D2 is divided into four sections by vertical and horizontal lines passing through the centroid coordinates of defect D2 that is the target of detection, and four region of interest IRs are extracted.

[0097] In the second embodiment, defects D1 or D2 are identified as contiguous regions, each designated as a single inspection target. As a method for extracting the contiguous region of defect D from the ground truth data GT, for example, it can be extracted from the geometric moment using an external open-source library, and the centroid can then be estimated. If a simple visual "center point" is sufficient, the circumscribing rectangle can be found and its center coordinates can be used. Note that the method for determining the centroid position is not limited to these methods.

[0098] Next, in the training data extraction step (step S4), four training data points TD can be extracted from each defect D (defects D1 and D2) by extracting patch images IM of patch size that include the defect pixels Dp present in each of the four target regions IR that divide the defect D into four parts. Therefore, the same number of training data points TD are extracted from regions containing small defects and regions containing other large defects. By utilizing this training data TD, the learning accuracy of the machine learning device 140 can be improved.

[0099] <Third Embodiment> Next, the extraction of the region of interest IR and the extraction of training data TD (not shown) in the third embodiment will be described. Figure 12 is a diagram illustrating the region extraction step (step S2) and the defective pixel determination step (step S3). In the extraction of multiple region of interest IRs in Figure 12, only specific regions from the training image P10 are extracted as region of interest IRs, and training data TD (not shown) is extracted from these region of interest IRs.

[0100] In 1200A, similar to 0700A in Figure 7, the region extraction unit 152 sets a virtual frame F to extract region AR from the training image P10, and extracts the region enclosed by frame F as region AR. The training image P10 shows, for example, a case where the system is trained to detect defects occurring on a weld line WL.

[0101] In this case, it is known in advance that the defects to be detected are distributed in one dimension along the weld line WL. Therefore, it is not necessary to extract the area outside of this, including the background region, as training data TD. Conversely, including the outer background region can lead to the model learning unexamined defects that occurred outside the weld line WL as part of the normal region, which can cause a decrease in accuracy and is therefore undesirable.

[0102] Therefore, as shown in 1200B, the frame F is scanned while imposing constraints such as including a welding line WL (a straight line with a one-dimensional width in the horizontal direction), and multiple regions AR are extracted. The range of the detection target is limited in advance to a part of the image to be inspected, and in this case, it is limited to one dimension in the horizontal direction. Within this range, the frame F is scanned in a random or exhaustive order to obtain regions AR, which are pixel ranges of a fixed size.

[0103] Next, in the defective pixel determination step (step S3), it is determined whether the region AR contains the defects D (defects D1 and D2) to be detected, and it is extracted as the region of interest IR. Then, in the training data extraction step (step S4), training data TD (not shown) can be extracted by extracting a patch image IM (not shown) of patch size that includes the defective pixels Dp present in the region of interest IR. In this example as well, since a patch image of patch size is extracted from the region of interest IR, the same number of training data TD are extracted for regions containing minute defects and regions containing other larger defects.

[0104] Furthermore, even with cast parts that have complex shapes, for example, a mask is used to separate the area where the subject is captured from the background, and multiple focus region IRs are extracted so that the background areas that are not captured are not treated the same as the "normal area." This makes it possible to extract training data TD only from the area where the part exists, thereby improving the learning accuracy of the machine learning device 140.

[0105] Next, we will discuss the case where the training image P10 contains multiple types of defects.

[0106] Figure 13 illustrates the defective pixel detection step (step S3) which includes defects of multiple types. As shown in 1300A, the training image P10 contains four defects D1, D2, D3, and D4. Defects D1 and D2 are classified as the same type of defect but differ in size. Similarly, defects D3 and D4 are classified as the same type of defect but differ in size. On the other hand, defects D1 and D2 are of different types than defects D3 and D4. The defective pixel detection unit 153 acquires information from the reference ground truth data GT (correct data) to determine the presence or absence of defective pixels.

[0107] As shown in 1300A, for the entire image (all pixels) of the training image P10, information such as location, whether or not it is a region with defects (defects D1, D2, D3, and D4), and the type of defect is acquired on a per-pixel basis. As shown in 1300B, for the training image P10 of 1300A, the ground truth data GT defines the correct answer for each pixel region, such as whether there is a defect, whether it is background (no defects), and the type of defect. The definition of the correct answer for each region can be defined in advance by the user. In 1300B, defects D1 and D2 are defined as defect type A, and defects D3 and D4 are defined as defect type B.

[0108] In step S3, the defective pixel determination unit 153 determines whether or not a defective pixel is present in any of the multiple regions AR acquired in step S2 shown in 1300C, based on the ground truth data GT, and extracts the region of interest IR.

[0109] Figure 14 illustrates the training data extraction step (step S4). As shown in Figure 14, it is determined whether or not a defective pixel is present in multiple areas of interest (IR), and patch images (IM) of the patch size of each of the four areas of interest (IR) are extracted for defect D1. Patch images (IM) of the patch size of each of the four areas of interest (IR) are extracted for defect D2. Four patch images (IM) are extracted from defect D1, and four patch images (IM) are extracted from defect D2. Patch images (IM) of the patch size of each of the four areas of interest (IR) are extracted for defect D3. Patch images (IM) of the patch size of each of the two areas of interest (IR) are extracted for defect D4. Four patch images (IM) are extracted from defect D3, and two patch images (IM) are extracted from defect D4. Note that the correct data (CD) is omitted for ease of understanding.

[0110] As shown in Figure 14, eight patch image IMs (four for defect D1 and four for defect D2) are extracted for defect type A, and six patch image IMs (four for defect D3 and two for defect D4) are extracted for defect type B. The number of patch image IMs extracted differs between defect types A and B, which are of different types. In order to make the learning accuracy equivalent between defect types A and B, it is preferable to apply oversampling and adjust the number of training data TDs extracted to be equivalent between the types of defects.

[0111] For example, the number of each type may be adjusted to a predetermined number. In this embodiment, the number of training data TDs extracted from defect type A and defect type B is set to 20. Twelve more training data TDs are extracted from defect type A, and fourteen more training data TDs are extracted from defect type B. The total number of training data TDs for defect type A and defect type B becomes 20.

[0112] Furthermore, the number of other types may be adjusted to be the maximum number among all types. In this embodiment, defect type A has 8 training data TDs. On the other hand, defect type B has 6 training data TDs. The 8 for defect type A is the maximum number among all types. Therefore, 2 more training data TDs are extracted from defect type B. The total number of training data TDs for defect type A and defect type B becomes 8.

[0113] The number of training data (TDs) extracted can be adjusted to be equal across different types, thereby improving the imbalance in sample size caused by the type of defect.

[0114] Next, the configuration of the defect inspection device 200 and the imaging system 300 shown in Figure 1 will be described.

[0115] As shown in Figure 15, a computer or workstation can be used as the defect inspection device 200. In this example, the defect inspection device 200 mainly consists of an input / output interface 212, a storage unit 216, an operation unit 218, a CPU 220, a RAM 222, a ROM 224, and a display control unit 226. A display device 230, which constitutes a display, is connected to the defect inspection device 200, and under the command of the CPU 220, the display control unit 226 controls the display device 230 to display information.

[0116] The input / output interface 212 can input various types of data (information) to the defect inspection device 200. For example, data stored in the memory unit 216 can be input via the input / output interface 212. Furthermore, the input / output interface 212 can communicate with external devices via a network. Wired communication or wireless communication (e.g., LAN (Local Area Network), WAN (Wide Area Network), Internet connection, etc.) can be used to send and receive data between the defect inspection device 200 and external devices.

[0117] The CPU (processor) 220 reads various programs, including the defect inspection program of the embodiment, stored in the memory unit 216 or ROM 224, and loads them into the RAM 222 to perform calculations, thereby providing overall control of each part. The CPU 220 also reads programs stored in the memory unit 216 or ROM 224, performs calculations using the RAM 222, and carries out various processes of the defect inspection device 200. The CPU 220 receives operation input from the operator via the operation unit 218 and transmits control signals corresponding to this operation input to each part of the defect inspection device 200 to control the operation of each part.

[0118] The storage unit 216 is a memory composed of a hard disk drive, eMMC, SSD, or other flash memory. The storage unit 216 stores data and programs that operate the defect inspection device 200, such as the operating system and programs that perform defect inspections. The storage unit 216 also stores information used in the embodiments described below.

[0119] The control unit 218 includes a keyboard and mouse, allowing the user to perform necessary operations on the defect inspection device 200 via these devices. By using a touch panel type device, the display device 230 also functions as a control unit. The display device 230 is, for example, a liquid crystal display and can display various types of information.

[0120] The CPU 220 includes an inspection target image acquisition unit 240 and a defect inspection unit 242. The inspection target image acquisition unit 240 acquires an inspection target image P30, which is an image of the object under inspection OBJ captured by the imaging system 300 or the like, via an input / output interface 212. The defect inspection unit 242 performs a defect inspection on the inspection target image P30. The inspection target image P30 is acquired by the imaging system 300, which will be described later. The defect inspection unit 242 determines the presence or absence (defective or non-defective) and type of defect using a learning model trained by the machine learning device 140 (see Figure 4). Under the control of the CPU 220, the results of the defect inspection by the defect inspection unit 242 are displayed on the display device 230. As previously described, the learning model is trained using training data TD generated by the training data generation device 100. The learning model is also updated according to the situation.

[0121] Next, an example of an imaging system 300 for capturing images of an object under examination (OBJ) will be described. Figure 16 is a schematic block diagram showing an example configuration of the imaging system 300. The imaging system 300 is for capturing images of an object under examination (OBJ) placed in an imaging room 314, and includes an imaging control unit 302, an imaging operation unit 304, an image storage unit 306, a camera 308, and radiation sources 310 and 312.

[0122] The shooting control unit 302 includes a CPU that controls the operation of each part of the shooting system 300. The shooting control unit 302 receives operation input from the operator (photographer) via the shooting operation unit 304 and transmits control signals corresponding to this operation input to each part of the shooting system 300 to control the operation of each part.

[0123] The imaging operation unit 304 includes an input device that receives operation input from the operator. The operator can input information about the object under inspection OBJ, input instructions for imaging conditions and instructions for performing imaging to the camera 308, input instructions for radiation irradiation conditions to the radiation sources 310 and 312, and input instructions to store the inspection target image P30 (not shown) obtained by imaging in the image storage unit 306. Imaging conditions include, for example, exposure time, focal length, imaging angle, and imaging location. Radiation irradiation conditions include irradiation start time, irradiation duration, irradiation angle, and irradiation intensity. Furthermore, conditions related to irradiation intensity include tube current and tube voltage.

[0124] The image storage unit 306 stores the inspection target image P30 (received light image data) of the object under inspection OBJ captured by the camera 308. The image storage unit 306 stores information for identifying the object under inspection OBJ, associated with the image data. The camera 308 is configured, for example, as a Digital Detector Array (DDA) system in which light receiving sensors are arranged on a flat surface and directly converted into digital values, or as a computed radiography (CR) system in which signals are stored on an imaging plate (IP) and digitized by an external reading device.

[0125] The camera 308, radiation sources 310 and 312 are located inside the imaging room 314. The radiation sources 310 and 312 are, for example, X-ray sources, and the partitions and entrances between the imaging room 314 and the outside are protected from X-rays by X-ray shielding materials (e.g., lead or concrete). However, when imaging is performed by irradiating the object under examination (OBJ) with visible light, it is not necessary to use the protected imaging room 314.

[0126] The radiation sources 310 and 312 irradiate the object under examination OBJ, which is placed in the imaging room 314, with radiation according to instructions from the imaging control unit 302.

[0127] Camera 308, in accordance with instructions from the imaging control unit 302 to perform imaging, receives radiation from the radiation source 310 that is irradiated onto the object under inspection OBJ and reflected by the object under inspection OBJ, or radiation from the radiation source 312 that is irradiated onto the object under inspection OBJ and passes through the object under inspection OBJ, and images the object under inspection OBJ. The object under inspection OBJ is held in the imaging chamber 314 by a holding member (not shown) (e.g., a manipulator, a mounting table, or a movable mounting table), and the distance and angle of the object under inspection OBJ relative to the camera 308, radiation sources 310 and 312 are adjustable. The operator can control the relative positions of the object under inspection OBJ, camera 308, radiation sources 310 and 312 via the imaging control unit 302, and can image a desired area of ​​the object under inspection OBJ.

[0128] The radiation sources 310 and 312 cease irradiating the object under inspection (OBJ) in synchronization with the completion of the imaging process by the camera 308.

[0129] In the example shown in Figure 16, one camera 308 and two radiation sources 310 and 312 are provided, but the number of cameras and radiation sources is not limited to these. For example, there may be multiple cameras and radiation sources, or there may be one of each. The shooting control unit 302, the shooting operation unit 304, and the image storage unit 306 can be implemented using a combination of computer hardware and software.

[0130] The defect inspection device 200 may be connected to the imaging system 300 in a communicative manner, or the defect inspection device 200 may be configured to function as the imaging control unit 302, imaging operation unit 304, and image storage unit 306 of the imaging system 300.

[0131] <Examples> The average number of samples (number of training data) was calculated before and after applying the training data generation method of the embodiment to images containing defects. The average number of samples was calculated by using 261 aluminum casting images as training images P10, and dividing the number of samples extracted from the 261 aluminum casting images by the total number of defects for each defect size. The aluminum casting images contained defects of different sizes: FMMD, FMLD, and Gas. The defect sizes were classified into three groups: "a group with defects that are on average large = FMMD," "a group with defects that are on average small = Gas," and "a group with defects that are on average between FMMD and Gas in size = FMLD."

[0132] Figure 17 conceptually shows the average sizes of Gas, FMLD, and FMMD. If the length of Gas is set to 1, then FMLD is approximately 2 to 6 times longer, and FMMD is approximately 30 to 50 times longer. For FMMD, FMLD, and Gas, the average number of samples extracted as training data was calculated before and after applying the training data generation method of the embodiment. Table 1 shows the results. No. 1 shows the average number of samples for FMMD, FMLD, and Gas before application, and No. 2 shows the average number of samples for FMMD, FMLD, and Gas after application.

[0133] [Table 1]

[0134] As shown in Table 1, No. 1, in FMMD before application, the number of samples per defect was calculated to be 174.6, due to the influence of a small number of large defects. In FMLD, the number of samples per defect was calculated to be 36.4, and in Gas, the number of samples per defect was calculated to be 21.8. Before application, with Gas set to 1, the ratios of FMMD:FMLD:Gas were 8:1.7:1, indicating a large imbalance in the average number of samples among FMMD, FMLD, and Gas. It can be understood that while Gas consists mostly of defects of 10px or less, FMLD and FMMD were affected by a small number of large defects.

[0135] On the other hand, as shown in No. 2 of Table 1, after application, FMMD yielded 3.1 samples per defect. FMLD yielded 2.3 samples per defect. Gas yielded 3.8 samples per defect. When Gas is set to 1, the ratios FMMD:FMLD:Gas = 0.8:0.6:1, confirming that FMLD and FMMD are less affected by a small number of large defects. After application, it can be seen that the average number of samples per defect is at a desirable ratio, both from defect groups with large average areas and defect groups with small average areas. This makes it possible to achieve equivalent learning accuracy.

[0136] <Other> In the above embodiment, the hardware structure of the processing unit that performs various processes is a variety of processors as shown below. These various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as a processing unit; a Programmable Logic Device (PLD), such as an FPGA (Field Programmable Gate Array), which is a processor whose circuit configuration can be changed after manufacturing; and a dedicated electrical circuit, such as an ASIC (Application Specific Integrated Circuit), which has a circuit configuration specifically designed to perform a particular process.

[0137] A single processing unit may be composed of one of these various processors, or it may be composed of two or more processors of the same or different type (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Furthermore, multiple processing units can be composed of a single processor. Examples of composing multiple processing units with a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as multiple processing units, as is typical of computers such as client and server systems. Secondly, a configuration using a processor that realizes the functions of the entire system, including multiple processing units, on a single IC (Integrated Circuit) chip, as is typical of System-on-a-Chip (SoC) systems. Thus, various processing units are configured, in terms of hardware structure, using one or more of the above-mentioned various processors.

[0138] Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit composed of circuit elements such as semiconductor devices.

[0139] Each of the above-described configurations and functions can be appropriately implemented using any hardware, software, or a combination thereof. For example, the present invention can also be applied to a program that causes a computer to execute the above-described processing steps (processing procedures), a computer-readable storage medium (non-temporary storage medium) that stores such a program, or a computer on which such a program can be installed.

[0140] Although examples of the present invention have been described above, it goes without saying that the present invention is not limited to the embodiments described above, and various modifications are possible without departing from the spirit of the invention. [Explanation of Symbols]

[0141] 10. Defect Inspection System 100 Training Data Generation Device 112 Input / Output Interfaces 116 Memory section 118 Operation section 120 CPU 126 Display Control Unit 130 Display device 140 Machine Learning Devices 142 Learning Models 142A Input Layer 142B Middle layer 142C Output Layer 144 Loss Value Calculation Unit 146 Parameter Control Unit 148 memory 151 Image acquisition unit 152 Region extraction part 153 Defective pixel detection unit 154 Training Data Extraction Unit 200 Defect Inspection Equipment 212 Input / Output Interfaces 216 Storage section 218 Operation section 220 CPU 226 Display Control Unit 230 Display device 240 Image acquisition unit for the subject of inspection 242 Defect Inspection Department 300 shooting systems 302 Imaging Control Unit 304 Shooting Control Unit 306 Image storage unit 308 Camera 310 Radiation Source 312 Radiation Source 314 Photography Room A Defective Variety B Defective variety AR area CD Correct Answer Data D, D1, D2, D3, D4 defects Dp Defective Pixels F frame GT Grand Truth Data IM patch image IR Focus Areas OBJ (Object under inspection) P10 Training Images P30 Images to be examined Px pixels S1 Step S2 Step S3 Step S4 Step TD, TD1, TD2 training data WL welding line

Claims

1. A training data generation device equipped with a processor, The aforementioned processor, For the image to be inspected, including the object to be inspected, a region of 2px or larger containing the defective pixels to be detected is extracted as a region of interest. A patch image of patch size containing defective pixels in the region of interest is extracted as training data. The number of training data points is equivalent across the types of defects. A training data generation device.

2. The aforementioned area of ​​interest is, A pixel region obtained according to a specific rule, As a result of determining whether each of the aforementioned pixel regions contains a defective pixel corresponding to the defect, the region determined to contain the defective pixel to be detected is: The training data generation device according to claim 1.

3. The aforementioned specific rule is, This involves scanning the image to be inspected in a random or exhaustive order to obtain a pixel range of a fixed size. The training data generation device according to claim 2.

4. The aforementioned specific rule is, The range of the detection target is predetermined to be limited to a part of the image to be inspected. Within the aforementioned range, scan in a random or exhaustive order. Obtain a pixel range of a fixed size. The training data generation device according to claim 2 or 3.

5. The aforementioned processor, A training data generation device according to any one of claims 1 to 3, which extracts a patch image of patch size containing pixels to be distinguished from the detection target from the image to be inspected, as training data.

6. The aforementioned area of ​​interest is, Identify a detection target that includes the aforementioned pixel, The method is determined by acquiring a certain number of pixel regions that include the pixels to be detected, A training data generation device according to any one of claims 1 to 3.

7. The training data generation device according to any one of claims 1 to 3, wherein the patch size is determined by the input of a machine learning device that creates a learning model using the training data.

8. The training data generation apparatus according to any one of claims 1 to 3, wherein the size of the area of ​​interest is determined based on the size of the defect to be detected or the distance between defects.

9. The training data generation apparatus according to any one of claims 1 to 3, wherein the processor determines whether or not each of the regions of interest contains the defective pixels based on the ground truth data based on the image to be inspected.

10. The training data generation apparatus according to any one of claims 1 to 3, wherein the distance between adjacent regions of interest is less than or equal to the patch size.

11. The training data generation apparatus according to any one of claims 1 to 3, wherein the processor determines whether each of the regions of interest contains the defective pixels, and makes a determination for each type of defect.

12. The training data generation apparatus according to claim 11, wherein the processor adjusts the number of training data for each type of defect.

13. The image to be examined is an X-ray transmission image. A training data generation device according to any one of claims 1 to 3.

14. The images to be inspected are X-ray transmission images of cast metal parts, forged metal parts, or welded metal parts. A training data generation device according to any one of claims 1 to 3.

15. A method for generating training data performed by a training data generation device equipped with a processor, The steps include: extracting a region of 2px or more containing the defective pixels to be detected from the image to be inspected, which includes the object to be inspected; The steps include: extracting a patch image of patch size containing defective pixels in the region of interest as training data; Includes, A method for generating training data, wherein the number of training data is equivalent across different types of defects.

16. A program that executes a training data generation method using a training data generation device equipped with a processor, The steps include: extracting a region of 2px or more containing the defective pixels to be detected from the image to be inspected, which includes the object to be inspected; The steps include: extracting a patch image of patch size containing defective pixels in the region of interest as training data; Make it run, A program in which the number of training data points is equivalent across different types of defects.

17. A non-temporary and computer-readable recording medium on which the program described in claim 16 is recorded.

18. A training data generation device according to any one of claims 1 to 3, An image acquisition unit that acquires the image to be inspected, A defect inspection unit that performs learning using the training data created by the training data generation device and detects defects in the image to be inspected, A defect inspection device equipped with the following features.

Images