Learning device, inspection device, learning method, and inspection method

By classifying and reclassifying substrate images to generate a learning dataset and using a weighting mechanism to select weight coefficients, the problems of large dataset size and low accuracy in printed wiring substrate inspection are solved, and high-precision substrate inspection is achieved.

CN112614086BActive Publication Date: 2024-11-01SCREEN HOLDINGS CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010947201.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-09-19
Filing Date
2020-09-10
Publication Date
2024-11-01
Estimated Expiration
2040-11-01

AI Technical Summary

Technical Problem

In the inspection of printed wiring boards, the existing deep learning technology requires a large dataset for learning, which results in a long learning time and insufficient inspection accuracy.

Method used

By classifying and reclassifying substrate images, a training dataset is generated. A weighting mechanism is used to select weight coefficients, reducing the number of assumed defect images and generating a fully learned model to achieve high-precision substrate inspection.

Benefits of technology

This reduces the size of the training dataset, improves the accuracy of substrate inspection, lowers the false alarm rate, and achieves efficient defect detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112614086B_ABST
    Figure CN112614086B_ABST
Patent Text Reader

Abstract

The present invention provides a learning device, an inspection device, a learning method and an inspection method, which can reduce the capacity of a learning data set and realize high-precision substrate inspection. In the learning device, a second input unit inputs second data that reclassifies a hypothetical defect image in the first data according to the presence or absence of a true defect. The learning unit generates a learning data set by weighting the hypothetical defect image in the second data. The learning unit generates a learning completion model by learning the relationship between the substrate image and the presence or absence of defects through machine learning using the learning data set. In the generation of the learning data set using the learning unit, when the hypothetical defect image is reclassified as having a true defect, the hypothetical defect image is classified as "defective" after a weight coefficient selected from a plurality of weight coefficients corresponding to the importance of the true defect is multiplied by the hypothetical defect image. Thereby, the capacity of the learning data set is reduced and high-precision substrate inspection is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a learning device, an inspection device, a learning method and an inspection method. Background Art

[0002] Conventionally, in the inspection of printed wiring boards, images of various regions of the printed wiring board are inspected by an inspection device, and images judged to have defects (hereinafter referred to as "defective images") are selected. Furthermore, an operator performs a defect confirmation operation (so-called verification operation) on the selected defective images. The defects selected by the inspection device include true defects with a high possibility of becoming quality problems and false reports with substantially no possibility of becoming quality problems. In the defect confirmation operation, the operator visually selects true defects from the defective images selected by the inspection device.

[0003] On the other hand, Japanese Patent Gazette No. 6512585 (Document 1) proposes that in a visual inspection device for parts such as metal bellows, a learning binary file is created by deep learning using a large number of good and bad image data prepared in advance, and the learning binary file is used to perform image processing on the inspected image to perform a good and bad judgment.

[0004] However, when deep learning technology such as that described in Document 1 is applied to the selection of defective images in a printed wiring board inspection device, if one wants to reduce the proportion of false positives contained in the defective images to improve the inspection accuracy, there is a concern that the number of images required for learning processing will become huge (i.e., the capacity of the learning data set will become huge), and learning will require a lot of time. Summary of the invention

[0005] The present invention is directed to a learning device for producing a learned model used when inspecting a substrate having a pattern on its surface, and its purpose is to reduce the capacity of a learning data set and realize high-precision substrate inspection.

[0006] A learning device according to a preferred embodiment of the present invention includes: a first input unit for inputting first data in which a substrate image is classified into a hypothetical defect image having a defect and a hypothetical good product image having no defect; a second input unit for inputting second data in which the hypothetical defect image in the first data is reclassified according to the presence or absence of a true defect; and a learning unit for generating a learning data set by weighting the hypothetical defect image in the second data, and learning the relationship between the substrate image and the presence or absence of a defect by machine learning using the learning data set to create a learning completion model. In the generation of the learning data set using the learning unit, when the hypothetical defect image is reclassified as having a true defect, a weight coefficient selected from a plurality of weight coefficients corresponding to the importance of the true defect is multiplied by the hypothetical defect image to classify the hypothetical defect image as having a defect, and when the hypothetical defect image is reclassified as not having a true defect, the hypothetical defect image is classified as having no defect.

[0007] According to the present invention, the capacity of the learning data set can be reduced and high-precision substrate inspection can be achieved.

[0008] Preferably, the plurality of weight coefficients increase as the importance of the region where the true defect is located on the substrate increases.

[0009] Preferably, the plurality of weight coefficients become larger as the importance of the type of inspection in which a true defect is detected among the plurality of types of inspection becomes higher.

[0010] The present invention is also directed to an inspection device for inspecting a substrate having a pattern on its surface. The inspection device in a preferred embodiment of the present invention includes an inspection unit that inspects an inspection image obtained by photographing the substrate using the learned model created by the learning device.

[0011] An inspection device according to another preferred embodiment of the present invention comprises: the learning device; and an inspection unit for inspecting an inspection image obtained by photographing a substrate using a learning model produced by the learning device.

[0012] Preferably, the second input unit of the learning device inputs new second data reclassified according to the presence or absence of a true defect by setting the defect image obtained by the inspection unit as the assumed defect image. The learning unit generates a new learning data set by weighting the assumed defect image in the new second data, and causes the learning completed model to relearn the relationship between the substrate image and the presence or absence of a defect by machine learning using the new learning data set.

[0013] Preferably, the device further includes an imaging unit for imaging the substrate to acquire the image to be inspected.

[0014] The present invention is also directed to a learning method for preparing a learning model used when inspecting a substrate having a pattern on its surface. A preferred learning method of an embodiment of the present invention includes: a) a step of inputting first data in which a substrate image is classified into a hypothetical defect image having a defect and a hypothetical good product image having no defect; b) a step of inputting second data in which the hypothetical defect image in the first data is reclassified according to the presence or absence of a true defect; and c) a step of generating a learning data set by weighting the hypothetical defect image in the second data, and learning the relationship between the substrate image and the presence or absence of a defect by machine learning using the learning data set to prepare a learning model. In the generation of the learning data set in the step c), when the hypothetical defect image is reclassified as having a true defect, a weight coefficient selected from a plurality of weight coefficients corresponding to the importance of the true defect is multiplied by the hypothetical defect image, and the hypothetical defect image is classified as having a defect; when the hypothetical defect image is reclassified as not having a true defect, the hypothetical defect image is classified as having no defect.

[0015] Preferably, the plurality of weight coefficients increase as the importance of the region where the true defect is located on the substrate increases.

[0016] Preferably, the plurality of weight coefficients become larger as the importance of the type of inspection in which a true defect is detected among the plurality of types of inspection becomes higher.

[0017] The present invention is also directed to an inspection method for inspecting a substrate having a pattern on its surface. A preferred embodiment of the inspection method of the present invention includes the step of inspecting an inspection image obtained by photographing the substrate using the learning model produced by the learning method.

[0018] The inspection method of another preferred embodiment of the present invention includes: d) the step of making a learning completion model by the learning method; and e) the step of inspecting the inspected image obtained by photographing the substrate using the learning completion model made in step d).

[0019] Preferably, it also includes: f) inputting a step of setting the defect image obtained in the step e) as the assumed defect image, and reclassifying new second data according to the presence or absence of true defects; and g) weighting the assumed defect image in the new second data to generate a new learning data set, and using machine learning of the new learning data set to enable the learning completion model to relearn the relationship between the substrate image and the presence or absence of defects.

[0020] Preferably, the method further comprises the step of photographing the substrate to obtain the inspected image.

[0021] The above-mentioned object and other objects, features, aspects and advantages will become clear from the following detailed description of the present invention with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a diagram showing the structure of the inspection device.

[0023] Figure 2 It is a diagram showing the structure of a first computer.

[0024] Figure 3 It is a diagram showing a functional structure implemented by the first computer.

[0025] Figure 4 It is a diagram showing the structure of the second computer.

[0026] Figure 5 It is a diagram showing a functional structure realized by the second computer.

[0027] Figure 6 This is a diagram showing the flow of creating a learning completion model.

[0028] Figure 7 This is a diagram showing the status of creation of a learning completion model.

[0029] Figure 8 It is a diagram showing various regions on a substrate.

[0030] Fig. 9 It is a diagram showing various regions on a substrate.

[0031] Fig.10 This is a diagram showing the flow of substrate inspection.

[0032] Fig.11 This is a diagram showing the process of relearning a learned model.

[0033] [Explanation of Symbols]

[0034] 1: Check the device

[0035] 4: Second Computer

[0036] 9: Substrate

[0037] 21: Photography Department

[0038] 301: Storage

[0039] 401: First input unit

[0040] 402: Second input unit

[0041] 403: Learning Department

[0042] S11~S15, S21~S24, S31~S33: Steps DETAILED DESCRIPTION

[0043] Figure 1 1 is a diagram showing the structure of an inspection device 1 according to an embodiment of the present invention. The inspection device 1 is, for example, a device for inspecting the appearance of a printed wiring substrate 9 (hereinafter, also simply referred to as "substrate 9") before electronic components are mounted. A pattern (for example, a wiring pattern or an electrode pattern formed of copper) is formed on the surface of the substrate 9. On the substrate 9, for example, there are areas where the copper plating is exposed, areas where the copper wiring is covered with a solder resist as a protective film, and areas where the solder resist is directly disposed on the surface of the base material.

[0044] The inspection device 1 includes: a device body 2 for photographing a substrate 9, a first computer 3, and a second computer 4. The first computer 3 and the second computer 4 are processing devices including a computing unit. The first computer 3 also controls the overall operation of the inspection device 1. The second computer 4 is a learning device for making a learning completion model described later. The device body 2 has: a photographing unit 21, a platform 22 for holding the substrate 9, and a platform moving mechanism 23. The photographing unit 21 photographs the substrate 9 to obtain an image. The image is, for example, a multi-grayscale color image. The platform moving mechanism 23 moves the platform 22 relative to the photographing unit 21.

[0045] The imaging unit 21 includes: an illumination unit 211 that emits illumination light, an optical system 212, and an imaging element 213. The optical system 212 guides the illumination light toward the substrate 9, and guides the light from the substrate 9 toward the imaging element 213. The imaging element 213 converts the image of the substrate 9 formed by the optical system 212 into an electrical signal. The illumination unit 211 includes a lamp such as a light emitting diode (LED) or a light bulb, and optical components such as a lens or a reflective member that adjusts the light from the lamp. The optical system 212 includes optical components such as a plurality of lenses or a half mirror. The imaging element 213 is, for example, a two-dimensional image sensor. The imaging element 213 may also be a one-dimensional image sensor, in which case imaging is performed while the platform 22 is moved.

[0046] The platform moving mechanism 23 includes a ball screw, a guide rail, a motor, etc. Of course, various mechanisms can be used as the platform moving mechanism 23, for example, a linear motor can be used. The first computer 3 controls the platform moving mechanism 23 and the shooting unit 21, whereby the platform 22 moves in the horizontal direction, and the desired area on the substrate 9 is photographed. The platform 22 is a holding unit that holds the substrate 9. The substrate 9 can be held by various methods. For example, a groove is formed on the platform 22, and suction is performed from a suction port formed in the groove, thereby adsorbing the substrate 9 on the platform 22. It is also possible to form many suction ports on the platform 22 to adsorb the substrate 9. It is also possible to form the platform 22 using a porous material and to perform suction from the porous material. The platform 22 can also use a mechanical mechanism to hold the substrate 9.

[0047] Figure 2 3 is a diagram showing the structure of the first computer 3. The first computer 3 has the structure of a general computer system, which includes: a central processing unit (CPU) 31, a read only memory (ROM) 32, a random access memory (RAM) 33, a fixed disk 34, a display 35, an input unit 36, a reading device 37, a communication unit 38, a graphics processing unit (GPU) 39, and a bus 30. The CPU 31 performs various calculations. The GPU 39 performs various calculations related to image processing. The ROM 32 stores basic programs. The RAM 33 stores various information. The fixed disk 34 stores information. The display 35 displays various information such as images. The input unit 36 ​​includes a keyboard 36a and a mouse 36b for receiving input from an operator. The reading device 37 reads information from a computer-readable recording medium 81 such as an optical disk, a magnetic disk, a magneto-optical disk, or a memory card. The display 35, keyboard 36a, mouse 36b, and reading device 37 are connected to the bus 30 via the interface I / F. The communication unit 38 sends and receives signals between other components of the inspection device 1 and external devices. The bus 30 is a signal circuit that connects the CPU 31, GPU 39, ROM 32, RAM 33, fixed disk 34, display 35, input unit 36, reading device 37, and communication unit 38.

[0048] In the first computer 3, the program 811 is read from the recording medium 81 via the reading device 37 in advance and stored in the fixed disk 34. The program 811 may also be stored in the fixed disk 34 via a network. The CPU 31 and the GPU 39 perform calculation processing while using the RAM 33 or the fixed disk 34 according to the program 811. The CPU 31 and the GPU 39 function as a calculation unit in the first computer 3. In addition to the CPU 31 and the GPU 39, other structures that function as a calculation unit may also be used.

[0049] Figure 3 8 is a diagram showing a functional structure realized by the first computer 3 executing arithmetic processing according to the program 811. These functional structures include a storage unit 301, an inspection unit 302, and a control unit 303. All or part of these functions may be realized by a dedicated circuit. In addition, these functions may be realized by multiple computers. Figure 3 Among the functional structures shown in FIG. 3 , the inspection unit 302 and the control unit 303 are realized by the CPU 31 , the GPU 39 , the ROM 32 , the RAM 33 , the fixed disk 34 , and their peripheral structures. In addition, the storage unit 301 is mainly realized by the RAM 33 and the fixed disk 34 .

[0050] The storage unit 301 stores the image of the substrate 9 acquired by the imaging unit 21, the learning model described later, etc. The inspection unit 302 is a classifier that inspects the image of the substrate 9 using the learning model stored in the storage unit 301 and classifies the image according to the presence or absence of defects. The control unit 303 controls the imaging unit 21, the stage moving mechanism 23, and the operation of each functional structure.

[0051] Figure 44 is a diagram showing the structure of the second computer 4. The second computer 4 has a general computer system structure similar to the first computer 3, and the general computer system includes: CPU 41, ROM 42, RAM 43, fixed disk 44, display 45, input unit 46, reading device 47, communication unit 48, GPU 49, and bus 40. CPU 41 performs various calculations. GPU 49 performs various calculations related to image processing. ROM 42 stores basic programs. RAM 43 stores various information. Fixed disk 44 stores information. Display 45 displays various information such as images. Input unit 46 includes keyboard 46a and mouse 46b for receiving input from the operator. Reading device 47 reads information from a computer-readable recording medium 82 such as an optical disk, a magnetic disk, a magneto-optical disk, and a memory card. Display 45, keyboard 46a, mouse 46b, and reading device 47 are connected to bus 40 via interface I / F. Communication unit 48 sends and receives signals between other structures of inspection device 1 and external devices. The bus 40 is a signal circuit that connects the CPU 41 , the GPU 49 , the ROM 42 , the RAM 43 , the fixed disk 44 , the display 45 , the input unit 46 , the reading device 47 , and the communication unit 48 .

[0052] In the second computer 4, the program 821 is read from the recording medium 82 via the reading device 47 in advance and stored in the fixed disk 44. The program 821 may also be stored in the fixed disk 44 via a network. The CPU 41 and the GPU 49 perform calculation processing while using the RAM 43 or the fixed disk 44 according to the program 821. The CPU 41 and the GPU 49 function as a calculation unit in the second computer 4. In addition to the CPU 41 and the GPU 49, other structures that function as a calculation unit may also be used.

[0053] Figure 5 821. This is a diagram showing a functional structure that is realized by the second computer 4 executing arithmetic processing according to the program 821. These functional structures include a first input unit 401, a second input unit 402, a learning unit 403, and a storage unit 404. All or part of these functions may be realized by a dedicated circuit. In addition, these functions may be realized by multiple computers. Figure 5 Among the functional structures shown in , the first input unit 401, the second input unit 402 and the learning unit 403 are implemented by the CPU 41, the GPU 49, the ROM 42, the RAM 43, the fixed disk 44 and their peripheral structures. In addition, the storage unit 404 is mainly implemented by the RAM 43 and the fixed disk 44.

[0054] The first input unit 401 inputs first data as a set of classified images of the substrate 9 (hereinafter referred to as "substrate images") to the storage unit 404. The second input unit 402 inputs second data as a set of reclassified substrate images to the storage unit 404. The storage unit 404 stores data input from the first input unit 401 and the second input unit 402, etc. The classification and reclassification of substrate images will be described in detail later.

[0055] The learning unit 403 generates a learning data set based on the second data, and produces a learning model for inspection using the inspection unit 302 by machine learning using the learning data set. In the learning unit 403, for the initial model for inspection, machine learning is performed on the relationship between the substrate image and the presence or absence of defects, thereby producing a learning model. The details of the learning data set will be described later. The machine learning in the learning unit 403 is performed, for example, by deep learning using a neural network. The learning using the deep learning can be performed, for example, using a residual network (Residual Network, ResNet). In addition, the machine learning can also be performed by methods other than deep learning.

[0056] Figure 6 It is a diagram showing the flow of creating a learning-completed model in the inspection device 1 . Figure 7 1 is a diagram showing the status of the preparation of the learning completion model. In the preparation of the learning completion model, first, a plurality of images of various regions selected from the images obtained by photographing the substrate 9 (i.e., substrate images) are prepared (step S11). In the following description, the plurality of substrate images are referred to as a "substrate image group". The substrate image group is preferably obtained by the photographing unit 21 of the inspection device 1. In addition, the substrate image group preferably includes images obtained by photographing a plurality of substrates 9, respectively.

[0057] In the substrate 9, a protective film called solder resist is formed on the copper wiring or the base material, and various regions exist on the substrate 9. For example, there are regions where the copper plating is exposed on the substrate 9, regions where the solder resist exists on the copper wiring, and regions where the solder resist exists on the base material.

[0058] like Figure 8As shown, in the region where the copper plating is exposed, there are small-area plating regions 93a where the area of ​​each exposed plating portion is small, large-area plating regions 93c where the area of ​​each exposed plating portion is large, and medium-area plating regions 93b where the area of ​​each exposed plating portion is moderate. In the small-area plating region 93a, the adverse effects on conduction etc. produced by the defects are large compared with the medium-area plating region 93b and the large-area plating region 93c. In addition, in the medium-area plating region 93b, the adverse effects on conduction etc. produced by the defects are large compared with the large-area plating region 93c. In other words, the importance of the region where the defect is located on the substrate 9 becomes high in the order of large-area plating region 93c, medium-area plating region 93b, and small-area plating region 93a.

[0059] like Fig. 9 As shown, the area covered by the solder resist includes a fine-line solder resist (SR) area 94a where the wiring under the solder resist is a fine-line pattern, a standard wiring SR area 94b where the wiring under the solder resist is a standard pattern, and a substrate SR area 94c where there is no wiring under the solder resist but only a substrate. In the fine-line SR area 94a, the adverse effect on conduction, etc. caused by the defect is greater than that in the standard wiring SR area 94b and the substrate SR area 94c. In addition, in the standard wiring SR area 94b, the adverse effect on conduction, etc. caused by the defect is greater than that in the substrate SR area 94c. In other words, the importance of the area where the defect is located on the substrate 9 becomes higher in the order of the substrate SR area 94c, the standard wiring SR area 94b, and the fine-line SR area 94a.

[0060] like Figure 7 As shown, the prepared substrate image group (i.e., a plurality of substrate images) is classified into assumed defective images having defects and assumed good images having no defects. In the second computer 4, the classified substrate image group is input as first data into the storage unit 404 using the first input unit 401 (step S12). The classification (i.e., inspection) of the substrate image group in step S12 can be performed by various known methods. For example, each substrate image is compared with a reference substrate image having no defects using a known appearance inspection device, and substrate images with obvious differences are classified as assumed defective images, and substrate images other than the images are classified as assumed good images.

[0061] In the classification of the substrate image group in step S12, various defect inspections are performed. For example, in the inspection of the plated portion, a plating foreign matter inspection for detecting foreign matter on the plated portion or a plating multi-value inspection for detecting the depth of plating in the plated portion is performed. Compared with the defects detected in the plating multi-value inspection (for example, discoloration, dirt, flaws, etc. of the plated portion), the defects detected in the plating foreign matter inspection (for example, foreign matter on the plated portion or lack of the plated portion) have a greater adverse effect on conduction, etc. In other words, the importance of the types of inspections becomes higher in the order of plating multi-value inspection and plating foreign matter inspection.

[0062] In addition, for example, in the solder resist inspection, an SR peeling inspection for detecting peeling of the solder resist or an SR spot inspection for detecting the depth of the solder resist is performed. Compared with defects detected in the SR spot inspection (for example, bright spots or dark spots of the solder resist), defects detected in the SR peeling inspection (for example, exposure of wiring caused by peeling of the solder resist) have a greater adverse effect on conduction, etc. In other words, the importance of the types of inspections increases in the order of SR spot inspection and SR peeling inspection.

[0063] In the first data, the type of area of ​​each assumed defect image on the substrate 9 and the type of inspection in which the defect was detected are combined with each assumed defect image. The "inspection area type" and "inspection type" associated with each assumed defect image in the first data are expressed, for example, by text data.

[0064] Then, all the assumed defect images (hereinafter also referred to as "the assumed defect image group") in the first data stored in the storage unit 404 in step S12 are selected and displayed on the display 45. Furthermore, through the operator's visual observation, as shown in FIG. Figure 7 As shown, an operation (so-called verification) of reclassifying the group of assumed defect images according to the presence or absence of true defects is performed. The so-called true defect refers to a defect that has a substantial adverse effect on the performance of the substrate 9, and therefore it is judged that the substrate 9 cannot be marketed as a product. In the verification, for example, when the operator visually observes the assumed defect image and determines that there is a true defect, right-click on the assumed defect image and select "True Defect" from the drop-down menu, thereby saving the assumed defect image in the true defect folder. In addition, when the operator visually observes the assumed defect image and determines that there is no true defect, right-click on the assumed defect image and select "False Report" from the drop-down menu, thereby saving the assumed defect image in the false report folder.

[0065] In the second computer 4, the group of assumed defect images reclassified by the verification is input as second data into the storage unit 404 by the second input unit 402 (step S13). In the second data, each assumed defect image is associated with the inspection area type and inspection type, and the verification result (i.e., information indicating the presence or absence of a true defect).

[0066] Then, a learning data set is generated based on the second data through the learning unit 403 (step S14). In step S14, weighting of the assumed defect image is performed in the second data. Specifically, when the assumed defect image is reclassified as having a true defect in step S13, in step S14, a weight coefficient is selected from a plurality of weight coefficients prepared in advance corresponding to the importance of the true defect, and multiplied with the assumed defect image. Moreover, the assumed defect image multiplied by the weight coefficient is classified as "defective". On the other hand, when the assumed defect image is reclassified as not having a true defect in step S13, in step S14, the assumed defect image is not multiplied by the weight coefficient, but is classified as "non-defective".

[0067] The plurality of weight coefficients are set in advance corresponding to the inspection area type and inspection type associated with the assumed defect image, and are stored in the storage unit 404. In the present embodiment, the weight coefficient increases as the importance of the inspection area type (i.e., the area where the true defect is located on the substrate 9) increases. In addition, the weight coefficient increases as the importance of the inspection type (i.e., the type of inspection in which the true defect is detected) increases. The plurality of weight coefficients are set, for example, between 0 and 1.

[0068] Specifically, for example, the weight coefficient when the inspection area type is a small-area plated area 93a and the inspection type is a plated foreign body inspection is greater than the weight coefficient when the inspection area type is a medium-area plated area 93b and the inspection type is a plated foreign body inspection. In addition, the weight coefficient when the inspection area type is a small-area plated area 93a and the inspection type is a plated foreign body inspection is greater than the weight coefficient when the inspection area type is a small-area plated area 93a and the inspection type is a plating multi-value inspection. The weight coefficient when the inspection area type is a small-area plated area 93a and the inspection type is a plated foreign body inspection is greater than the weight coefficient when the inspection area type is a large-area plated area 93c and the inspection type is a plating multi-value inspection.

[0069] For example, the weight coefficient when the inspection area type is the fine line SR area 94a and the inspection type is the SR peeling inspection is greater than the weight coefficient when the inspection area type is the standard wiring SR area 94b and the inspection type is the SR peeling inspection. In addition, the weight coefficient when the inspection area type is the fine line SR area 94a and the inspection type is the SR peeling inspection is greater than the weight coefficient when the inspection area type is the fine line SR area 94a and the inspection type is the SR spot inspection. The weight coefficient when the inspection area type is the fine line SR area 94a and the inspection type is the SR peeling inspection is greater than the weight coefficient when the inspection area type is the base material SR area 94c and the inspection type is the SR spot inspection.

[0070] In addition, the weight coefficient may be set in advance corresponding to only one of the inspection area type and the inspection type. In addition, the weight coefficient may be set in advance based on other information associated with the assumed defect image (i.e., information other than the inspection area type and the inspection type). The weight coefficient does not necessarily need to be set between 0 and 1 and may be changed as appropriate.

[0071] When the learning data set is generated, the learning unit 403 uses the learning data set to perform machine learning on the relationship between the substrate image and the presence or absence of defects for the initial model for inspection, thereby creating a learned model used when inspecting the substrate 9 (step S15).

[0072] Fig.10 2 is a diagram showing the process of inspecting the substrate 9 using the learning completion model. Figure 1 In the inspection device 1 shown in FIG. 1 , first, as described above, the second computer 4 as the learning device creates a learning model (step S21). The learning model is sent from the second computer 4 to the first computer 3 and stored in the storage unit 301 of the first computer 3 (see FIG. 1 ). Figure 3 )middle.

[0073] Next, the control unit 303 controls the imaging unit 21 and the stage moving mechanism 23, etc., so that the imaging unit 21 images the substrate 9 and obtains an image of the substrate 9 to be inspected (hereinafter also referred to as "inspected image") (step S22).

[0074] Then, the inspection unit 302 uses the learning model stored in the storage unit 301 to inspect the image to be inspected (step S23). In step S23, for example, the plating foreign matter inspection, plating multi-value inspection, SR peeling inspection, and SR spot inspection are performed. Moreover, among the substrate images of each area selected from the inspected image, the substrate image that is judged to be similar to the substrate image classified as "defective" in the learning model (that is, the substrate image with a true defect) is classified as a defective image. In addition, the substrate image that is not classified as a defective image is classified as a good image.

[0075] Thereafter, the substrate image classified as a defective image by the inspection unit 302 is verified by the operator in substantially the same manner as in step S13 to determine whether there is a true defect in the substrate 9 (i.e., whether the substrate 9 can be put on the market) (step S24). In step S24, the substrate image classified as a defective image by the inspection unit 302 is further classified into a substrate image having a true defect and a substrate image having no true defect (i.e., a false report).

[0076] In the inspection device 1 , the learned model may be re-learned using the results of the inspection of the substrate 9 by the inspection device 1 (steps S21 to S24 ). Fig.11 1 is a diagram showing the flow of relearning the learned model in the inspection device 1. In relearning the learned model, first, the second input unit 402 (see Figure 5 ), the defect image reclassified in step S24 (i.e., the defect image obtained in step S23 is set as a hypothetical defect image, and the defect image is reclassified according to the presence or absence of a true defect) is input into the storage unit 404 as new second data (step S31).

[0077] Next, a new learning data set is generated based on the new second data by the learning unit 403 (step S32). In step S32, weighting of the assumed defect image is performed in the new second data in substantially the same manner as in step S14. Specifically, when the assumed defect image is reclassified as having a true defect in step S31, in step S32, a weight coefficient is selected from the plurality of weight coefficients prepared in advance corresponding to the importance of the true defect, and is multiplied with the assumed defect image. Moreover, the assumed defect image multiplied by the weight coefficient is classified as "defective". On the other hand, when the assumed defect image is reclassified as not having a true defect in step S31, in step S32, the assumed defect image is not multiplied by the weight coefficient, but is classified as "non-defective".

[0078] When the new learning data set is generated, the learning unit 403 re-learns the relationship between the substrate image and the presence or absence of defects for the learned model through machine learning using the new learning data set (step S33 ).

[0079] As described above, the second computer 4 is a learning device for producing a learning model used when inspecting a substrate having a pattern on its surface. The learning device includes: a first input unit 401, a second input unit 402, and a learning unit 403. The first input unit 401 inputs first data in which a substrate image is classified into a hypothetical defect image with defects and a hypothetical good image without defects. The second input unit 402 inputs second data in which the hypothetical defect images in the first data are reclassified according to the presence or absence of true defects. The learning unit 403 generates a learning data set by weighting the hypothetical defect images in the second data. The learning unit 403 generates a learning model by learning the relationship between the substrate image and the presence or absence of defects through machine learning using the learning data set.

[0080] In the generation of the learning data set by the learning unit 403, when the assumed defect image is reclassified as having a true defect, the assumed defect image is classified as "having a defect" after a weight coefficient selected according to the importance of the true defect among a plurality of weight coefficients is multiplied by the assumed defect image. In addition, when the assumed defect image is reclassified as not having a true defect, the assumed defect image is classified as "not having a defect".

[0081] By using the learning model created using the learning data set to inspect the substrate 9, an image similar to a substrate image with a high importance level is more likely to be detected as a defect image than an image similar to a substrate image with a low importance level true defect. As a result, the possibility of an image similar to a substrate image with a low importance level but not actually having a true defect (i.e., a substrate image determined to be a false alarm in verification) being detected as a defect image in the inspection of the substrate 9 can be reduced. In other words, the false alarm rate in the defect inspection of the substrate 9 can be reduced, and high-precision inspection can be achieved.

[0082] In addition, as described above, in the generation of the learning data set by the learning unit 403, the assumed defect image having a true defect is multiplied by the weight coefficient corresponding to the importance of the true defect, thereby reducing the number of assumed defect images (i.e., the capacity of the assumed defect image group), and generating a learning data set for the preparation of the learning-completed model that can realize the high-precision inspection. That is, in the learning device, the capacity of the learning data set can be reduced, and the high-precision inspection of the substrate 9 can be realized.

[0083] As described above, it is preferred that the plurality of weight coefficients become larger as the importance of the region where the true defect is located on the substrate 9 becomes higher. Thus, defects in a region with high importance (i.e., a high sensitivity region) on the substrate 9 can be preferentially detected compared to defects in a region with low importance. As a result, the false alarm rate in defect inspection of the substrate 9 can be further reduced, and a higher-precision inspection can be achieved.

[0084] As described above, it is preferred that the plurality of weight coefficients become larger as the importance of the type of inspection for detecting true defects among the plurality of inspections becomes higher. Thus, defects in inspections of high importance (i.e., inspections with high lethality) can be preferentially detected compared to defects in inspections of low importance. As a result, the false alarm rate in defect inspection of the substrate 9 can be further reduced, and a higher-precision inspection can be achieved.

[0085] As described above, the inspection device 1 is a device for inspecting a substrate 9 having a pattern on its surface. The inspection device 1 includes the learning device and the inspection unit 302. The inspection unit 302 inspects the inspected image obtained by photographing the substrate 9 using the learning completed model produced by the learning device. Thus, as described above, high-precision inspection of the substrate 9 can be achieved.

[0086] As described above, it is preferred that the second input unit 402 inputs new second data that is reclassified according to the presence or absence of true defects by setting the defect image obtained by the inspection unit 302 as a hypothetical defect image. In addition, the learning unit 403 weights the hypothetical defect image in the new second data to generate a new learning data set. Moreover, the learning unit 403 relearns the relationship between the substrate image and the presence or absence of defects for the learning completed model through machine learning using the new learning data set. In this way, the capacity of the learning data set used for relearning can be reduced, and the inspection accuracy of the substrate 9 can be improved.

[0087] In addition, in the inspection device 1, if the inspection unit 302 inspects the inspection image using the learned model, it is not necessary to provide a learning device. In this case, the inspection unit 302 inspects the inspection image obtained by photographing the substrate 9 using the learned model created by the learning device provided independently of the inspection device 1. In this case, the inspection of the substrate 9 can also be achieved with high accuracy as described above.

[0088] As described above, the inspection device 1 preferably further includes an imaging unit 21 that captures the inspection image by imaging the substrate 9. Thus, the acquisition of the inspection image and the inspection can be performed continuously by one device. As a result, the inspection efficiency of the substrate 9 can be improved.

[0089] As described above, the learning method for producing a completed learning model includes: a step of inputting first data in which a substrate image is classified into a hypothetical defect image with defects and a hypothetical good image without defects (step S12); a step of inputting second data in which the hypothetical defect images in the first data are reclassified according to the presence or absence of true defects (step S13); and a step of weighting the hypothetical defect images in the second data to generate a learning data set, and producing a completed learning model by learning the relationship between the substrate image and the presence or absence of defects through machine learning using the learning data set (steps S14 to S15).

[0090] In the generation of the learning data set in steps S14 to S15, when the assumed defect image is reclassified as having a true defect, the assumed defect image is multiplied by a weight coefficient selected according to the importance of the true defect among a plurality of weight coefficients, and the assumed defect image is classified as having a defect. When the assumed defect image is reclassified as not having a true defect, the assumed defect image is classified as not having a defect. Thus, the capacity of the learning data set can be reduced in the same manner as described above, and high-precision inspection of the substrate 9 can be achieved.

[0091] Various changes can be made to the learning device, the inspection device 1, the learning method, and the inspection method.

[0092] For example, the substrate image group prepared in step S11 may be acquired outside the inspection device 1 and input to the inspection device 1 via a recording disk or a network. In step S22, the inspected image may be acquired outside the inspection device 1 and input to the inspection device 1. In this case, the imaging unit 21 may be omitted from the inspection device 1.

[0093] The inspection of the substrate 9 in the step S12 and the step S23 is not limited to the inspection of foreign matter in the plating or the inspection of SR peeling, and various inspections can be performed on the substrate 9. In addition, the inspection area on the substrate 9 is not limited to the small area plating area or the thin line SR area, and various areas can be set on the substrate 9.

[0094] In the generation of the learning data set in step S14, the weight coefficient to be multiplied by the assumed defect image does not necessarily need to be set in advance, and can be set based on experience by the operator visually observing the assumed defect image during the verification in step S13. The same is true in the re-learning of the learned model using the inspection results obtained by the inspection unit 302 (steps S31 to S33).

[0095] In the relearning of the learned model (steps S31 to S33), in the generation of the new learning data set in step S32, weighting of the assumed defect image (i.e., multiplication of the weight coefficient corresponding to the importance of the defect) may not be performed. Even in this case, the inspection accuracy of the substrate 9 can be improved by relearning the learned model. In addition, it is not necessarily necessary to relearn the learned model (steps S31 to S33).

[0096] The first computer 3 and the second computer 4 may be accommodated in the housing of the apparatus main body 2. On the contrary, the second computer 4 may be used as a single learning device independent of other components of the inspection apparatus 1.

[0097] The configurations in the above-described embodiment and various modifications may be appropriately combined as long as they do not contradict each other.

[0098] The invention has been described and explained in detail, but the above description is illustrative rather than restrictive. Therefore, it can be said that many modifications and aspects can be implemented without departing from the scope of the present invention.

Claims

1. A learning device for preparing a learning model used when inspecting a substrate having a pattern on its surface, characterized in that: include: A first input unit for inputting first data in which a substrate image is classified into a presumed defective image having a defect and a presumed good image having no defect; A second input unit inputs second data obtained by reclassifying the assumed defect image in the first data according to the presence or absence of a true defect; as well as A learning unit that generates a learning data set by weighting the assumed defect image in the second data, and learns the relationship between the substrate image and the presence or absence of defects by machine learning using the learning data set to create a learning completed model; In the generation of the learning data set using the learning unit, When the assumed defect image is reclassified as having a true defect, a weight coefficient selected from among a plurality of weight coefficients corresponding to the importance of the true defect is multiplied by the assumed defect image to classify the assumed defect image as having a defect, When the assumed defect image is reclassified as not having a true defect, the assumed defect image is classified as non-defective.

2. The learning device according to claim 1, characterized in that The plurality of weight coefficients become larger as the importance of the region where the true defect is located on the substrate becomes higher.

3. The learning device according to claim 1, characterized in that: The plurality of weight coefficients become larger as the importance of the type of inspection in which a true defect is detected among the plurality of types of inspections becomes higher.

4. An inspection device for inspecting a substrate having a pattern on its surface, characterized in that: include: A learning device as claimed in any one of claims 1 to 3; as well as The inspection section inspects the inspection image obtained by photographing the substrate using the learned model created by the learning device.

5. The inspection device according to claim 4, characterized in that: The second input unit of the learning device inputs new second data which is reclassified according to the presence or absence of a true defect by setting the defect image obtained by the inspection unit as the assumed defect image. The learning unit generates a new learning data set by weighting the assumed defect image in the new second data, and causes the learned model to re-learn the relationship between the substrate image and the presence or absence of defects through machine learning using the new learning data set.

6. The inspection device according to claim 4, characterized in that: The system also includes a photographing unit for photographing the substrate to obtain the inspected image.

7. A learning method for preparing a learning model used when inspecting a substrate having a pattern on its surface, characterized in that: include: a) a step of inputting first data in which substrate images are classified into assumed defective images having defects and assumed good images having no defects; b) inputting second data obtained by reclassifying the assumed defect images in the first data according to the presence or absence of true defects; as well as c) generating a learning data set by weighting the assumed defect image in the second data, and learning the relationship between the substrate image and the presence or absence of defects by machine learning using the learning data set to create a learning completed model; In the generation of the learning data set in step c), When the assumed defect image is reclassified as having a true defect, a weight coefficient selected from among a plurality of weight coefficients corresponding to the importance of the true defect is multiplied by the assumed defect image to classify the assumed defect image as having a defect, When the assumed defect image is reclassified as not having a true defect, the assumed defect image is classified as non-defective.

8. The learning method according to claim 7, characterized in that: The plurality of weight coefficients become larger as the importance of the region where the true defect is located on the substrate becomes higher.

9. The learning method according to claim 7, characterized in that: The plurality of weight coefficients become larger as the importance of the type of inspection in which a true defect is detected among the plurality of types of inspections becomes higher.

10. An inspection method for inspecting a substrate having a pattern on its surface, characterized in that include: d) the step of preparing a learning completion model by the learning method according to any one of claims 7 to 9; as well as e) A step of inspecting the inspection image obtained by photographing the substrate using the learned model created in step d).

11. The inspection method according to claim 10, characterized in that: Also includes: f) inputting new second data which is reclassified according to the presence or absence of a true defect by setting the defect image obtained in step e) as a hypothetical defect image; as well as g) generating a new learning data set by weighting the assumed defect image in the new second data, and causing the learned model to re-learn the relationship between the substrate image and the presence or absence of defects through machine learning using the new learning data set.

12. The inspection method according to claim 10, characterized in that: Also includes: The step of photographing the substrate to obtain the inspected image.