Determination device, learning device, and teacher data creation device

Abstract

To provide a determination device, a learning device, and a teacher data creation device that perform class classification with high accuracy and derive an influence degree of each feature amount.SOLUTION: A determination device includes: a two-dimensional information acquisition unit that acquires two-dimensional information about an object to be determined and incidental information associated with the two-dimensional information; an extraction unit that extracts a first feature amount from the two-dimensional information acquired by the two-dimensional information acquisition unit; a second feature amount acquisition unit that acquires a second feature amount related to the object and different from the first feature amount; a third feature amount acquisition unit that acquires a third feature amount different from the first feature amount and the second feature amount; an operation unit that performs a calculation related to class classification of the object using as input information the first feature amount extracted by the extraction unit and at least one of the second feature amount acquired by the second feature amount acquisition unit and the third feature amount acquired by the third feature amount acquisition unit; and an output unit that outputs a result of the calculation performed by the operation unit.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a determination device, a learning device, and a teacher data creation device. [Background technology]

[0002] Conventionally, methods have been proposed for analyzing yield factors of products manufactured through multiple manufacturing processes (see, for example, Patent Document 1). Such techniques are used, for example, in semiconductor manufacturing processes. At the semiconductor manufacturing process site, each manufacturing lot is analyzed to determine whether it is a normal lot group with a high yield or an abnormal lot group with a low yield. Furthermore, based on the analysis results, the degree of influence on yield reduction for each process or each piece of equipment is extracted by focusing on the differences in processing equipment in each process. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2005-012095 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the conventional technology described above, machine learning such as deep learning is used to classify failure patterns in a wafer map (patterns of defective chips appearing on a wafer). However, the accuracy of classifying failure patterns based on a wafer map is not sufficient, and there is a problem that the accuracy drops significantly, especially when multiple failure patterns are superimposed. In other words, the analysis method according to the conventional technology has a problem that the accuracy of classifying is not sufficient.

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and aims to provide a determination device, a learning device, and a teacher data creation device that are capable of performing class classification with high accuracy and deriving the influence of each feature. [Means for solving the problem]

[0006] (1) One aspect of the present invention is a determination device including a two-dimensional information acquisition unit that acquires two-dimensional information about an object to be determined and additional information associated with the two-dimensional information; an extraction unit that extracts a first feature from the two-dimensional information acquired by the two-dimensional information acquisition unit; a second feature acquisition unit that acquires a second feature related to the object, the second feature being different from the first feature; a third feature acquisition unit that acquires a third feature different from the first feature and the second feature; a calculation unit that performs a calculation related to a class classification of the object using as input information the first feature extracted by the extraction unit and at least one of the second feature acquired by the second feature acquisition unit or the third feature acquired by the third feature acquisition unit; and an output unit that outputs a result of the calculation performed by the calculation unit.

[0007] (2) One aspect of the present invention is that, in the above-mentioned determination device, a predetermined process is performed on the object, the first feature is information about the object obtained after the process is performed, the second feature is information while the process is being performed or before the process is performed, and the third feature is information after the process is performed, while the process is being performed, or before the process is performed.

[0008] (3) In one aspect of the present invention, in the above-described determination device, a predetermined process is performed on the object, and the second feature amount is a value that is controlled in the process.

[0009] (4) In one aspect of the present invention, in the above-mentioned determination device, a predetermined process is performed on the object, and the second feature amount is a value that influences the process.

[0010] (5) One aspect of the present invention is that in the above-mentioned judgment device, the two-dimensional information acquisition unit acquires the two-dimensional information about the object that has been determined to be a judgment target as a result of a prior determination of whether or not the object is a judgment target.

[0011] (6) In one aspect of the present invention, in the above-mentioned determination device, the two-dimensional information is an image of the object.

[0012] (7) In one aspect of the present invention, in the above-mentioned determination device, the extraction unit extracts the first feature using at least one of a convolutional neural network, a Transformer, a decision tree, and ensemble learning, using the two-dimensional information, which is an image, as input information.

[0013] (8) In one aspect of the present invention, in the above-mentioned determination device, the calculation unit calculates a likelihood for each class of the object for each combination of the second feature amount and the third feature amount.

[0014] (9) In one aspect of the present invention, in the above-mentioned determination device, the object is a printed label, and the class of the object is a classification of types of defects that occurred when printing the label.

[0015] (10) In one aspect of the present invention, in the above-mentioned judgment device, the two-dimensional information is information that represents areas of the object that have defects and areas that do not have defects using two or more values ​​on a two-dimensional coordinate system.

[0016] (11) In one aspect of the present invention, in the above-mentioned determination device, the calculation unit determines the likelihood of each class of the object.

[0017] (12) In one aspect of the present invention, in the above-mentioned determination device, the calculation unit has an output layer that receives the extracted first feature and at least one of the acquired second feature or third feature as input information and outputs a likelihood for each class of the object or a class corresponding to the maximum likelihood.

[0018] (13) In one aspect of the present invention, in the above-described determination device, the object is a wafer for semiconductor manufacturing, and the class of the object is a classification of defects for each wafer.

[0019] (14) One aspect of the present invention is a learning device including: a two-dimensional information acquisition unit that acquires two-dimensional information about an object to be determined and associated information associated with the two-dimensional information; a class information acquisition unit that acquires information about a class into which the object is classified based on a predetermined determination criterion; a second feature acquisition unit that acquires a second feature related to the object, the second feature being different from a first feature extracted from the two-dimensional information; a third feature acquisition unit that acquires a third feature different from the first feature and the second feature; and a learning unit that trains a machine learning model to estimate the class based on the first feature extracted based on the two-dimensional information acquired by the two-dimensional information acquisition unit and at least one of the second feature acquired by the second feature acquisition unit or the third feature acquired by the third feature acquisition unit.

[0020] (15) One aspect of the present invention is a learning device as described above, further comprising an extension unit that generates multiple extended two-dimensional information based on the acquired two-dimensional information, and the learning unit trains a machine learning model to estimate the class based on the first feature extracted based on the generated extended two-dimensional information and the second feature acquired by the second feature acquisition unit.

[0021] (16) One aspect of the present invention is a teacher data creation device including a two-dimensional information acquisition unit that acquires multiple pieces of two-dimensional information about an object to be determined and associated information associated with the two-dimensional information; a selection unit that selects two or more of the multiple pieces of two-dimensional information based on a predetermined selection algorithm; a superposition unit that creates teacher data to be used for training or testing a machine learning model by superimposing the two or more selected pieces of two-dimensional information; and at least one of a second feature acquisition unit that acquires a second feature related to the object, the second feature being different from a first feature extracted from the two-dimensional information; a third feature acquisition unit that acquires a third feature different from the first feature and the second feature; and a second feature combination unit that combines the second feature corresponding to the superimposed samples or a third feature combination unit that combines the third feature corresponding to the superimposed samples, and an integration unit that integrates at least one of the combined second feature or the acquired third feature into the teacher data.

[0022] (17) In one aspect of the present invention, the above-mentioned teacher data creation device further includes a defect assignment unit that assigns information indicating that each coordinate or feature of the teacher data has a defect based on a predetermined probability distribution or random number. Effect of the Invention

[0023] According to the present invention, it is possible to perform class classification with high accuracy and to derive the influence of each feature amount. [Brief description of the drawings]

[0024] [Figure 1] FIG. 1 is a diagram for explaining an overview of a system according to a first embodiment. [Diagram 2] 2 is a functional configuration diagram showing an example of a functional configuration of a determination device according to the first embodiment. FIG. [Diagram 3] FIG. 4 is a diagram for explaining pre-processing performed by the system according to the first embodiment. [Figure 4]FIG. 11 is a diagram for explaining an overview of a determination method according to a second embodiment. [Diagram 5] FIG. 11 is a diagram showing a specific example of defect image data according to the second embodiment. [Figure 6] FIG. 11 is a diagram showing a specific example of a log feature amount according to the second embodiment. [Figure 7] FIG. 11 is a diagram showing a specific example of an environmental variable feature amount according to the second embodiment. [Figure 8] FIG. 11 is a diagram showing a specific example of a calculation result according to the second embodiment. [Figure 9] FIG. 11 is a diagram for explaining an overview of a determination method according to a third embodiment. [Figure 10] FIG. 11 is a diagram showing an example of two-dimensional information according to the third embodiment. [Figure 11] FIG. 13 is a diagram for explaining extraction of feature amounts according to the third embodiment. [Figure 12] FIG. 13 is a diagram for explaining an example of a Radon feature according to the third embodiment. [Figure 13] FIG. 11 is a functional configuration diagram showing an example of the functional configuration of a learning device according to a fourth embodiment. [Figure 14] FIG. 13 is a diagram for explaining main types of data extension according to the fourth embodiment. [Figure 15] FIG. 13 is a functional configuration diagram showing an example of the functional configuration of a teacher data creation device according to a fifth embodiment. [Figure 16] 23 is a first example of two-dimensional information obtained by performing overlay according to the sixth embodiment. [Figure 17] 23 is a second example of two-dimensional information obtained by performing overlay according to the sixth embodiment. [Figure 18] 13 is a diagram for explaining details of a calculation performed by a superimposing unit according to the sixth embodiment. FIG. [Figure 19] FIG. 23 is a diagram showing raw information of a third feature amount according to the sixth embodiment. [Figure 20] FIG. 23 is a diagram showing an example of a classification result of overlay information according to the sixth embodiment. [Figure 21] FIG. 23 is a diagram showing an example of an overlap classification problem according to the sixth embodiment. [Figure 22] FIG. 23 is a diagram showing an example of a classification result of overlay information according to the sixth embodiment. [Figure 23] FIG. 23 is a diagram showing an example of an overlap classification problem according to the sixth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] [Embodiment] Hereinafter, the system 1 according to the embodiment of the present invention will be described in detail with reference to the accompanying drawings, showing preferred embodiments. Note that the aspects of the present invention are not limited to these embodiments, and include various modifications or improvements. In other words, the components described below include those that a person skilled in the art can easily imagine and those that are substantially the same, and the components described below can be appropriately combined. In addition, various omissions, substitutions, or modifications of the components can be made without departing from the gist of the present invention. In addition, in the following drawings, the scale and number of each structure may be different from the scale and number of the actual structure in order to make each structure easier to understand.

[0026] First, the matters that are the premise of the embodiment described below will be described. The system 1 according to this embodiment is applied to the result of a predetermined process. The predetermined process that is the target of the system 1 according to this embodiment broadly includes processing such as the manufacture of products and the inspection of manufactured products. The result of a predetermined process may be a final result or a result obtained as an intermediate process. The predetermined process is based on a temporal concept, and involves a change after a predetermined period of time (which may be any short period or any long period). The change includes no change before and after the predetermined process.

[0027] The system 1 according to this embodiment performs judgment on an object obtained as a result of, for example, a predetermined process. The judgment broadly includes highly accurate judgment on quality and the like, and identification of the cause of defect. The cause of defect may include the absence of a defect (i.e., a non-defective product). The cause of defect may include a superposition problem in which multiple causes of defect are superimposed. The system 1 according to this embodiment realizes a series of these analysis procedures by a set of (the same) analysis configuration. Furthermore, the system 1 according to this embodiment is capable of handling realistic data scale and complexity.

[0028] [Embodiment 1] First, the first embodiment will be described with reference to FIGS.

[0029] FIG. 1 is a diagram for explaining an overview of a system according to a first embodiment. First, an overview of the system 1 according to this embodiment will be explained with reference to the diagram. The system 1 includes a processing device 10, a two-dimensional information acquisition device 20, a feature acquisition device 30, and a determination device 50. The functions of each device included in the system 1 may be provided in any manner. Furthermore, each device included in the system 1 may be an independent device, or the functions of each device may be provided by a single device or multiple devices.

[0030] The system 1 performs a predetermined process. The process is, for example, a predetermined process performed by the processing device 10. As a result of the process performed by the processing device 10, an object S is obtained. Here, the object S is not limited to being obtained as a result of the predetermined process performed by the processing device 10. For example, the processing device 10 may perform a predetermined process on an object S that exists in advance. That is, it can be said that the object S is an object S that exists after the predetermined process is performed by the processing device 10. More specifically, the process performed by the processing device 10 may be a process for manufacturing the object S or a process for inspecting the object S. In the following description, a case where the object S is a tangible object will be described. However, this embodiment is not limited to this example, and the object S may be an intangible object.

[0031] The two-dimensional information acquisition device 20 acquires information about the object S after a predetermined process is performed by the processing device 10. The two-dimensional information acquisition device 20 acquires, for example, two-dimensional information about the object S. Hereinafter, the information about the object S acquired by the two-dimensional information acquisition device 20 will be referred to as two-dimensional information. Note that the two-dimensional information according to this embodiment is not limited to information being expressed two-dimensionally (for example, being planar information). The two-dimensional information according to this embodiment only needs to be expressed at least two-dimensionally, and may broadly include, for example, three-dimensional or higher-dimensional information including a time element or a spatial element. The two-dimensional information acquisition device 20 may directly acquire the external characteristics of the object S as two-dimensional information, for example, by performing imaging or the like. The two-dimensional information acquisition device 20 may also acquire the results of a predetermined measurement performed on the object S as two-dimensional information. The measurement may be a measurement of the external characteristics, or may be a measurement related to the function of the object S. The results of the measurement may be quantified, for example, for each two-dimensional coordinate indicating the external appearance of the object S, to generate two-dimensional information. More specifically, the two-dimensional information may be pixel information corresponding to xy plane coordinates, or information indicating the presence or absence and the degree of a defect.

[0032] The feature acquisition device 30 acquires a feature. The feature acquired by the feature acquisition device 30 may be any feature. Preferably, the feature acquired by the feature acquisition device 30 may be related to the processing device 10. The feature related to the processing device 10 is preferably an feature that affects the object S obtained as a result of processing performed by the processing device 10. However, the present embodiment is not limited to this example, and a feature not related to the processing device 10 may be acquired. In addition, the feature acquisition device 30 may acquire multiple types of feature. In this case, at least one of the multiple types of feature may preferably be related to the processing device 10. In addition, the multiple types of feature may widely include other information not related to the object S.

[0033] The determination device 50 performs a determination based on the two-dimensional information obtained by the two-dimensional information acquisition device 20 and the feature amount obtained by the feature amount acquisition device 30. The determination performed by the determination device 50 may be, for example, related to class classification of the object S. Class classification may be classifying the object S into any of the predetermined classes, or may be outputting the likelihood for each class. Note that the present embodiment is not limited to this example, and the determination device 50 can perform a determination based on various calculations. In other words, the determination performed by the determination device 50 may be outputting a calculation result obtained as a result of performing various calculations.

[0034] FIG. 2 is a functional configuration diagram showing an example of the functional configuration of the determination device according to the first embodiment. An example of the functional configuration of the determination device 50 will be described with reference to FIG. 2. The determination device 50 includes a two-dimensional information acquisition unit 51, a second feature amount acquisition unit 52, an extraction unit 53, a calculation unit 54, an output unit 55, a third feature amount acquisition unit 56, and a class information acquisition unit 57. Each of these functional units is realized, for example, by using an electronic circuit. Each functional unit may include a storage unit such as a semiconductor memory or a magnetic hard disk device, as necessary. Each function may be realized by a computer having a CPU (Central Processing Unit) and software. All or a part of each functional unit may be realized by hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field-Programmable Gate Array). All or a part of each functional unit may be realized by a combination of software and hardware. In addition, all or part of the functions of the determination device 50 may be realized using hardware or the like provided in an external device such as a cloud server. In addition, the determination device 50 may be configured with multiple devices, and each functional block may be distributed among the multiple devices.

[0035] The two-dimensional information acquisition unit 51 acquires two-dimensional information from the two-dimensional information acquisition device 20. The two-dimensional information acquisition unit 51 acquires two-dimensional information about an object S that is the object to be determined by the determination device 50. The two-dimensional information acquisition unit 51 outputs the acquired two-dimensional information to the extraction unit 53 as two-dimensional information DI. In addition to acquiring the two-dimensional information DI, the two-dimensional information acquisition unit 51 acquires additional information associated with the two-dimensional information DI. The additional information may be information indicating attributes of the two-dimensional information DI, etc. The additional information may be, for example, the number of pixels of the two-dimensional information DI, etc.

[0036] The extraction unit 53 acquires the two-dimensional information DI from the two-dimensional information acquisition unit 51. The extraction unit 53 extracts a first feature based on the acquired information. That is, the first feature is a feature based on the two-dimensional information of the object S. The first feature may be any feature extracted from the two-dimensional information of the object S. For example, the first feature may be a feature that can be recognized by humans, or may be a feature that cannot be recognized by humans. The extraction unit 53 may extract the first feature by performing inference using a machine learning algorithm, for example, or may extract the first feature by performing a calculation using a mathematical formula or the like. The extraction unit 53 outputs the extracted feature to the calculation unit 54 as the first feature C1.

[0037] The second feature acquisition unit 52 acquires the feature from the feature acquisition device 30. The feature acquired by the second feature acquisition unit 52 is also referred to as the second feature. The second feature may include various feature. That is, the second feature acquisition unit 52 may acquire a plurality of types of second feature C2. As an example, the second feature is preferably a value controlled in a process performed by the processing device 10. In addition, the second feature is preferably a value that has an effect on a process performed by the processing device 10 (or on the object S in the process). The second feature acquisition unit 52 outputs the acquired feature to the calculation unit 54 as the second feature C2. Here, the second feature C2 is a feature related to the object S, but is different from the first feature C1. The second feature may be extracted by a second feature extraction unit (not shown).

[0038] In addition, the extraction of the second feature may be performed using a device (method) based on screening such as Sure Independence Screening (SIS), which is a representative example of interaction modeling, or a factorization machine (SFM) with feature selection. Among SFMs, an interaction modeling device (method) may be used that includes a single feature extraction unit (step) that estimates the individual effect of a feature using L1 norm regularization, a combination feature extraction unit (step) that estimates a combination feature, which is a combination of multiple features, using L2 norm regularization, and a total feature extraction unit (step) that estimates all features using a regularization term that introduces an upper bound. In addition, the effectiveness of the application of the Cauchy-Schwarz (CS) inequality and the Triangle-Inequality (TI) upper bound as an upper bound has been confirmed in previous research (K. Atarashi et al., 2021). Of these, SFM using the TI upper bound (called TI-SFM(L1)) is a new form proposed in this invention and is one of the preferred forms. However, previous research has also used the L2 norm to estimate the effect of a single feature, so a device (method) including L2 norm regularization may be used in the single feature extraction section (step). In addition, the CS upper bound or other upper bounds may be used in the overall feature extraction section (step).

[0039] Here, the differences between the first feature C1 and the second feature C2 will be described. First, the first feature C1 and the second feature C2 are obtained by different routes. That is, the first feature C1 is a feature extracted from the two-dimensional information DI (that is, it can be said that the feature is indirectly obtained via the two-dimensional information DI), whereas the second feature C2 is a feature obtained directly from some device or environment, and thus the two are different. The first feature C1 and the second feature C2 may be independent features or may be related features.

[0040] In addition, the first feature C1 and the second feature C2 are different from each other in terms of the time of acquisition of the feature. The first feature C1 is information extracted from the two-dimensional information DI obtained by observing the object S after a predetermined process performed by the processing device 10 has been performed. Observation of the object S broadly includes imaging, measurement, inspection, and the like. That is, the first feature C1 is acquired after the process has been performed. On the other hand, the second feature C2 is not limited to being acquired after the process has been performed. For example, the second feature C2 may be acquired while the process is being performed, or may be acquired before the process is performed. Note that the second feature C2 does not prevent being acquired after the process, like the first feature C1.

[0041] The determination device 50 may further include a third feature acquisition unit 56 to acquire the third feature. In this case, the third feature acquisition unit 56 acquires the third feature from the feature acquisition device 30 or another storage device. The third feature is a feature different from both the first feature and the second feature. The third feature may be information after a predetermined process performed by the processing device 10 is performed, during the process, or before the process is performed. The third feature may be, for example, auxiliary information of the two-dimensional information, or may be a feature generated from the two-dimensional information DI by a different method from the first feature, or may be a feature generated from the auxiliary information of the two-dimensional information DI or the feature generated from the two-dimensional information DI. The third feature acquisition unit 56 outputs the acquired feature to the calculation unit 54 as the third feature C3.

[0042] The class information acquisition unit 57 acquires information about the class of the object S. The class information acquisition unit 57 outputs the information about the class of the object S to the calculation unit 54 as class information CL.

[0043] The calculation unit 54 acquires the first feature C1 extracted by the extraction unit 53, and acquires the second feature C2 acquired by the second feature acquisition unit 52. The calculation unit 54 may also acquire the third feature C3 from the third feature acquisition unit 56, and acquire class information CL from the class information acquisition unit 57. The calculation unit 54 performs a calculation related to the class classification of the object S using the acquired information as input information. The calculation performed by the calculation unit 54 may be performed by inference using a machine learning algorithm, or may be performed by calculation using a mathematical formula or the like. The calculation unit 54 outputs information obtained as a result of the calculation to the output unit 55 as result information RI.

[0044] The output unit 55 outputs the result of the calculation performed by the calculation unit 54. The output unit 55 may visually present the determination result to the user by outputting the result of the calculation to a display device such as a liquid crystal display or an organic EL (Electroluminescence) display. The output unit 55 may also present the determination result to the user by other methods. The output unit 55 may accumulate information by outputting the result of the calculation to a storage device (not shown).

[0045] 3 is a diagram for explaining the pre-processing performed by the system according to the first embodiment. With reference to the figure, an example of the pre-processing performed by the determination device 50 will be described. In the pre-processing, a preliminary determination is made on the object S processed by the processing device 10 as to whether or not it is a target for determination by the determination device 50. Note that the pre-processing performed with reference to the figure does not necessarily have to be performed in this embodiment. For example, the determination device 50 may perform a determination on all objects S processed by the processing device 10 without performing a preliminary determination by the pre-processing.

[0046] As shown in the figure, the preliminary judgment by the preprocessing is performed by the preliminary judgment device 60. When the preprocessing is performed, the system 1 further includes the preliminary judgment device 60 to perform the preprocessing. The preliminary judgment device 60 observes the object S processed by the processing device 10. As a result of the observation, the preliminary judgment device 60 performs a preliminary judgment as to whether or not the object S is a judgment target. The preliminary judgment as to whether or not the object S is a judgment target may be performed, for example, based on a judgment as to whether or not the object S is a good product. More specifically, as a result of the preliminary judgment performed by the preliminary judgment device 60, an object S judged to be a defective product may be regarded as a judgment target by the judgment device 50, and an object S judged to be a good product may not be regarded as a judgment target by the judgment device 50. That is, according to such a configuration, the system 1 may be said to perform a primary judgment as to whether or not the object is a defective product by the preliminary judgment device 60, and to perform a class classification of the cause of the defect by the judgment device 50.

[0047] [Summary of embodiment 1] According to the embodiment described above, the determination device 50 includes the two-dimensional information acquisition unit 51 to acquire two-dimensional information DI about the object S to be determined, the extraction unit 53 to extract a first feature C1 from the two-dimensional information DI acquired by the two-dimensional information acquisition unit 51, the second feature acquisition unit 52 to acquire a second feature C2, which is a feature about the object D and is different from the first feature C1, the calculation unit 54 to perform a calculation related to the class classification of the object S using the first feature C1 extracted by the extraction unit 53 and the second feature C2 acquired by the second feature acquisition unit 52 as input information, and the output unit 55 to output the result of the calculation performed by the calculation unit 54. That is, the determination device 50 performs a calculation related to the class classification of the object S based on the second feature C2 about the object S in addition to the first feature C1 extracted from the two-dimensional information DI about the object S. Therefore, according to this embodiment, a more accurate judgment can be made by performing multimodal processing based on various feature amounts, instead of making a judgment based only on two-dimensional information as in the conventional technology. Therefore, according to this embodiment, it is possible to perform class classification with high accuracy, and the calculation unit 54 and the output unit 55 can also identify the influence of the second feature amount C2.

[0048] Moreover, according to the above-described embodiment, when the system 1 performs preliminary judgment, the system 1 further includes a preliminary judgment device 60. The two-dimensional information acquisition unit 51 acquires two-dimensional information about the object S judged to be a judgment target as a result of a previous judgment as to whether or not the object S is a judgment target. That is, according to the above-described embodiment, a calculation related to class classification of the object S is performed as a secondary judgment only for the object for which a primary judgment as to whether or not it is a judgment target has been performed in advance. Therefore, according to this embodiment, it is possible to perform efficient and highly accurate class classification.

[0049] [Embodiment 2] Next, a second embodiment will be described with reference to Fig. 4 to Fig. 8. The second embodiment is a specific example in which the system 1 according to the first embodiment is applied to an industrial printing process.

[0050] FIG. 4 is a diagram for explaining an outline of the determination method according to the second embodiment. An outline of the determination model used in this embodiment will be explained with reference to the same figure. In this embodiment, the object S to be determined by the determination device 50 is a printed label. The printing process for the object S is performed by, for example, a processing device 10. In this embodiment, the processing device 10 may specifically be an industrial printing line for PET bottle film or the like, which performs four-color printing at a predetermined speed (for example, several hundred meters per minute). Also, in this embodiment, the class of the object to be determined by the determination device 50 may be a classification of types of defects during label printing.

[0051] As shown in the figure, two-dimensional information DI is input to the extraction unit 53. In the example shown in the figure, the two-dimensional information DI is image information of a defect image obtained by capturing an image of an object S that has been determined to be defective as a result of a preliminary determination performed in advance by a preliminary determination device 60. Specifically, the image information is a printed matter that has been subjected to a printing process by a processing device 10. The system 1 can obtain the two-dimensional information DI by capturing an image of the printed matter with an imaging device (not shown).

[0052] FIG. 5 is a diagram showing a specific example of defect image data according to the second embodiment. With reference to the same figure, a specific example of defect image data to be judged by the judgment device 50 will be described. FIG. 5(A) is a diagram showing an example of a case where a printed label as the object S has a defect of "fog". FIG. 5(B) is a diagram showing an example of a case where a printed label as the object S has a defect of "two-two stain". FIG. 5(C) is a diagram showing an example of a case where a printed label as the object S has a defect of "fish eye". FIG. 5(D) is a diagram showing an example of a case where a printed label as the object S has a defect of "doctor line". FIG. 5(E) is a diagram showing an example of a case where a printed label as the object S has a defect of "ink splash". FIG. 5(F) is a diagram showing an example of a case where a printed label as the object S has a defect of "swim". Note that the defect image data to be judged by the judgment device 50 is not limited to this example. The judgment device 50 may judge other defects not shown. In addition, the judgment by the judgment device 50 may include that there is no defect. Furthermore, the determination device 50 may make a determination regarding the case where a plurality of these defects are present at the same time.

[0053] Returning to FIG. 4, the two-dimensional information DI, which is a defect image, is input to the extraction unit 53. The extraction unit 53 may include a convolutional neural network that has been trained in advance using a large amount of image data set. In the illustrated example, the extraction unit 53 includes VGG-16. The extraction unit 53 extracts features by performing, for example, a convolutional operation based on the two-dimensional information DI. In the illustrated example, the extraction unit 53 extracts 512 features as a result of repeating the convolutional operation multiple times. The features extracted by the extraction unit 53 are illustrated as first features C1. That is, the extraction unit 53 extracts the first features C1 using a convolutional neural network with the two-dimensional information DI, which is an image, as input information. Note that the extraction by the extraction unit 53 is not limited to an example in which a convolutional neural network is used, and known methods such as Transformer, decision tree, or ensemble learning may be used. The ensemble learning may be, for example, boosting, bagging, stacking, or the like.

[0054] In addition, in the judgment according to the present embodiment, a second feature C2 and a third feature C3 different from the first feature C1 are used. In the illustrated example, the second feature C2 includes an environmental variable feature. Furthermore, the third feature C3 includes a log feature. Note that the present embodiment is not limited to an example in which both the environmental variable feature of the second feature C2 and the log feature of the third feature C3 are included, and it is sufficient that either the log feature or the environmental variable feature is included. Furthermore, the second feature C2 and the third feature C3 may include features other than the environmental variable feature and the log feature.

[0055] FIG. 6 is a diagram showing a specific example of a log feature according to the second embodiment. A specific example of a log feature, which is one of the third feature C3, will be described with reference to the same figure. The log feature is a feature based on an inspection log obtained as a result of a predetermined inspection. The log feature includes, for example, a position on a product where a candidate estimated to be a defect exists, a rough size of the candidate, and a rough classification level of abnormality of the candidate. In the illustrated example, the position on a product where a candidate estimated to be a defect exists is shown in XY relative coordinates. The X coordinate specifies the position in the horizontal direction (direction perpendicular to the conveying direction) of a PET bottle film or the like printed by an industrial printing line, and may be information specified in [mm (millimeter)]. The Y coordinate specifies the position in the conveying direction, and may be information specified in [m (meter)]. Note that the dimensions of printed products on an industrial printing line are, for example, 140 [cm (centimeter)] in width and 4000 [m] in length, with one roll as the reference unit. The XY coordinates may be converted into standardized data (values ​​that are not dependent on roll size) based on the measured position data. The size of the candidate is shown in [mmsq. (millimeters per square meter)]. The degree of abnormality is shown as "heavy", "medium", or "light", etc.

[0056] In the illustrated example, five third feature amounts C3 are shown. Each third feature amount C3 corresponds to a different two-dimensional information DI. Each third feature amount C3 is assigned a serial number (No.) of 1 to 5. The number of defects present in the second feature amount C2 having the serial number "1" is "3", the X coordinate of the position on the product where a candidate estimated to be a defect exists is "585.3 [mm]", the Y coordinate is "0 [m]", the size is "0.9 [mmsq.]", and the degree of abnormality is "minor defect (minor)". Furthermore, the number of defects present in the second feature C2 having the serial number "2" is "5", the X coordinate of the position on the product where the candidate estimated to be a defect exists is "314.6 [mm]", the Y coordinate is "0.3 [m]", the size is "3.5 [mmsq.]", and the degree of abnormality is "serious defect (serious / medium / light / streaks)". Furthermore, the number of defects present in the second feature C2 having the serial number "3" is "5", the X coordinate of the position on the product where the candidate estimated to be a defect exists is "314.6 [mm]", the Y coordinate is "0.8 [m]", the size is "4.1 [mmsq.]", and the degree of abnormality is "serious defect (serious / medium / light / streaks)". Furthermore, the number of defects present in the second feature C2 having the serial number "4" is "3", the X coordinate of the position on the product where the candidate estimated to be a defect exists is "333.2 [mm]", the Y coordinate is "3.2 [m]", the size is "1.8 [mmsq.]", and the degree of abnormality is "serious defect (serious / medium / light)". Furthermore, the number of defects present in the second feature C2 having the serial number "5" is "5", the X coordinate of the position on the product where the candidate estimated to be a defect exists is "267.9 [mm]", the Y coordinate is "3.3 [m]", the size is "2.1 [mmsq.]", and the degree of abnormality is "serious defect (serious / medium / light)".

[0057] 7 is a diagram showing a specific example of an environmental variable feature amount according to the second embodiment. With reference to the figure, a specific example of an environmental variable feature amount, which is one of the second feature amounts C2, will be described. The environmental variable feature amount is a parameter or a measurement result used in processing the object S. Specifically, the environmental variable feature amount is a measurement result of the printing speed, the presence or absence of wear of the blade that scrapes off the ink, and the material state (e.g., the viscosity of the printing ink, etc.) measured on the line, and is part of the causal system of the printing defect.

[0058] In the illustrated example, six second feature values ​​C2 are shown. The third feature values ​​C3 are assigned serial numbers IMG01 to IMG06. The serial numbers correspond to different two-dimensional information DI. In addition, in the figure, "roll" is a number that identifies a roll to be printed on in an industrial printing line. In addition, in the figure, "blade wear", "low speed", "low ink viscosity", "high ink viscosity", and "highest ink viscosity" are each indicated by a dummy variable of 0 or 1. "Blade wear" indicates whether or not the blade is worn, with "1" indicating wear and "0" indicating no wear. In addition, "low speed" indicates whether or not the printing speed is lower than normal, with "1" indicating lower than normal and "0" indicating normal. The ink viscosity is judged in three stages compared to normal. If it differs from normal, either "low ink viscosity", "high ink viscosity", or "highest ink viscosity" will be "1", and the others will be "0".

[0059] The second feature quantity C2 corresponding to "IMG01" corresponds to roll "R4", where blade wear is "0", low speed is "0", low ink viscosity is "1", high ink viscosity is "0", and maximum ink viscosity is "0". The second feature quantity C2 corresponding to "IMG02" corresponds to roll "R6", where blade wear is "0", low speed is "0", low ink viscosity is "0", high ink viscosity is "0", and maximum ink viscosity is "1". The second feature quantity C2 corresponding to "IMG03" corresponds to roll "R6", where blade wear is "0", low speed is "0", low ink viscosity is "0", high ink viscosity is "0", and maximum ink viscosity is "1". The second feature quantity C2 corresponding to "IMG04" corresponds to the roll "R2", where the blade wear is "1", the low speed is "1", the low ink viscosity is "1", the high ink viscosity is "0", and the maximum ink viscosity is "0". The second feature quantity C2 corresponding to "IMG05" corresponds to the roll "R6", where the blade wear is "0", the low speed is "0", the low ink viscosity is "0", the high ink viscosity is "0", and the maximum ink viscosity is "1". The second feature quantity C2 corresponding to "IMG06" corresponds to the roll "R6", where the blade wear is "0", the low speed is "0", the low ink viscosity is "0", the high ink viscosity is "0", and the maximum ink viscosity is "1".

[0060] FIG. 8 is a diagram showing a specific example of the calculation result according to the second embodiment. The figure shows the result calculated by the calculation unit 54. The numerical values ​​shown in the figure are coefficient vectors, and indicate the relative magnitude of the influence of each feature on the inference result (i.e., the importance of each feature). The calculation result shows the relationship between the second feature C2 (log feature) and the third feature C3 (environment variable feature) and the classification of the defect. Note that in the figure, the part with a relatively large coefficient value (specifically, 0.2 or more) is emphasized. According to this embodiment, it is possible to specify the variables required for classification from these calculation results. Note that the calculation unit 54 may calculate the likelihood for each class of the object for each combination of the second feature C2 and the third feature C3.

[0061] [Summary of embodiment 2] According to the embodiment described above, the two-dimensional information DI is an image of an object. The extraction unit 53 extracts the first feature C1 using at least one of a convolutional neural network, a Transformer, and a decision tree, using the two-dimensional information DI, which is an image, as input information. The calculation unit 54 performs class classification based on at least one of a second feature C2, which is a feature different from the first feature C1, or a third feature that acquires a feature different from the first feature C1 or the second feature, in addition to the first feature C1 extracted from the image. Therefore, according to the present embodiment, class classification can be performed with higher accuracy than when class classification is performed based only on an image. Furthermore, the importance of each feature or a combination thereof, and the required feature can be specified based on the relationship between the class classification and the second feature C2 or the third feature C3.

[0062] [Embodiment 3] Next, a third embodiment will be described with reference to Figures 9 to 12. The third embodiment is a specific example in which the system 1 according to the first embodiment is applied to a semiconductor manufacturing process.

[0063] FIG. 9 is a diagram for explaining an outline of the determination method according to the third embodiment. An outline of the determination model used in this embodiment will be explained with reference to the same figure. In this embodiment, the object S to be determined by the determination device 50 is a wafer for semiconductor manufacturing. A processing step for the object S, which is a wafer for semiconductor manufacturing, is performed by a processing device 10. The processing step performed by the processing device 10 may be any process in the wafer manufacturing process. Specific examples of the processing step may be an exposure step, an etching step, a wafer inspection step, etc.

[0064] (Step S11) First, the determination device 50 acquires wafer map data as two-dimensional information DI. The wafer map may be acquired by the processing device 10, or may be acquired by an inspection device (not shown) after processing by the processing device 10. The wafer map is information in which defective and non-defective areas are represented by two or more values ​​on a two-dimensional coordinate system.

[0065] (Step S12) Next, a first judgment is made based on the two-dimensional information DI. In the first judgment, a rough classification is made as to whether or not the wafer map belongs to one of the defect classes representing defects. In other words, the first judgment can be said to be a two-class classification of a defect class or a non-defective class. If the first judgment determines that the wafer does not have a defect, the processing ends for that wafer. If the first judgment determines that the wafer does have a defect, the processing proceeds to step S13.

[0066] (Step S13) Next, a second judgment is performed. In the second judgment, a judgment is made regarding the type of defect in the wafer map. The second judgment is performed by the judgment device 50. That is, the object class judged by the judgment device 50 according to this embodiment is a classification of defects for each wafer. The judgment device 50 may judge the likelihood for each object class. The classes are classified, for example, based on which part on the wafer the defect occurs in, and may be represented by "Loc", "Edge-Loc", "Center", etc., and may be classified into, for example, about 7 to 100 types.

[0067] Here, the difference between the first judgment and the second judgment will be described. The first judgment is a rough judgment based only on the two-dimensional information DI. The second judgment is a detailed judgment based on the two-dimensional information DI and additional features. That is, the first judgment and the second judgment do not need to use the same input information. In addition, the first judgment is a rough two-class classification to determine whether or not the defect belongs to any of the defect classes, whereas the second judgment is a detailed class classification to classify the types of defects. The second judgment is not limited to an example based on the additional features in addition to the two-dimensional information DI, and may be based only on the two-dimensional information DI. The input information for the first judgment and the input information for the second judgment may be the same. In this case, the classifier used for the first judgment and the classifier used for the second judgment may be configured to have different numbers of classes to be judged. In this case, the defective class in the first judgment may be labeled as a "defective class" (not a non-defective class=none) regardless of the type of defect.

[0068] FIG. 10 is a diagram showing an example of two-dimensional information according to the third embodiment. An example of the two-dimensional information DI according to the third embodiment will be described with reference to the same figure. The figure shows a classification of defects occurring in a wafer map as the two-dimensional information DI. Each of FIGS. 10(A) to 11(H) has a different classification of defects. The circle shown in each of FIGS. 10(A) to 11(H) indicates the outer circle of the wafer. Inside the wafer, light-colored areas indicate the locations where defects occur. Also, inside the wafer, dark-colored areas indicate the locations where there are no defects.

[0069] FIG. 10(A) is classified as “none” (i.e., not a defective class) and shows that the number of defects is less than a predetermined value. FIG. 10(B) is classified as “Loc” and shows that defects are concentrated at a location other than the outer circle or the center of the wafer. FIG. 10(C) is classified as “Edge-Loc” and shows that defects are concentrated at a location on the wafer near the edge of the wafer (near the outer circle). FIG. 10(D) is classified as “Center” and shows that defects are occurring at the center of the wafer. FIG. 10(E) is classified as “Edge-Ring” and shows that defects are occurring along the outer circle of the wafer. FIG. 10(F) is classified as “Scratch” and shows that a scratch-like line is present somewhere on the wafer. FIG. 10(G) is classified as “Random” and shows that the number of defects is greater than a predetermined value, but the defective locations are randomly occurring all over the wafer. FIG. 10(H) is classified as "Donut" and shows that the defective portion occurs in a doughnut shape centered on the center of the wafer.

[0070] FIG. 11 is a diagram for explaining extraction of a feature amount according to the third embodiment. As shown in the figure, the two-dimensional information DI is input to the extraction unit 53. Specifically, the two-dimensional information DI is a wafer map as described above. Only the two-dimensional information DI determined to have a defect by the first determination in step S11 described with reference to FIG. 9 may be input to the extraction unit 53. In this embodiment, the extraction unit 53 extracts the first feature amount C1 by a convolutional neural network. The extraction unit 53 includes a plurality of convolution operation layers and a plurality of pooling layers. The extraction unit 53 can obtain the first feature amount C1 as a result of performing a plurality of convolution operations and pooling operations. In addition, the feature amount C3 obtained by performing Radon transform on the two-dimensional information DI is obtained. Furthermore, according to this embodiment, a determination is made based on the extracted first feature amount C1, second feature amount C2, and third feature amount C3. The first feature amount C1, the second feature amount C2, and the third feature amount C3 are different features from each other. As an example of the second feature C2, various manufacturing parameters in a semiconductor manufacturing process, environmental information, etc. can be used. As an example of the third feature C3, a Radon feature can be used. The Radon feature may be Rand transformed from the two-dimensional information DI. The first feature C1 and the second feature C2 are input to a fully connected layer or an output layer included in the calculation unit 54. The fully connected layer generates output information converted into a predetermined dimension using the first feature C1 extracted by the extraction unit 53 and the acquired second feature C2 as input information. The output layer makes a judgment regarding the class of the object using the output information or the first feature C1 extracted by the extraction unit 53 and the acquired second feature C2 as input information. The output layer may, for example, judge the likelihood for each class of the object.

[0071] FIG. 12 is a diagram for explaining an example of the Radon feature according to the third embodiment. According to the present embodiment, in order to further improve the classification accuracy, a calculation may be performed in the fully connected layer or the output layer to which the Radon feature is added. FIGS. 12(A) to 12(H) are obtained as a result of Radon transforming the wafer maps shown in FIGS. 10(A) to 10(H), respectively. In addition, features derived using a method such as Tensor voting may be used instead of the Radon feature. The determination device 50 may perform class classification based on the feature obtained from the two-dimensional information DI and its associated information in this way. Class classification may be classification into any class, or may be determining the likelihood of each class.

[0072] [Summary of embodiment 3] According to the embodiment described above, the two-dimensional information DI is information that indicates the defective and non-defective parts of the object S by two or more values ​​on a two-dimensional coordinate system. Specifically, the two-dimensional information DI may be a wafer map. The extraction unit 53 uses the wafer map as input information and extracts the first feature C1 using at least one of a convolutional neural network, a Transformer, a decision tree, and ensemble learning. The calculation unit 54 performs class classification based on the first feature C1 extracted from the wafer map, as well as the second feature C2 and the third feature C3, which are features different from the first feature C1. Therefore, according to the present embodiment, class classification can be performed more accurately than when class classification is performed based only on the wafer map.

[0073] [Embodiment 4] Next, the fourth embodiment will be described with reference to Figs. 13 and 14. The fourth embodiment is a specific example of the learning stage of the calculation unit 54. When the determination device 50 according to this embodiment is applied to an industrial printing process as shown in the second embodiment, or to a semiconductor manufacturing process as shown in the third embodiment, the number of samples that can be used as learning data may be very small, and there may be few defect features. When there are few defect features, for example, there may be cases where the defective portion is very small and the color difference is small. In addition, for types of defects that occur rarely, it is difficult to obtain samples. Therefore, in this embodiment, a method that allows learning from a small number of samples will be described. The learning of the calculation unit 54 is performed by, for example, a learning device 70.

[0074] FIG. 13 is a functional configuration diagram showing an example of the functional configuration of the learning device according to the fourth embodiment. With reference to the same figure, an example of the functional configuration of the learning device 70 according to this embodiment will be described. The learning device 70 includes a two-dimensional information acquisition unit 71, a second feature acquisition unit 72, an extension unit 73, an extraction unit 74, a class information acquisition unit 75, a learning unit 76, and a third feature acquisition unit 77. Each of these functional units is realized, for example, by using an electronic circuit. Each functional unit may include a storage means such as a semiconductor memory or a magnetic hard disk device, as necessary. Each function may be realized by a computer having a CPU and software. All or a part of each functional unit may be realized by using hardware such as an ASIC, PLD, or FPGA. All or a part of each functional unit may be realized by a combination of software and hardware. All or a part of the functions of the learning device 70 may be realized by using hardware provided in an external device such as a cloud server. The learning device 70 may be configured with multiple devices, and each functional block may be distributed among multiple devices.

[0075] The two-dimensional information acquisition unit 71 acquires two-dimensional information from the two-dimensional information acquisition device 20. The two-dimensional information acquisition unit 71 acquires two-dimensional information about an object S to be learned. For example, when applied to an industrial printing process as shown in the second embodiment, the two-dimensional information may be an image of a label. When applied to a semiconductor manufacturing process as shown in the third embodiment, the two-dimensional information may be a wafer map. The two-dimensional information acquisition unit 71 outputs the acquired two-dimensional information to the extension unit 73 as two-dimensional information DI.

[0076] The second feature amount acquisition unit 72 acquires a feature amount from the feature amount acquisition device 30. The feature amount acquired by the second feature amount acquisition unit 72 is also referred to as a second feature amount. The second feature amount acquisition unit 72 outputs the acquired feature amount to the extension unit 73 as a second feature amount C2-1. The second feature amount C2-1 corresponds to the two-dimensional information DI acquired by the two-dimensional information acquisition unit 71.

[0077] The third feature amount acquisition unit 77 acquires a third feature amount. The third feature amount is, for example, a feature amount acquired by performing a Radon transform on the two-dimensional information DI. The third feature amount acquisition unit 77 outputs the acquired feature amount to the extension unit 73 as a third feature amount C3-1. The third feature amount C3-1 corresponds to the two-dimensional information DI acquired by the two-dimensional information acquisition unit 71.

[0078] The extension unit 73 generates a plurality of pieces of extended two-dimensional information based on the acquired two-dimensional information DI. The generation of the extended two-dimensional information may be performed, for example, by amplifying the original two-dimensional information DI. By generating the extended two-dimensional information, the extension unit 73 can improve the size and quality of the learning dataset.

[0079] 14 is a diagram for explaining main types of data extension of two-dimensional information according to embodiment 4. A specific example of generation of extended two-dimensional information will be described with reference to the same figure. First, there can be given an example of "horizontal inversion" of the two-dimensional information DI. This is to create data different from the original two-dimensional information DI by inverting the two-dimensional information DI in the horizontal direction. Secondly, we can exemplify "cropping" two-dimensional information DI. This is to create multiple data from the original two-dimensional information DI by cropping a small area of ​​4 [px (pixels)] on each side of the two-dimensional information DI. Thirdly, the "rotation" of the two-dimensional information DI can be exemplified. This is to create data different from the original two-dimensional information DI by rotating the two-dimensional information DI 10[° (degrees)] in the CW or CCW direction. A fourth example is "random zoom" of the two-dimensional information DI. This is to create data different from the original two-dimensional information DI by enlarging or reducing the two-dimensional information DI in a range of, for example, about 75[%] to 125[%]. Fifthly, a "color shift" of the two-dimensional information DI can be exemplified. This is to create data having pixel values ​​different from the original two-dimensional information DI by varying the pixel values ​​of the original two-dimensional information DI within a range of plus or minus 50. Sixthly, a "random deletion" of the two-dimensional information DI can be exemplified. This is to create data in which pixel information of the original two-dimensional information DI is missing by deleting a 4[px] x 12[px] rectangle from the original two-dimensional information DI with a probability of, for example, 50[%]. Seventhly, a "mix-up" of two-dimensional information DI can be exemplified. This is to generate new data by synthesizing two or more pieces of two-dimensional information DI as original data. The synthesis of two-dimensional information DI may be performed according to a beta distribution, for example, with a parameter of 0.5. An eighth example is "auto-contrast" of the two-dimensional information DI. This is to create data different from the original two-dimensional information DI by performing auto-contrast with a probability of, for example, 50%.

[0080] The extension unit 73 may create one or more new data from the original two-dimensional information DI by a method other than the above-mentioned method. Also, the extension unit 73 may create one or more new data from the original two-dimensional information DI by combining a plurality of the above-mentioned methods.

[0081] Returning to FIG. 13, the extension unit 73 outputs the original two-dimensional information DI and data extended from the two-dimensional information DI to the extraction unit 74 as extended two-dimensional information EDI. The extension unit 73 also outputs the second feature C2 corresponding to the extended two-dimensional information EDI to the learning unit 76 as the second feature C2-2. Specifically, the second feature C2-2 may be the second feature C2 corresponding to the original two-dimensional information DI. The extension unit 73 also outputs the third feature C3 corresponding to the extended two-dimensional information EDI to the learning unit 76 as the third feature C3-2. Specifically, the third feature C3-2 may be the third feature C3 corresponding to the original two-dimensional information DI.

[0082] For samples for which the second feature does not exist, artificial data can be generated and used. (This artificial data can also be used to improve accuracy.) The generation of artificial data will be described. First, from the two-dimensional information DI and the class information corresponding to the two-dimensional information DI, the density of elements that match a specific value in the two-dimensional information DI (for example, defect rate / Defect Rate = number of corresponding elements / all elements) is calculated for each class. Next, a new attribute (new attribute) is derived based on a distribution in which the attribute values ​​of the two-dimensional information DI including the calculated density are aggregated for multiple samples. Furthermore, the derived new attribute is added as a new attribute of the sample by the new attribute adding unit. The new attribute may be aggregated from a continuous value to a discrete value such as an interval variable (for example, a cluster of the GMM clustering result of the defect rate). For a pair of a class and a new attribute, a feature common within the class (common feature) and a feature specific to the new attribute (specific feature) are defined, and the common feature and the specific feature are associated with each other for each level of the class and the new attribute. The associated feature is described as a feature by class and new attribute. A class-new attribute feature corresponding to the class to which a certain sample belongs and the level of the new attribute derived from the two-dimensional information may be associated with each other and set as the second feature C2-2.

[0083] The extraction unit 74 extracts a feature from each of the acquired extended two-dimensional information EDI. A convolutional neural network or the like may be used to extract the feature. In addition, a Radon feature or the like may be separately extracted from each of the extended two-dimensional information EDI. The extraction unit 74 outputs the feature extracted from the extended two-dimensional information EDI to the learning unit 76 as a first feature C1.

[0084] The class information acquisition unit 75 acquires information about classes into which the object S is classified based on a predetermined criterion. Classification as a learning data set may be performed, for example, by a human or mechanically based on a predetermined criterion. The class information acquisition unit 75 outputs information about the classified classes to the extension unit 73 and the learning unit 76 as class information CL.

[0085] The learning unit 76 trains a machine learning model to estimate a class indicated in the class information CL based on the first feature amount C1 and the second feature amount C2-2 extracted based on the extended two-dimensional information EDI. The machine learning model may be included in the calculation unit 54 at the time of inference.

[0086] In this embodiment, the learning device 70 does not necessarily have to include the expansion unit 73. For example, if the learning data set is sufficient, learning may be performed based on the acquired data set without expanding the data set by including the expansion unit 73.

[0087] [Summary of the fourth embodiment] According to the embodiment described above, the learning device 70 includes the two-dimensional information acquisition unit 71 to acquire two-dimensional information DI about the object S to be judged, includes the class information acquisition unit 75 to acquire information about the class into which the object S is classified based on a predetermined judgment criterion, includes the second feature acquisition unit 72 to acquire a second feature C2 that is a feature about the object S and is different from the first feature C1 extracted from the two-dimensional information DI, and includes the learning unit 76 to train a machine learning model to estimate a class based on the first feature C1 extracted based on the two-dimensional information DI acquired by the two-dimensional information acquisition unit 71 and the second feature C2 acquired by the second feature acquisition unit 72. That is, according to this embodiment, the machine learning model is trained to estimate a class based not only on the two-dimensional information DI but also on the second feature C2 corresponding to the two-dimensional information DI. The machine learning model trained by the learning device 70 according to this embodiment can estimate class classification with higher accuracy than when trained based only on the two-dimensional information DI. Also, as in the second embodiment, the influence of the second feature C2 can be specified.

[0088] Moreover, according to the above-described embodiment, by further including the extension unit 73, a plurality of extended two-dimensional information EDI is generated based on the acquired two-dimensional information DI. The learning unit 76 trains a machine learning model to estimate a class based on the first feature C1 extracted based on the generated extended two-dimensional information EDI and the second feature C2 acquired by the second feature acquisition unit 72. Therefore, according to this embodiment, data can be extended from a small number of samples. The learning device 70 performs learning based on the extended data. Therefore, according to this embodiment, a machine learning model can be trained even from a small number of samples.

[0089] [Embodiment 5] Next, a fifth embodiment will be described with reference to FIG. 15. The fifth embodiment is a specific example of the stage of creating teacher data when learning is performed by the learning device 70. Depending on the dataset used for learning by the learning device 70 according to this embodiment, defects may occur rarely, making it difficult to obtain a sample. In addition, there are also cases where these defects are combined, and it is statistically very difficult to obtain such a dataset. Therefore, in this embodiment, a method capable of creating a dataset having rare defects will be described. The creation of teacher data is performed by, for example, a teacher data creation device 80.

[0090] FIG. 15 is a functional configuration diagram showing an example of the functional configuration of the teacher data creation device according to the fifth embodiment. With reference to the same figure, an example of the functional configuration of the teacher data creation device 80 according to this embodiment will be described. The teacher data creation device 80 includes a two-dimensional information acquisition unit 81, a second feature acquisition unit 82, a selection unit 831, a superposition unit 832, an integration unit 84, a missing part assignment unit 85, an output unit 86, a class information acquisition unit 87, a third feature acquisition unit 881, and a third feature combination unit 882. Each of these functional units is realized, for example, by using an electronic circuit. Each functional unit may include a storage means such as a semiconductor memory or a magnetic hard disk device, as necessary. Each function may be realized by a computer having a CPU and software. All or a part of each functional unit may be realized by using hardware such as an ASIC, a PLD, or an FPGA. All or a part of each functional unit may be realized by a combination of software and hardware. In addition, all or part of the functions of the teacher data creation device 80 may be realized using hardware or the like provided in an external device such as a cloud server. In addition, the teacher data creation device 80 may be composed of multiple devices, and each functional block may be distributed among the multiple devices.

[0091] In this embodiment, the training data broadly refers to a data set used for training a machine learning model. The use of the training data is not limited to training, and may be used for verifying a trained model.

[0092] The two-dimensional information acquisition unit 81 acquires a plurality of pieces of two-dimensional information from the two-dimensional information acquisition device 20. The two-dimensional information acquisition unit 81 acquires two-dimensional information about an object S for which teacher data is to be created. For example, when applied to an industrial printing process as shown in the second embodiment, the two-dimensional information may be an image of a label. When applied to a semiconductor manufacturing process as shown in the third embodiment, the two-dimensional information may be a wafer map. The two-dimensional information acquisition unit 81 outputs the acquired plurality of pieces of two-dimensional information to the selection unit 831 as two-dimensional information DI.

[0093] The class information acquisition unit 87 acquires information about the class of the object S. The class information acquisition unit 87 outputs the information about the class of the object S as class information CL to the selection unit 831, the superposition unit 832, the second feature amount combination unit 821, the third feature amount acquisition unit 881, the third feature amount combination unit 882, and the integration unit 84.

[0094] The selection unit 831 selects two or more pieces of two-dimensional information DI from the acquired plurality of pieces of two-dimensional information DI based on a predetermined selection algorithm. The plurality of pieces of two-dimensional information DI selected by the selection unit 831 are two-dimensional information DI to be the target of superimposition. The selection unit 831 outputs the selected two or more pieces of two-dimensional information DI to the superimposition unit 832 as selected two-dimensional information SDI.

[0095] The overlay unit 832 creates overlaid two-dimensional information OLDI by overlaying two or more pieces of two-dimensional information DI selected by the selection unit 831. The overlaid two-dimensional information OLDI is obtained by overlaying the features of the respective pieces of two-dimensional information DI that were the source of the overlay. Note that the overlaid two-dimensional information OLDI may be used as it is as training data for learning or testing a machine learning model.

[0096] The second feature amount acquisition unit 82 acquires a feature amount from the feature amount acquisition device 30. The feature amount acquired by the second feature amount acquisition unit 82 is also described as a second feature amount. The second feature amount acquisition unit 82 outputs the acquired feature amount to the integration unit 84. The second feature amount C2 corresponds to the two-dimensional information DI acquired by the two-dimensional information acquisition unit 81.

[0097] The second feature amount combining unit 821 creates a combined second feature amount OLC2 corresponding to the superimposed two-dimensional information OLDI. The created combined second feature amount OLC2 is output to the integration unit .

[0098] The third feature amount acquisition unit 881 acquires a third feature amount. The third feature amount acquisition unit 881 outputs the acquired feature amount to the integration unit 84. The third feature amount C3 corresponds to the two-dimensional information DI acquired by the two-dimensional information acquisition unit 81.

[0099] Furthermore, the third feature amount combining unit 882 creates a combined third feature amount OLC3 corresponding to the superimposed two-dimensional information OLDI. The created combined third feature amount OLC3 is output to the integrating unit 84.

[0100] The integration unit 84 acquires the overlaid two-dimensional information OLDI from the overlay unit 832, acquires the combined second feature amount OLC2 from the second feature amount combination unit 821, and acquires the combined third feature amount OLC3 from the third feature amount combination unit 882. The integration unit 84 integrates the acquired combined second feature amount OLC2 and combined third feature amount OLC3 into the overlaid two-dimensional information OLDI. The integration unit 84 outputs the integrated information to the loss adding unit 85 as integrated information CI.

[0101] The missing part adding unit 85 acquires the integrated information CI from the integration unit 84, and adds a missing part to the acquired information. Adding a missing part may be, for example, changing a feature value or a pixel value. The missing part adding unit 85 may add information indicating that each of the feature value and pixel of the teacher data has a missing part based on a predetermined probability distribution or random number. Furthermore, the missing part adding unit 85 may change the feature value and pixel value to a suitable range depending on the type of missing part to be added to the teacher data. Note that the processing by the missing part adding unit 85 does not necessarily have to be performed.

[0102] The output unit 86 outputs the information after the missing part is added by the missing part adding unit 85 as teacher data TD. The teacher data TD output by the output unit 86 may be stored in a predetermined storage device (not shown). The teacher data TD stored in the storage device may be used for learning by the learning device 70, or may be used for verification of the determination device 50.

[0103] [Summary of embodiment 5] According to the embodiment described above, the teacher data creation device 80 includes a two-dimensional information acquisition unit 81 to acquire a plurality of pieces of two-dimensional information about the object S to be determined, a selection unit 831 to select two or more pieces of two-dimensional information from the acquired plurality of pieces of two-dimensional information based on a predetermined selection algorithm, and a superimposition unit 832 to superimpose the selected two or more pieces of two-dimensional information to create superimposed two-dimensional information OLDI. The second feature acquisition unit 82 acquires a second feature C2 that is a feature related to the object S and is different from the first feature C1 extracted from the two-dimensional information, the second feature combination unit 821 creates a combined second feature OLC2 corresponding to the teacher data, and the integration unit 84 integrates the second feature C2 into the teacher data TD to create teacher data used for learning or testing a machine learning model. That is, according to this embodiment, teacher data having features of multiple classifications can be created. Furthermore, since the second feature C2 is integrated into the teacher data created in this embodiment, teacher data for learning based on the two-dimensional information DI and the second feature C2 can be created.

[0104] [Embodiment 6] Next, a specific example of overlap classification will be described with reference to FIGS.

[0105] Fig. 16 shows an example of the two-dimensional information obtained by the superposition, specifically, a case where the defect "Center" shown in Fig. 10(D) is combined with the defect "Scratch" shown in Fig. 10(F) is shown.

[0106] In this embodiment, an open data set, WM-811K, is used. This data set is recorded in the final product test in actual mass production. A single wafer is equipped with hundreds to tens of thousands of small rectangular parallelepiped (so-called die) chips, and the original label of each die is 1 (good), 2 (bad), or 0 (outside the product). In addition, class labels are attached to a total of 172950 samples by engineers with domain knowledge. This data is widely used to study the classification problem of defect patterns. Many different solutions are difficult, such as product decomposition by product combinations and technology combinations, data imbalance between classes, and random cause defects appear in all samples even for wafers of a certain defect class. In addition, the upper parts of two or more classes correspond to common or different causes. In addition to the data of two-dimensional information with class labels, wafer attributes such as the number of columns and rows, die size, and defect rate are also recorded in the test log. However, there is no strict overlapping class label added to WM-811K. Therefore, in this embodiment, a synthetic data set is generated as shown in the figure.

[0107] FIG. 17 is an example of two-dimensional information after superimposition. This figure shows an example of superimposition of all seven defect classes. The superimposition unit 832 randomly samples a predetermined number (for example, one at a time) of samples of a predetermined class to be superimposed, and performs a logical OR operation on each die (pixel) of the wafer map. However, dies outside the wafer are left with labels outside the wafer. The superimposition unit 832 generates a data set as shown in the figure by such an operation. Note that, before the logical OR operation, a pre-processing is performed in advance to convert the labels of each die into 0 for defects, 1 for non-defective products, and other values ​​(such as -1) for those outside the wafer area.

[0108] 18 is a diagram for explaining the details of the calculation performed by the overlapping unit 832. With respect to the features attached to the samples of a predetermined class to be overlapped, the overlapping unit 832 performs a) logical OR calculation or majority calculation on 0-1 variables, and b) takes the values ​​of basic statistics or the like on numerical variables.

[0109] The features X1 to X116 in the table are 0-1 variables. Of the features X1 to X116, X1 to X5 are features common to all classes (=incorrect random features unrelated to defects), X42 to X47 are features related to the Center class, and X93 to X101 are features related to the Scratch class. Specifically, the features of the overlap class are as shown in the table at the bottom of FIG.

[0110] Here, each row represents the correspondence of features for each class and cluster (failure Class, Cluster) pair. The clusters here are clusters organized by performing clustering based on a mixed Gaussian distribution using DefectRate (failure pixel rate), which is auxiliary information of the two-dimensional information of samples of each class, and defining the optimal number of clusters based on AIC (Akaike Information Criterion) or BIC (Bayes Information Criterion). In addition, for each original class, a corresponding DefectRate interval is associated with each cluster. For each learning sample, the corresponding feature can be associated from the original class and the cluster that can be identified from the DefectRate (this is one example). Note that if feature data is available for the overlay failure sample, it is sufficient to simply combine them and use them. If an appropriate feature organization is available, feature data for the overlay failure sample similar to the above can be obtained.

[0111] The third feature amount corresponds to, for example, an inspection log. This is common to both the above-mentioned embodiment 2 and embodiment 3. Hereinafter, the combination of the third feature amount will be specifically described using an example of a semiconductor wafer in embodiment 3.

[0112] The raw information of the third feature amount is, for example, as shown in FIG.

[0113] The overlapping process of features attached to selected samples of a given class (c∈C(SP)) that is the target of an overlapping SP is as shown in (1) to (3) below. Note that there are several variations depending on the difference in features.

[0114] (1) Regarding the number of pixels of the semiconductor wafer map, CountPixel (total number of dies), CountRow (number of rows), and RowntColumn (number of columns), the number of pixels of all the wafer maps WM to be superimposed is unified by resizing. For example, unify to S1 rows, S2 columns, pixel count S1xS2.

[0115] (2) Regarding DefectRate, which indicates the ratio of defective dies on the semiconductor wafer map, for each pixel after resizing, the logical sum v(i,j) of the element values ​​(e.g., good dies: 0, defective dies: non-zero values ​​converted to 1, etc.) or some other calculation of the color system is performed, and then the following equation (1) for the wafer of the overlay SP is derived for each overlay sample.

[0116]

number

[0117] The logical sum is derived, for example, by the following formula (2).

[0118]

number

[0119] The size of the print log feature can also be derived as a result of overlapping based on the result of derivation for each pixel in the same manner as described above. A rank can be derived based on the result of this processing and a predetermined criterion.

[0120] (3) The number of defects in the print log feature is the sum of the number of defects in the original samples. Basic statistics such as the maximum value may also be used.

[0121] Next, the results of learning the overlapped data will be described.

[0122] In the case of two-class superposition (2-class SP, 21 classes from Class No. 8 to 28 in Fig. 23), there are many combinations and the density of defective dies is low, making it difficult to distinguish between superposed classes or from one class. In fact, the numerical verification results (Fig. 20) show that without features (missing rate 100%), the classification accuracy is low at less than 70% (0.6939, at the far left of the bottom row of the same table). On the other hand, when features are added, classification accuracy of 90% or more is achieved even when the missing rate of features is very high at 80%. Furthermore, when the missing rate is 40% or less, the classification accuracy is over 99.6%, close to 100%. (The bottom row of the table, from the third column onwards: “The rate of missing value on the causal variables” is 0.8, 0.6, 0.4, 0.2, 0.0.)

[0123] Fig. 21 shows the class definitions for a classification problem that combines 6-class superposition and 7-class superposition. In the figure, the classes excluded from the 7-class superposition (All) in the 6-class superposition (6-class SP) are marked as "-XXX". For example, in the case of class ClassNo.=101 when all six classes other than "Loc" are overlapped, the column indicating the overlapping class definition is written as SP Definition="All - Loc".

[0124] FIG. 22 shows an example of the classification result of the overlay information, which shows the classes of all overlay combinations of 7 defect classes and 6 classes. The classification problem of combining 6 classes and 7 classes is a problem with high defect density and high difficulty in discrimination. Even in this case, when the second feature is used, a highly accurate classification is achieved with a classification accuracy of approximately 87% or more when the missing rate of each feature is 60% or less, and 95% or more when the missing rate is 40% or less. On the other hand, when no feature is used (missing rate 100%), the accuracy is low at 40.5%.

[0125] The set value indicates, for example, clustering based on a Gaussian mixture model is performed on the distribution of the defect rate (defective pixel rate on two-dimensional information) of samples corresponding to a certain class, and the number of clusters is optimized based on the Akaike Information Criterion or the Bayes Information Criterion, and the cluster identifier and identification condition (e.g., the range of the defect rate corresponding to each cluster) derived under the condition of the optimal number of clusters for the class. This is different from a form in which two-dimensional information and a second feature value linked to each sample (ID) are simply added.

[0126] The two-dimensional information represents the location and shape of the defect, while the set value provides relative information on the density and size of the defect for the sample set. By using these two, it is possible to improve the accuracy of class discrimination and identify the cause of the defect.

[0127] Let me explain a little more specifically. Generally, there is often a correspondence between a defect class and the process that causes it. Therefore, it is possible to narrow down the candidates for the process that caused the defect by identifying the class (e.g. 1:m). However, the class hierarchy is insufficient to obtain a correspondence with the "process variable" (material, method, processing equipment, information, etc.) that is the direct cause of the defect. Here, the cluster (or the feature obtained from the distribution of the defect rate and other incidental information) located at the lower level of the class corresponds well to the effect of a specific level of the process variable that represents the direct cause of the defect or a combination of specific levels of multiple variables. This has been made clear by research on virtual measurement (soft sensors) and the like. In other words, it is only by using both the class (classification identifier based on the position and shape of the defect) and the cluster (representing the density, dimensions, severity, etc. of the defect) that it becomes possible to associate with the process variable that is the direct cause of the defect.

[0128] In addition, in superposition classification, only the superposition result of a mixture of various classes and defect rates appears in the two-dimensional information. The original class cannot be identified from this two-dimensional information alone (the first feature derived from it). Furthermore, as mentioned above, simply adding the second feature does not provide a cluster that associates the class with the process variable, so the effect of improving accuracy by adding the second feature is limited.

[0129] (Definition of the second and third features) The second feature is an external feature such as a causal feature collected independently of the two-dimensional information, whereas the third feature is an internal feature related to the two-dimensional information, derived from the two-dimensional information and its accompanying information. This is the same for both semiconductor cases and printing cases.

[0130] [Embodiment 7] The second feature may be extracted by a second feature acquisition unit or a second feature extraction unit (not shown). For the extraction of the second feature, a device (method) based on a screening method such as Sure Independence Screening (SIS), which is a representative example of interaction modeling, or a factorization machine (Sparse Factorization Machines, SFM) with feature selection may be used. Among SFMs, an interaction modeling device (method) may be used that includes a single feature extraction unit (step) that estimates the independent effect of a feature using L1 norm regularization, a combined feature extraction unit (step) that estimates a combined feature, which is a combination of a plurality of features, using L2 norm regularization, and a total feature extraction unit (step) that estimates all features using a regularization term with an upper bound. Among these, an SFM using the TI upper bound (referred to as TI-SFM(L1)) is a new form proposed in the present invention and is one of the preferable forms.

[0131] When each part corresponds to a formula, it becomes as shown in the following formula (3).

[0132]

number

[0133] The second term on the right side is the independent feature extraction unit, the third term on the right side is the combined feature extraction unit, and the fourth term on the right side is the total feature extraction unit. Here, Ω TI (P) represents the upper bound of the Triangle-Inequality (TI). However, in previous studies, the L2 norm was used to estimate the effect of individual features, and a device (method) including L2 norm regularization may be used in the individual feature extraction unit (step). Also, the CS upper bound or other upper bounds may be used in the overall feature extraction unit (step).

[0134] In addition, all or part of the functions of each unit of the system 1 in the above-mentioned embodiment may be realized by recording a program for realizing these functions on a computer-readable recording medium, reading the program recorded on the recording medium into a computer system, and executing it. Note that the "computer system" referred to here includes hardware such as an OS and peripheral devices.

[0135] In addition, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage units such as hard disks built into computer systems. Furthermore, the term "computer-readable recording medium" may also include those that dynamically hold a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, and those that hold a program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or client in such cases. Furthermore, the above program may be one that realizes part of the above-mentioned functions, or may be one that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0136] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the spirit of the present invention.

[0137] The invention described in the above embodiment will be summarized and disclosed as an addendum.

[0138] (Appendix 1) a two-dimensional information acquisition unit that acquires two-dimensional information and associated information about an object to be determined; an extraction unit that extracts a first feature amount from the two-dimensional information acquired by the two-dimensional information acquisition unit; a second feature amount acquisition unit that acquires a second feature amount that is a feature amount related to the object and is different from the first feature amount; a third feature amount generation unit that associates a set value (e.g., an identifier and an identification condition of a sample set) derived from the two-dimensional information and the additional information about a plurality of samples with information about each sample, and assigns information about a (relative) set value to each sample; and a second feature amount extraction unit that extracts an important second feature amount from the second feature amounts based on the third feature amount that is information about the set value generated by the third feature amount generation unit. a calculation unit that performs a calculation related to class classification of the object using the first feature extracted by the extraction unit and the second feature extracted by the second feature extraction unit as input information; an output unit that outputs a result of the calculation performed by the calculation unit; A determination device comprising:

[0139] (Appendix 2) In the invention described in Appendix 1, The set value indicates, for example, cluster identifiers and identification conditions (e.g., range of defect rates corresponding to each cluster) derived under conditions of the optimal number of clusters for the class by performing clustering based on a Gaussian mixture model on the distribution of the defect rates (defective pixel rates in two-dimensional information) of samples corresponding to a certain class and optimizing the number of clusters based on the Akaike Information Criterion or the Bayes Information Criterion. This differs from the previous patent, which simply adds two-dimensional information and a second feature associated with each sample (ID). The two-dimensional information represents the location and shape of the defect, while the set value provides relative information on the density and size of the defect for the sample set. By using these two, it is possible to improve the accuracy of class discrimination and identify the cause of the defect.

[0140] Generally, there is a correspondence between a defect class and the process that causes the defect. Therefore, it is possible to narrow down the candidates for the process that caused the defect by identifying the class (generally a one-to-many relationship). However, the class hierarchy is insufficient to obtain a correspondence with the "process variable" (material, method, processing equipment, information, etc.) that is the direct cause of the defect. Here, the cluster (or the feature obtained from the distribution of defect rate and other auxiliary information) located at the lower level of the class corresponds well to the effect of a specific level of the process variable that represents the direct cause of the defect or a combination of specific levels of multiple variables. This has been made clear by research on virtual measurement (soft sensors) that estimates and predicts measured values, etc., using data drive to replace physical measurement. In other words, it is only by using both the class (classification identifier based on the position and shape of the defect) and the cluster (representing the density, size, severity, etc. of the defect) that it becomes possible to associate with the process variable that is the direct cause of the defect. In addition, in superposition classification, only the superposition result of a mixture of various classes and defect rates appears in the two-dimensional information. The original class cannot be identified from this two-dimensional information alone (the first feature derived from it). Furthermore, as mentioned above, simply adding the second feature does not provide a cluster that associates the class with the process variable, so the effect of improving accuracy by adding the second feature is limited.

[0141] The invention described in Appendix 1 is expected to have the following effects: (1) It is possible to achieve single-class or overlapping-class classification using the same learning machine. (2) It is possible to improve the accuracy of class classification (both single class and overlapping class). (3) It is possible to identify causal features and extract the impact (importance) of features (when combining the second feature at the output layer).

[0142] The second feature amount and the third feature amount differ in the following points. The difference is that the second feature is an external feature such as a causal feature collected independently of the two-dimensional information, whereas the third feature is an internal feature related to the two-dimensional information, derived from the two-dimensional information and its associated information. This is also true in the semiconductor and printing embodiments.

[0143] The second feature is information such as an identifier of the process, various variables representing the state or events of the process (temperature, humidity, pressure, voltage, amount of material input, operating speed, degree of cooling or heating, processing time, whether after maintenance or not, whether after a breakdown or not, whether operational or not, etc. at each point in the process), or an identifier of the processing medium or an identifier of the processing target.

[0144] The third feature is a feature derived from a single or multivariate distribution (e.g., distribution of defect rates by class) of the two-dimensional information or additional information (e.g., defect rates) associated with the two-dimensional information (e.g., a cluster resulting from clustering the distribution of defect rates by class (including multi-modal distribution) based on a Gaussian mixture distribution model, an identification condition and an identifier for that cluster, etc.).

[0145] (Appendix 3) The second feature may be obtained in the second feature acquisition unit or the second feature extraction unit using screening such as Sure Independence Screening (SIS), which is a representative example of interaction modeling, or a device (method) based on Sparse Factorization Machines (SFM) with feature selection. Among SFMs, the following device (method) is particularly one of the preferred ones proposed by the present invention. An independent feature extraction unit (step) that estimates the individual effect of a feature using L1 norm regularization, a combined feature extraction unit (step) that estimates combined features, which are combinations of multiple features, using L2 norm regularization, and a total feature extraction unit (step) that estimates all features using a regularization term that introduces an upper bound. An interaction modeling device (method) equipped with a factorization machine with feature selection, characterized by including

[0146] (Appendix 4) The inventions described in Supplementary Notes 1 to 3 may also be modified to include the inventions described in the claims. [Explanation of symbols]

[0147] 1...system, 10...processing device, 20...two-dimensional information acquisition device, 30...feature acquisition device, 50...judgment device, 51...two-dimensional information acquisition section, 52...second feature acquisition section, 53...extraction section, 54...calculation section, 55...output section, 56...third feature acquisition section, 57...class information acquisition section, 60...preliminary judgment device, 70...learning device, 71...two-dimensional information acquisition section, 72...second feature acquisition section, 73...extension section, 74...extraction section, 75...class information acquisition section, 76...learning section, 77...third feature feature acquisition unit, 80...teacher data creation device, 81...two-dimensional information acquisition unit, 82...second feature acquisition unit, 831...selection unit, 832...superimposition unit, 84...integration unit, 85...missing part, 86...output unit, 87...class information acquisition unit, 881...third feature acquisition unit, 882...third feature combination unit, 821...second feature combination unit, C1...first feature, C2...second feature, C3...third feature, S...object, DI...two-dimensional information, EDI...extended two-dimensional information, CL...class information

Claims

1. a two-dimensional information acquisition unit that acquires two-dimensional information about an object to be determined and additional information associated with the two-dimensional information; an extraction unit that extracts a first feature amount from the two-dimensional information acquired by the two-dimensional information acquisition unit; a second feature amount acquiring unit that acquires a second feature amount related to the object, the second feature amount being different from the first feature amount; a third feature amount acquiring unit that acquires a third feature amount different from the first feature amount and the second feature amount; a calculation unit that performs a calculation related to class classification of the object using as input information the first feature amount extracted by the extraction unit and at least one of the second feature amount acquired by the second feature amount acquisition unit and the third feature amount acquired by the third feature amount acquisition unit; an output unit that outputs a result of the calculation performed by the calculation unit; A determination device comprising:

2. A predetermined process is performed on the object; the first feature amount is information about the object acquired after the step is performed, the second characteristic amount is information during or before the process is performed, The third characteristic amount is information after the step is performed, while the step is performed, or before the step is performed. The determination device according to claim 1 .

3. A predetermined process is performed on the object; The second characteristic amount is a value that is controlled in the step. The determination device according to claim 1 or 2.

4. A predetermined process is performed on the object; The second characteristic amount is a value that affects the process. The determination device according to claim 1 or 2.

5. The two-dimensional information acquisition unit acquires the two-dimensional information about the object determined to be a determination target as a result of a previous determination of whether the object is a determination target. The determination device according to claim 1 or 2.

6. The two-dimensional information is an image of the object. The determination device according to claim 1 or 2.

7. The extraction unit extracts the first feature amount using at least one of a convolutional neural network, a Transformer, a decision tree, and ensemble learning, using the two-dimensional information, which is an image, as input information. The determination device according to claim 6.

8. The calculation unit calculates a likelihood for each class of the object for each combination of the second feature amount and the third feature amount. The determination device according to claim 6.

9. the object being a printed label; The object class is a classification of types of defects that occur when printing the label. The determination device according to claim 6.

10. The two-dimensional information is information that expresses the defective and non-defective portions of the object using two or more values ​​on two-dimensional coordinates. The determination device according to claim 1 or 2.

11. The calculation unit determines a likelihood for each class of the object. The determination device according to claim 10.

12. The calculation unit has an output layer that receives the extracted first feature amount and at least one of the acquired second feature amount and the acquired third feature amount as input information and outputs a likelihood for each class of the object or a class corresponding to a maximum likelihood. The determination device according to claim 11.

13. the object being a wafer for semiconductor manufacturing; The object class is a classification of the types of defects for each wafer. The determination device according to claim 10.

14. a two-dimensional information acquisition unit that acquires two-dimensional information about an object to be determined and associated information associated with the two-dimensional information; a class information acquisition unit that acquires information about a class into which the object is classified based on a predetermined judgment criterion; a second feature amount acquiring unit that acquires a second feature amount related to the object, the second feature amount being different from the first feature amount extracted from the two-dimensional information; a third feature amount acquiring unit that acquires a third feature amount different from the first feature amount and the second feature amount; a learning unit that trains a machine learning model to estimate the class based on the first feature amount extracted based on the two-dimensional information acquired by the two-dimensional information acquisition unit and at least one of the second feature amount acquired by the second feature amount acquisition unit and the third feature amount acquired by the third feature amount acquisition unit; A learning device comprising:

15. An extension unit that generates a plurality of pieces of extended 2D information based on the acquired 2D information, The learning unit trains a machine learning model to estimate the class based on the first feature amount extracted based on the generated extended two-dimensional information and the second feature amount acquired by the second feature amount acquisition unit. The learning device according to claim 14.

16. a two-dimensional information acquisition unit that acquires a plurality of pieces of two-dimensional information about an object to be determined and associated information associated with the two-dimensional information; A selection unit that selects two or more pieces of two-dimensional information from the acquired plurality of pieces of two-dimensional information based on a predetermined selection algorithm; a superposition unit that creates teacher data to be used for learning or testing a machine learning model by superposing two or more selected pieces of two-dimensional information; a second feature amount acquiring unit that acquires a second feature amount related to the object, the second feature amount being different from the first feature amount extracted from the two-dimensional information; a third feature amount acquiring unit that acquires a third feature amount different from the first feature amount and the second feature amount; At least one of a second feature amount combining unit that combines the second feature amounts corresponding to the superimposed samples and a third feature amount combining unit that combines a third feature amount corresponding to the superimposed samples; an integration unit that integrates at least one of the combined second feature amount and the acquired third feature amount into the teacher data; A teacher data creation device comprising:

17. The method further includes a defect adding unit that adds information indicating that each coordinate or feature of the teacher data has a defect based on a predetermined probability distribution or random number. The teacher data creation device according to claim 16.

Images