Model generation device, inspection device, model generation method, and program

Abstract

This model generation device (11) comprises: an acquisition unit (111) that acquires a first variance-covariance matrix (A) generated by first learning data including a plurality of pieces of image data, and a second variance-covariance matrix (B) generated by second learning data including a plurality of pieces of image data; and a model coupling unit (112) in which the first variance-covariance matrix (A) and the second variance-covariance matrix (B) are linearly combined to generate a third variance-covariance matrix (W).

Description

[Technical Field]

[0001] This disclosure relates to a model generation device, an inspection device, a model generation method, and a program. [Background technology]

[0002] Machine learning can be broadly categorized into supervised learning and unsupervised learning. One example of supervised learning is the problem of predicting categories, or "classification." Another example of unsupervised learning is the problem of finding groups, or "clustering." Artificial intelligence that performs "classification" or "clustering" on images includes neural networks such as CNNs (Convolutional Neural Networks).

[0003] Furthermore, some anomaly detection methods identify defects in the object being inspected from image data, and machine learning classification and clustering techniques are sometimes used in this type of anomaly detection. Specifically, image features are extracted from image data and represented as a set of feature vectors. The set of feature vectors is analyzed using a variance-covariance matrix, and clustering is performed to determine whether the inspected item is defective or not, based on whether it belongs to the good product cluster or the defective product cluster.

[0004] A variance-covariance matrix is ​​a matrix that represents the variance and correlation of data. The diagonal elements store the variance of the features extracted from the data, and the off-diagonal elements store the correlation of the features extracted from the data. For example, a variance-covariance matrix is ​​trained using a training dataset prepared for clustering, and clustering is performed using this variance-covariance matrix.

[0005] A common learning model in clustering is an algorithm for classifying observations within a dataset into groups. However, variance-covariance matrices are sometimes used as part of statistical models to capture patterns of data variation or relationships between variables. Therefore, in the following explanation, the variance-covariance matrix will be referred to as the "model" as appropriate.

[0006] High-precision clustering is possible by training a variance-covariance matrix using a large amount of image data. However, in actual operation, it is not possible to prepare a large amount of image data from the beginning of operation. Therefore, an initial model is generated from a relatively small amount of training images, and a new model is generated later by adding training image data to improve accuracy from the initial stage.

[0007] For example, Non-Patent Document 1 describes a technique that applies a pre-trained general-purpose CNN to anomaly detection using a method called PaDiM (Patch Distribution Modeling). The anomaly detection described in Non-Patent Document 1 assumes that the probability of a sample belonging to a certain class occurring in the feature space follows a normal distribution, and is performed based on the Mahalanobis distance, which takes into account the estimated result of the normal distribution. [Prior art documents] [Non-patent literature]

[0008] [Non-Patent Document 1] Thomas Defard et al., “PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization”, (https: / / arxiv.org / abs / 2011.08785). [Overview of the project] [Problems that the invention aims to solve]

[0009] Conventional techniques, such as those described in Non-Patent Document 1, have the problem that when updating the variance-covariance matrix from the initially set one to a different matrix in order to improve the accuracy of estimation, a large amount of training data must be prepared each time.

[0010] This disclosure aims to solve the above-mentioned problems and to provide a model generation device that can generate different variance-covariance matrices without collecting new training data. [Means for solving the problem]

[0011] The model generation apparatus according to this disclosure comprises: an acquisition unit that acquires a first variance-covariance matrix generated by first training data including multiple image data and a second variance-covariance matrix generated by second training data including multiple image data; and a model combining unit that generates a third variance-covariance matrix by linearly combining the first variance-covariance matrix and the second variance-covariance matrix. The model combination part processes the model so that the column vectors of the second variance-covariance matrix are acute in relation to the column vectors of the first variance-covariance matrix, and then performs a linear combination. . [Effects of the Invention]

[0012] According to this disclosure, a third variance-covariance matrix different from the first variance-covariance matrix is ​​generated by linearly combining the first and second variance-covariance matrices. Therefore, the model generation device according to this disclosure can generate different variance-covariance matrices without collecting new training data. [Brief explanation of the drawing]

[0013] [Figure 1] This is a block diagram showing an example configuration of the inspection apparatus according to Embodiment 1. [Figure 2] This is a block diagram showing an example of the configuration of the model coupling section of the model generation device according to Embodiment 1. [Figure 3] This is a flowchart showing the operation of the inspection device according to Embodiment 1. [Figure 4] This is a flowchart showing the model generation method according to Embodiment 1. [Figure 5] This is a flowchart showing the model linear combination process in Embodiment 1. [Figure 6] Figures 6A and 6B are block diagrams showing the hardware configuration that realizes the functions of the model generation device according to Embodiment 1. [Figure 7]It is a block diagram showing a configuration example of a model generation apparatus according to Embodiment 2. [Figure 8] It is a flowchart showing a model generation method according to Embodiment 2. MODE FOR CARRYING OUT THE INVENTION

[0014] Embodiment 1. (Mathematical formulas used for model generation) First, mathematical formulas used for generating a model (variance-covariance matrix) will be described. An image feature amount x extracted from an image serving as learning data c is represented by the following formula (1). The following formula (1) shows that the image feature amount x c is an N f dimensional real-valued column vector (vertical vector). N f represents the length of a feature amount (the number of dimensions of a feature amount vector), and is the number of feature amounts extracted from image data. The subscript "f" at the lower right of N f is an initial of "feature". N d is the number of pieces of learning data. The subscript "d" at the lower right of N d is an initial of "data". Further, the variable c is an initial of "column" which means a column. For example, when processing is performed on N d images, a loop from c=1 to N d is performed. R represents the set of all real numbers, and N f ×1 indicates that the vector is a column vector having N f elements. TIFF0007915925000001.tif13166

[0015] A data matrix X formed by arranging respective image feature amounts x for a plurality of images belonging to one certain class c | Nd is represented by the following formula (2). Hereinafter, unless otherwise specified, the number of data is sufficient, and Nd >N f That is the case. TIFF0007915925000002.tif20166

[0016] Image features x belonging to the above class c The expected value μ| Nd It is expressed by the following equation (3). In the following equation (3), the function E() represents the expected value. Generally, the expected value and the mean are different concepts, but in this case, the image feature x c The expected value μ| Nd N belongs to the class d Individual image features x c It is equal to the average of (hereinafter referred to as "average feature") of the image feature x. c The expected value μ| Nd is, N f It is a column vector of real values ​​with a given dimension. TIFF0007915925000003.tif31166

[0017] Data matrix X| Nd The variance-covariance matrix Σ| Nd This is the image feature x c The expected value μ| Nd Using this, it is expressed by the following equation (4). The variance-covariance matrix is ​​sometimes simply called the "covariance matrix". In the following equation (4), E(X| Nd X T | Nd ) is the correlation matrix. X| Nd X T | Nd This is the Gram matrix. The superscript "T" indicates the transpose. TIFF0007915925000004.tif23166

[0018] Variance-covariance matrix Σ| Nd By rearranging equation (4) above, the deviation vector (x) can be obtained as shown in equation (5) below. c -μ| Nd It can be expressed using ). TIFF0007915925000005.tif35166

[0019] The deviation vector of column c (x c -μ| Nd ) to y c By setting this, equation (5) above can be transformed into equation (6) below. Equation (6) below is the variance-covariance matrix Σ| Nd This indicates that it is a positive semi-definite matrix. As shown in equation (6) below, matrix Q| Nd and its transpose matrix Q T | Nd By substituting these values, equation (10), which will be described later, can be derived. TIFF0007915925000006.tif36166

[0020] Target sample x target The Mahalanobis distance d is used as an index to show the degree of similarity or outlier between two groups. M There is a Mahalanobis distance d. M This is the variance-covariance matrix Σ| Nd Using this, it is expressed by the following equation (7). In the following equation (7), the Mahalanobis distance d M This is the variance-covariance matrix Σ -1 | Nd When the reciprocal of exists, that is, the variance-covariance matrix Σ| Nd This is a distance that can be defined when is positive definite. The Mahalanobis distance d is expressed by equation (7) below. M The dimension is N f This is the distance defined in the feature space. TIFF0007915925000007.tif23166

[0021] Any matrix can be represented by its singular values ​​and singular vectors. The process of decomposing a matrix into its singular values ​​and singular vectors is called singular value decomposition (SVD). Singular value decomposition allows any p × q matrix Z to be represented as Z = USV TThis is a method of breaking it down into its constituent parts.

[0022] The singular values ​​Z obtained by singular value decomposition of a p×q matrix (Z≠0) are expressed by the following equation (8). In the following equation (8), r is the rank of the singular value Z, and is the number of elements of the singular values ​​that make up the vector singular value Z. The singular value Z is a vector having p×q real-valued elements. Singular values ​​σ1,···,σ r For a diagonal matrix S whose diagonal elements are u1,···,u, the left singular vectors are u1,···,u r This is a matrix U whose elements are v1. The right singular vector for a diagonal matrix S is v1 T ,···,u r T A matrix V whose elements are T That is the case. Note that the left singular vector and the right singular vector together are called singular vectors. TIFF0007915925000008.tif21166

[0023] When a matrix (Z, Z≠0) can be expressed by singular value decomposition as in equation (8) above, its Moore-Penrose generalized inverse (hereinafter simply referred to as "generalized inverse") is given by equation (9) below. In equation (9) below, when Z is an invertible matrix (p=q), the generalized inverse of Z is given by Z - This is the inverse matrix of Z. -1 This is consistent with the statement. While the inverse matrix is ​​defined only for invertible matrices, the generalized inverse matrix is ​​defined for non-zero matrices as well. However, in order to calculate the generalized inverse matrix, the rank r must be known. Since vectors are a type of matrix, the generalized inverse matrix can also be defined for vectors. TIFF0007915925000009.tif36166

[0024] (Basic configuration of the inspection device) Next, the inspection apparatus according to Embodiment 1 will be described. The inspection apparatus according to Embodiment 1 performs an abnormality inspection to determine defects in the object to be inspected from image data. More specifically, the inspection apparatus according to Embodiment 1 extracts image features by performing edge detection, contour extraction, or texture analysis on the image data, and determines defects in the object to be inspected based on the extracted image features.

[0025] Figure 1 is a block diagram showing an example configuration of the inspection device 1 according to Embodiment 1. In Figure 1, the inspection device 1, for example, acquires a model (variance-covariance matrix) from the learning unit 2, updates the model for the purpose of improving the accuracy of clustering, generates a new model as an inspection model, and determines defects in the inspection target from image data based on the inspection model. Furthermore, when the inspection device 1 generates a new model from the model acquired from the learning unit 2, it can generate a model different from the model acquired from the learning unit 2 without having to collect new training data.

[0026] (Learning Department) Learning unit 2 uses the training data to perform singular value decomposition to obtain the variance-covariance matrix Σ| Nd We find the variance-covariance matrix Σ| represented by equation (6) above. Nd Since this is a positive semi-definite matrix, the variance-covariance matrix Σ| Nd This can be expressed by the following equation (10). In the following equation (10), Q| Nd This is a matrix representing the transformation of the original data, and the column vectors of this matrix correspond to the basis vectors of the original data space. The variance-covariance matrix Σ| Nd is matrix Q| Nd And its transpose matrix Q T | Nd It is calculated by the product of the two. TIFF0007915925000010.tif9166

[0027] Matrix Q| in equation (10) above Nd By performing singular value decomposition on the matrix Q| Ndis represented by the following formula (11). In the following formula (11), the singular values are arranged in descending order of magnitude, such that σ1≧σ2≧···≧σ NF holds. In general, sigma (especially "σ 2 ") is often used as a symbol representing variance, but in the present description, sigma represents a singular value. TIFF0007915925000011.tif19166

[0028] From the properties of singular vectors, the matrix U| Nd and the matrix V| Nd satisfy the relationship of the following formula (12) in the above formula (11). In the following formula (12), I is an N f ×N f identity matrix. TIFF0007915925000012.tif19166

[0029] By substituting the singular value decomposition represented by the above formula (11) into the above formula (10), the variance-covariance matrix Σ| Nd is represented by the following formula (13). In the following formula (13), (σ1) 2 | Nd , which is a diagonal element of the matrix S 2 , (σ2) 2 , ···, (σ Nf ) 2 are respectively singular values of the variance-covariance matrix Σ| Nd the variance-covariance matrix Σ| TIFF0007915925000013.tif21166

[0030] Based on the property represented by the above formula (12) and the above formula (13), the inverse matrix Σ Nd of the variance-covariance matrix Σ| -1 | Nd is represented by the following formula (14). As described above, when the matrix is a regular matrix, the generalized inverse matrix matches the inverse matrix. Hereinafter, a form in which a matrix is decomposed and expressed as a product of matrices is referred to as a "decomposition form". Singular value decomposition is one type of decomposition form, and is a special form thereof. The right-hand side of the following formula (14) is also a singular value decomposition. TIFF0007915925000014.tif30166

[0031] As described above, the learning unit 2 generates a first variance-covariance matrix A and a second variance-covariance matrix B, and outputs the generated first variance-covariance matrix A and second variance-covariance matrix B to the inspection apparatus 1. Here, the first variance-covariance matrix A is a matrix generated based on first learning data. The second variance-covariance matrix B is a matrix generated based on second learning data that is different from the first learning data.

[0032] As shown in FIG. 1, the inspection apparatus 1 includes a model generation apparatus 11, an evaluation unit 12, and a determination unit 13. The model generation apparatus 11 linearly combines the first variance-covariance matrix A and the second variance-covariance matrix B to generate a new third variance-covariance matrix W that is different from both the matrices A and B.

[0033] (Evaluation Unit) The evaluation unit 12 calculates an evaluation index for the third variance-covariance matrix W. The evaluation index is for a sample x target which is an index indicating the degree of similarity or outlierness. For example, the evaluation unit 12 calculates, as the similarity, a Mahalanobis distance d M for an inspection target. The Mahalanobis distance d M is an index indicating the degree of similarity or outlierness of a given sample x target . The Mahalanobis distance d M can take correlation into account through the variance-covariance matrix, so distance calculation based on the distribution of actual data is possible. Note that, instead of the Mahalanobis distance d M , d obtained by squaring the Mahalanobis distance d represented by the following formula (15) M may be used as d M 2 . In the following formula (15), (x target -μ| Nd ) is a sample y target , and the transpose thereof is (x target -μ| Nd )T That is. U| Nd S -1 | Nd U T | Nd is Σ -1 | Nd That is the case. TIFF0007915925000015.tif10166

[0034] The target can be represented simply as a sample feature, with the mean feature μ set to zero. In this case, x target =y target That is the case. Also, target y target This is the deviation vector (y) defined as the deviation from the mean feature μ. target =x target It can also be expressed as -μ). Furthermore, the average feature vector μ may be constructed by a linear combination, similar to the variance-covariance matrix W. In this case, the ratio of p and q, which are the parameters for the linear combination of the average feature vector μ, may be the same as or different from that of the variance-covariance matrix W. In addition, in the evaluation of the variance-covariance matrix W in Embodiment 2 described later, the mean feature quantity μ may be set to zero, or a value calculated by linear combination may be used.

[0035] (Judgment Department) The determination unit 13 determines whether the object to be inspected is defective or not based on the similarity of the images to be inspected calculated by the evaluation unit 12 based on the third variance-covariance matrix W. For example, the similarity of images is calculated using the Mahalanobis distance d, which is calculated using the variance-covariance matrix W of the normal image data set generated by the model generation device 11. M Let it be so. The determination unit 13 determines the Mahalanobis distance d M Whether or not the object being inspected is defective is determined based on whether or not it exceeds a threshold. Typically, the threshold is set based on a normal distribution, for example, using the 95% confidence interval of a group of normal image data. Mahalanobis distance d MIf the value is below the threshold, the subject of the test is determined to be normal, and the Mahalanobis distance d M If the value exceeds the threshold, the item being inspected is determined to be defective.

[0036] The information indicating the inspection result from the determination unit 13 is output to an external device. For example, if the external device is a display device, the determination unit 13 generates display information for displaying the inspection result and outputs the generated display information to the display device. The display device displays the inspection result based on the display information.

[0037] Although Figure 1 shows the evaluation unit 12 and determination unit 13 located outside the model generation device 11, these may also be included within the model generation device 11. Furthermore, although the example shows the learning unit 2 located outside the inspection device 1, the learning unit 2 may also be included within the inspection device 1.

[0038] (Basic configuration of the model generation system) As shown in Figure 1, the model generation device 11 comprises an acquisition unit 111, a model merging unit 112, and a setting unit 113. For example, the model generation device 11 is implemented by a computer. The computer's memory stores programs that constitute information processing applications for realizing the functions of the acquisition unit 111, the model merging unit 112, and the setting unit 113. The computer's processor executes the information processing applications read from the memory, thereby realizing the functions of the acquisition unit 111, the model merging unit 112, and the setting unit 113.

[0039] (Acquisition Department) The acquisition unit 111 acquires the first variance-covariance matrix A and the second variance-covariance matrix B. The first variance-covariance matrix A is the variance-covariance matrix generated by the first training data. The first training data is a dataset N containing multiple image data of the subject being examined. A Furthermore, the second variance-covariance matrix B is the variance-covariance matrix generated by the second training data. The second training data is a data set N containing multiple image data of the subject being examined.A Data set N is different from B That is the case.

[0040] For example, if the inspection device 1 includes a communication unit (not shown in Figure 1), the acquisition unit 111 accesses the learning unit 2 via the communication unit and acquires the first variance-covariance matrix A and the second variance-covariance matrix B from the learning unit 2. Furthermore, if the learning unit 2 stores the first variance-covariance matrix A and the second variance-covariance matrix B in a storage device provided separately from the inspection device 1, the acquisition unit 111 may access the storage device via the communication unit to acquire the first variance-covariance matrix A and the second variance-covariance matrix B.

[0041] When the learning unit 2 stores the first variance-covariance matrix A and the second variance-covariance matrix B in a storage device provided by the inspection device 1 or the model generation device 11, the acquisition unit 111 may access the storage device to acquire the first variance-covariance matrix A and the second variance-covariance matrix B. In this case, the inspection device 1 and the model generation device 11 do not need to be equipped with the communication unit described above.

[0042] (Settings section) The setting unit 113 sets the ratios p and q of the linear combination to the model combination unit 112. For example, the setting unit 113 sets the ratios p and q specified by the user using an input device (not shown in Figure 1) in the model coupling unit 112. Alternatively, the setting unit 113 may set the ratios p and q specified by an external device in the model coupling unit 112. For example, if the determination result of whether or not there is a defect in the inspection target does not match the actual state of the inspection target, the setting unit 113 may set the ratios p and q specified by the inspection device 1 in the model coupling unit 112. By providing the setting unit 113 in this way, it is possible to set the ratios p and q of the linear combination to various values.

[0043] Furthermore, if the ratios p and q of the linear combination are pre-set in the model combination unit 112, the setting unit 113 is unnecessary. In this case, the model generation device 11 does not need to have the setting unit 113. That is, the model generation device 11 only needs to have at least the acquisition unit 111 and the model combination unit 112.

[0044] (Model connection part) The model coupling unit 112 generates a third variance-covariance matrix W by linearly combining the first variance-covariance matrix A and the second variance-covariance matrix B. For example, in addition to the first variance-covariance matrix A and the second variance-covariance matrix B, the model coupling unit 112 is set to ratios p and q, which are parameters related to their linear combination. The model coupling unit 112 performs a linear combination of the first variance-covariance matrix A and the second variance-covariance matrix B according to the following equation (16). In equation (16) below, the ratio p is the ratio of the linear combinations to the first variance-covariance matrix A, and the ratio q is the ratio of the linear combinations to the second variance-covariance matrix B. TIFF0007915925000016.tif8166

[0045] The ratio p or q of a linear combination can take any value within the real number range. In other words, p or q can take negative values. Furthermore, the ratio p or q of the linear combination may be set such that the value of p+q is 1. In the following case, where p+q is 1, the ratio q is 1-p. Furthermore, the ratio p is called the doping / extrapolation ratio, and the range between 0 and 1 is called doping, while the range outside this range is called extrapolation. Doping and extrapolation can also be collectively called doping / extrapolation.

[0046] Here, the second variance-covariance matrix B is generated by the learning unit 2 using the second training data N. BBased on this, the first variance-covariance matrix A is learned and updated. The setting unit 113 uses information about the pre-update model (first variance-covariance matrix A) and the updated model (second variance-covariance matrix B) to set the extrapolation ratio p, which is a parameter, in the model coupling unit 112. This allows the model coupling unit 112 to generate a new feature distribution without having to obtain it from the training data.

[0047] (Dimensionality reduction of the variance-covariance matrix) As described above, the first variance-covariance matrix A and the second variance-covariance matrix B calculated by singular value decomposition are highly likely to contain unnecessary components such as noise in the ranks of the latter half of the singular values. Here, the ranks of the latter half of the singular values ​​refer to the components of the variance-covariance matrix that are arranged in order of magnitude, and whose latter half singular values ​​are small. The learning unit 2 outputs the fourth variance-covariance matrix generated by singular value decomposition based on the first training data to the model generation device 11, and outputs the fifth variance-covariance matrix generated by singular value decomposition based on the second training data to the model generation device 11.

[0048] The model connector 112 performs dimensionality reduction by removing the ranks of the latter half of the singular values ​​that constitute the fourth variance-covariance matrix, and treats the resulting matrix as the first variance-covariance matrix A. Similarly, the model connector 112 performs dimensionality reduction by removing the ranks of the latter half of the singular values ​​that constitute the fifth variance-covariance matrix, and treats the resulting matrix as the second variance-covariance matrix B.

[0049] The model coupling unit 112 generates a third variance-covariance matrix W by linearly combining the first variance-covariance matrix A and the second variance-covariance matrix B. Thus, by using a variance-covariance matrix from which noise and other unwanted elements have been removed, it is possible to generate a high-quality third variance-covariance matrix W.

[0050] Furthermore, although the case where the model coupling unit 112 performs dimensionality reduction was shown, dimensionality reduction may also be performed by the acquisition unit 111 or the learning unit 2. In this case as well, a variance-covariance matrix from which noise and other factors have been removed can be used, and a high-quality third variance-covariance matrix W can be generated.

[0051] (The angle between the column vectors is acute.) For example, when performing a linear combination of variance-covariance matrices using the relation shown in equation (16) above, the angle between the column vectors of the first variance-covariance matrix A and the column vectors of the second variance-covariance matrix B can be either acute or obtuse. This is because it is uncertain whether the direction of the column vectors obtained by singular value decomposition is reversed by 180 degrees. In particular, if the angle between the column vectors of the first variance-covariance matrix A and the column vectors of the second variance-covariance matrix B is obtuse, the characteristics of the first variance-covariance matrix A and the second variance-covariance matrix B are lost through the linear combination.

[0052] Therefore, it is necessary to adjust the second variance-covariance matrix B so that the angle between the column vectors of the first variance-covariance matrix A and the column vectors of the second variance-covariance matrix B is acute. Therefore, the model coupling unit 112 reverses the direction of the column vectors of the variance-covariance matrices so that the angle between the column vectors of the first variance-covariance matrix A and the column vectors of the second variance-covariance matrix B is acute, and then performs a linear combination.

[0053] Figure 2 is a block diagram showing an example configuration of the model coupling unit 112 of the model generation device 11. As shown in Figure 2, the model coupling unit 112 comprises an inner product calculation unit 1121, a sign inversion unit 1122, and a linear combination unit 1123. The inner product calculation unit 1121 calculates the inner product of the column vectors of the two input variance-covariance matrices. The sign inversion unit 1122 reverses the direction of the column vectors of the input variance-covariance matrices if the angle between the column vectors of the two variance-covariance matrices is obtuse. The linear combination unit 1123 performs a linear combination of the two input variance-covariance matrices and outputs a variance-covariance matrix which is the result of the linear combination.

[0054] For example, the dot product calculation unit 1121 calculates the column vector a of the first variance-covariance matrix A according to the following equation (17). n and the column vector b of the second variance-covariance matrix B n The inner product d n Calculate. In the following equation (17), a n ' is a column vector of the first variance-covariance matrix A' after passing through the sign inversion section 1122. The column vectors of the first variance-covariance matrix A' are a1',···,a as shown in equation (17) below. n ' is an element. TIFF0007915925000017.tif31166

[0055] dot product d n If it is negative, the sign inversion unit 1122 takes one of the variance-covariance matrices input to the model coupling unit 112, in this case the column vector a of the first variance-covariance matrix A. n The sign indicating the direction is reversed (a n '=-a n ) and output it as the first variance-covariance matrix A'. On the other hand, the inner product d n If a is not negative, the sign inversion unit 1122 outputs the input first variance-covariance matrix A as the first variance-covariance matrix A'. In this case, a n '=a n This is the result.

[0056] The linear combination unit 1123 outputs the third variance-covariance matrix W by linearly combining the first variance-covariance matrix A' and the second variance-covariance matrix B according to equation (18) below. This allows us to generate a feature distribution space that includes the components of both the first variance-covariance matrix A and the second variance-covariance matrix B, without losing any of their features. Furthermore, under these conditions, the component contributions of the first variance-covariance matrix A and the second variance-covariance matrix B can be adjusted by the extrapolation ratio. TIFF0007915925000018.tif9166

[0057] Next, the operation of the inspection device 1 will be explained. Figure 3 is a flowchart illustrating the operation of the inspection device 1, showing a series of processes for inspecting the object to be inspected by the inspection device 1. The model generation device 11 generates a new third variance-covariance matrix W, which is different from the first variance-covariance matrix A and the second variance-covariance matrix B, by linearly combining the two matrices A and B (step ST1). For example, the model generation device 11 obtains the first variance-covariance matrix A and the second variance-covariance matrix B from the learning unit 2, and then calculates the third variance-covariance matrix W according to equation (16) above, using the set ratios p and q of the linear combination.

[0058] The evaluation unit 12 calculates an evaluation index for the third variance-covariance matrix W generated by the model generation device 11 (step ST2). For example, the evaluation unit 12 calculates the sample x according to the above formula (7). target The Mahalanobis distance d indicates the similarity between the two. M Calculate.

[0059] The determination unit 13 determines whether the object to be inspected is defective or not based on the similarity calculated by the evaluation unit 12 (step ST3). For example, if the similarity of the images is the Mahalanobis distance d M If this is the case, the determination unit 13 determines the Mahalanobis distance d M If the value is below the threshold, the subject of the test is determined to be normal, and the Mahalanobis distance d M If the threshold is exceeded, the inspected item is determined to be defective. This allows the inspection device 1 to generate different variance-covariance matrices without collecting new training data, and to perform inspections based on evaluation information of the variance-covariance matrices.

[0060] (Model generation method) Figure 4 is a flowchart showing the model generation method according to Embodiment 1. The acquisition unit 111 acquires the first variance-covariance matrix A and the second variance-covariance matrix B (step ST11). Here, the first variance-covariance matrix A is obtained by the learning unit 2 from the first training data N AThis is a matrix (model) learned using the second variance-covariance matrix B, which is obtained by the learning unit 2 from the second training data N. B This is a matrix (model) that was trained using [the specified method / tool].

[0061] The model coupling unit 112 generates a third variance-covariance matrix W by linearly combining the first variance-covariance matrix A and the second variance-covariance matrix B (step ST12). The third variance-covariance matrix W is a matrix (model) that is different from both the first variance-covariance matrix A and the second variance-covariance matrix B. By performing the above method, the model generation device 11 can generate a third variance-covariance matrix W, which is a different variance-covariance matrix, without collecting new training data.

[0062] Figure 5 is a flowchart illustrating the model linear combination process, showing the detailed processing of step ST12 in Figure 4. As shown in Figure 5, the processing of step ST12 shown in Figure 4 can be divided into steps ST121 to ST125, which are the processes performed by the model combination unit 112 in Figure 2.

[0063] The dot product calculation unit 1121 calculates the column vector a of the first variance-covariance matrix A. n and the column vector b of the second variance-covariance matrix B n The inner product d n The dot product calculation unit 1121 calculates the dot product d. n The information indicating this, along with the first variance-covariance matrix A, is output to the sign inversion unit 1122.

[0064] Next, the sign inversion unit 1122 performs the inner product d n Check whether it is negative or not (step ST122). Inner product d n If it is negative (step ST122; YES), the sign inversion unit 1122 sets the column vector a of the first variance-covariance matrix A. n The column vector a with the sign indicating the direction reversed. n (Step ST123) dot product d nIf the value is not negative (step ST122; NO), the sign inversion unit 1122 outputs the column vector a of the input first variance-covariance matrix A. n As is, column vector a n Let's assume that.

[0065] The dot product calculation unit 1121 checks whether the processing in steps ST121 and ST122 has been completed for all column vectors (step ST124). If the processing has not been completed for all column vectors (step ST124; NO), the unit returns to the processing in step ST121 and repeats the processing in steps ST121 and ST122 for the remaining column vectors.

[0066] If processing is completed for all column vectors (step ST124; YES), the linear combination unit 1123 outputs the third variance-covariance matrix W by linearly combining the first variance-covariance matrix A' and the second variance-covariance matrix B (step ST125). The model coupling unit 112 can generate a feature distribution space that includes the components of both variance-covariance matrices without losing any features of the first variance-covariance matrix A and the second variance-covariance matrix B, by performing the series of processes described above.

[0067] Next, we will describe the hardware configuration that realizes the functions of the model generation device 11. The functions of the model generation device 11—the acquisition unit 111, the model merging unit 112, and the setting unit 113—are realized by a processing circuit. That is, the model generation device 11 includes a processing circuit for executing the processes from step ST11 to step ST12 shown in Figure 4. The processing circuit may be dedicated hardware, or it may be a CPU (Central Processing Unit) that executes a program stored in memory.

[0068] Figure 6A is a block diagram showing the hardware configuration for realizing the functions of the model generation device 11. Figure 6B is a block diagram showing the hardware configuration for executing the software that realizes the functions of the model generation device 11. In Figures 6A and 6B, the acquisition unit 111 acquires model A and model B from the learning unit 2 via the input interface 100. The setting unit 113 may acquire the ratio p of the linear combination from outside the device via the input interface 100. The model combination unit 112 outputs the third variance-covariance matrix W to the outside of the device via the output interface 101.

[0069] When the processing circuit is a dedicated hardware processing circuit 102 as shown in Figure 6A, the processing circuit 102 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof. The acquisition unit 111, the model coupling unit 112, and the setting unit 113 of the model generation device 11 may be implemented by separate processing circuits, or these functions may be implemented together by a single processing circuit.

[0070] When the processing circuit is the processor 103 shown in Figure 6B, the functions of the acquisition unit 111, model coupling unit 112, and setting unit 113 of the model generation device 11 are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in memory 104.

[0071] The processor 103 reads and executes the program stored in the memory 104, thereby realizing the functions of the acquisition unit 111, the model merging unit 112, and the setting unit 113 of the model generation device 11. For example, the model generation device 11 includes a memory 104 for storing a program in which the processes from step ST11 to step ST12 shown in Figure 4 are ultimately executed when implemented by the processor 103. These programs cause the computer to execute the procedures or methods of processing performed by the acquisition unit 111, the model merging unit 112, and the setting unit 113. The memory 104 may be a computer-readable storage medium on which programs for functioning as the acquisition unit 111, the model matching unit 112, and the setting unit 113 are stored.

[0072] Memory 104 includes, for example, non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), and EEPROM (Electrically-EPROM) (registered trademark), as well as magnetic disks, flexible disks, optical disks, compact disks, minidiscs, and DVDs.

[0073] Some of the functions of the acquisition unit 111, model merging unit 112, and setting unit 113 of the model generation device 11 may be implemented by dedicated hardware, while other parts may be implemented by software or firmware. For example, the functions of the acquisition unit 111 and the setting unit 113 may be implemented by a processing circuit 102, which is dedicated hardware, and the function of the model merging unit 112 may be implemented by a processor 103 reading and executing a program stored in memory 104. In this way, the processing circuit can implement the above functions by hardware, software, firmware, or a combination thereof.

[0074] As described above, the model generation device 11 according to Embodiment 1 includes an acquisition unit 111 that acquires a first variance-covariance matrix A and a second variance-covariance matrix B, and a model combining unit 112 that generates a third variance-covariance matrix W by linearly combining the first variance-covariance matrix A and the second variance-covariance matrix. Since the third variance-covariance matrix W, which is different from both matrices A and B, is generated by linearly combining the first variance-covariance matrix A and the second variance-covariance matrix B, the model generation device 11 can generate different variance-covariance matrices without collecting new training data.

[0075] In the model generation device 11 according to Embodiment 1, the model coupling unit 112 processes the column vectors of the second variance-covariance matrix B so that the column vectors of the first variance-covariance matrix A and the column vectors of the second variance-covariance matrix B form an acute angle, and then performs a linear combination. As a result, the model generation device 11 can generate a feature distribution space that includes the components of both variance-covariance matrices without losing the features of the first variance-covariance matrix A and the second variance-covariance matrix B.

[0076] The model generation device 11 according to Embodiment 1 includes a setting unit 113 for setting the ratios p and q of the linear combination. This makes it possible for the model generation device 11 to set the ratios p and q of the linear combination to various values.

[0077] In the model generation device 11 according to Embodiment 1, the first variance-covariance matrix A is a matrix obtained by dimensionality reduction of the fourth variance-covariance matrix generated by singular value decomposition based on the first training data. The second variance-covariance matrix B is a matrix obtained by dimensionality reduction of the fifth variance-covariance matrix generated by singular value decomposition based on the second training data. By using a variance-covariance matrix from which noise and other imperfections have been removed, it is possible to generate a high-quality third variance-covariance matrix W.

[0078] In the model generation device 11 according to Embodiment 1, the second variance-covariance matrix B is a matrix obtained by learning and updating the first variance-covariance matrix A based on the second training data. As a result, the model generation device 11 can generate a new feature distribution without obtaining it from the training data.

[0079] The inspection device 1 according to Embodiment 1 comprises a model generation device 11, an evaluation unit 12, and a determination unit 13. This makes it possible to provide an inspection device 1 that can generate different variance-covariance matrices without collecting new training data.

[0080] In the inspection device 1 according to Embodiment 1, the evaluation unit 12 uses the Mahalanobis distance d of the object to be inspected as the similarity. M Calculate the Mahalanobis distance d. M Since correlation can be considered through the variance-covariance matrix, distance calculations based on the actual data distribution are possible.

[0081] The model generation method according to Embodiment 1 includes the following steps: an acquisition unit 111 acquires a first variance-covariance matrix generated from first training data containing multiple image data, and a second variance-covariance matrix generated from second training data containing multiple image data (step ST11); and a model combining unit 112 generates a third variance-covariance matrix by linearly combining the first variance-covariance matrix and the second variance-covariance matrix (step ST12). By executing the above method, the model generation device 11 can generate different variance-covariance matrices without collecting new training data.

[0082] A computer executing the program according to Embodiment 1 performs the following processes: obtaining a first variance-covariance matrix generated from first training data containing multiple image data, and a second variance-covariance matrix generated from second training data containing multiple image data; and generating a third variance-covariance matrix by linearly combining the first and second variance-covariance matrices. By having the computer execute the above program, different variance-covariance matrices can be generated without collecting new training data.

[0083] Embodiment 2. The model generation device according to Embodiment 2 comprehensively generates a variance-covariance matrix (model) for a variable p of any interval and timeframe, and searches for the optimal model using the test data set. As a result, in addition to the effects described in Embodiment 1, the model generation device according to Embodiment 2 can adaptively create a model that reduces degradation and over-detection.

[0084] Figure 7 is a block diagram showing an example configuration of the model generation device 11A according to Embodiment 2. In Figure 7, the model generation device 11A comprehensively searches for the extrapolation ratio p in the first variance-covariance matrix A and the second variance-covariance matrix B to generate the optimal third variance-covariance matrix W best The system generates a third variance-covariance matrix W, which is a better evaluation metric for the first variance-covariance matrix A and the second variance-covariance matrix B obtained through unsupervised learning, using a set of inspection data containing both good and defective products. best It is possible to generate it. Furthermore, the inspection device 1 according to Embodiment 2 includes a model generation device 11A. Furthermore, the inspection device 1 according to Embodiment 2 may include an evaluation unit 114 of the model generation device 11A instead of the evaluation unit 12.

[0085] As an evaluation metric, you may use AUROC (Area Under the ROC Curve) or the number of false positives. Other evaluation metrics may also be used. This function is intended for situations where degradation occurs after updating from the first variance-covariance matrix A to the second variance-covariance matrix B, or when the number of false positives does not improve under the condition that there are zero missed detections.

[0086] As shown in Figure 7, the model generation device 11A includes an acquisition unit 111, a model coupling unit 112, a setting unit 113, an evaluation unit 114, and an output unit 115. For example, the model generation device 11A is implemented by a computer. The computer's memory stores programs that constitute information processing applications for realizing the functions of the acquisition unit 111, the model coupling unit 112, the setting unit 113, the evaluation unit 114, and the output unit 115. The computer's processor executes the information processing applications read from the memory, thereby realizing the functions of the acquisition unit 111, the model coupling unit 112, the setting unit 113, the evaluation unit 114, and the output unit 115.

[0087] (Acquisition Department) The acquisition unit 111 acquires the first variance-covariance matrix A and the second variance-covariance matrix B. For example, if the model generation device 11A includes a communication unit (not shown in Figure 7), the acquisition unit 111 accesses the learning unit 2 via the communication unit and acquires the first variance-covariance matrix A and the second variance-covariance matrix B from the learning unit 2. Furthermore, if the learning unit 2 stores the first variance-covariance matrix A and the second variance-covariance matrix B in a storage device provided separately from the model generation device 11A, the acquisition unit 111 may access the storage device via the communication unit to acquire the first variance-covariance matrix A and the second variance-covariance matrix B.

[0088] When the learning unit 2 stores the first variance-covariance matrix A and the second variance-covariance matrix B in the storage device provided by the model generation device 11A, the acquisition unit 111 may access the storage device to acquire the first variance-covariance matrix A and the second variance-covariance matrix B. In this case, the model generation device 11A does not need to be equipped with the communication unit described above.

[0089] The display device 3 displays information output from the model generation device 11A. For example, the display device 3 displays information indicating the evaluation index of the third variance-covariance matrix W calculated by the evaluation unit 114. By referring to the evaluation index displayed on the display device 3, the user can decide which linear combination ratios p and q to use for the linear combination. Alternatively, the model combination unit 112 may automatically determine the linear combination ratios p and q that yield the best evaluation index values. The display device 3 may be a display unit provided by the inspection device 1 or the model generation device 11A, or it may be a display unit provided by a terminal that is connected to the inspection device 1 or the model generation device 11A via a network.

[0090] (Settings section) The setting unit 113 sets the ratios p and q of the linear combination to the model combination unit 112. For example, the setting unit 113 sets the ratios p and q specified by the user using an input device (not shown in Figure 7) in the model coupling unit 112. Alternatively, the setting unit 113 may set the ratios p and q specified by an external device in the model coupling unit 112. For example, if the determination result of whether or not there is a defect in the inspected object does not match the actual state of the inspected object, the setting unit 113 may set the specified ratios p and q in the model coupling unit 112. By providing the setting unit 113 in this way, it is possible to set the ratios p and q of the linear combination to various values.

[0091] Furthermore, if the ratios p and q of the linear combination are pre-set in the model combination unit 112, the setting unit 113 is unnecessary. In this case, the model generation device 11A does not need to have the setting unit 113. That is, the model generation device 11A only needs to have the acquisition unit 111, the model combination unit 112, the evaluation unit 114, and the output unit 115.

[0092] (Model connection part) The model coupling unit 112 generates a third variance-covariance matrix W by linearly combining the first variance-covariance matrix A and the second variance-covariance matrix B. For example, in addition to the first variance-covariance matrix A and the second variance-covariance matrix B, the model coupling unit 112 is set to ratios p and q, which are parameters related to their linear combination. The model coupling unit 112 performs a linear combination of the first variance-covariance matrix A and the second variance-covariance matrix B according to equation (16) above.

[0093] (Evaluation Department) The evaluation unit 114 calculates an evaluation index for the third variance-covariance matrix W. For example, the evaluation unit 114 uses the Mahalanobis distance d of the subject under test as an evaluation index for the third variance-covariance matrix W. M The determination unit 13 calculates the Mahalanobis distance d. M Based on this, it is possible to determine whether or not the object being inspected has a defect.

[0094] Furthermore, the evaluation unit 114 may calculate an evaluation index for the third variance-covariance matrix W, in which the linear combination ratios p and q have been changed. For example, the model combination unit 112 is set with the first variance-covariance matrix A and the second variance-covariance matrix B obtained from the learning unit 2 by the acquisition unit 111, and the interval and spacing to which the extrapolation ratio p is applied are set by the setting unit 113. The first variance-covariance matrix A and the second variance-covariance matrix B may be generated by the learning unit 2 and stored in a storage unit of the model generation device 11A, which is not shown in Figure 7.

[0095] The first variance-covariance matrix A is given by the training data set N A This is the variance-covariance matrix generated by learning using the following method: the second variance-covariance matrix B is the training data set N. B This is the variance-covariance matrix generated by learning using [the specified method]. The training dataset N A For example, the number of data points A This is a set of data, and the training data set N B The number of data A The number of data points to which image data has been added from the data set. B This is a set of data.

[0096] The model coupling unit 112 comprehensively performs the linear combination process shown in Figure 5 on the first variance-covariance matrix A and the second variance-covariance matrix B stored in the memory unit, for all cases of the extrapolation ratio p. The evaluation unit 114 evaluates the third variance-covariance matrix W obtained by the linear combination by the model coupling unit 112. p The following is evaluated sequentially. At this time, the third variance-covariance matrix W p This is temporarily stored in the memory unit mentioned above. On the other hand, if the extrapolation ratio p of the search target is large, it puts a strain on the memory capacity of the memory unit, so after evaluation by the evaluation unit 114, the third variance-covariance matrix W p The data may be discarded from the above-mentioned storage unit.

[0097] The model coupling unit 112 determines the maximum interpolation ratio p calculated by the evaluation unit 114. best The information is stored in the above memory unit, and after the search is complete, the interpolation ratio p with the highest evaluation value is selected. best The third variance-covariance matrix W best It may be regenerated. Also, the model coupling part 112 is the third variance-covariance matrix W that had the best evaluation metric. best Only the above memory may be temporarily stored. The third variance-covariance matrix W of the best evaluation metric is ultimately used. best The above information is read from the memory unit.

[0098] (Output section) The output unit 115 outputs an evaluation index to the display device 3 for the third variance-covariance matrix W, in which the ratio p of the linear combination has been changed. For example, the evaluation unit 114 outputs the evaluation index for the third variance-covariance matrix W best To obtain this, the third variance-covariance matrix W generated by the model coupling unit 112 p The evaluation is performed comprehensively. At this time, the evaluation unit 114 outputs evaluation indicators such as AUROC or the number of false positives for the ratios p and q of the linear combination to the output unit 115. The output unit 115 generates display information for displaying the evaluation indicators and outputs it to the display device 3. The display device 3 displays the evaluation indicators based on the display information. This allows the evaluation indicators to be presented to the user via the screen of the display device 3.

[0099] Depending on the models to be linearly combined and the ratios p and q of the linear combinations, it may be possible to generate a model (variance-covariance matrix) that yields better evaluation metrics for the first variance-covariance matrix A and the second variance-covariance matrix B before linear combination, or it may generate a model that yields worse evaluation metrics. By referring to the evaluation metrics displayed on the display device 3, the user can decide which ratios p and q to use for linear combination. Alternatively, the model combination unit 112 may automatically determine the ratios p and q of the linear combination that yield the best evaluation metric values.

[0100] The evaluation unit 114 evaluates whether the third variance-covariance matrix W is degraded with respect to the first variance-covariance matrix A or the second variance-covariance matrix B. For example, the determination unit 13 determines whether the object to be inspected is defective based on the first variance-covariance matrix A generated using the correct image data. The number of times the determination unit 13 has determined the object to be inspected to be defective, and the image data for which the object to be inspected has been determined to be defective, are stored in a storage unit (not shown in Figure 7) provided by the model generation device 11A.

[0101] Next, the determination unit 13 determines whether the object to be inspected is defective or not based on the second variance-covariance matrix B generated using the correct image data. Similarly, the number of times the determination unit 13 has determined the object to be inspected to be defective, and the image data for which the object to be inspected has been determined to be defective, are stored in a storage unit (not shown in Figure 7) provided by the model generation device 11A. The second variance-covariance matrix B is derived from the second training data N. B Based on this, the first variance-covariance matrix A is learned and updated.

[0102] Alternatively, the determination unit 13 may determine whether the object to be inspected is defective based on a third variance-covariance matrix W obtained by linearly combining the first variance-covariance matrix A and the second variance-covariance matrix B, which are generated using the correct image data. In this case as well, the number of times the determination unit 13 has determined the object to be inspected to be defective, and the image data for which the object to be inspected has been determined to be defective, are stored in a storage unit (not shown in Figure 7) provided by the model generation device 11A.

[0103] The evaluation unit 114 compares the number of image data determined to be defective in the first variance-covariance matrix A and the second variance-covariance matrix B based on the judgment results stored in the memory unit. At this time, if the number of image data determined to be defective in the second variance-covariance matrix B is greater than that in the first variance-covariance matrix A, the evaluation unit 114 evaluates that the third variance-covariance matrix W has degraded with respect to either the first variance-covariance matrix A or the second variance-covariance matrix B.

[0104] Furthermore, if the number of image data determined to be good products with respect to the second variance-covariance matrix B is greater than that of the first variance-covariance matrix A, and the determination unit 13 only determines whether the second variance-covariance matrix B is defective or not, the evaluation unit 114 may evaluate that the third variance-covariance matrix W has not degraded with respect to the first variance-covariance matrix A or the second variance-covariance matrix B. In this case, if there is new image data of the product to be inspected that is determined to be defective with respect to the second variance-covariance matrix B, the evaluation unit 114 may evaluate that a degradation has occurred even if the number of image data of the product to be inspected that is determined to be defective decreases.

[0105] Next, a model generation method according to Embodiment 2 will be described. Figure 8 is a flowchart showing the model generation method according to Embodiment 2, and the third variance-covariance matrix W is generated by the model generation device 11A. best This shows a series of steps to obtain [the result]. The acquisition unit 111 acquires the first variance-covariance matrix A and the second variance-covariance matrix B (step ST1A). The first variance-covariance matrix A is obtained by the learning unit 2 from the first training data N AThis is a matrix (model) learned using the second variance-covariance matrix B, which is obtained by the learning unit 2 from the second training data N. B This is a matrix (model) that was trained using [the specified method / tool].

[0106] The model coupling unit 112 generates a third variance-covariance matrix W by linearly combining the first variance-covariance matrix A and the second variance-covariance matrix B using the linear combination ratios p and q set by the setting unit 113 (step ST2A). The third variance-covariance matrix W is a matrix (model) that is different from both the first variance-covariance matrix A and the second variance-covariance matrix B.

[0107] The evaluation unit 114 calculates the third variance-covariance matrix W using the ratios p and q of the linear combination by the model coupling unit 112. p The evaluation metrics are calculated (Step ST3A). For example, AUROC or false positives may be calculated as evaluation metrics.

[0108] The model coupling unit 112 checks whether the processing in steps ST2A and ST3A has been completed for all p search ranges (step ST4A). If the processing has not been completed for all p search ranges (step ST4A; NO), the unit returns to the process of step ST2A and repeats the processing of steps ST2A and ST3A for the remaining p search ranges.

[0109] If processing is completed for all p search ranges (step ST4A; YES), the evaluation unit 114 outputs evaluation indicators such as AUROC or the number of false positives for the ratios p and q of the linear combination to the output unit 115. The output unit 115 generates display information for displaying the evaluation indicators and outputs it to the display device 3. The display device 3 displays the evaluation indicators based on the display information (step ST5A). For example, the progress of the evaluation indicators may be visually displayed by plotting, or the ratios p and q of the linear combinations corresponding to the best scores of the evaluation indicators may be presented.

[0110] Next, the model to be adopted is determined by referring to the display on the display device 3 (step ST6A). For example, the third variance-covariance matrix W displayed by the display device 3. p Based on the changes in the evaluation index, the third variance-covariance matrix W p From among them, the third variance-covariance matrix W corresponding to the best score of the evaluation metric is selected. best However, it was decided to adopt it as the model for adoption.

[0111] For example, using an input device (not shown in Figure 7), the user selects the ratios p and q of the linear combination corresponding to the adopted model, and the corresponding third variance-covariance matrix W. p The third variance-covariance matrix W best This will be decided. Furthermore, the evaluation unit 114 automatically selects the ratios p and q of the linear combination corresponding to the best score of the evaluation index, and the corresponding third variance-covariance matrix W p The third variance-covariance matrix W best It may be decided as such.

[0112] The evaluation unit 114 automatically or by the user selects the third variance-covariance matrix W best The output is (step ST7A). For example, the evaluation unit 114 outputs the third variance-covariance matrix W to the determination unit 13. best The evaluation index is output. The determination unit 13 outputs the third variance-covariance matrix W best The inspection target may be determined to be defective based on the evaluation indicators. This allows the inspection device 1 to generate different variance-covariance matrices without collecting new training data, and to perform inspections based on the evaluation information of the variance-covariance matrices.

[0113] As described above, the model generation device 11A according to Embodiment 2 includes an evaluation unit 114 that calculates an evaluation index for the third variance-covariance matrix W. This allows the model generation device 11A to determine whether or not there are defects in the object being inspected based on the evaluation index.

[0114] In the model generation device 11A according to Embodiment 2, the evaluation unit 114 calculates an evaluation index for the third variance-covariance matrix W with the linear combination ratios p and q changed. The output unit 115 outputs the evaluation index for the third variance-covariance matrix W with the linear combination ratios p and q changed to the display device 3. By referring to the display on the display device 3, the user can adopt the third variance-covariance matrix W corresponding to the evaluation index for the best score.

[0115] In the model generation device 11A according to Embodiment 2, the evaluation unit 114 evaluates whether the third variance-covariance matrix W is degraded with respect to the first variance-covariance matrix A or the second variance-covariance matrix B. As a result, the model generation device 11A can suppress the occurrence of degradation of the third variance-covariance matrix W.

[0116] The various aspects of this disclosure are summarized below as an appendix.

[0117] (Note 1) An acquisition unit that acquires a first variance-covariance matrix generated by first training data containing multiple image data, and a second variance-covariance matrix generated by second training data containing multiple image data, The system comprises a model coupling unit that generates a third variance-covariance matrix by linearly combining the first variance-covariance matrix and the second variance-covariance matrix. A model generation device characterized by the following features. (Note 2) The model combination unit processes the column vectors of the second variance-covariance matrix so that the column vectors of the first variance-covariance matrix are at an acute angle to the column vectors of the first variance-covariance matrix, and then performs the linear combination. A model generation apparatus as described in Appendix 1, characterized by the features described herein. (Note 3) The system includes a setting unit for setting the ratio of the linear combination. A model generation apparatus as described in Appendix 1 or Appendix 2, characterized by the above. (Note 4) The system includes an evaluation unit that calculates an evaluation index for the third variance-covariance matrix. A model generation apparatus as described in any one of the appendices 1 to 3, characterized by the features described herein. (Note 5) The evaluation unit calculates the evaluation index for the third variance-covariance matrix in which the ratio of the linear combination has been changed. The system includes an output unit that outputs the evaluation index for the third variance-covariance matrix, in which the ratio of the linear combination has been changed, to a display device. A model generation apparatus as described in Appendix 4, characterized by the features described herein. (Note 6) The evaluation unit evaluates whether the third variance-covariance matrix is ​​degraded with respect to the first variance-covariance matrix or the second variance-covariance matrix. A model generation apparatus as described in Appendix 4 or Appendix 5, characterized by the features described herein. (Note 7) The first variance-covariance matrix is ​​a matrix obtained by dimensionality reduction of the fourth variance-covariance matrix generated by singular value decomposition based on the first training data. The second variance-covariance matrix is ​​a matrix obtained by dimensionality reduction of the fifth variance-covariance matrix, which is generated by singular value decomposition based on the second training data. A model generation apparatus as described in any one of the appendices 1 to 6, characterized by the features described herein. (Note 8) The second variance-covariance matrix is ​​the matrix obtained by learning and updating the first variance-covariance matrix based on the second training data. A model generation apparatus as described in any one of the appendices 1 to 7, characterized by the features described herein. (Note 9) A model generation device described in any one of the items from Appendix 1 to Appendix 3, An evaluation unit that calculates the similarity of the images to be inspected based on the third variance-covariance matrix, The system includes a determination unit that determines whether or not the object to be inspected is defective based on the similarity calculated by the evaluation unit. An inspection device characterized by the following features. (Note 10) The evaluation unit calculates the Mahalanobis distance of the subjects under inspection as a measure of similarity. The inspection apparatus described in Appendix 9, characterized by the features described herein. (Note 11) A model generation method executed by a model generation device, The acquisition unit obtains a first variance-covariance matrix generated by first training data containing multiple image data, and a second variance-covariance matrix generated by second training data containing multiple image data. The model coupling unit includes the step of generating a third variance-covariance matrix by linearly combining the first variance-covariance matrix and the second variance-covariance matrix. A model generation method characterized by the following: (Note 12) On the computer, The process involves obtaining a first variance-covariance matrix generated from the first training data containing multiple image data, and a second variance-covariance matrix generated from the second training data containing multiple image data. A process to generate a third variance-covariance matrix by linearly combining the first variance-covariance matrix and the second variance-covariance matrix, A program to execute.

[0118] Furthermore, it is possible to combine each embodiment, modify any component of each embodiment, or omit any component in each embodiment. [Industrial applicability]

[0119] The model generation apparatus described herein can be used, for example, in a defect inspection apparatus for photomasks used in semiconductors. [Explanation of Symbols]

[0120] 1 Inspection device, 2 Learning unit, 3 Display device, 11,11A Model generation device, 12 Evaluation unit, 13 Judgment unit, 100 Input interface, 101 Output interface, 102 Processing circuit, 103 Processor, 104 Memory, 111 Acquisition unit, 112 Model coupling unit, 113 Setting unit, 114 Evaluation unit, 115 Output unit, 1121 Dot product calculation unit, 1122 Sign inversion unit, 1123 Linear coupling unit.

Claims

1. An acquisition unit that acquires a first variance-covariance matrix generated by first training data containing multiple image data, and a second variance-covariance matrix generated by second training data containing multiple image data, The system comprises a model coupling unit that generates a third variance-covariance matrix by linearly combining the first variance-covariance matrix and the second variance-covariance matrix, The model combination unit processes the column vectors of the second variance-covariance matrix so that they form an acute angle with respect to the column vectors of the first variance-covariance matrix, and then performs the linear combination. A model generation device characterized by the following features.

2. The system includes a setting unit for setting the ratio of the linear combination. The model generation apparatus according to feature 1.

3. The system includes an evaluation unit that calculates an evaluation index for the third variance-covariance matrix. The model generation apparatus according to feature 1.

4. The evaluation unit calculates the evaluation index for the third variance-covariance matrix in which the ratio of the linear combination has been changed. The system includes an output unit that outputs the evaluation index for the third variance-covariance matrix, in which the ratio of the linear combination has been changed, to a display device. The model generation apparatus according to feature 3.

5. The evaluation unit evaluates whether the third variance-covariance matrix is ​​degraded with respect to the first variance-covariance matrix or the second variance-covariance matrix. The model generation apparatus according to feature 3.

6. The first variance-covariance matrix is ​​a matrix obtained by dimensionality reduction of the fourth variance-covariance matrix generated by singular value decomposition based on the first training data. The second variance-covariance matrix is ​​a matrix obtained by dimensionality reduction of the fifth variance-covariance matrix generated by singular value decomposition based on the second training data. A model generation apparatus according to any one of claims 1 to 5.

7. The second variance-covariance matrix is ​​a matrix obtained by learning and updating the first variance-covariance matrix based on the second training data. A model generation apparatus according to any one of claims 1 to 5.

8. A model generation apparatus according to claim 1 or claim 2, An evaluation unit that calculates the similarity of the images to be inspected based on the third variance-covariance matrix, The system includes a determination unit that determines whether or not the object to be inspected is defective based on the similarity calculated by the evaluation unit. An inspection device characterized by the following features.

9. The evaluation unit calculates the Mahalanobis distance of the subject under inspection as the similarity. The inspection apparatus according to feature 8.

10. A model generation method executed by a model generation device, The acquisition unit obtains a first variance-covariance matrix generated by first training data containing multiple image data, and a second variance-covariance matrix generated by second training data containing multiple image data. The model coupling unit includes the step of generating a third variance-covariance matrix by linearly combining the first variance-covariance matrix and the second variance-covariance matrix, The model coupling unit processes the column vectors of the second variance-covariance matrix so that they form an acute angle with respect to the column vectors of the first variance-covariance matrix, and then performs the linear combination. A model generation method characterized by the following:

11. On the computer, The process involves obtaining a first variance-covariance matrix generated from first training data containing multiple image data, and a second variance-covariance matrix generated from second training data containing multiple image data. A process to generate a third variance-covariance matrix by linearly combining the first variance-covariance matrix and the second variance-covariance matrix, Make it run, A program for performing the linear combination after processing such that the column vectors of the second variance-covariance matrix have an acute angle with respect to the column vectors of the first variance-covariance matrix.

Images