Data processing apparatus and data processing method

Abstract

A data processing device of the present technology has a processing circuit (120) that sequentially updates a Gramer matrix in the form of SVD in a learning phase, and that calculates a variance covariance matrix from the SVD of the Gramer matrix in the form of SVD in the end of the learning phase.

Description

Technical Field

[0001] The disclosed technology relates to a data processing device and a data processing method. Background Art

[0002] Data processing devices are used, for example, in the field of machine learning. Machine learning primarily involves supervised learning and unsupervised learning. One type of supervised learning involves predicting categories, known as "classification." Unsupervised learning also involves finding groups, known as "clustering." Data processing devices are used as artificial intelligence devices for "classification" or "clustering."

[0003] Artificial neural networks, such as CNNs (Convolutional Neural Networks), have achieved significant success as artificial intelligence for image classification and clustering. Artificial neural networks generate image features based on image data. Image features are vector quantities that can be represented as vectors in a feature space. Data processing devices determine similarity or deviation based on distances defined in this feature space, such as the Mahalanobis distance. Both similarity and deviation are important quantities required for classification and clustering.

[0004] Classification and clustering techniques in machine learning are applied to anomaly detection, which detects anomalies in images. More specifically, anomaly detection is performed based on the "Mahalanobis distance," which assumes that the probability of occurrence of samples belonging to a certain class in feature space can be expressed as a normal distribution and takes into account the estimated results of the normal distribution. For example, Non-Patent Document 1 discloses a technique that uses a method called Patch Distribution Modeling (sometimes abbreviated as "PaDiM") to apply a trained general-purpose CNN to anomaly detection.

[0005] Prior art literature

[0006] Non-patent literature

[0007] Non-Patent Literature 1: Thomas Defard et al., “PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization” (https: / / arxiv.org / abs / 2011.08785) Summary of the Invention

[0008] Problems to be solved by the invention

[0009] The Mahalanobis distance is the distance defined by the variance-covariance matrix (Σ). If the dimension of the feature space is set to N f , then the variance-covariance matrix (Σ) is of size N f ×N f In the learning phase, the variance-covariance matrix (∑) is updated based on the learning data. In the inference phase, the calculation of the Mahalanobis distance usually requires a matrix of size N. f ×N f The matrix is ​​the inverse matrix of the variance-covariance matrix (Σ).

[0010] As an algorithm for updating the variance-covariance matrix (Σ) in the learning phase, an algorithm compatible with singular value decomposition, and more specifically, an algorithm compatible with low-rank approximation based on singular value decomposition, is required.

[0011] Means for solving problems

[0012] The data processing device of the disclosed technology has a processing circuit. During the learning phase, the processing circuit updates the Gram matrix in a sequential manner while maintaining the SVD form. During the finalization of the learning phase, the processing circuit calculates the variance-covariance matrix based on the SVD of the Gram matrix described later while maintaining the SVD form.

[0013] Effects of the Invention

[0014] The data processing device of the disclosed technology has the above-mentioned structure, and the algorithm for updating the variance-covariance matrix (Σ) in the learning phase is compatible with singular value decomposition, or is also compatible with low-rank approximation based on singular value decomposition.

[0015] Therefore, the data processing device and data processing method of the disclosed technology can benefit from the benefits of singular value decomposition and the benefits of low-rank approximation based on singular value decomposition during the learning phase. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a hardware configuration diagram showing the hardware configuration of the data processing device according to the first embodiment.

[0017] Figure 2 This is a block diagram that diagrammatically illustrates functions used by the data processing device of the disclosed technology.

[0018] Figure 3 This is a block diagram schematically illustrating a parameter updating method used by the data processing device according to the first embodiment.

[0019] Figure 4 This is one of the flowcharts showing the processing steps for updating parameters of the data processing device according to the first embodiment.

[0020] Figure 5 This is the second flowchart showing the processing steps for parameter update of the data processing device according to the first embodiment.

[0021] Figure 6 This is a flowchart showing the processing steps of distance calculation in the data processing device according to the first embodiment.

[0022] Figure 7 This is an explanatory diagram showing the data processing method of the second embodiment.

[0023] Figure 8 This is an explanatory diagram showing the data processing method of the third embodiment. DETAILED DESCRIPTION

[0024] Introduction 1

[0025] As described below, it is assumed that the image feature quantity (x) processed by the disclosed technology is given by a vertical quantity (column vector).

[0026]

[0027] Among them, N f Indicates the length of the feature. N f The subscript "f" in N comes from the first letter of feature. d Indicates the number of learning data. In this specification, learning data is referred to as "data" below. d The subscript "d" in the variable "data" is derived from the first letter of data. In addition, the variable "c" is derived from the first letter of column (see the mathematical formula (2) described later).

[0028] For a plurality of images belonging to a certain category, a data matrix (X) in which the feature values ​​of each image are arranged is given by the following mathematical formula.

[0029]

[0030] In this manual, except for special cases, the data quantity is sufficient. d >N f .

[0031] The expected value (μ) of the image feature quantity belonging to this category is expressed by the following mathematical formula.

[0032]

[0033] Here, E() of the function represents the expected value. Generally speaking, the expected value and the average value are different concepts, but in this case, the expected value (μ) of the image feature quantity is equivalent to the N number of images belonging to the category. dThe average value of the feature values ​​of the images (hereinafter referred to as the “average feature value”) is obtained.

[0034] The variance-covariance matrix (Σ) of the data matrix (X) is given by the following mathematical formula using μ.

[0035]

[0036] The superscript "T" in equation (4) indicates transposition. In addition, the variance-covariance matrix is ​​sometimes referred to as the "covariance matrix." In addition, the XX in the first term on the right side of equation (4) is T It is called the Gram matrix. In addition, the expected value of the Gram matrix is ​​the first term on the right side of the mathematical formula (4) E(XX T ) is called the correlation matrix.

[0037] The variance-covariance matrix (Σ) can be derived by transforming the mathematical formula (4), but it is also possible to use the deviation vector (x c -μ) indicates.

[0038]

[0039] Here, if the deviation vector (x c -μ) is reset to y c , then the mathematical formula (5) can be transformed as follows.

[0040]

[0041] The second line of formula (6) represents the variance-covariance matrix (Σ) as QQ T , that is, the variance-covariance matrix (Σ) is a semi-positive definite matrix.

[0042] It is better to use Mahalanobis distance (d M ), using the measurement results of the normal distribution, measure the target sample (x target ) is the similarity or deviation of the two. Mahalanobis distance (d M ) is given by the following mathematical formula using the variance-covariance matrix (Σ).

[0043]

[0044] As shown in formula (7), the Mahalanobis distance (d M ) is the inverse matrix (Σ) of the variance-covariance matrix (Σ) -1 ) is the distance that can be defined when the variance-covariance matrix (Σ) is a positive definite value. The Mahalanobis distance (d M ) is in dimension N f The distance defined in the feature space of .

[0045] The disclosed technology focuses on the data belonging to a certain category from N d Increase to N d +1, how to update the variance-covariance matrix (Σ), this is clarified. If the data can be given in the form of singular value decomposition to increase to N d +1, the variance-covariance matrix (Σ) used to calculate the Mahalanobis distance (d M )'s variance-covariance matrix (Σ) and the inverse matrix (Σ -1 ).

[0046] Introduction 2

[0047] It's known that any matrix can be represented by its singular values ​​and singular vectors. The form obtained by decomposing a matrix into its singular values ​​and singular vectors is called singular value decomposition. The singular value decomposition of a p×q matrix (Z≠0) is shown below.

[0048]

[0049] Where r represents the rank of Z. The {σ1, ..., σ r} are the singular values ​​of Z. S is the singular value {σ1, ..., σ r} is a diagonal matrix of elements. The {u1, ..., u r} is called the singular value {σ1,…,σ r}. In addition, {v1, ..., v r} is called the singular value {σ1,…,σ r The left and right singular vectors are collectively referred to as singular vectors.

[0050] When the singular value decomposition of a p×q matrix (Z≠0) is given, Z+AB is obtained. T Research on the singular value decomposition of . For example, the algorithm is disclosed in the following non-patent literature.

[0051] Matthew Brand, "Fast Low-Rank Modifications of the Thin SingularValue Decomposition", MERL Technical Report, TR2006-059, May 2006.

[0052] Find Z+AB T The algorithm for the singular value decomposition of is described as a program and can be set as a function in a function library. In this specification, it is assumed that Z+ABT The singular value decomposition function is represented as "IncrSVD" as follows.

[0053]

[0054] Among them, Incr comes from the first four characters representing Incremental, and SVD comes from the first letters of Singular Value Decomposition.

[0055] Introduction 3

[0056] When a certain matrix (Z, Z≠0) can be expressed as a singular value decomposition (see mathematical formula (8)), the Moore-Penrose type generalized inverse matrix (also called a pseudo-inverse matrix, hereinafter referred to as the "generalized inverse matrix") is given by the following mathematical formula.

[0057]

[0058] When Z is a regular matrix (p = q), the generalized inverse matrix of Z (Z - ) and the inverse matrix of Z (Z -1 ). The inverse is defined only with respect to regular matrices, whereas the generalized inverse is defined with respect to nonzero matrices. However, in order to compute the generalized inverse, the rank must be known.

[0059] In addition, a vector is also a type of matrix, so the generalized inverse matrix is ​​also defined relative to the vector.

[0060] Generally speaking, since calculations in physics and engineering use observation data obtained by measuring devices and sensors, they inevitably contain calculation errors. This is also true in the technical field to which the data processing device 100 of the disclosed technology belongs. Therefore, when calculating the singular value decomposition for a matrix calculated based on the observation data, in numerical calculation, all singular values ​​(σ i , i is 1 to N f When the original singular value becomes non-zero due to the error of numerical calculation, when the inverse matrix of the matrix (including the generalized inverse matrix) is directly calculated, the 1 / σ i , thus becoming a severely distorted value.

[0061] To determine the rank of the p×q matrix Z obtained from the measurement data, the following method is used: first, a temporary rank (l=min(p,q)) is set, and the following singular value decomposition is calculated.

[0062]

[0063] The rank determination method then examines the last singular value to determine which singular value can be approximated to 0. The tolerance for the rank of a matrix is ​​specified using, for example, a minimum limit value (called "machine epsilon") that can be handled by a computer.

[0064] σ r+1 ≈0,…σ l ≈0...(12)

[0065] (Z) when the rank of Z is set to r r It is given by the following mathematical formula.

[0066] (Z) r :=σ1u1v1 T +…+σ r u r v r T ...(13)

[0067] In the case of (Z) defined by mathematical formula (13) r Another example of a similar method is the "low-rank approximation based on singular value decomposition." The generalized inverse matrix of a matrix for low-rank approximation is called the "rank-constrained generalized inverse matrix." The rank-constrained generalized inverse matrix is ​​also called the rank-constrained pseudo-inverse matrix.

[0068] About Z and (Z) r , the following relationship holds.

[0069]

[0070] Z and (Z) r The error is given by the following mathematical formula.

[0071]

[0072] The symbol appearing on the left side of the mathematical formula (15) is the Frobenius norm or the Euclidean norm.

[0073] The rank-constrained generalized inverse matrix is ​​a generalized inverse matrix that uses a low-rank approximation based on singular value decomposition. The low-rank approximation based on singular value decomposition has the same properties (disadvantages and advantages) as the general approximation. The disadvantage of the approximation is that it produces errors from the true value. The advantage of the approximation is that it can refine the information to the required amount. The low-rank approximation based on singular value decomposition also has the disadvantage of producing errors from the true value and the advantage of being able to refine the information to the required amount.

[0074] Most of the matters described in the third introduction are cited from the following cited document 1. In addition, the proof of deriving the error shown in the mathematical formula (15) is shown in cited document 1.

[0075] Reference 1: Kenichi Kanaya, "Linear Algebra Seminar, Projection, Singular Value Decomposition, Generalized Inverse Matrix," Kyoritsu Publishing, ISBN 978-4-320-11340-4.

[0076] Implementation method 1.

[0077] The data processing device 100 according to the first embodiment shows the minimum necessary configuration required for the data processing device 100 of the disclosed technology.

[0078] Figure 1 1 is a hardware configuration diagram showing the hardware configuration of the data processing device 100 according to the first embodiment.

[0079] Figure 1 A is a hardware configuration diagram showing a hardware configuration in which the functions of the data processing device 100 of the first embodiment are executed by dedicated hardware. Figure 1 As shown in A, the hardware structure when implemented by dedicated hardware includes an input interface 110, a processing circuit 120, and an output interface 130.

[0080] The processing circuit 120 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof. The various functions of the data processing device 100 may be implemented by separate processing circuits 120 or may be implemented collectively by a single processing circuit 120.

[0081] Figure 1 B is a hardware configuration diagram showing a hardware configuration in which the functions of the data processing device 100 of the first embodiment are executed by software. Figure 1 As shown in B, the hardware structure when executed by software includes an input interface 110, a processor 122, a memory 124, and an output interface 130.

[0082] Generally speaking, the processor 122 is also called a CPU (Central Processing Unit), a central processing device, a processing device, an arithmetic device, a microprocessor, a microcomputer, or a DSP (Digital Signal Processor).

[0083] The hardware structure of the data processing device 100 is Figure 1In the case of the hardware structure shown in Figure B, the various functions of the data processing device 100 are implemented using software, firmware, or a combination of software and firmware. Software and firmware are described as programs and stored in memory 124. Processor 122 implements the various functions by reading and executing the programs stored in memory 124. Specifically, data processing device 100 includes memory 124, which, when executed by processor 122, stores the programs that ultimately execute the various processing steps. Alternatively, these programs can be said to cause processor 122 to execute the processes and methods of data processing device 100 (the data processing method of the disclosed technology). Here, memory 124 may be, for example, non-volatile or volatile semiconductor memory such as RAM, ROM, flash memory, or EPROM. Furthermore, memory 124 may be a magnetic disk, floppy disk, optical disk, compact disk, minidisc, or DVD. Furthermore, memory 124 may be in the form of an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0084] Furthermore, each function of the data processing device 100 according to the first embodiment may be partially implemented by dedicated hardware, and the remaining functions may be implemented by software or firmware.

[0085] As described above, the data processing device 100 according to the first embodiment implements the above-described functions through hardware, software, firmware, or a combination thereof.

[0086] The technical feature of the data processing device 100 of the disclosed technology is that it does not have N f ×N f The variance-covariance matrix (Σ) is saved and used in this way as a matrix of size 1.

[0087] Since the variance-covariance matrix (Σ) represented by the mathematical formula (6) is a semi-positive definite matrix, the variance-covariance matrix (Σ) can be transformed as follows.

[0088]

[0089] Next, assume that the matrix Q appearing in equation (16) is expressed in singular value decomposition form.

[0090]

[0091] In this specification, the singular values ​​are arranged in order of size, and σ1≧σ2≧…≧σ NF In general, sigma (especially “σ 2 ”) is often used as a symbol to represent variance, but in this specification, as described above, sigma is used to represent a singular value.

[0092] According to the properties of singular vectors, the following equations hold for U and V appearing in equation (17).

[0093]

[0094] Here, I in the mathematical formula (18) is the size of N f ×N f The identity matrix of .

[0095] By substituting the singular value decomposition shown in equation (17) into equation (16), the variance-covariance matrix (Σ) can be decomposed as follows.

[0096]

[0097] As shown in mathematical formula (19), {σ1 2 ,σ2 2 ,…,σ Nf 2} are the singular values ​​of the variance-covariance matrix (Σ).

[0098] According to the properties shown in equation (18) and equation (19), the inverse matrix (Σ) of the variance-covariance matrix (Σ) -1 ) is given by the following mathematical formula.

[0099]

[0100] As mentioned above, in the case of a regular matrix, the generalized inverse matrix is ​​consistent with the inverse matrix. In addition, in the derivation of mathematical formula (20), it is assumed that all singular values ​​of the variance-covariance matrix (Σ) are non-zero. The method for dealing with the case where the variance-covariance matrix (Σ) is not full rank will become clear from the following description.

[0101] In this specification, the form in which a matrix is ​​decomposed and expressed as a matrix product is called a "decomposition form." Singular value decomposition is a special form of decomposition. The right side of mathematical formula (20) also gives the singular value decomposition. In this way, if the variance-covariance matrix (Σ) is given by singular value decomposition, it is possible to deal with the problem of appearing to be full rank due to numerical errors (see the third introduction). In this specification, singular value decomposition is sometimes also referred to as SVD.

[0102] The data processing device 100 of the disclosed technology uses the decomposition form given on the right side of the mathematical formula (20) to calculate the Mahalanobis distance (d M ).

[0103]

[0104] As shown in equation (21), in order to measure a sample (xtarget ) is similar or deviated from the Mahalanobis distance (d M ) and use the distance (d M 2 ), there is no problem. In addition, as shown in mathematical formula (21), the characteristic quantity of the target (x target ) can also be represented by a deviation vector (y target =x target -μ) indicates.

[0105] (Update method when data increases)

[0106] Figure 2 This is a block diagram that diagrams the functions used by the data processing device 100 of the disclosed technology. More specifically, Figure 2 The function IncrSVD() represented by the mathematical formula (9) is represented as a block diagram.

[0107] Figure 3 This is a block diagram illustrating a method for updating data used by the data processing device 100 of the first embodiment. More specifically, Figure 3 is to change the number of data from N d Increase to N d A block diagram schematically showing a method of updating data by the data processing device 100 according to the first embodiment at time +1.

[0108] It would be ideal, but not necessary, to be able to directly update the singular value decomposition form of the variance-covariance matrix (Σ), namely U, S, and V, as data increases. Figure 3 As shown, the data processing device 100 of the first embodiment uses the first IncrSVD() to update the SVD of the Gram matrix when data increases, and uses the second IncrSVD() to obtain the SVD of the variance-covariance matrix (Σ) based on the SVD of the correlation matrix. Figure 3 In FIG, the first IncrSVD() is represented by a vertically long block, and the second IncrSVD() is represented by a horizontally long block.

[0109] When the number of data is N d When , let the SVD of the Gram matrix be given as follows.

[0110]

[0111] The right subscript "0" added to the matrix on the right side of equation (22) is simply a number to identify the matrix for which SVD is to be performed.

[0112] When the number of data increases to N dWhen +1, the Gram matrix is ​​updated using the first IncrSVD().

[0113]

[0114] Here, similar to the "0" in mathematical formula (22), the right subscript "1" added to the matrix appearing on the left side of mathematical formula (23) is a number that identifies which matrix the SVD is for.

[0115] The relationship between the Gram matrix and the correlation matrix is ​​described in equation (4). That is, the expected value of the Gram matrix is ​​the correlation matrix. Based on this relationship, the SVD of the correlation matrix is ​​given by:

[0116]

[0117] When adding x to the data nd+1 When the average feature value (μ|N d +1) Updated as follows.

[0118]

[0119] The procedure shown in equation (25) is called "updating the average feature value."

[0120] Finally, the number of data is N d The SVD of the variance-covariance matrix (Σ) at +1 uses the updated average feature value (μ|N d +1) and the 2nd IncrSVD(), are given as follows.

[0121]

[0122] Above, according to Figure 3 It is clear that the number of data starts from N d To N d The method for updating data when the value of +1 is incremented by 1 is described, but the disclosed technology is not limited thereto. The data processing device 100 of the disclosed technology may also, for example, pre-divide the data into multiple batches and update the data in batches. The concept of updating data in batches is the same as the method for incrementing the data by 1. That is, when updating data in batches, the SVD of the Gram matrix is ​​updated using the first IncrSVD() method, and the SVD of the variance-covariance matrix (Σ) is calculated using the SVD of the correlation matrix using the second IncrSVD() method.

[0123] (Mahalanobis distance using rank-constrained generalized inverse matrix)

[0124] As shown in equation (6), the variance-covariance matrix (Σ) is a semi-positive definite matrix. When the variance-covariance matrix (Σ) is not full rank, there is no inverse matrix (Σ) of the variance-covariance matrix (Σ). -1 Even when the variance-covariance matrix (Σ) is numerically full rank, if any of its singular values ​​approaches 0, division by zero will occur numerically. Furthermore, the problem of appearing to be full rank due to numerical errors is described in the third section of the introduction.

[0125] In such a case, the Mahalanobis distance can be obtained by reducing the dimension of the feature space within the range of an allowable error.

[0126] The data processing device 100 of the disclosed technology intentionally reduces the rank of the variance-covariance matrix (Σ) to k (k≦r). d When , the rank of the variance-covariance matrix is ​​reduced to k by the following mathematical formula.

[0127]

[0128] The “(·)” in mathematical formula (27) k ” represents an approximation of reducing the rank to k. At this time, the error caused by the low rank of the variance-covariance matrix (Σ) can be evaluated as follows.

[0129]

[0130] Finally, the Mahalanobis distance when the rank is set to k (k≦r) is given as follows.

[0131]

[0132] Thus, the data processing method of the disclosed technology is highly compatible with low-rank approximations based on singular value decomposition during the learning phase, thus benefiting from the low-rank approximation's ability to reduce the number of parameters that must be stored. However, when calculating the Mahalanobis distance of the reduced-dimensional subspace during the inference phase, it is important to determine the projection matrix used to project the subspace and the dimensions of the space. The reason why this is important becomes clear from the explanation of "meaningful zero singular values" described in Implementation 5.

[0133] The data processing device 100 of the disclosed technology outputs the following data in combination with the calculated Mahalanobis distance.

[0134] Output data: {(d M ) k or (d M 2 ) k , (U)k ,(S 2 ) k or (S 2 ) k -1}...(30)

[0135] That is, the data processing device 100 of the disclosed technology outputs the calculated Mahalanobis distance, and the singular vectors and singular values ​​involved in the variance-covariance matrix (Σ).

[0136] Figure 4 This is one of the flowcharts showing the processing steps in the parameter update of the data processing device 100 according to the first embodiment. Figure 4 As shown, the processing steps of the data processing device 100 of the first embodiment can be divided into ST01 to ST07.

[0137] The processing step (ST01) described as "acquiring a new image feature value (x)" is a processing step performed via the input interface 110. In ST01, the data processing device 100 acquires, for example, the Nth image feature value (x) via the input interface 110. d +1 data x Nd+1 .

[0138] Recorded as "Number of data (N d The processing step (ST02) of "updating the number of data" is a processing step implemented by the processing circuit 120 or the processor 122. In ST02, the processing circuit 120 or the processor 122 counts the number of data, for example, from N d Update to N d +1.

[0139] The processing step (ST03) described as "updating the average feature value (μ)" is a processing step performed by the processing circuit 120 or the processor 122. In ST03, the processing circuit 120 or the processor 122 performs "updating the average feature value" expressed in equation (25).

[0140] The processing step (ST04) described as "updating U, S, and V of the Gram matrix" is a processing step performed by processing circuit 120 or processor 122. In ST04, processing circuit 120 or processor 122 updates the SVD of the Gram matrix using the first IncrSVD() shown in mathematical formula (23).

[0141] Recorded as "U, S about Σ 2 The processing step (ST05) of "calculation of" is a processing step implemented by the processing circuit 120 or the processor 122. In ST05, the processing circuit 120 or the processor 122 uses the second IncrSVD() shown in mathematical formula (26) to calculate the number of data N dSVD of the variance-covariance matrix (Σ) at +1.

[0142] Recorded as "About Σ -1 U.S. -2 The processing step of "saving" (ST06) is a processing step performed using the memory 124 or an external storage device. In ST06, the data processing device 100 saves the latest Σ -1 The singular vectors (U) and singular values ​​(S -2 ) is stored in the memory 124 or an external storage device.

[0143] Figure 5 This is the second flowchart showing the processing steps for parameter update in the data processing device 100 according to the first embodiment.

[0144] Figure 5 The processing steps shown are similar to Figure 4 The same processing steps are shown. However, Figure 5 As shown, Figure 5 The processing steps shown use the fixed (U) k T The k-dimensional feature space of the map. Figure 5 The image feature quantity processed by the flowchart shown (the image feature quantity with the phonetic symbol underlined for x) is given by the following mathematical formula.

[0145]

[0146] However, (U) in equation (31) k T It is a matrix obtained based on experience, which is obtained from N f The mapping from dimensional space to k-dimensional space.

[0147] (U) k T When the data is sufficiently rich, that is, when it contains information sufficient to reproduce the properties of the category to which the data belongs, it can be a matrix consisting of the first to kth right singular vectors associated with the variance-covariance matrix (Σ) (see equation (27)). The method for determining whether the data is sufficiently rich has become clear from the description of embodiment 6.

[0148] Figure 6 This is a flowchart showing the processing procedure for distance calculation by the data processing device 100 according to the first embodiment. Figure 4 and Figure 5 The processing steps performed during the learning phase of the data processing device 100 are shown. Figure 6 FIG. 1 shows the processing steps performed in the inference phase of the data processing device 100. Figure 6As shown, the processing steps in the inference phase can be divided into ST11 to ST13.

[0149] Although not specifically shown, the data processing device 100 in the learning phase and the data processing device 100 in the inference phase may be different devices. In other words, a data processing device 100 that performs inference may be provided separately from the data processing device 100 that performs learning.

[0150] Recorded as "sample (x target The processing step of "acquiring" (ST11) is a processing step performed by the data processing device 100 for performing the inference. In ST11, the data processing device 100 for performing the inference obtains the image feature quantity x for which the Mahalanobis distance is to be calculated. target .

[0151] Recorded as "About Σ -1 U.S. -2 The processing step of "reading" (ST12) is a processing step performed by the data processing device 100 that performs the inference. In ST12, the data processing device 100 that performs the inference reads the Σ stored in ST06 of the learning phase. -1 U.S. -2 .

[0152] Recorded as "Mahalanobis distance (d M The processing step (ST13) of "calculating the Mahalanobis distance" is a processing step performed by the data processing device 100 that performs the inference. The data processing device 100 that performs the inference calculates the Mahalanobis distance (d M ) or the square of the Mahalanobis distance (d M 2 )(Refer to mathematical formulas (7), (21), and (29)).

[0153] The data processing device 100 that performs the inference outputs the data shown in the mathematical formula (30) together with the calculated Mahalanobis distance in order to clearly define the feature space having the calculated Mahalanobis distance. k and (S 2 ) k or (S -2 ) k .

[0154] (Simple numerical example)

[0155] The data processing method of the disclosed technology will be made clearer by the following simple numerical example: Assume that the data belonging to a certain category is given as follows.

[0156]

[0157] The numerical example shown in formula (32) is the number of data (Nd ) is increased from 3 to 4.

[0158] Assuming that the number of data is 3, the average feature value and SVD of the Gram matrix are given below.

[0159]

[0160] Due to space limitations, numerical examples in this manual are displayed to 7 decimal places.

[0161] Through the first IncrSVD(), the SVD of the Gram matrix is ​​updated.

[0162]

[0163] As shown in mathematical formula (34), in the Gram matrix, due to its symmetry, the left singular vectors are consistent with the right singular vectors.

[0164] The SVD obtained by the first IncrSVD() is the SVD of the Gram matrix when the number of data is 4.

[0165]

[0166] The average feature value is updated as follows.

[0167]

[0168] By the second IncrSVD(), the SVD of the variance-covariance matrix (Σ) is obtained as follows.

[0169]

[0170] The SVD obtained by the second IncrSVD() becomes the SVD of the variance-covariance matrix (Σ) when the number of data is 4.

[0171]

[0172] The technical feature of the data processing device and data processing method of embodiment 1 is that the algorithm for updating the variance-covariance matrix (Σ) during the learning phase is compatible with singular value decomposition. More specifically, the data processing device and data processing method of embodiment 1 include a function compatible with singular value decomposition such as IncrSVD() in the algorithm for updating the variance-covariance matrix (Σ) during the learning phase.

[0173] By having this technical feature, the data processing device and data processing method of embodiment 1 can benefit from the singular value decomposition that can deal with the problem of appearing to be full rank due to numerical errors (see the third introduction).

[0174] The technical feature of the data processing device and data processing method of embodiment 1 is that the algorithm for updating the variance-covariance matrix (Σ) in the learning phase is also compatible with a low-rank approximation based on singular value decomposition. More specifically, the data processing device and data processing method of embodiment 1 include a function that is compatible with a low-rank approximation based on singular value decomposition such as IncrSVD() in the algorithm for updating the variance-covariance matrix (Σ) in the learning phase.

[0175] By having this technical feature, the data processing device and data processing method of embodiment 1 can benefit from the low-rank approximation based on singular value decomposition, which allows information to be refined to a required amount during the learning phase.

[0176] The data processing device and data processing method of embodiment 1 are applied to anomaly detection using PaDiM, and their effectiveness is verified. In this application example, the image is divided into small areas of 56×56. For each small area, the variance covariance matrix (Σ) is updated separately. As described in non-patent document 1 related to PaDiM, Wide ResNet50 is used in CNN, and the feature length (N f ) is 1792. Even with simple calculations, the memory required for updating during the learning phase is 56×56×1792×1792=40 [GB]. Even considering that the variance-covariance matrix (Σ) is a symmetric matrix, about 20 [GB] of memory is still required.

[0177] The data processing device and data processing method of the first embodiment have demonstrated that, in this application example, the dimension of updates in the learning phase can be reduced to k=20.

[0178] Implementation method 2.

[0179] The data processing device and data processing method of Embodiment 2 are variations of the data processing device and data processing method of the disclosed technology. Unless otherwise specified, the same reference numerals as those used in Embodiment 1 are used in Embodiment 2. Furthermore, descriptions that overlap with those in Embodiment 1 are omitted as appropriate in Embodiment 2.

[0180] In order to distinguish it from the method shown in the first embodiment, the data processing method of the second embodiment is referred to as "IncrPCA" in this specification. Figure 7 This is an explanatory diagram showing the data processing method of the second embodiment.

[0181] However, the core idea of ​​IncrSVD() is to introduce the following extension system.

[0182]

[0183] Compared with the case of the disclosed technology, the method of importing the expansion system is also effective. d When , the SVD of the Gram matrix is ​​given as in Mathematical Formula (22). However, when considering that the Gram matrix is ​​a semi-positive definite matrix represented by a quadratic form, the SVD of the Gram matrix can be expressed as follows.

[0184]

[0185] Furthermore, low-rank approximation can also be performed on the SVD of the Gram matrix. The conditions under which low-rank approximation can be performed are clarified by the fifth embodiment.

[0186] After obtaining the Nth d When there is +1 data, the Gram matrix can be expressed by the following expansion system.

[0187]

[0188] Mathematical formula (41) gives the following insights.

[0189]

[0190] Here, the script font "SVD" appearing in mathematical formula (42) represents a function for obtaining singular value decomposition.

[0191] Above, the number of data from N d To N d The data update method for +1, when incremented by 1, becomes clear by introducing an extended system. However, the disclosed technology is not limited to this. For example, the data processing device 100 of the disclosed technology may also pre-divide data into multiple batches and append data in batches to generate an extended system. The concept of introducing the extended system in batches is the same as the method for incrementally incrementing data by 1.

[0192] Batch processing is equivalent to the following situation: when the number of data is N d When the new unified join n b When such data processing is performed, the mathematical formula (42) is rewritten as follows.

[0193]

[0194] Here, I(n b ) is of size n b ×n b The identity matrix of .

[0195] As described above, the SVD of the Gram matrix can also perform a low-rank approximation for k (k < r) dimensions. Performing a low-rank approximation for k dimensions means using only the top k of the singular values of the Gram matrix arranged in descending order (for example, refer to Mathematical Formula (27)). However, since the update of the Gram matrix is a cyclic process, the error caused by the low-rank approximation is cumulative.

[0196] In the anomaly detection process based on PaDiM, the inventors of the present disclosure reduced the dimension of the original N f = 1792 to k = 20 and updated the Gram matrix based on a cyclic process. The inventors of the present disclosure found that in this application example, this cumulative error is within an allowable range.

[0197] (Simple numerical example)

[0198] By using the same numerical example as the numerical example shown in Embodiment 1, the data processing method of Embodiment 2 becomes clearer. When the number of data is 3, the U G0 and S G0 of the Gram matrix are given as follows.

[0199]

[0200] In addition, U G0 is the same as U0.

[0201] If the numerical example is applied, Mathematical Formula (42) is calculated as follows.

[0202]

[0203] And the matrix calculated by Mathematical Formula (45) can be represented by the following SVD form.

[0204]

[0205] Among them, the subscript "3" appearing in Mathematical Formula (46) is only a number for identifying the SVD of which matrix.

[0206] The first singular vector and the second singular vector of U3 calculated by Mathematical Formula (46) are the same as the first singular vector and the second singular vector of U1 calculated by Mathematical Formula (34). And when the first singular value and the second singular value of S3 calculated by Mathematical Formula (46) are squared respectively, they are respectively consistent with the first singular value and the second singular value of S1 calculated by Mathematical Formula (34).

[0207]

[0208] As shown in mathematical formula (47), the data processing device of embodiment 2 enlarges the dimension of the matrix to (N f +n b )×(N f +n b ), but since in the SVD of the Gram matrix, n b The singular values ​​are zero, so the final size is N f ×N f decomposition form.

[0209] The processing steps in the data processing method after mathematical formula (47) are the same as the data processing method of embodiment 1.

[0210] A unique technical feature of the data processing device and data processing method of the second embodiment is that they include an algorithm for introducing an expansion system and updating the Gram matrix in the learning phase.

[0211] According to this technical feature, in addition to the effects shown in the first embodiment, the data processing device and the data processing method of the second embodiment further achieve the effect of being able to be implemented using a general SVD function instead of IncrSVD().

[0212] Implementation method 3.

[0213] The data processing device and data processing method of Embodiment 3 are variations of the data processing device and data processing method of the disclosed technology. Unless otherwise specified, the same reference numerals as those used in Embodiment 1 are used in Embodiment 3. Furthermore, in Embodiment 3, descriptions that overlap with those in the already described embodiments are omitted as appropriate.

[0214] In order to distinguish it from the method shown in the embodiment already described, in this specification, the data processing method of embodiment 3 is referred to as "GPU-oriented IncrPCA".

[0215] The data processing device of the disclosed technology can also use a GPU (Graphics Processing Unit) instead of a CPU (Central Processing Unit). The advantage of using a GPU is that a function for calculating the SVD (hereinafter referred to as the "SVD function") is already available as a library (for example, NVidia's cuSOLVER library). If the size of the matrix is ​​32×32 or less, the SVD function on the GPU is very fast.

[0216] The scenario assumed in Implementation 3 is, for example, a scenario where the error is sufficiently small to be negligible even when low-rank approximation is performed below k = 32. However, to facilitate understanding of the data processing method of Implementation 3, this specification first describes the mathematical formula without low-rank approximation.

[0217] (Prerequisite Knowledge for the Data Processing Method of Implementation 3)

[0218] The singular value decomposition of any matrix (Z) can be obtained by T Eigenvalue decomposition of Z and ZZ T The eigenvalue decomposition of is obtained.

[0219] Assume that the singular value decomposition of Z is in the form of USV T . Z T Z can be deformed as follows.

[0220]

[0221] Here, if we assume that the i-th column of V is v i , then according to mathematical formula (48), we get the following relationship.

[0222] (Z T Z)v i =σ i 2 v i ...(49)

[0223] That is, v i and σ i 2 Become Z T The eigenvector and eigenvalue of Z.

[0224] Likewise, ZZ T The following deformations are possible.

[0225]

[0226] Here, if the i-th column of U is set to ui, the following relationship is obtained according to mathematical formula (50).

[0227] (ZZ T )u i =σ i 2 u i ...(51)

[0228] As mentioned above, the singular value decomposition of any matrix (Z) can be obtained by T Eigenvalue decomposition of Z and ZZ T It can be found by eigenvalue decomposition.

[0229] For example, assume that the size of Z is p×q, and for a horizontally long matrix, q > p. In this case, Z T has a size of p×p and is small, however, ZZ T has a size of q×q and is large. In such a case, only performing the eigenvalue decomposition of Z T to calculate the matrix (V) related to the right singular vectors and the matrix (S) related to the singular values is sufficient. The matrix (U) related to the left singular vectors can be calculated through the following calculation.

[0230]

[0231] Here, if a low-rank approximation can be performed for k (k < r) dimensions, it is assumed that all k singular values are non-zero, and it is certain that there exists S -1 . Additionally, in the eigenvalue decomposition of Z T Z, if there are no non-zero eigenvalues, then even without performing a low-rank approximation, S -1 can be calculated.

[0232] (Details of the data processing method in Embodiment 3)

[0233] Figure 8 is an explanatory diagram showing the data processing method in Embodiment 3.

[0234] The data processing apparatus in Embodiment 3 defines the following matrix appearing in Mathematical Formula (43) as M.

[0235]

[0236] Additionally, M is a matrix (X) composed of data up to the data quantity of N d +n b . Regarding the lower right subscript b of X, it is a consecutive number assigned to each batch. The data quantity in the b-th batch is n b .

[0237] Assume that M can perform SVD as follows.

[0238]

[0239] However, at this stage, assume that the SVD form has not been obtained yet. Additionally, the subscript "G1" appearing in Mathematical Formula (54) is only a symbol for identifying the SVD of which matrix.

[0240] As shown in Mathematical Formulas (48) and (49), the matrix (V G1 ) related to the right singular vectors of M and the matrix (S G1 ) related to the singular values can be obtained through the eigenvalue decomposition of M T .

[0241] In addition, the matrix (U G1 ) is calculated according to the following relationship.

[0242]

[0243] In addition, if M T If all the intrinsic values ​​of M are non-zero, then there exists a matrix (S G1 ) is the inverse matrix of .

[0244] The data processing device of the third embodiment updates M every time the amount of data increases. The updated M is given as follows.

[0245]

[0246] In mathematical formula (56), the number of data in the b+1th batch is n b+1 .

[0247] The updating of M shown in the mathematical formula (56) corresponds to the updating of the Gram matrix using the first IncrSVD() in the first embodiment.

[0248] Low-rank approximation can also be used when decomposing the singular value of M. However, since the update of M is a cyclic process, the error caused by the low-rank approximation is cumulative. The inventors of the disclosed technology use the original N f =1792 is reduced to k=20, and M is updated through a loop process. The inventors of the disclosed technology have found that in this application example, the cumulative error is within the allowable range.

[0249] After reflecting a sufficient amount of data, the operation of obtaining the SVD of the variance-covariance matrix (Σ) from the SVD of the Gram matrix is ​​also called closing. In the closing stage, the SVD of the Gram matrix is ​​calculated as follows.

[0250]

[0251] The subscript "G" in equation (57) is a symbol used to emphasize that it is the SVD of the Gram matrix. In addition, as shown in equation (57), the SVD of the Gram matrix is ​​reduced to a k-dimensional approximation. N all It is the total number of data at the end.

[0252] In conclusion, the SVD of the variance-covariance matrix (Σ) is performed using the SVD function, which is given below.

[0253]

[0254] However, it is assumed that the SVD shown in equation (58) also includes errors caused by dimensionality reduction approximation. In addition, the average feature value (μ) appearing on the right side of equation (58) also takes into account the matrix size in matrix addition and subtraction operations, reducing the dimension to k dimensions. Equation (58) can also be said to give the SVD of the variance-covariance matrix (Σ) when the feature space is reduced to k dimensions. This SVD function can also be a GPU-based SVD function.

[0255] The unique technical feature of the data processing device and data processing method of the third embodiment is that, according to M T Eigenvalue decomposition of M, computes the singular value decomposition of M.

[0256] According to this technical feature, the data processing device and data processing method of the third embodiment have the following effects, in addition to the effects described in the embodiments already shown: they can be realized by finding a general eigenvalue decomposition function.

[0257] Implementation method 4.

[0258] The data processing device and data processing method of Embodiment 4 are variations of the data processing device and data processing method of the disclosed technology. Unless otherwise specified, the same reference numerals as those used in Embodiment 1 are used in Embodiment 4. Furthermore, in Embodiment 4, descriptions that overlap with those in the already described embodiments are omitted as appropriate.

[0259] (Calculation function)

[0260] The data processing device and data processing method of the disclosed technology may also have a verification function. The data processing device and data processing method of the disclosed technology may also calculate the correlation matrix for verification using the following sequential method that is not in the SVD format.

[0261]

[0262] The sequential update formula of the correlation matrix shown in equation (59) has the same form as the sequential update formula of the average feature value (μ) shown in equation (25). If the expected value function E() is used, equation (25) is expressed as follows.

[0263]

[0264] If mathematical formula (59) and mathematical formula (60) are used, the variance-covariance matrix (Σ) used for verification can be obtained by the following sequential method that is not in SVD form.

[0265]

[0266] In the updated variance-covariance matrix (Σ) obtained by mathematical formula (61), the SVD form obtained by the data processing method shown in the embodiment already shown can be used to verify whether it is correctly obtained.

[0267] The unique technical feature of the data processing device and data processing method of the fourth embodiment is that the correlation matrix used for verification can be directly and sequentially obtained. The term "directly" here means "not in SVD form."

[0268] According to this technical feature, the data processing device and data processing method of embodiment 4, in addition to the effects described in the embodiments already shown, also have the following effects: it is possible to verify the correctness of the variance-covariance matrix (Σ) calculated in the form of SVD in the learning phase.

[0269] Implementation method 5.

[0270] The data processing device and data processing method of Embodiment 5 are variations of the data processing device and data processing method of the disclosed technology. Unless otherwise specified, the same reference numerals as those used in Embodiment 1 are used in Embodiment 5. Furthermore, in Embodiment 5, descriptions that overlap with those in the already described embodiments are omitted as appropriate.

[0271] When the variance-covariance matrix (Σ) after the tail is assumed to be full rank, the generalized inverse matrix (Σ - ), the following represents the target sample (x target ) squared Mahalanobis distance (d M 2 )(See also mathematical formula (21)).

[0272]

[0273] Among them, the size of the singular value is σ1≧…≧σ Nf In this way, the Mahalanobis distance can be obtained by the generalized inverse matrix (Σ) of the variance-covariance matrix (Σ) - ) is found, but the generalized inverse matrix (Σ - )'s first singular value (1 / σ1 2 ) is the smallest, and the final singular value (1 / σ Nf 2 ) is the largest. The final singular value (1 / σ Nf 2 ) is the largest, about the final singular value (1 / σ Nf 2 ) has the greatest impact on the Mahalanobis distance.

[0274] During the learning phase, the variance-covariance matrix (Σ) representing the properties of a particular class can also be said to be in the order of the size of the singular values, i.e., σ1,…,σ Nf Therefore, in the learning phase, it is not a problem to assume that the feature space is a k-dimensional partial space.

[0275] On the other hand, in the inference phase, the feature space is set to be full N f Dimensional space is more important.

[0276] (meaningful zero singular values)

[0277] Even if we assume that all the deviation vectors (y1, ..., y Nall ) spans a space with a dimension of r(r <N f ), the deviation vector about the target (y target ) does not necessarily belong to the r-dimensional space. For example, when considering a class consisting of images in a normal state for a certain object, let the deviation vector (y1, ..., y Nall )The dimension of the space opened is r(r <N f Even so, the deviation vector (y target ) does not necessarily belong to the r-dimensional space. In this case, the zero singular value of the variance-covariance matrix (Σ) of the class consisting of images in the normal state is a "meaningful zero singular value". On the contrary, even for the deviation vector (y target ), when it always belongs to the r-dimensional space, the dimension larger than r is also lengthy, and the zero singular value of the variance-covariance matrix (Σ) is a meaningless zero singular value.

[0278] Assume that the rank of the variance-covariance matrix (Σ) involved in the class composed of normal state images is r(r <N f In this case, the data processing device and the data processing method in the inference stage may also calculate the Mahalanobis distance (d M ), the judgment processing shown below is implemented as a response method for "meaningful zero singular values".

[0279]

[0280] However, ε (epsilon) appearing in equation (63) is a machine ε or other threshold.

[0281] When the condition shown in equation (63) is true, the deviation vector (y target ) does not belong to the category composed of normal state images. If it is clear that the target does not belong to the category, there is no need to specifically calculate the Mahalanobis distance (dM )。In addition, the Mahalanobis distance in this case can be said to be the result of a division operation based on a zero singular value being ∞ (infinity). When the condition shown in the mathematical formula (63) is true, the zero singular value of the variance-covariance matrix (Σ) of the class composed of images in the normal state is a non-truncatable singular value, that is, a "meaningful zero singular value".

[0282] Only when the condition shown in the mathematical formula (63) is false, the data processing device and the data processing method in the inference stage perform the process of calculating the Mahalanobis distance (d M ) defined in the r-dimensional feature space.

[0283] When the data processing device and the data processing method of the present disclosed technology perform a low-rank approximation considering the dimension as k (k < r), the mathematical formula (63) is applied to perform the following judgment process.

[0284]

[0285] When a low-rank approximation considering the dimension as k can be performed, the approximate value of the Mahalanobis distance is given by the following mathematical formula according to the rank-constrained generalized inverse matrix.

[0286]

[0287] In this specification, the "variable with a circumflex over y" defined in the second formula of the mathematical formula (65) is called the "normalized deviation vector with respect to the target" or simply the "normalized deviation vector" for short. In addition, even when no low-rank approximation is performed, this vector is also called the "normalized deviation vector" (refer to the mathematical formula (62)).

[0288] The mathematical formula (64) gives the condition for performing a low-rank approximation when calculating the Mahalanobis distance. The condition for performing a low-rank approximation is as follows.

[0289]

[0290] The mathematical formula (66) represents that when observing the coordinates of y Nf in the N f -dimensional space defined by the basis vectors from u1 to u target , the absolute values of all coordinate elements from the (k + 1)-th to the N<00002​​​​) is an element of the k-dimensional subspace spanned by . Therefore, as long as there is no deviation vector (y target ) is the deviation vector (y1,…,y Nall )”, low-rank approximation should not be implemented.

[0291] In this specification, the condition given by mathematical formula (66) is referred to as the "feasibility condition for low-rank approximation." In the case where it is unknown whether the object (application example) to which the disclosed technology is to be applied satisfies the feasibility condition for low-rank approximation, the data processing device and the data processing method may simply implement the judgment process shown in mathematical formula (63) or mathematical formula (64) during the inference stage.

[0292] The inventors of the disclosed technology have found that in the abnormality detection process based on PaDiM, even if the original N f The dimension of =1792 is reduced to k=20, which also meets the “feasibility condition of low-rank approximation” shown in mathematical formula (66).

[0293] A unique technical feature of the data processing device and data processing method of the fifth embodiment is that they have a method for dealing with "meaningful zero singular values".

[0294] Through this technical feature, the data processing device and data processing method of embodiment 5 also have the following effect: even if the variance-covariance matrix (Σ) involved in a certain category is not full rank, the full N can be used in the inference stage. f dimensional space as a feature space to solve learning problems such as "classification" and "clustering".

[0295] Implementation method 6.

[0296] The data processing device and data processing method of Embodiment 6 are variations of the data processing device and data processing method of the disclosed technology. Unless otherwise specified, the same reference numerals as those used in Embodiment 1 are used in Embodiment 6. Furthermore, in Embodiment 6, descriptions that overlap with those in the already described embodiments are omitted as appropriate.

[0297] However, in the first embodiment Figure 5 In the figure, we show the use of fixed singular vectors ((U) k T As described above, in order to use a fixed k-dimensional subspace mapped by a fixed singular vector, it is necessary to obtain sufficient data, that is, to obtain information sufficient to reproduce the properties of the category to which the data belongs.

[0298] The data processing device and the data processing method of the disclosed technology include a unit for determining whether data is sufficiently abundant, and more specifically, include an end condition for a loop process related to data update.

[0299] (About the end condition of the loop processing related to data update)

[0300] When the number of data is N d +1, according to the number of data N d The information about the end condition of the loop processing related to data update is given as follows.

[0301]

[0302] Among them, ε in mathematical formula (67) σ is the threshold value of singular values, ε u is the threshold value for singular vectors. σ and ε u It can be the same value or different values. In addition, the singular value (σ i ) and singular vectors (u i ) is related to the variance-covariance matrix (Σ) shown below.

[0303]

[0304] To determine whether the data is sufficiently rich, even if the singular values ​​and singular vectors of the variance-covariance matrix (Σ) cannot be directly observed, it is sufficient to indirectly observe the singular values ​​and singular vectors of the variance-covariance matrix (Σ). Specifically, indirectly observing the singular values ​​and singular vectors of the variance-covariance matrix (Σ) involves observing the correlation matrix.

[0305] In the case of the data processing device and data processing method of embodiment 1, the termination condition shown in mathematical formula (67) can be achieved by using the singular values ​​and singular vectors of the SVD of the correlation matrix derived from the SVD of the Gram matrix shown in mathematical formula (23).

[0306] In the case of the data processing device and data processing method of embodiment 2, the termination condition shown in mathematical formula (67) can be achieved by using the singular values ​​and singular vectors of the SVD of the correlation matrix derived from the SVD of the Gram matrix shown in mathematical formula (42) or mathematical formula (43).

[0307] In the case of the data processing device and data processing method of embodiment 3, the termination condition shown in mathematical formula (67) can be achieved by using the singular values ​​and singular vectors of the SVD of the correlation matrix derived from the SVD of the Gram matrix obtained by eigenvalue decomposition of MTM, etc.

[0308] A unique technical feature of the data processing device and data processing method of the sixth embodiment is that they have an end condition for the loop processing related to data updating.

[0309] According to this technical feature, the data processing device and data processing method of the sixth embodiment also have the following effect: the end of the loop processing can be determined based on whether the data is sufficiently rich.

[0310] Industrial applicability

[0311] The data processing device and data processing method of the disclosed technology can be applied to a defect inspection device that performs abnormality determination (for example, a defect inspection device for a semiconductor photomask), and thus has industrial applicability.

[0312] Label Description

[0313] 100: data processing device; 110: input interface; 120: processing circuit; 122: processor; 124: memory; 130: output interface.

Claims

1. A data processing device, wherein: The data processing device has a processing circuit, During the learning phase, the processing circuit updates the Gram matrix in a sequential manner while maintaining the SVD form. At the end of the learning phase, the processing circuit calculates a variance-covariance matrix based on the SVD of the Gram matrix while maintaining the SVD form.

2. A data processing device, wherein: The data processing device has a processor for executing a program. The processor executes the program, and in the learning phase, the Gram matrix is ​​updated successively while maintaining the SVD form. The processor executes the program, and at the end of the learning phase, a variance-covariance matrix is ​​calculated based on the SVD of the Gram matrix while maintaining the SVD form.

3. The data processing apparatus according to claim 2, wherein: The procedure has the following features: the SVD of Z, A and B as input and Z + AB T The SVD of is used as the output function, Where A, B, and Z are matrices respectively.

4. The data processing apparatus according to claim 2, wherein: The program has a function that outputs the SVD of the expanded system.

5. The data processing apparatus according to claim 2, wherein: The program has the output matrix M represented by the quadratic form T Function for the eigenvalue decomposition of M.

6. The data processing device according to any one of claims 2 to 5, wherein: The program has a function that calculates the correlation matrix for verification not in the form of SVD but sequentially.

7. The data processing device according to any one of claims 2 to 5, wherein: The program has functions to handle zero singular values ​​that have meaning.

8. The data processing device according to any one of claims 2 to 5, wherein: The program includes a function for determining the end of the loop processing based on a judgment of whether the learning data is sufficiently rich.

9. A data processing method, wherein: During the learning phase, the Gram matrix is ​​updated successively in the form of SVD. At the end of the learning phase, the variance-covariance matrix is ​​calculated based on the SVD of the Gram matrix while maintaining the SVD form.

10. The data processing method according to claim 9, wherein: The data processing method includes taking the SVD of Z, A and B as input and Z+AB T The SVD is calculated numerically as output, Where A, B, and Z are matrices respectively.

11. The data processing method according to claim 9, wherein: The data processing method includes numerical calculation of the SVD of the output expansion system.

12. The data processing method according to claim 9, wherein: The data processing method includes outputting a matrix M represented by a quadratic form T Numerical computation of the eigenvalue decomposition of M.

13. The data processing method according to any one of claims 9 to 12, wherein: The data processing method further includes numerical calculations that do not use SVD but sequentially calculate the correlation matrix for verification.

14. The data processing method according to any one of claims 9 to 12, wherein: The data processing method includes processing to deal with meaningful zero singular values.

15. The data processing method according to any one of claims 9 to 12, wherein: The data processing method includes a process of determining the end of a loop process based on a determination of whether the learning data is sufficiently rich.

Images