DATA PROCESSING DEVICE AND DATA PROCESSING METHOD

Abstract

A data processing device according to the presently disclosed technology comprises a processing circuit (120), wherein the processing circuit (120) sequentially updates a Gram matrix in SVD form in a learning phase, and the processing circuit (120) calculates a variance-covariance matrix in SVD form in the finalization phase of the learning phase based on the SVD with respect to the Gram matrix.

Description

TECHNICAL AREA

[0001] The technology disclosed herein relates to a data processing device and a data processing method. BACKGROUND ON THE STATE OF THE TECHNOLOGY

[0002] Data processing devices are used, for example, in the field of machine learning. Problems solved by machine learning can be broadly divided into supervised and unsupervised learning. One form of supervised learning is the problem of predicting a category, i.e., "classification." Furthermore, unsupervised learning also includes the problem of group identification, i.e., "clustering." The data processing device is used as artificial intelligence that performs "classification" or as artificial intelligence that performs "clustering."

[0003] As an artificial intelligence that performs tasks such as image classification or clustering, an artificial neural network like a Convolutional Neural Network (CNN) has achieved excellent results. The CNN generates a set of image features from the image data. This set of features is a vector quantity that can be expressed as a vector in feature space. The data processing device determines the degree of similarity or the degree of dissimilarity based on a distance definable in feature space, such as the Mahalanobis distance. Both the degree of similarity and the degree of dissimilarity are important quantities required for classification or clustering.

[0004] A classification and clustering technique from machine learning is used to detect anomalies in images. More specifically, it is assumed that the probability of a sample belonging to a particular class in the feature space can be expressed by a normal distribution, and anomaly detection is performed based on the "Mahalanobis distance," taking into account the determination of the normal distribution. For example, non-patent literature 1 discloses a technique in which a trained general-purpose CNN is used to detect anomalies using a technique called patch distribution modeling (sometimes simply referred to as "PaDiM"). REFERENCE LIST NON-PATENT LITERATURE

[0005] Non-patent literature 1: by Thomas Defard et al. "PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization" (https: / / arxiv.org / abs / 2011.08785) SUMMARY OF THE INVENTIONAL PROBLEM

[0006] The Mahalanobis distance is a distance defined by a variance-covariance matrix (Σ). If the dimension of the feature space is N f The variance-covariance matrix (Σ) is a matrix of size N. f ×N r The variance-covariance matrix (Σ) is updated based on the training data during the learning phase. In the inference phase, calculating the Mahalanobis distance typically requires an inverse variance-covariance matrix (Σ) of size N. f ×N r it.

[0007] The algorithm required to update the variance-covariance matrix (Σ) during the learning phase must have an affinity for singular value decomposition, more precisely an algorithm that has an affinity for low-ranking approximation based on singular value decomposition. SOLUTION TO THE PROBLEM

[0008] A data processing device according to the present technology comprises a processing circuit in which, during a learning phase, the processing circuit sequentially updates a Gram matrix in SVD form and computes a variance-covariance matrix in SVD form based on an SVD that refers to a Gram matrix described later during the finalization of the learning phase. ADVANTAGEOUS EFFECTS OF THE INVENTION

[0009] A data processing device according to the present technology comprises the above-mentioned configuration and an algorithm for updating a variance-covariance matrix (Σ) in a learning phase, which has an affinity for singular value decomposition and also an affinity for low-ranking approximation based on singular value decomposition.

[0010] Therefore, according to the present technology, the data processing device and the data processing method can benefit from singular value decomposition and the low-ranking approximation based on singular value decomposition during the learning phase. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1A and Fig. Figure 1B shows representations of the hardware configuration illustrating a hardware configuration of a data processing device according to a first embodiment. Fig. Figure 2 is a block diagram that schematically represents a function used by the data processing device according to the present disclosed technology. Fig. Figure 3 is a block diagram that schematically represents a parameter update technique used by the data processing device according to the first embodiment. Fig. Figure 4 is a first flowchart illustrating the processing steps during parameter updating of the data processing device according to the first embodiment. Fig. Figure 5 is a second flowchart illustrating the processing steps during parameter updates of the data processing device according to the first embodiment. Fig. Figure 6 is a flowchart illustrating the processing steps in the distance calculation of the data processing device according to the first embodiment. Fig. Figure 7 is an explanatory representation which describes a data processing method according to a second embodiment. Fig. Figure 8 is an explanatory representation which describes a data processing method according to a third embodiment. DESCRIPTION OF THE EXECUTION FORMS << Introduction 1> >

[0011] It is assumed that an image feature set (x) processed by the present technology is given by a vertical vector (column vector) as follows. c=1 to Nd for xc∈ℝNf×1

[0012] Here stands N f for the length of the feature set. The lower right subscript "f" in N f It is derived from the first letter of the feature. N drepresents the number of training data parts. It should be noted that in this specification, the training data will be referred to simply as "data". The lower right subscript "d" in N d is derived from the first letter of the data. In addition, c of the variable is derived from the first letter of the column that designates a column (see the following mathematical formula (2)).

[0013] A data matrix (X) formed by arranging the set of image features of a large number of images of a certain class is given by the following mathematical formula. X|Nd=[x1,⋯xc,⋯xNd]therefore X|Nd⊂ℝNf×Nd

[0014] Unless otherwise stated, the present description assumes that the number of data elements is sufficient and N d >N f applies.

[0015] The expected value (µ) of the image features belonging to a class is expressed by the following mathematical formula. μ|Nd=E(X|Nd)=1Nd∑c=1Ndxctherefore μ|Nd∈ℝNf×1

[0016] Here, E() of the function represents an expected value. Generally, expected value and average value are distinct concepts, but in this case, the expected value (µ) of the image feature set corresponds to the average (hereafter referred to as the "average feature set") of the N. d -Image feature sets that belong to the class.

[0017] The variance-covariance matrix (Σ) for the data matrix (X) is given by the following mathematical formula using µ. ∑|Nd︷Variance−Covariance Matrix=E(X|Nd︷Gram's)︷Correlation Matrix−μ|NdμT|Ndtherefore ∑|Nd∈ℝNf×Nf

[0018] Here, the letter "T" in the upper right index of mathematical formula (4) stands for transposition. Note that the variance-covariance matrix can also simply be called the "covariance matrix". Furthermore, XX T The term appearing on the right-hand side of mathematical formula (4) is called the Gram matrix. Furthermore, the expected value of the Gram matrix, i.e., E(XX), is given by... T ) of the first term on the right-hand side of the mathematical formula (4), is called the correlation matrix.

[0019] The variance-covariance matrix (Σ) can be derived by modifying the mathematical formula (4), but can also be obtained using a deviation vector (x) c -µ) express the mean value. ∑|Nd=E([x1−μ]Nd'⋯xNd−μ|Nd[(x1−μ|Nd)T⋮(xNd−μ|Nd)T])=1Nd[x1−μ|Nd'⋯xNd−μ|Nd][(x1−μ|Nd)T⋮(xNd−μ|Nd)T]

[0020] If the deviation vector (x) is here c-µ) of the c-th column again y c The mathematical formula (5) can be modified as follows. yc:=xc−μ|Nd then ∑|Nd=1Nd[y1,⋯yNd]︷=:Q1Nd[y1T⋮yNdT]︷=:QT≽0

[0021] The second line of the mathematical formula (6) shows that the variance-covariance matrix (Σ) can be expressed as QQT, i.e. the variance-covariance matrix (Σ) is a positive semidefinite matrix.

[0022] The Mahalanobis distance (d M ) is frequently used to express the degree of similarity or the degree of deviation of the target sample (x target ) to measure based on the measurement result of the normal distribution. The Mahalanobis distance (d M ) is given using the following mathematical formula using a variance-covariance matrix (Σ). dM︷∈ℝ=(xtarget−μ|Nd)T∑−1|Nd(xtarget−μ|Nd) if ∑|Nd≻0

[0023] As shown in mathematical formula (7), the Mahalanobis distance (d) M ) a distance that can be defined if an inverse matrix (Σ-1) of the variance-covariance matrix (Σ) exists, i.e., if the variance-covariance matrix (Σ) is a positive definite value. The Mahalanobis distance (d M ), represented by the mathematical formula (1) to mathematical formula (7), is a distance that is in a feature space with a dimension of N f is defined.

[0024] The present technology addresses the question of how a variance-covariance matrix (Σ) is updated when the number of data elements of a given class of N changes. d on N d +1 increases, and demonstrates this. If the variance-covariance matrix (Σ) increases with an increase in the number of data elements to N d Given +1 in the form of a singular value decomposition, the inverse matrix (Σ- 1) of the variance-covariance matrix (Σ) for calculating the Mahalanobis distance (d M ) easily determine. << Introduction 2 >>

[0025] It is known that every matrix can be represented by a singular value and a singular vector. A form in which a matrix is ​​decomposed into a singular value and a singular vector is called a singular value decomposition. A singular value decomposition of a particular pxq matrix (Z≠0) is expressed as follows. Z︷∈ℝp×q=[u1,⋯ur]︷U∈ℝp×r[σ1 ⋱ σr]︷S∈ℝr×r[v1T⋮vrT]︷VT∈ℝr×q

[0026] Here, r represents a rank of Z. {σ1,..., σ r The values ​​}, which appear in mathematical formula (8), are singular values ​​of Z. S is a diagonal matrix whose components are the singular values ​​{σ1,..., σ r} are. The {u1,..., u appearing in mathematical formula (8) r} are considered the left singular vector with respect to the singular value {σ1,..., σr} denotes. The {v1,..., v} appearing in mathematical formula (8) r} are considered the right singular vector with respect to the singular value {σ1,..., σ r The left singular vector and the right singular vector are together referred to as the singular vector.

[0027] Studies were conducted to determine the singular value decomposition of Z+AB. T to determine, given the singular value decomposition of a certain pxq matrix (Z≠0). For example, the following non-patented literature discloses the algorithm.

[0028] Matthew Brand, “Fast Low-Rank Modifications of the Thin Singular Value Decomposition,” MERL Technical Report, TR2006 - 059, May 2006.

[0029] An algorithm for determining the singular value decomposition of Z+ABT is described as a program and can be a function of a function library. In the present specification, the function for determining the singular value decomposition of Z+AB is described. T as follows, expressed as "IncrSVD". {Uout,Sout,Vout}=IncrSVD︷Function{Uin,Sin,Vin,Ain,Bin} where UoutSoutVoutT=UinSinVinT︷Z+AinBinT

[0030] Here, “Incr” is derived from the first four characters of “Incremental” and “SVD” from the initial letters of “Singular Value Decomposition”. << Introduction 3 >>

[0031] If a certain matrix (Z, Z≠0) can be expressed by singular value decomposition (see mathematical formula (8)), the general inverse matrix of the Moore-Penrose type (hereinafter referred to simply as the "general inverse matrix", also called the pseudoinverse matrix) is obtained from the following mathematical formula. Z−︷∈ℝq×p=V︷∈ℝq×r[1σ1 ⋱ 1σr]︷S−1∈ℝr×rUT︷∈ℝr×p=1σ1v1u1T︷∈ℝq×p+⋯+1σrvrurT︷∈ℝq×p

[0032] If Z is a regular matrix (p=q), then the general inverse matrix (Z-) of Z is formed with the inverse matrix (Z- 1 ) of Z. Inverse matrices are defined only for regular matrices, while general inverse matrices are defined for non-zero matrices. However, to compute the general inverse matrix, the rank must be known.

[0033] Since the vector is also a type of matrix, the general inverse matrix for the vector is also defined.

[0034] In general, calculations in physics and engineering always contain computational errors because they use observational data acquired by measuring devices and sensors. This is no exception in the technical field, to which the data processing device 100 according to the present disclosed technology belongs. Therefore, when calculating the singular value decomposition for a matrix calculated on the basis of observational data, all singular values ​​(σ) i , i is a natural number from 1 to N f ) is positive in the numerical calculation. In a case where a singular value that is originally zero assumes a non-zero value due to an error in the numerical calculation, the singular value is positive due to 1 / σ. i to an unrealistic value if an inverse matrix (including a general inverse matrix) of the matrix is ​​calculated unchanged.

[0035] To determine the rank of Z in the pxq matrix obtained from the measurement data, a procedure is used in which a preliminary rank (I=min(p, q)) is first determined and the following singular value decomposition is calculated. Z=σ1u1v1T+⋯+σlulvlT where l=min(p,q)σ1≥⋯≥σl

[0036] The procedure for determining the rank then examines which singular value can be used to approximate the final singular value to zero. For example, a minimum limit that can be processed by a computer (referred to as the "machine epsilon") is used to specify a tolerance for the rank of the matrix. σr+1≈0, ⋯ σl≈0 (Z) r in a case where the rank of Z is given by the following mathematical formula. (Z)r:=σ1u1v1T+⋯+σrurvrT

[0037] A technique used to create (Z) rAccording to mathematical formula (13), it is similar to a “low-ranking approximation based on a singular value decomposition”. The general inverse matrix for the low-ranking approximation is called the “rank-restricted general inverse matrix”. The rank-restricted general inverse matrix is ​​also called the rank-restricted pseudoinverse matrix.

[0038] Between Z and (Z) r The following relational expression is created. Z−(Z)r=U[0⋱0σrr+1⋱σl]VT

[0039] The error between Z and (Z) r is given by the following mathematical formula. ‖(Z)r−Z‖=σr+12+⋯+σl2

[0040] Here, the symbol to the left of the mathematical formula (15) is the Frobenius norm or Euclidean norm.

[0041] The rank-restricted general inverse matrix is ​​a general inverse matrix that uses a low-ranking approximation based on singular value decomposition. The singular value decomposition-based low-ranking approximation has similar properties (advantages and disadvantages) to the general approximation. The disadvantage of this approximation is that it introduces an error relative to the actual value. The advantage is that information can be reduced to the essentials. The singular value decomposition-based low-ranking approximation has the disadvantage of an error relative to the actual value and the advantage of reducing information to the necessary level.

[0042] Many of the facts described in Introduction 3 are taken from prior art document 1. In addition, prior art document 1 describes a proof for the derivation of an error given in mathematical formula (15).

[0043] Citation 1: Kenichi Kanaya, “linear algebraic seminar, projection, singular value decomposition, general inverse matrix,” Kyoritsu Shuppan, ISBN978-4-320-11340-4. First embodiment.

[0044] The data processing device 100 according to a first embodiment illustrates a minimum required configuration that is necessary for the data processing device 100 according to the present disclosed technology.

[0045] Fig. 1A and Fig. Figure 1B shows representations of the hardware configuration illustrating a hardware configuration of a data processing device 100 according to a first embodiment.

[0046] Fig. Figure 1A is a hardware configuration diagram illustrating a hardware configuration in a case where functions of the data processing device 100 according to the first embodiment are performed by dedicated hardware. As shown in Fig. As shown in Figure 1A, the hardware configuration when executed by dedicated hardware comprises an input interface 110, a processing circuit 120 and an output interface 130.

[0047] The processing circuit 120 corresponds, for example, to a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof. The functions of the data processing device 100 can be implemented by separate processing circuits 120 or jointly by a single processing circuit 120.

[0048] Fig. Figure 1B is a hardware configuration diagram illustrating a hardware configuration in a case where functions of the data processing device 100 according to the first embodiment are performed by dedicated hardware. As shown in Fig. As shown in Figure 1B, the hardware configuration when executed by software includes the input interface 110, a processor 122, a memory 124 and the output interface 130.

[0049] The 122 processor is also generally referred to as a central processing unit (CPU), processing unit, arithmetic unit, microprocessor, microcomputer or digital signal processor (DSP).

[0050] In a case where the hardware configuration of the data processing device is 100 as in Fig. As shown in Figure 1B, each function of the data processing device 100 is implemented by software, firmware, or a combination of software and firmware. The software and firmware are described as programs and stored in the main memory 124. The processor 122 executes each function by reading and executing the program stored in the main memory 124. That is, the data processing device 100 includes the main memory 124 for storing a program which, when executed by the processor 122, causes each processing step to be carried out. Furthermore, it can also be said that these programs cause the processor 122 to execute the process and procedure of the data processing device 100 (data processing procedure according to the present disclosed technology).Here, the 124-bit memory can be, for example, non-volatile or volatile semiconductor memory such as RAM, ROM, flash memory, EPROM, or EEPROM. The 124-bit memory can also be a storage medium such as a magnetic hard disk, a flexible hard disk, an optical hard disk, a compact disc, a mini-disc, or a DVD. Furthermore, the 124-bit memory can be in the form of a hard disk drive (HDD) or a solid-state drive (SSD).

[0051] It should be noted that some of the functions of the data processing device 100 according to the first embodiment may be implemented by dedicated hardware and the others may be implemented by software or firmware.

[0052] In the data processing device 100 according to the first embodiment, the functions described above are implemented by hardware, software, firmware or a combination thereof.

[0053] A technical feature of the data processing device 100 according to the present technology is that the variance-covariance matrix (Σ) is neither in the form of a matrix with a size of N f ×N r is neither saved nor used.

[0054] Since the variance-covariance matrix (Σ) expressed by mathematical formula (6) is a positive semidefinite matrix, the variance-covariance matrix (Σ) can be modified as follows. ∑|Nd=Q|NdQT|Nd

[0055] Next, it is assumed that Q of a matrix appearing in mathematical formula (16) is expressed in a singular value decomposition format. Q|Nd=U|Nd[σ1⋱σNf]︷S|NdVT|Nd

[0056] In the present specification, the singular values ​​are arranged in order of their magnitude, and it is assumed that σ1 ≥ σ2 ≥... ≥ σ NFThis applies. In general, Sigma (especially “σ”) is used. 2 “) is often used as a symbol for variance, but in the present specification, as described above, Sigma represents a singular value.

[0057] The following mathematical formulas apply to U and V, which appear in mathematical formula (17) due to the property of the singular vector. {UT|NdU|Nd=I︷∈ℝNf×NfVT|NdV|Nd=I︷∈ℝNf×Nf

[0058] Here, the unit matrix appearing in mathematical formula (18) is a matrix of size N. f ×N f .

[0059] By substituting the singular value decomposition shown in mathematical formula (17) into mathematical formula (16), the variance-covariance matrix (Σ) can be decomposed as follows. ∑|Nd=U|Nd[(σ1)2⋱(σNf)2]︷S2|NdUT|Nd

[0060] As shown in mathematical formula (19), {σ1 2 ,σ2 2 ,...,σNf 2} Singular values ​​of the variance-covariance matrix (Σ).

[0061] The inverse matrix (Σ-1) of the variance-covariance matrix (Σ) is obtained from the following mathematical formula due to the property expressed by mathematical formula (18) and mathematical formula (19). ∑−1|Nd=∑−|Nd =U|Nd[(σ1)2⋱(σNf)−2]︷S−2|NdUT|Nd

[0062] As described above, for a regular matrix, the general inverse matrix is ​​compared with the inverse matrix. Note that when deriving mathematical formula (20), all singular values ​​of the variance-covariance matrix (Σ) are assumed to be non-zero. A countermeasure for the case where the variance-covariance matrix (Σ) does not have full rank will become apparent from the following description.

[0063] In this description, a form in which a matrix is ​​decomposed and expressed by a product of the matrices is called a "decomposition form." Singular value decomposition is a type of decomposition form and represents a special case. Singular value decomposition is also given on the right-hand side of mathematical formula (20). If the variance-covariance matrix (Σ) is given by singular value decomposition as described above, a problem can be solved where the matrix appears to assume full rank due to a numerical error (see Introduction 3). Here, singular value decomposition can also be called SVD.

[0064] The data processing device 100 according to the present disclosed technology calculates the Mahalanobis distance (d M ) using the decomposition form given on the right-hand side of mathematical formula (20). dM2=(xtarget−μ|Nd)T︷ytargetTU|NdS-2|NdUT|Nd︷∑−1|Nd(xtarget−μ|Nd)︷ytarget

[0065] As shown in mathematical formula (21), there is no problem if, instead of the Mahalanobis distance (d M ) a square (d M 2 ) of the Mahalanobis distance (d M ) is used to determine the degree of similarity or the degree of deviation of a given sample (x target ) to measure. Furthermore, as shown in mathematical formula (21), the set of features (x) can be measured. target ) for the target also by a deviation vector (y) target =x target -µ) is represented, which is defined as the deviation from the average set of characteristics (µ). (Update procedure when new data is added)

[0066] Fig. Figure 2 is a block diagram schematically representing a function used by the data processing device 100 according to the present disclosed technology. More precisely, it is Fig. 2 a block diagram illustrating IncrSVD(), a function expressed by the mathematical formula (9).

[0067] Fig. Figure 3 is a block diagram schematically representing a data update technique used by the data processing device 100 according to the first embodiment. More precisely, Fig. 3 a block diagram schematically representing a data update technique performed by the data processing device 100 according to the first embodiment when the number of data elements of N d on N d +1 increases.

[0068] It is ideal if U, S, and V, the matrices in the singular value decomposition form of the variance-covariance matrix (Σ), can be directly updated when the number of data elements increases, but this is not strictly necessary. As in Fig. As shown in Figure 3, the data processing device 100 according to the first embodiment updates the SVD of the Gram matrix using the first IncrSVD() when the number of data elements increases, and obtains the SVD of the variance-covariance matrix (Σ) from the SVD of the correlation matrix using the second IncrSVD(). Fig. 3 shows the first IncrSVD() as a vertically long block and the second IncrSVD() as a horizontally long block.

[0069] If the number of data elements is N d The SVD of the Gram matrix is ​​calculated as follows. X|NdXT|Nd=U0S0V0T

[0070] Here, the subscript number “0” in the lower right corner after the matrix on the right side of the mathematical formula (22) is simply a number that indicates for which matrix the SVD is valid.

[0071] If the number of data elements is N d When the value is increased by +1, the Gram matrix is ​​updated using the first IncrSVD(). {U1,S1,V1}=IncrSVD{U0,S0,V0, xNd+1,xNd+1}whereU1S1V1T=X|Nd+1XT|Nd+1

[0072] Here, the lower right ending “1”, which was added to the matrix on the left side of the mathematical formula (23), is a number that indicates for which matrix the SVD is valid, similar to “0” in the mathematical formula (22).

[0073] The relationship between the Gram matrix and the correlation matrix is ​​described by mathematical formula (4). That is, the expected value of the Gram matrix is ​​a correlation matrix. Using this relationship, the SVD of the correlation matrix is ​​obtained as follows. E(X|Nd+1XT|Nd+1)=1Nd+1X|Nd+1XT|Nd+1 =U1(1Nd+1S1)V1T

[0074] If x nd+1 When something new is added to the data, the average feature set (µ|N) is changed. d +1) updated as follows. μ|Nd+1=NdNd+1μ|Nd+1Nd+1xNd+1

[0075] The process represented in mathematical formula (25) is called the “update of the average set of features”.

[0076] Finally, the SVD of the variance-covariance matrix (Σ) is calculated for a data element count of N. d +1 using the updated average feature set (µ| N d+1) and the second IncrSVD() as follows. {U2,S2,V2}=IncrSVD{U1,1Nd+1S1,V1−μ|Nd+1,μ|Nd+1} where U2S2V2T=∑|Nd+1

[0077] As described above, the analysis was based on Fig. 3. A data update technique is explained in which the number of data elements is N. d to one on N dThe number of data elements is increased by one, but the technology described here is not limited to this. For example, according to the present technology, the data processing device 100 can pre-divide data into a multitude of batches and update the data in batch units. The technique of updating data in batch units corresponds to the technique of sequentially increasing the number of data elements by one at a time. That is, even when the number of data elements is updated in batch units, the SVD of the Gram matrix can be updated using the first IncrSVD(), and the SVD of the variance-covariance matrix (Σ) can be determined from the SVD of the correlation matrix using the second IncrSVD(). (Mahalanobis distance using a rank-restricted general inverse matrix)

[0078] As shown in mathematical formula (6), the variance-covariance matrix (Σ) is a positive semidefinite matrix. If the variance-covariance matrix (Σ) is not a full-rank matrix, there is no inverse matrix (Σ-1) of the variance-covariance matrix (Σ). Even if the variance-covariance matrix (Σ) is a full-rank matrix in the numerical computation, a so-called zero division occurs numerically when one of the singular values ​​of the variance-covariance matrix (Σ) is close to zero. Furthermore, the problem of apparent full-ranking arises due to numerical error, as described in Introduction 3.

[0079] To prepare for such a case, the Mahalanobis distance can be determined by reducing the dimension of the feature space within the tolerance range.

[0080] The data processing device 100 according to the present disclosed technology intentionally lowers the rank of the variance-covariance matrix (Σ) to k (k ≤ r). If the number of data elements N d The value obtained by lowering the rank of the variance-covariance matrix to k is calculated using the following mathematical formula. ∑|Nd∼(∑|Nd)k︷∈ℝNf×Nf=U|Nd︷∈ℝNf×Nf[σ12⋱σk20⋱0]︷∈ℝNf×NfUT|Nd︷∈ℝNf×Nf =[u1,⋯uk]︷(U)k∈ℝNf×k[σ12⋱σk2]︷(S2)k∈ℝk×k[u1T⋮ukT]︷(U)kT∈ℝk×Nf where k≤r (•) k “In mathematical formula (27), “ represents an approximation where the rank in dimension is reduced to k. At this point, the error due to the reduction of the rank of the variance-covariance matrix (Σ) can be evaluated as follows. ‖(∑|Nd)k−∑|Nd‖=σk+14+⋯+σNf4.

[0081] If the rank k is (k ≤ r)S, then the Mahalanobis distance is finally given as follows. (dM2)k=(xtarget−μ|Nd)T︷ytargetT(U)k(S2)k−1(U)kT︷(∑|Nd)k−1(xtarget−μ|Nd)︷ytarget

[0082] As described above, the data processing method according to the present technology exhibits a high affinity for the low-ranking approximation based on singular value decomposition during the learning phase and can thus benefit from the low-ranking approximation, which can reduce the number of parameters to be stored. However, when determining the Mahalanobis distance of a subspace reduced in the inference phase, it is important that the subspace is a space projected by a specific projection matrix and that the number of dimensions of the subspace is known. The reason for this becomes clear from the description of the "singular value zero with meanings" in a fifth embodiment.

[0083] The data processing device 100 according to the present technology outputs the following data in combination with the calculated Mahalanobis distance.

[0084] Output data: {(dM)kor(dM2)k,(U)k,(S2)kor(S2)k−1}

[0085] That is, the data processing device 100 according to the present technology outputs the calculated Mahalanobis distance as well as the singular vector and the singular value with respect to the variance-covariance matrix (Σ).

[0086] Fig. Figure 4 is a first flowchart illustrating the processing steps during the parameter update of the data processing device 100 according to the first embodiment. As shown in Fig. As shown in Figure 4, the processing steps of the data processing device 100 according to the first embodiment can be divided into steps ST01 to ST07.

[0087] The processing step (ST01) described as "acquisition of a new image feature set (x)" is a processing step that is performed via the input interface 110. In ST01, the data processing device 100 acquires, for example, N d +1st data x Nd +1 via input interface 110.

[0088] A feature called "update of the number of data elements (N)" d The processing step designated ST02 is a processing step executed by the processing circuit 120 or the processor 122. In ST02, for example, the processing circuit 120 or the processor 122 updates the count of the number of data elements of N. d on N d +1.

[0089] A processing step (ST03) designated as "Update of average feature set (µ)" is a processing step performed by the processing circuit 120 or the processor 122. In ST03, the processing circuit 120 or the processor 122 performs the "Update of average feature set" specified in mathematical formula (25).

[0090] A processing step (ST04) designated as “U, S, V Gram Matrix Update” is a processing step performed by the processing circuit 120 or the processor 122. In ST04, the processing circuit 120 or the processor 122 updates the SVD for the Gram matrix using the first IncrSVD(), which is expressed in mathematical formula (23).

[0091] A term described as "calculation of U and S" 2The processing step designated for Σ (ST05) is a processing step executed by the processing circuit 120 or the processor 122. In ST05, the processing circuit 120 or the processor 122 calculates the SVD of the variance-covariance matrix (Σ) when the number of data elements N d +1 is, using the second IncrSVD(), which is expressed by the mathematical formula (26).

[0092] The one referred to as "storage of U and S -2 for Σ -1 The processing step described (ST06) is a processing step that is performed using the working memory 124 or an external storage device. In ST06, the data processing device 100 stores the singular vector (U) and the singular value (S). -2 ) for the last Σ -1 in RAM 124 or in the external storage device.

[0093] Fig. Figure 5 is a second flowchart illustrating the processing steps during the parameter update of the data processing device 100 according to the first embodiment.

[0094] The in Fig. The 5 processing steps shown are the same as those in Fig. 4 processing steps shown. However, as in Fig. 5 shown, used in Fig. The processing step shown in Figure 5 uses a k-dimensional feature space defined by a fixed (U) k T is depicted. The set of image features (x with an accent mark in the form of a line), which is represented by the in Fig. The following mathematical formula describes how the process is handled as shown in the flowchart 5. fu¨rc=1 to Nd xc¯︷∈ℝk×1:=[u1T⋮ukT]︷(U)kT∈ℝk×Nfxc︷∈ℝNf×1

[0095] However, (U) k T, which appears in the mathematical formula (31), an empirically determined matrix that maps from N f makes a conversion from -dimensional space to k-dimensional space.

[0096] (U) k T The matrix can contain the first to k-th right singular vectors with respect to the variance-covariance matrix (Σ) if the data are sufficiently rich, i.e., contain enough information to reproduce the properties of the class to which the data belong (see mathematical formula (27)). A technique for determining whether the data are sufficiently rich is given in the description of a sixth embodiment.

[0097] Fig. Figure 6 is a flowchart illustrating the processing steps in the distance calculation of the data processing device 100 according to the first embodiment. While the Fig. 4 and Fig. Illustrating 5 processing steps that are carried out in the learning phase of the data processing device 100, illustrated Fig. Six processing steps are performed in the inference phase of the data processing device 100. As described in Fig. As shown in Figure 6, the processing step in the inference phase can be divided into ST11 to ST13.

[0098] Note that although the data processing device 100 in the learning phase and the data processing device 100 in the inference phase are not specifically shown, they can be different devices. That is, there can be a data processing device 100 that performs inference separately from the data processing device 100 that performs learning.

[0099] The “sample capture (x)” targetThe processing step designated (ST11) is a processing step performed by the data processing device 100, which performs the inference. In ST11, the data processing device 100, which performs the inference, records xt arget , which is a set of image features for which the Mahalanobis distance is to be determined.

[0100] A term used to describe “reading U and S” -2 for Σ -1 The processing step described (ST12) is a processing step performed by the data processing device 100, which performs the inference. In ST12, the data processing device 100, which performs the inference, reads U and S. -2 for Σ -1 , which are stored in ST06 of the learning phase.

[0101] A calculation of the Mahalanobis distance (d MThe processing step designated (ST13) is a processing step performed by the data processing device 100, which performs the inference. The data processing device 100, which computes the inference, calculates the Mahalanobis distance (d M ) or the square (d M 2 ) of the Mahalanobis distance (see mathematical formulas (7), (21) and (29)).

[0102] To clarify the feature space in which the calculated Mahalanobis distance is defined, the data processing device 100, which performs the inference, provides the data specified in mathematical formula (30), i.e., (U) k and (S 2 )k or (S -2 ) k , together with the calculated Mahalanobis distance. (Simple numerical example)

[0103] The data processing method according to the present technology is explained in more detail using the following simple numerical examples. It is assumed that data belonging to a specific class is specified as follows. X|3=[121110119314020] x4=

[13020]

[0104] The numerical example shown in mathematical formula (32) is a case in which the number of data elements (N) d ) is increased from 3 to 4.

[0105] It is assumed that the average feature set and the SVD of the Gram matrix with a data element count of 3 are given as follows. μ|3=13(

[12131] +

[11040] +

[11929] )=[116.666733.33333]X|3XT|3=[121110119314029][121311104011929] =[4090211602116023402] =[−0.9618652+0.2735238−0.2735238−0.9618652]︷U0[44,201.2400102.7618]︷S0[−0.9618652−0.2735238+0.2735238−0.9618652]︷V0T

[0106] Please note that due to space limitations on paper, the numerical examples in this specification are shown with a maximum of seven digits.

[0107] The first IncrSVD() update the SVD of the Gram matrix. {U1,S1,V1}=IncrSVD{U0,S0,V0,x4,x4} where U1=[−0.9708330+0.2397568−0.2397568−0.9708330] S1=[61.309.3200294.6762] V1T=[−0.9708330−0.2397568+0.2397568−0.9708330]

[0108] As shown in mathematical formula (34), the left singular vector and the right singular vector coincide due to the symmetry of the Gram matrix.

[0109] The SVD obtained by the first IncrSVD() is the SVD of the Gram matrix if the number of data elements is 4. U1S1V1T=[5780214202142023802] where X|4XT|4=[12111011913031402920][12131110401192913020] =[5780214202142023802]

[0110] The average set of features is updated as follows. μ|4=14(3[116.666733.3333]+

[13020] ) =

[12030]

[0111] The second IncrSVD() yields the SVD of the variance-covariance matrix (Σ) as follows. {U2,S2,V2}=IncrsSVD{U1,14S1,V1,−μ|4,μ|4} where U2=[+0.7071068−0.7071068−0.7071068−0.7071068]S2=[100001]V2T=[+0.7071068−0.7071068−0.7071068−0.7071068]

[0112] The SVD obtained by the second IncrSVD() is the SVD of the variance-covariance matrix (Σ) when the number of data elements is 4. U2S2V2T=[+50.5−49.5−49.5+50.5] where ∑|4=14[+1−10−1+10+1+10−1−10][+1+1−10+10−1−1+10−10] =14[+202−198−198+202]=[+50.5−49.5−49.5+50.5]︷=U2S2V2T

[0113] A technical feature of the data processing device and the data processing method according to the first embodiment is that an algorithm for updating a variance-covariance matrix (Σ) during a learning phase has an affinity for singular value decomposition. More precisely, the data processing device and the data processing method according to the first embodiment include a function with an affinity for singular value decomposition, called IncrSVD(), in an algorithm for updating a variance-covariance matrix (Σ) during a learning phase.

[0114] With this technical feature, the data processing device and data processing method according to the first embodiment can benefit from singular value decomposition, which can overcome the problem of apparent full rank due to numerical errors (see Introduction 3).

[0115] A technical feature of the data processing device and the data processing method according to the first embodiment is that an algorithm for updating a variance-covariance matrix (Σ) during a learning phase has an affinity for the low-ranking approximation based on a singular value decomposition. More precisely, the data processing device and the data processing method according to the first embodiment include a function that has an affinity for the low-ranking approximation based on the singular value decomposition of IncrSVD() in the algorithm for updating the variance-covariance matrix (Σ) during the learning phase.

[0116] With this technical feature, the data processing device and the data processing method according to the first embodiment can benefit from the low-ranking approximation based on singular value decomposition, such that the information can be reduced to the required amount during the learning phase.

[0117] The data processing device and data processing method according to the first embodiment are applied to the detection of anomalies using PaDiM, and their effects are examined. In this application, the image is divided into 56×56 small regions. For each small region, the variance-covariance matrix (Σ) is updated. As described in the non-patent literature 1 on PaDiM, Wide ResNet50 is used for CNN, and the length (N) fThe number of features in the generated image feature set is 1792. Even with a simple calculation, the storage requirement for updating during the learning phase is 56 × 56 × 1792 × 1792 = 40 GB. Even considering that the variance-covariance matrix (Σ) is a symmetric matrix, a storage requirement of approximately 20 GB is still needed.

[0118] The data processing device and the data processing method according to the first embodiment have shown that the dimension of the update in the learning phase in this application example can be reduced to k=20. Second embodiment.

[0119] A data processing device and a data processing method according to a second embodiment are modifications of the data processing device and the data processing method according to the present disclosed technology. In the second embodiment, the same reference numerals are used as in the first embodiment, unless otherwise specified. In addition, in the second embodiment, the description that overlaps with the first embodiment is optionally omitted.

[0120] In order to differentiate it from the technology described in the first embodiment, the data processing method according to the second embodiment is referred to as “IncrPCA” in the present description. Fig. Figure 7 is an explanatory representation which describes a data processing method according to a second embodiment.

[0121] Incidentally, a core idea of ​​IncrSVD() is the introduction of the following extended system. Z+ABT=[UA][S00I][VTBT]

[0122] The technique for introducing the extended system is also effective for cases handled with the present technology. It is assumed that the SVD of the Gram matrix is ​​as given in mathematical formula (22) when the number of data elements N is... d The value is [value missing]. However, since the Gram matrix is ​​a positive semidefinite matrix in square form, the SVD of the Gram matrix can be expressed as follows. X|NdXT|Nd=UG0︷U0SG0SG0T︷S0UG0T︷V0T

[0123] Note that the low-ranking approximation can be performed for the SVD of the Gram matrix. The condition under which the low-ranking approximation can be performed is clearly evident from the fifth embodiment.

[0124] Upon receiving the N d With +1 data, the Gram matrix can be represented by the following extended system. X | NdXT |

[0125] The mathematical formula (41) suggests the following. SVD{[UG0xNd+1][SG0001]}=SVD{X|Nd+1}

[0126] Here, “SVD” refers to the font of the script in the mathematical formula (42) for a function to determine the singular value decomposition.

[0127] As described above, the data update technique was used when the number of data elements increased from Nd to N. dThe process of increasing the value by one is illustrated by the introduction of an extended system, but the present technology is not limited to this. For example, according to the present disclosed technology, the data processing device 100 can pre-divide data into a plurality of batches, add data in batch units, and create an extended system. The technique of introducing the extended system in batch units corresponds to the concept of sequentially increasing data by one value at a time.

[0128] Batch processing corresponds to a situation in which n b New data elements are added when the number of data elements exceeds N. d is. When such data processing is carried out, the mathematical formula (42) is rewritten as follows. SVD{[UG0xNd+1⋯xNd+nb]︷∈ℝNf×(Nf+nb)[SG000I(nb)]︷∈ℝ(Nf+nb)×(Nf+nb)}=SVD{X|Nd+nb}.

[0129] The n b,, which appears in the mathematical formula (43), is an identity matrix with a size of n b ×n b .

[0130] As described above, the low-ranking approximation on the SVD of the Gram matrix can be approximated to dimension k (k <r) angewendet werden. Die Tatsache, dass die Low-Ranking-Approximation auf die k-Dimension durchgeführt wird, bedeutet, dass nur die ersten k Singulärwerte, die in absteigender Reihenfolge angeordnet sind, unter den Singulärwerten der Gramschen Matrix verwendet werden (siehe beispielsweise mathematische Formel (27)). Da die Aktualisierung der Gramschen Matrix jedoch in einer Schleife erfolgt, summiert sich der durch die Low-Ranking-Approximation verursachte Fehler.

[0131] In the detection of anomalies by PaDiM, the inventor of the present technology has the original dimension of N fThe value of k = 1792 was reduced to k = 20, and the Gram matrix was updated by loop processing. In this application example, the inventor of the present technology found that this cumulative error lies within an acceptable range. (Simple numerical example)

[0132] Using the same numerical examples as in the first embodiment, the data processing method according to the second embodiment is further clarified. If the number of data elements is three, UG0 and SG0 of the Gram matrix are specified as follows. UG0=[−0.9618652+0.2735238−0.2735238−0.9618652]SG0=[+210.240900+10.1315] Note that U G0 is the same as U0.

[0133] By applying the numerical example, the mathematical formula (42) is calculated as follows. [UG0x4][SG0001]=[−0.9618652+0.2735238130−0.2735238−0.961865220][+210.2409000+10.137150001] =[−202.2234+2.772752130−57.50588−9.75057320]

[0134] Furthermore, the matrix calculated using the mathematical formula (45) can be expressed in the following SVD format. SVD{[−202.2234+2.772752130−57.50588−9.75057320]}=U3S3V3T=SVD{X|4} where U3=[−0.9708330−0.23975680−0.2397568+0.9708330]=[U10]S3=[+247.6072000+17.16614000+0]=[S10.50]V3 T=[+0.8485722−0.0014301−0.5290776−0.4278291−0.5901715−0.6845874−0.3112675+0.8072766−0.5014150]

[0135] Here, the number "3" in the mathematical formula (46) is simply a number that indicates for which matrix the SVD is valid.

[0136] The first singular vector and the second singular vector of U3, calculated according to mathematical formula (46), are identical to the first singular vector and the second singular vector of U1, calculated according to the X|4XT|4=(U3S3V3T)(U3S3V3T)T=U3S32U3T=[U10][S10.5 0]2[U1T0T]=U1S1U1T mathematical formula (34). Furthermore, the first singular value and the second singular value of S3, calculated using mathematical formula (46), are squared to the first singular value and the second singular value of S1, calculated using mathematical formula (34).

[0137] As shown in mathematical formula (47), the data processing device according to the second embodiment extends the dimension of the matrix to (N f +n b )×(N f +n b ), but n bSingular values ​​become 0 in the SVD of the Gram matrix, resulting in a decomposition form with a size of N. f ×N r will be received.

[0138] The processing steps in the data processing procedure according to the mathematical formula (47) and the following expressions are the same as in the data processing procedure according to the first embodiment.

[0139] A technical feature unique to the data processing device and data processing method according to the second embodiment is that it includes an algorithm for introducing an extended system and updating a Gram matrix in a learning phase.

[0140] With this technical feature, the data processing device and the data processing method according to the second embodiment have, in addition to the effect described in the first embodiment, the effect that they can be implemented using a general SVD function instead of IncrSVD(). Third embodiment.

[0141] A data processing device and a data processing method according to a third embodiment are modifications of the data processing device and the data processing method according to the present disclosed technology. In the third embodiment, unless otherwise specified, the same reference numerals are used as in the preceding embodiments. In the third embodiment, descriptions that overlap with those of the previously described embodiments are expediently omitted.

[0142] To enable differentiation from the methods described in the previously described embodiments, the data processing method according to the third embodiment is referred to in the present description as “GPU-oriented IncrPCA”.

[0143] The data processing device according to the present technology can use a graphics processing unit (GPU) instead of a central processing unit (CPU). One advantage of using a GPU is that a function for determining the SVD (hereinafter referred to simply as the "SVD function") is already provided as a library (e.g., the cuSOLVER library from Nvidia). The GPU-based SVD function is fast when the matrix size is 32 × 32 or less.

[0144] The scenario assumed in the third embodiment is one in which, for example, even when the low-ranking approximation is performed on k=32 or less, the error is so small as to be negligible. However, to facilitate understanding of the data processing procedure according to the third embodiment, a mathematical formula without performing the low-ranking approximation is first described in this report. (Prior knowledge of the data processing procedures according to the third embodiment)

[0145] The singular value decomposition of an arbitrary matrix (Z) can be given by the eigenvalue decomposition of Z T Z and the eigenvalue decomposition of ZZ T will be obtained... It is assumed that the form of the singular value decomposition of Z UPS is... T is. Z T Z can be deformed as follows. ZTZ=(USVT)T(USVT)=VSTSVT for this reason ZTZV=VSTS

[0146] If here the i-th column of V vi The following relationship results from the mathematical formula (48). (ZTZ)vi=σi2vi

[0147] That means, v i and σ i 2 are eigenvectors or eigenvalues ​​of Z T Z.

[0148] Similarly, ZZ can T as follows. ZZT=(USVT)(USVT)T=USSTUT for this reason ZZTU=USST

[0149] If the i-th column of U is ui, the following relationship results from the mathematical formula (50). (ZZT)ui=σi2ui

[0150] As described above, the singular value decomposition of an arbitrary matrix (Z) can be achieved by the eigenvalue decomposition of Z T Z and the eigenvalue decomposition of ZZ T will be obtained.

[0151] For example, let's assume that the size of Z is pxq and that the matrix is ​​horizontally long, i.e., q > p. In this case, the size of Z is T Z with p×p small, the size of ZZ T However, with q×q being large. In such a case, only the eigenvalue decomposition of Z is possible. T The following calculations are performed to compute matrix (V) with respect to the right singular vector and matrix (S) with respect to the singular value. Matrix (U) with respect to the left singular vector can be computed by the following calculation. if S -1 exists U=ZVS−1 because Z=USVT

[0152] If the low-ranking approximation for dimension k (k <r) durchgeführt wird, sind alle angenommenen k Singulärwerte ungleich Null, und es kann sichergestellt werden, dass S- 1 exists. In the eigenvalue decomposition of ZTZ, S- 1can be calculated without performing the low-ranking approximation if there are no non-zero eigenvalues. (Details of the data processing procedure according to the third embodiment)

[0153] Fig. Figure 8 is an explanatory illustration which presents a data processing method according to the third embodiment.

[0154] In the data processing device according to the third embodiment, the following matrix, which appears in the mathematical formula (43), is defined as M. M|Nd+nb:=[UG0xNd+1⋯xNd+nb][SG000I(nb)]︷∈ℝNfx(Nf+n0)=:[UG0Xb][SG000I(nb)] where Xb:=[xNd+1⋯xNd+nb]

[0155] Note that M is a matrix (X) containing data, where the number of data elements can be up to N. d +n bThe lower right subscript b for X is a serial number assigned to each batch. The number of data elements in the b-th batch is n. b .

[0156] M can be SVD as follows. M|Nd+nb=UG1︷∈ℝNf×NfSG1︷∈ℝNf×NfVG1T︷∈ℝNf×(Nf+nb)

[0157] At present, however, it is assumed that the SVD format is not yet available. Note that the index "G1" in mathematical formula (54) is merely a symbol to indicate for which matrix the SVD applies.

[0158] The matrix (V G1 ) with respect to the right singular vector and the matrix (S G1 ) with respect to the singular value for M can be determined by eigenvalue decomposition of M T M can be determined according to the mathematical formulas (48) and (49).

[0159] Furthermore, the matrix (U G1) with respect to the left singular vector for M, calculated using the following relational formula. M|Nd+nbVG1=UG1SG1

[0160] If all eigenvalues ​​of M T If M are not equal to zero, then the matrix (SG1) has an inverse matrix with respect to the singular value.

[0161] The data processing device according to the third embodiment updates M each time the number of data elements is increased. The updated formula M is as follows. M|Nd+nb+nb+1=[UG1xNd+nb+1⋯xNd+nb+nb+1][SG100I(nb+1)]=:[UG1Xb+1][SG100I(nb+1)] where Xb+1:=[xNd+nb+1⋯xNd+nb+nb+1]︷∈ℝNf×nb+1 In mathematical formula (56), the number of data elements in the (b+1)th batch n is b+1 .

[0162] The update of M, represented in the mathematical formula (56), corresponds to the update of the Gram matrix using the first IncrSVD() in the first embodiment.

[0163] A low-ranking approximation can be used in the singular value decomposition of M. However, since updating M is a loop process, the error caused by the low-ranking approximation accumulates. When detecting anomalies using PaDiM, the inventor of the present technology has the original dimension of N f =1792 reduced to k=20 and M updated by loop processing. In this application example, the inventor of the present technology has determined that this cumulative error lies within an acceptable range.

[0164] The process of determining the SVD of the variance-covariance matrix (Σ) from the SVD of the Gram matrix after considering a sufficient number of data elements is also called finalization. In the finalization phase, the SVD of the Gram matrix is ​​determined as follows. (X|Nall)k=(UG|Nall)k(SG|Nall)k(SG|Nall)k︷∈ℝk×k(VG|Nall)kT

[0165] The subscript 'G' in mathematical formula (57) is a symbol to emphasize that it is the SVD of the Gram matrix. Furthermore, as shown in mathematical formula (57), the SVD of the Gram matrix is ​​assumed to be approximated by reducing the dimension to k-dimensionality. all is the total number of data elements in the finalization.

[0166] The SVD is specified as follows when finalizing a variance-covariance matrix (Σ) using an SVD function. SVD{(∑|Nall)k}=SVD{1Nall(UG|Nall)k(SG|Nall)k2(UG|Nall)kT−(μ|Nall)k(μ|Nall)kT}

[0167] However, the SVD represented in mathematical formula (58) also contains an error due to the reduced-dimensional approximation. Furthermore, the average feature set (µ), which appears on the right-hand side of mathematical formula (58), is also reduced to the k-dimensional when considering the matrix size during addition and subtraction. One can also say that mathematical formula (58) gives the SVD for a variance-covariance matrix (Σ) when the feature space is reduced to the k-dimensional. This SVD function can be a GPU-based SVD function.

[0168] A technical feature unique to the data processing device and data processing method according to the third embodiment is that the singular value decomposition of M is based on the eigenvalue decomposition of M. T M is calculated.

[0169] With this technical feature, the data processing device and the data processing method according to the third embodiment have, in addition to the effects described in the embodiments described above, the advantage that they can be implemented by a function for obtaining a general eigenvalue decomposition. Fourth embodiment.

[0170] A data processing device and a data processing method according to a fourth embodiment are modifications of the data processing device and the data processing method according to the present disclosed technology. Unless otherwise specified, the same reference numerals are used in the fourth embodiment as in the embodiments described above. In the fourth embodiment, the description that overlaps with the embodiments described above is omitted accordingly. (Verification function)

[0171] The data processing device and data processing method according to the present disclosed technology may include a verification function. The data processing device and data processing method according to the present disclosed technology may obtain the correlation matrix for verification by the following sequential procedure instead of the SVD format. E(X|NdXT|Nd)︷∈ℝNf×Nf=1Nd[x1⋯xNd][x1T⋮xNdT]

[0172] For this reason E(X|Nd+1XT|Nd+1)=NdNd+1E(X|NdXT|Nd)︷∈ℝNf×Nf+1Nd+1xNd+1xNd+1T︷∈ℝNf×Nf

[0173] The formula for the sequential update of the correlation matrix in mathematical formula (59) has the same form as the formula for the sequential update of the average characteristic value (µ) in mathematical formula (25). By using E() of the expected value function, mathematical formula (25) can be expressed as follows. E(X|Nd+1)=NdNd+1E(X|Nd)+1Nd+1xNd+1

[0174] Using the mathematical formulas (59) and (60), the variance-covariance matrix (Σ) can be determined for verification using the following sequential method instead of the SVD format. ∑|Nd+1=E(X|Nd+1XT|Nd+1)−E(X|Nd+1)E(X|Nd+1)T where E(X|Nd+1XT|Nd+1)=NdNd+1E(X|NdXT|Nd)+1Nd+1xNd+1xNd+1T E(X|Nd+1)=NdNd+1E(X|Nd)+1Nd+1xNd+1

[0175] The updated variance-covariance matrix (Σ) obtained by mathematical formula (61) can be used to verify that the format of the SVD obtained by the data processing procedure described in the embodiments described above is correct.

[0176] A technical feature unique to the data processing device and data processing method of the fourth embodiment is that a correlation matrix for verification can be obtained directly and sequentially. The term "directly" used here means "not in SVD format".

[0177] With this technical feature, the data processing device and the data processing method according to the fourth embodiment, in addition to the effects described in the embodiments described above, also have the effect of allowing the correctness of the variance-covariance matrix (Σ) calculated in the SVD format during the learning phase to be checked. Fifth embodiment.

[0178] A data processing device and a data processing method according to a fifth embodiment are modifications of the data processing device and the data processing method according to the present disclosed technology. Unless otherwise specified, the same reference numerals are used in the fifth embodiment as in the embodiments described above. In the fifth embodiment, descriptions that overlap with those of the embodiments described above are expediently omitted.

[0179] Assuming that the variance-covariance matrix (Σ) has full rank after finalization, the square (d) M 2 ) of the Mahalanobis distance for the target sample (x target ) using the general inverse matrix (Σ-) as follows (see also mathematical formula (21)). dM2=ytargetTU|Nail[1σ12 ⋱ 1σNf2]UT|Nail︷∑−ytarget =:y¯targetTy¯target where ytarget=xtarget−μ|Naily¯target:=[1σ1⋱1σNf]UT|Nallytarget

[0180] Here, the size of the singular value σ1 ≥... ≥ (σ Nf As described above, the Mahalanobis distance is determined by the general inverse matrix (Σ-) of the variance-covariance matrix (Σ), where the first singular value (1 / σ1) in the general inverse matrix (Σ-) 2 ) the smallest and the last singular value (1 / σ Nf 2 ) is the largest. The fact that the last singular value (1 / σ) Nf 2 ) is the largest, meaning that the term (the component) for the last singular value (1 / σ) Nf 2 ) has the greatest influence on the Mahalanobis distance.

[0181] During the learning phase, it can also be said that the variance-covariance matrix (Σ), which represents the property of a certain class, is arranged in the order of the magnitudes of the singular values, i.e., in the order σ1,...,σ Nf This is important. Therefore, during the learning phase, the assumed feature space is a k-dimensional subspace, and there is no problem.

[0182] On the other hand, it is important in the inference phase to expand the feature space to a complete N f to create a -dimensional space. (Meaningful zero singular value)

[0183] Even if the dimension of the space defined by all deviation vectors (y1,...,y) Nall ) is formed, which belong to the teacher data, r (r <Nf) ist, gehört der Abweichungsvektor (y target) for the goal does not necessarily correspond to this r-dimensional space. For example, if we consider a class that contains a normal-state image for a particular object, we assume that one dimension of a space defined by a deviation vector (y1,...,y) Nall ) for the image in the normal state, r (r <N f ). However, the deviation vector (y) belongs target ) for the target representing the image in the abnormal state, does not necessarily correspond to this r-dimensional space. In such a case, the zero singular value for the variance-covariance matrix (Σ) of the class consisting of images in the normal state is a "meaningful zero singular value". Conversely, the same applies to a deviation vector (y). target ) for any target: If it always belongs to this r-dimensional space, the dimension larger than r is redundant, and the zero singular value for the variance-covariance matrix (Σ) is a non-significant zero singular value.

[0184] It is assumed that the rank of the variance-covariance matrix (Σ) with respect to the class to which the image belongs in the normal state is r (r <N f ). In this case, the data processing device and the data processing procedure can be used in the inference phase before the calculation of the Mahalanobis distance (d). M ) perform the following determination processing as a coping procedure for the “meaningful zero singular value”. for ∀i∈{k+1,⋯,Nf} if |uiTytarget︷∈ℝ|≥ε then not included in the class (dM=∞) else continue with the calculation of dM2 in dim=r where U|Nall=:[u1⋯uNf]

[0185] However, the ε (epsilon) appearing in mathematical formula (63) is a threshold for a machine epsilon or similar.

[0186] If the condition represented in mathematical formula (63) is satisfied, the deviation vector (y) belongs target) for the target not belonging to the class that contains the image in its normal state. If it is clear that the target does not belong to the class, it is not necessary to determine the Mahalanobis distance (d M ) to determine intentionally. Note that in this case, to interpret it broadly, the Mahalanobis distance is zero ∞ (infinity) due to division by the singular value. If the condition represented in mathematical formula (63) is satisfied, the zero singular value for the variance-covariance matrix (Σ) of the class containing the normal-state image is a singular value that cannot be truncated, i.e., a “meaningful zero singular value”.

[0187] Only if the condition specified in mathematical formula (63) is not met, do the data processing device and the data processing procedure perform the calculation of the Mahalanobis distance (d) during the inference phase. M ) by, which is defined in the r-dimensional feature space.

[0188] The data processing device and data processing method according to the present disclosed technology perform the following determination processing by applying the mathematical formula (63) in a case where the low-ranking approximation is performed, where the dimension is defined as k (k <r) betrachtet wird. for ∀i∈{k+1,⋯,Kf} if |uiTytarget︷∈ℝ|≥ε then not included in the class and terminate the process; otherwise, continue with the calculation of dM2 in dim=k where U|Nall:=[u1⋯uNf]

[0189] In a case where the low-ranking approximation can be performed, where the dimension is considered to be k, the approximation to the Mahalanobis distance is obtained from the following mathematical formula, which is based on the rank-restricted general inverse matrix. dM2∼(dM)k2=(y¯target)kT(y¯target)k where (y¯target)k︷∈ℝk×1:=[1σ1 ⋱ 1σk]︷∈ℝk×k[u1T⋮ukT]︷Truncated UT∈ℝk×Nfytarget︷∈ℝNf×1

[0190] The variable defined in the second formula (65), in which y is accented with a dash, is referred to in this description as the normalized deviation vector for the target or simply as the normalized deviation vector. Note that even in a case where the low-ranking approximation is not performed, the same vector is referred to as the normalized deviation vector (see mathematical formula (62)).

[0191] Mathematical formula (64) specifies a condition for performing the low-ranking approximation when calculating the Mahalanobis distance. The conditions under which the low-ranking approximation can be performed are as follows. for ∀i∈{k+1,⋯,Nf} and ∀Ytarget |uiTytarget︷∈ℝ|<ε

[0192] The mathematical formula (66) expresses that the absolute values ​​of all (k+1) up to N f -th coordinate components are smaller than ε (epsilon) if the coordinates of y target in which the basis vectors from u1 to uN f defined N f The low-ranking approximation must be considered in a n-dimensional space. That is, the condition under which the low-ranking approximation can be performed is that the deviation vectors (y) target ) for all targets are elements of a k-dimensional subspace defined by the deviation vectors (y1,...,y Nall ). That is, the condition under which the low-ranking approximation can be performed is that the “deviation vectors (y) target ) for all targets are elements of a k-dimensional subspace defined by the deviation vectors (y1,...,y Nall ) is formed".

[0193] In this description, the condition specified by mathematical formula (66) is referred to as the “condition under which a low-ranking approximation can be performed”. In a case where it is not known whether an object (application example) to which this technology is to be applied satisfies an implementable condition of low-ranking approximation, the data processing device and the data processing procedure in the inference phase only need to perform the determination processing represented by mathematical formula (63) or mathematical formula (64).

[0194] The inventor of the present technology has determined that when anomalies are detected by PaDiM, even when the original dimension of N f =1792 is reduced to k=20, which satisfies the “condition under which a low-ranking approximation can be performed” as represented in mathematical formula (66).

[0195] A technical feature specific to the data processing device and data processing method of the fifth embodiment is that the data processing device and data processing method include a coping method for “a meaningful zero singular value”.

[0196] With this technical feature, the data processing device and the data processing method according to the fifth embodiment also have the effect of being able to solve learning problems such as "classification" and "clustering" by providing a complete N f Use a -dimensional space as a feature space in an inference phase, even if a variance-covariance matrix (Σ) is not full-rank with respect to a particular class. Sixth embodiment.

[0197] A data processing device and a data processing method according to a sixth embodiment are modifications of the data processing device and the data processing method according to the present disclosed technology. Unless otherwise specified, the same reference numerals are used in the sixth embodiment as in the embodiments described above. In the sixth embodiment, descriptions that overlap with those of the embodiments described previously are expediently omitted.

[0198] Meanwhile, it shows Fig. 5 according to the first embodiment a flowchart for the case of using a k-dimensional feature space defined by a fixed singular vector ((U) k T). As described above, the use of the fixed k-dimensional subspace projected by the fixed singular vector required that the data be sufficiently rich, i.e., that enough information be captured to reproduce the property of the class to which the data belong.

[0199] The data processing device and data processing method according to the present disclosed technology include means for determining whether data is sufficiently rich, in particular a termination condition for loop processing with respect to data updating. (End condition for loop processing with respect to data update)

[0200] Based on the information, if the number of data elements is N d +1 and the number of data elements N dThe end condition for loop processing with respect to data updates is specified, for example, as follows. for ∀i∈{1,⋯Nf}|σi|Nd+1−σi|Nd|<εσ ∩‖ui|Nd+1−ui|Nd‖<εu

[0201] Here, εσ in mathematical formula (67) is a threshold for the singular value and ε u a threshold value for the singular vector. ε σ and εu can have the same or different values. Furthermore, the singular value (σ) appearing in mathematical formula (67) i ) and the singular vector (u i ) in relation to the variance-covariance matrix (Σ) described below. ∑|Nd=[u1|Nd⋯uNf|Nf][σ1|Nd ⋱ σNf|Nd][u1T|Nd⋮uNfT|Nd]∑|Nd+1=[u1|Nd+1⋯uNf|Nd+1][σ1|Nd+1 ⋱ σNf|Nd+1][u1T|Nd+1⋮uNfT|Nd+1]

[0202] To determine whether the data are sufficiently rich, it is enough if the singular value and singular vector for the variance-covariance matrix (Σ) can be observed indirectly, even if they cannot be observed directly. One method for indirectly observing the singular value and singular vector for the variance-covariance matrix (Σ) is to observe the correlation matrix.

[0203] In the case of the data processing device and data processing method according to the first embodiment, the singular value and the singular vector for the SVD of the correlation matrix derived from the SVD of the Gram matrix shown in mathematical formula (23) are preferably used as the final condition shown in mathematical formula (67).

[0204] In the case of the data processing device and data processing method according to the second embodiment, the singular value and singular vector for the SVD of the correlation matrix, which is derived from the SVD of the Gram matrix shown in mathematical formulas (42) or (43), can be used as the final condition according to mathematical formula (67).

[0205] In the case of the data processing device and data processing method according to the third embodiment, the singular value and the singular vector for the SVD of the correlation matrix are derived from the SVD of the Gram matrix, which is determined by the eigenvalue decomposition or the like of M T M was obtained, preferably used as the final condition shown in mathematical formula (67).

[0206] A technical feature unique to the data processing device and data processing method according to the sixth embodiment includes a termination condition for loop processing with respect to data updating.

[0207] With this technical feature, the data processing device and the data processing method according to the sixth embodiment also have the effect that the end of the loop processing can be determined based on the determination of whether the data are sufficiently rich or not. INDUSTRIAL APPLICABILITY

[0208] The data processing device and data processing method according to the present technology can be applied to a defect detection device that performs an anomaly detection, for example a defect detection device for a photomask for a semiconductor, and are industrially applicable. REFERENCE MARK LIST

[0209] 100: Data processing device, 110: Input interface, 120: Processing circuit, 122: Processor, 124: Memory, 130: Output interface QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited non-patent literature

[0000] Thomas Defard et al. "PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization" (https: / / arxiv.org / abs / 2011.08785

[0005] Matthew Brand, “Fast Low-Rank Modifications of the Thin Singular Value Decomposition,” MERL technical report, TR2006 - 059, May 2006

[0028] Kenichi Kanaya, „linear algebraic seminar, projection, singular value decomposition, general inverse matrix“, Kyoritsu Shuppan, ISBN978-4-320-11340-4

[0043]

Claims

[1] Data processing device comprising a processing circuit, wherein During a learning phase, the processing circuit sequentially updates a Gram matrix in the form of an SVD, and The processing circuit calculates a variance-covariance matrix in SVD form based on the SVD in relation to the Gram matrix during the finalization of the learning phase. [2] Data processing device with a processor that executes a program, wherein The processor executes the program to sequentially update a Gram matrix in SVD form during a learning phase, and The processor executes the program to compute a variance-covariance matrix in SVD form based on the SVD in relation to the Gram matrix during the finalization of the learning phase. [3] Data processing device according to claim 2, wherein the program includes a function that takes SVD of Z, A and B as inputs and SVD of Z+ABT as output, where A, B and Z are each a matrix. [4] Data processing device according to claim 2, wherein the program includes a function that outputs the SVD of an extended system. [5] Data processing device according to claim 2, wherein the program includes a function that performs an eigenvalue decomposition of a matrix (M T outputs M) in square form. [6] Data processing device according to any one of claims 2 to 5, wherein the program includes a function that sequentially computes a correlation matrix for verification, not in SVD form. [7] Data processing device according to any one of claims 2 to 5, wherein the program includes a function for handling a zero singular value with a meaning. [8] Data processing device according to any one of claims 2 to 5, wherein the program includes a function that determines an end to the loop processing based on the determination of whether training data is sufficiently rich or not. [9] Data processing procedures, in full: Sequential updating of a Gram matrix in SVD form during a learning phase; and Calculating a variance-covariance matrix in SVD form based on the SVD in relation to the Gram matrix in the finalization of the learning phase. [10] Data processing method according to claim 9, comprising a numerical calculation with SVD of Z, A and B as inputs and SVD of Z+AB T as output, where A, B and Z are each a matrix. [11] Data processing method according to claim 9, comprising a numerical calculation for outputting the SVD of an extended system. [12] Data processing method according to claim 9, comprising a numerical calculation for outputting an eigenvalue decomposition of a matrix (M T M), which is represented in a square shape. [13] Data processing method according to any one of claims 9 to 12, further comprising a numerical calculation for sequentially calculating a correlation matrix for verification, which is not in SVD form. [14] Data processing method according to any one of claims 9 to 12, comprising processing for handling a zero singular value that has meaning. [15] Data processing method according to any one of claims 9 to 12, comprising processing to determine an end of the loop processing based on a determination of whether training data are sufficiently rich or not.

Images