Data processing device, abnormality detection device, and data processing method

Abstract

A data processing device (10) is configured to include: a training unit (11) that acquires a projective transformation matrix and a singular value matrix that are generated on the basis of a plurality of training data items for each of the training data items, and that have a smaller number of dimensions than that of the corresponding training data item; a combining unit (13) that calculates a new projective transformation matrix and a new singular value matrix by performing weighted combination processing on the plurality of projective transformation matrices and singular value matrices acquired by the training unit (11); and a calculation unit (15) that, by using the new projective transformation matrix and the new singular value matrix calculated by the combining unit (13), approximately calculates a covariance matrix obtained by performing weighted combination processing on covariance matrices respectively corresponding to the plurality of training data items.

Description

Data processing device, anomaly detection device, and data processing method

[0001] This disclosure relates to a data processing device, an anomaly detection device, and a data processing method.

[0002] Patent Document 1 discloses a feature similarity determination device for evaluating whether a sample of time-series data to be judged is similar to a population, and a technique (hereinafter referred to as "prior art") that effectively determines similarity with the population even for events where it is difficult to determine characteristic behavior. In the prior art, when determining the similarity between the population and the target sample based on the distance between them in the feature space, the feature quantities of the target sample are compressed using Incremental SVD, and the Mahalanobis distance to the population is calculated using the compressed feature quantities. Here, since compressed feature quantities are used when determining similarity, the computational load is reduced, and since the feature quantities are compressed using Incremental SVD, the compression process is also lightweight.

[0003] Incidentally, there is a need to obtain a better learning model by combining multiple existing learning models. There is also a need to utilize the information contained in multiple existing learning models in order to reduce the effort required to retrain learning models. For example, when integrating or combining distribution data used for training each of multiple learning models, the covariance matrix and mean vector contained in each distribution data may be multiplied by predetermined weights before being combined. Such weighted combination processing is the same even when the features are compressed in the prior art described in Patent Document 1, for example. Even when the features are compressed in the prior art, the covariance matrix and mean vector are obtained from the compressed features, and these are multiplied by predetermined weights in the same way as above before being combined with other distribution data.

[0004] Japanese Patent Publication No. 2006-235669

[0005] However, the weighted merging process described above has the problem of requiring a large number of multiplications. For example, when performing weighted merging on R covariance matrices, it is necessary to calculate the covariance matrix R times before processing. In this case, the number of multiplications required to calculate the covariance matrix is ​​proportional to the number of matrices to be merged, R. For example, as shown in Figure 17, the length of the feature N ｆ When calculating R covariance matrices Σ for Nd data points, the number of multiplications required is R × Nd × N. ｆ ２ This is the result. Furthermore, when combining R covariance matrices Σ, a predetermined weight coefficient is multiplied by each covariance matrix, and the number of multiplications is proportional to the square of the feature length. In the example in Figure 17, R × N ｆ ２ The number of multiplications required is as follows.

[0006] Thus, in conventional weighted covariance processing, it is necessary to recalculate the covariance matrix before performing the covariance processing, which increases the number of multiplications required for calculating the covariance matrix and assigning weight coefficients. As a result, conventional methods have had the problem of being time-consuming to calculate the covariance matrix when weighted covariance processing is involved.

[0007] This disclosure was made to solve the above-mentioned problems and aims to provide a data processing device that can obtain a matrix corresponding to a new covariance matrix obtained by weighted joining of multiple covariance matrices in a shorter time than conventional methods.

[0008] The data processing device according to this disclosure includes: a learning unit that acquires projection transformation matrices and singular value matrices generated for each set of training data based on a plurality of training data, wherein the projection transformation matrices and singular value matrices have a reduced number of dimensions compared to the corresponding training data; a coupling unit that calculates new projection transformation matrices and singular value matrices by applying weighted coupling processing to the plurality of projection transformation matrices and singular value matrices acquired by the learning unit; and a calculation unit that approximates a covariance matrix obtained by applying weighted coupling processing to the covariance matrices corresponding to each of the plurality of training data using the new projection transformation matrices and singular value matrices calculated by the coupling unit.

[0009] According to this disclosure, by configuring it as described above, a matrix corresponding to a new covariance matrix obtained by weighted joining of multiple covariance matrices can be obtained in a shorter time than before.

[0010] Figure 12A and Figure 12B are diagrams illustrating the overview of processing by the data processing device according to Embodiment 1. Figure 12A and Figure 12B are diagrams illustrating the hardware configuration of the data processing device according to Embodiment 1. Figure 12A and Figure 12B are diagrams illustrating the hardware configuration of the data processing device according to Embodiment 1. Figure 12A and Figure 12B are diagrams illustrating the configuration of the data processing device according to Embodiment 2. Figure 12A and Figure 12B are diagrams illustrating the configuration of the data processing device according to Embodiment 2. Figure 12A and Figure 12B are diagrams illustrating the hardware configuration of the data processing device according to Embodiment 2. Figure 12A and Figure 12B are diagrams illustrating the configuration of the data processing device according to Embodiment 2. Figure 12A and Figure 12B are diagrams illustrating the components corresponding to the first and second terms of equation (5) calculated by the evaluation unit in the data processing device according to Embodiment 2. Figure 12A and Figure 12B are diagrams illustrating the configuration of the evaluation unit in the data processing device according to Embodiment 2. Figure 12A and Figure 12B are diagrams illustrating the hardware configuration of the data processing device according to Embodiment 1. Figure 12A and Figure 12B are diagrams illustrating the configuration of the data processing device according to Embodiment 2. Figure 12A and Figure 12B are diagrams illustrating the hardware hardware configuration of the data processing device according to Embodiment 2. Figure 1 This figure illustrates the challenges in calculating the covariance matrix using conventional weighted concatenation methods.

[0011] The embodiments will be described in detail below with reference to the drawings. Embodiment 1. First, an overview of the processing by the data processing device 10 according to Embodiment 1 will be described with reference to Figure 1.

[0012] The data processing device 10 does not obtain a final covariance matrix by directly performing weighted combination processing on a plurality of covariance matrices as they are. Instead, it performs singular value decomposition on the feature quantity matrix X that is the basis of each covariance matrix, and uses the obtained result to approximately represent each covariance matrix. Then, the data processing device 10 performs weighted combination processing on the data obtained through approximation, and approximately calculates the above-mentioned final covariance matrix from the result. If the number of dimensions of the feature quantity matrix X is reduced by singular value decomposition, the number of dimensions of the matrix to be subjected to weighted combination processing can be made lower than the number of dimensions of the original covariance matrix, enabling the weighted combination processing to be performed in a shorter time than conventional methods.

[0013] For example, consider the case where the length (number of dimensions) of the feature quantity is N ｆ in the process of calculating the covariance matrix Σ. The covariance matrix is calculated by Σ=XX Ｔ where X denotes the feature quantity matrix that is the basis of the covariance matrix, and T denotes the transposed matrix. When singular value decomposition is performed on the original feature quantity matrix X, this feature quantity matrix X can be approximated as X=USV Ｔ by using the top k singular values. That is, the number of dimensions of the original feature quantity matrix X can be reduced (the downward white arrow in FIG. 1).

[0014] Substituting this X=USV Ｔ into Σ=XX Ｔ , the covariance matrix Σ can be approximated, as shown in FIG. 1, as U ｋ diag(s １ ２ , ..., s ｋ ２ ) U ｋ Ｔ (the leftward white arrow in FIG. 1). U ｋ is the left singular matrix obtained by singular value decomposition, and diag(s １ ２ , ..., s ｋ ２ ) is the singular value matrix obtained by singular value decomposition. k is the length (number of dimensions) of the feature quantity of the left singular matrix.

[0015] As shown in FIG. 1, in the data processing device 10, both the number of rows and the number of columns are N ｆThe covariance matrix Σ is N ｆ ×k matrix U ｋ and k s ｉ ２ This is approximated by (i = 1, ..., k) (upward white arrow in Figure 1). In other words, the number of data points to be stored in the learning model as a result of learning is the original N ｆ ×N ｆ From, N ｆ Since it can be expressed as ×k + k, the data processing device 10 can calculate the covariance matrix Σ with far fewer operations than the original matrix operation. In addition, the data processing device 10 sets the target of the weighted combination process to the left singular matrix U ｋ and singular value matrix diag(s １ ,…, s １ｋ Therefore, the number of weight multiplications is R × N. ｆ ２ Rather, R(N) ｆ ×k + k) = R(N ｆ +1) k can be set to k < N ｆ This allows for weight assignment with fewer calculations.

[0016] Furthermore, the data processing device 10 approximates the final covariance matrix described above, that is, the covariance matrix obtained by directly applying the weighted fusion process to the multiple covariance matrices, based on the results of the weighted fusion process. This makes it possible for the data processing device 10 to calculate the final covariance matrix with weighted fusion in a shorter time than before.

[0017] In the following explanation, the left singular matrix U obtained by singular value decomposition is referred to as U. ｋ This is sometimes called the projection transformation matrix. In the following explanation, the projection transformation matrix is ​​the left singular matrix U obtained by singular value decomposition. ｋ This is synonymous with the projection transformation matrix, which is used to project the elements of the original feature matrix X onto the principal component space after dimensionality reduction.

[0018] Figure 2 shows an example of the configuration of a data processing device 10 according to Embodiment 1. As shown in Figure 1, the data processing device 10 is configured to include a learning unit 11, a coupling unit 13, a calculation unit 15, an input interface 110, and an output interface 130.

[0019] The learning unit 11 obtains projection transformation matrices and singular value matrices that are generated for each set of training data based on the training data used for each of the multiple learning models, and which have a reduced number of dimensions compared to the corresponding training data. For example, the learning unit 11 may obtain such projection transformation matrices and singular value matrices from an external device (not shown) other than the data processing device 10.

[0020] Alternatively, the learning unit 11 may generate a projection transformation matrix and a singular value matrix for each set of training data based on the training data used for each of the multiple training models, and obtain the generated projection transformation matrix and singular value matrix. In this case, the learning unit 11 may, for example, obtain the training data used for each of the multiple training models from the external device described above, and obtain the left singular matrix generated by singular value decomposition of the obtained training data as the projection transformation matrix. The learning unit 11 may also obtain the singular value matrix generated by singular value decomposition of the obtained training data. Here, the left singular matrix (projection transformation matrix) and singular value matrix generated by singular value decomposition have a reduced dimensionality compared to the training data that was subjected to singular value decomposition.

[0021] The coupling unit 13 calculates new projection transformation matrices and singular value matrices by applying a weighted coupling process to the multiple projection transformation matrices and singular value matrices obtained by the learning unit 11.

[0022] Furthermore, the coupling unit 13 may perform a correction to prevent the loss of corresponding column vector components in multiple projection transformation matrices when performing the weighted coupling process described above. An example of the configuration of the coupling unit 13 for performing such a correction is shown in Figure 3. For example, the coupling unit 13 is configured to include a weighted coupling processing unit 131, as shown in Figure 3.

[0023] The weighted coupling processing unit 131 is configured to include, for example, an inner product calculation unit 1311, a sign inversion unit 1312, and a weighted coupling unit 1313. The inner product calculation unit 1311 calculates the inner product of a plurality of projection transformation matrices, using one of the corresponding column vectors as the reference vector, between that reference vector and the other column vectors. The sign inversion unit 1312 performs the above correction by inverting the sign of the other column vectors if the result calculated by the inner product calculation unit 1311 is negative. The weighted coupling unit 1313 performs a weighted coupling process on a plurality of projection transformation matrices, including the projection transformation matrices after the above correction by the sign inversion unit 1312, and on a plurality of singular value matrices. The symbols of the English letters shown in Figure 3 will be described later.

[0024] The calculation unit 15 uses the new projection transformation matrix and singular value matrix calculated by the coupling unit 13 to approximately calculate the covariance matrix obtained by applying a weighted coupling process to the covariance matrices corresponding to each of the multiple training data. For example, the calculation unit 15 can approximately calculate the above-mentioned covariance matrix using the new projection transformation matrix and singular value matrix calculated by the coupling unit 13 by a known method.

[0025] The input interface 110 is a functional unit that functions as an interface for receiving input from the user of the data processing device 10, and is composed of, for example, a keyboard, mouse, and touch panel.

[0026] The output interface 130 is a functional unit that functions as an interface for outputting data from the data processing device 10 to the user, and is composed of, for example, a display.

[0027] In addition, the data processing device 10 may include a storage unit (not shown). The storage unit is composed of, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive). The storage unit stores various types of data handled by the data processing device 10.

[0028] Figure 4 shows an overview of the processing performed by the coupling unit 13. Following the dimensionality reduction by singular value decomposition described above, a projection transformation matrix U projects the elements of the original feature matrix X onto the principal component space after dimensionality reduction. ｋ , and the singular value matrix D ｋ Each of the multiple learning models possesses one of these properties.

[0029] The coupling section 13 is the projection transformation matrix U that each learning model possesses. ｋ and singular value matrix D ｋ For this, the weight coefficient w is as shown in equation (1) below. ｉ The projection space is modified by applying a weighted combination (linear combination).

[0030] Figure 4 shows the distribution space of the training data used to train learning model 1, the distribution space of the training data used to train learning model 2, and the distribution space of the training data obtained by weighted coupling. According to Figure 4, for a given test vector x, the first component (principal component) and the second component (components other than the principal component) are changed by weighted coupling.

[0031] When there are two learning models, the coupling unit 13 performs a weighted coupling process, for example, so that the ratio of the weight coefficients is p:q. Since it is not possible to assign negative weights to the covariance matrix, p, q are usually set to ≥ 0. However, the coupling unit 13 does not necessarily have to set p, q ≥ 0; for example, one or more of p and q may be negative. Also, the sum of the ratios (p + q) is usually 1, but it does not have to be 1.

[0032] Furthermore, the coupling unit 13 performs weighted coupling on the mean value μ of each training data as needed. For example, if the training data is not normalized so that the sum of the mean values ​​μ of each training data is zero, the coupling unit 13 performs weighted coupling on the mean value μ as well.

[0033] Next, an example of the operation of the data processing device 10 shown in Figure 2 will be described. Figure 5 is a flowchart illustrating an example of the operation of the data processing device 10.

[0034] First, the learning unit 11 obtains a projection transformation matrix and a singular value matrix that are generated for each set of training data based on the training data used for each of the multiple learning models, and which have a reduced number of dimensions compared to the corresponding training data (step ST1).

[0035] Next, the coupling unit 13 calculates a new projection transformation matrix and a new singular value matrix by applying a weighted coupling process to the multiple projection transformation matrices and singular value matrices obtained by the learning unit 11 (step ST2).

[0036] Next, the calculation unit 15 uses the new projection transformation matrix and singular value matrix calculated by the coupling unit 13 to approximately calculate a covariance matrix obtained by applying a weighted coupling process to the covariance matrices corresponding to each of the multiple training data (step ST3). For example, the calculation unit 15 can approximately calculate the above-mentioned covariance matrix using the new projection transformation matrix and singular value matrix calculated by the coupling unit 13 by a known method.

[0037] Next, step ST1 performed by the learning unit 11 will be explained in detail with reference to Figure 6. Figure 6 is a flowchart showing the details of step ST1. For the sake of clarity, the following explanation will describe an example in which the learning unit 11 acquires the learning data used for learning each of the two learning models from the external device mentioned above, obtains the left singular matrix generated by singular value decomposition of the acquired learning data as a projection transformation matrix, and also obtains the singular value matrix generated by singular value decomposition.

[0038] First, the learning unit 11 obtains the feature quantities x contained in the training data used to train each of the two learning models from the external device described above (step ST11). When the respective training data are, for example, training data A and B, training data A and B each consist of multiple feature quantities x (for example, height, weight, sitting height, age, etc.).

[0039] Next, the learning unit 11 checks whether it has completed acquiring the feature x for all the training data (in this case, two training data) (step ST12). If it finds that it has not completed acquiring the feature x for all the training data (step ST12; NO), it returns to step ST11, and the learning unit 11 acquires the missing feature x. If it has completed acquiring the feature x for all the training data (step ST12; YES), the process moves to step ST13.

[0040] In step ST13, the learning unit 11 calculates the average value μ of the feature x included in the acquired training data for each feature (step ST13). The learning unit 11 performs this process for all the training data.

[0041] Next, the learning unit 11 calculates a feature matrix X, which is a matrix in which the values ​​obtained by subtracting the mean value μ from the feature quantity x are arranged in columns (step ST14). The learning unit 11 performs this process for all the training data.

[0042] Next, the learning unit 11 performs singular value decomposition on the feature matrix X calculated in step ST14, thereby obtaining the left singular matrix U ｋ and singular value matrix D ｋ The left singular matrix calculated as described above is used as the projection transformation matrix U of the training data A. ｋ , and the projection transformation matrix U of the training data B. ｋ It is obtained as follows. The learning unit 11 also uses the singular value matrix calculated as described above as the singular value matrix D of the learning data A. ｋ , and the singular value matrix D of training data B ｋ It will be acquired as follows.

[0043] The projection transformation matrix and singular value matrix obtained by the learning unit 11 both have a reduced dimensionality compared to the training data that was subjected to singular value decomposition. For example, the learning unit 11 can obtain a projection transformation matrix and singular value matrix with a reduced dimensionality compared to the training data that was subjected to singular value decomposition by pre-selecting features whose singular values ​​are among the top k and reducing the other features. The learning unit 11 obtains a projection transformation matrix U for each training data set. ｋ and singular value matrix D ｋ This is output to the coupling unit 13.

[0044] Next, step ST2 by the joint 13 will be explained in detail with reference to Figure 7. Figure 7 is a flowchart showing the details of step ST2.

[0045] First, the weighted coupling processing unit 131 of the coupling unit 13 processes the projection transformation matrix U for each training data acquired by the learning unit 11 in step ST15 described above. ｋ and singular value matrix D ｋ The weighted coupling processing unit 131 obtains the projection transformation matrix U for each acquired training data. ｋ and singular value matrix D ｋ From among these, select the reference projection transformation matrix U ｋ and singular value matrix D ｋ Select the selected projection transformation matrix U ｋ and singular value matrix D ｋ The first projection transformation matrix U １ and the first singular value matrix D １ It is obtained as (step ST21).

[0046] For example, the weighted coupling processing unit 131 uses the projection transformation matrix of training data A and the projection transformation matrix of training data B, and the projection transformation matrix U of training data A. ｋ When used as a reference, the projection transformation matrix U of the training data A ｋ The first projection transformation matrix U １ It is obtained as follows. In addition, the weighted coupling processing unit 131 selects the singular value matrix D of the learning data A from the singular value matrix of the learning data A and the singular value matrix of the learning data B. ｋ When used as a reference, the singular value matrix D of training data A ｋ The first singular value matrix D１ It is obtained as follows. Note that in Figure 3 above, the first projection transformation matrix is ​​denoted by U １ Represented by the first singular value matrix, denoted by D １ It is represented as follows.

[0047] Next, the weighted combination processing unit 131 uses the first projection transformation matrix U １ and the first singular value matrix D １ The first projection transformation matrix U is the target of the join with １ and the first singular value matrix D １ Projection transformation matrix U other than ｉ and singular value matrix D ｉ The inner product calculation unit 1311 obtains the projection transformation matrix U of the learning data B in the example described above. ｋ and the singular value matrix D of training data B ｋ Obtain the following. Note that in Figure 3 above, projection transformation matrices other than the first projection transformation matrix are denoted by U. ２…ｉ Represented by , the singular value matrices other than the first singular value matrix are denoted by D ｉ It is represented as follows.

[0048] Next, the dot product calculation unit 1311 of the weighted coupling processing unit 131 calculates the first projection transformation matrix U obtained in step ST21. １ The column vectors contained therein and the projection transformation matrix U obtained in step ST22 ｉ For the column vectors included in the expression, calculate the dot product of the corresponding column vectors and check whether the calculated dot product is negative or not (step ST23).

[0049] As a result, if the calculated inner product is negative (step ST23; YES), the sign inversion unit 1312 of the weighted coupling processing unit 131 performs a projection transformation matrix U that is not the reference among the column vectors for which the inner product was calculated. ｉ The column vector u contained within ｎ The sign of is reversed (step ST24). On the other hand, if the calculated dot product is not negative (step ST23; NO), the sign reversal unit 1312 of the weighted coupling processing unit 131 processes the column vector u ｎ The sign remains unchanged, and the process moves to step ST25.

[0050] Here, step ST24 will be described with reference to FIG. 8. As already described, the weighted combination processing unit 131 obtains the first projection transformation matrix U acquired in step ST21 １ and the projection transformation matrix U acquired in step ST22 ｉ , performs weighted combination processing on them. Here, the first projection transformation matrix U １ and this first projection transformation matrix U １ and the projection transformation matrix U to be combined ｉ singular value decomposition is used for calculation, but when these two, the first projection transformation matrix U １ and the projection transformation matrix U ｉ are compared in units of corresponding column vectors, the angle formed by some column vectors may be an obtuse angle as shown in FIG. 8A.

[0051] Here, if the angle formed between column vectors is an obtuse angle, when the above-described weighted combination processing is performed, cancellation of the column vector components occurs, and there are cases where the combined matrix cannot be generated correctly. In particular, when the angle formed between column vectors is 180 degrees, the component of the column vector included in the matrix after weighted combination becomes 0, and the component is lost.

[0052] Therefore, in order to avoid the occurrence of such a problem, the sign inversion unit 1312 of the weighted combination processing unit 131 is configured such that when the angle formed between column vectors is an obtuse angle, as shown in FIG. 8B, the reference projection transformation matrix (the first projection transformation matrix U １ ) performs correction to invert the sign of column vectors included in projection transformation matrices other than the above.

[0053] In addition, when there are three or more projection transformation matrices to be subjected to weighted combination processing, the sign inversion unit 1312 may use any one projection transformation matrix as a reference. In that case, the sign inversion unit 1312 may perform correction to invert the signs of all column vectors for which the inner product with the column vectors included in the reference projection transformation matrix is negative, or may perform correction to invert only the signs of predetermined column vectors instead of all column vectors.

[0054] Returning to FIG. 7, in step ST25, the weighted combination processing unit 131 checks whether the first projection transformation matrix U １ the inner products of all corresponding column vectors included between the column vectors of said first projection transformation matrix U ｉ and the column vectors included in the projection transformation matrix U have all been calculated (step ST25). As a result, if the inner products of all corresponding column vectors have all been calculated (step ST25: YES), the inner product calculation unit 1311 outputs the first projection transformation matrix U １ to the weighted combination unit 1313, and the sign inversion unit 1312 outputs the corrected projection transformation matrix U' ２…ｉ to the weighted combination unit 1313, and the process proceeds to step ST26. On the other hand, if the inner products of all corresponding column vectors have not all been calculated (step ST25: NO), the process returns to step ST23, and the inner product calculation unit 1311 calculates the inner product for the column vectors for which the inner product has not been calculated yet. Note that in FIG. 3 described above, the corrected projection transformation matrix is denoted by reference sign U' ２…ｉ .

[0055] In step ST26, the weighted combination unit 1313 receives input of a combination ratio (weight) (step ST26). For example, a user of the data processing device 10 inputs the combination ratio (weight) to the data processing device 10 using the input interface 110. The weighted combination unit 1313 receives the input of the combination ratio (weight) by the user of the data processing device 10 via the input interface 110. Note that in FIG. 3 described above, the combination ratio (weight) is denoted by reference sign w ｉ .

[0056] Next, the weighted combination unit 1313 performs weighted combination processing on the first projection transformation matrix U １ and the corrected projection transformation matrix U' ２…ｉ according to the combination ratio (weight) w received in step ST26 ｉ . Similarly, the weighted combination unit 1313 performs weighted combination processing on the first singular value matrix D １ and the singular value matrix D ｉ according to the combination ratio (weight) w received in step ST26 ｉA weighted combination process is performed accordingly (step ST27).

[0057] Next, the weighted combination processing unit 131 processes all projection transformation matrices U'. ２…ｉ and singular value matrix D ｉ Regarding the first projection transformation matrix U, １ and the first singular value matrix D １ Check whether the weighted join process with has been completed (step ST28). As a result, all projection transformation matrices U' are checked. ２…ｉ and singular value matrix D ｉ Regarding the first projection transformation matrix U, １ and the first singular value matrix D １ If the weighted combination process has been completed (step ST28; YES), the weighted combination processing unit 131 outputs the projective transformation matrix Uc and the singular value matrix Dc after weighted combination, and terminates the process. Meanwhile, all projective transformation matrices U' ２…ｉ and singular value matrix D ｉ Regarding the first projection transformation matrix U, １ and the first singular value matrix D １ If the weighted join process with has not been completed (step ST28; NO), the process returns to step ST22. Then, the inner product calculation unit 1311 calculates the first projection transformation matrix U １ and the first singular value matrix D １ The projective transformation matrix U' has not yet completed the weighted join process with the other matrix. ２…ｉ and singular value matrix D ｉ The data is obtained, and the following processes from step ST22 onwards are carried out.

[0058] The following equation (2) represents the processing steps ST21 to ST28 described above in mathematical terms.

[0059] In equation (2), d ｎ This represents the inner product. Also, in equation (2), b ｎ This is the first projection transformation matrix U １ Represents the column vector contained in u ｎ This is the first projection transformation matrix U １ The projection transformation matrix U is joined with it. ｉ This represents the column vectors contained within. Also, n is the first projection transformation matrix U１ and projection transformation matrix U ｉ This represents the number of column vectors contained in (i.e., the number of feature quantities x). The calculation and correction (sign inversion) of the inner product is performed by the first projection transformation matrix U １ and projection transformation matrix U ｉ This is performed for all n column vectors contained in . Also, in equation (2), u ｎ ' represents the corrected column vector, U ｉ ' represents the corrected projection transformation matrix. Also, in equation (2), w ｉ represents the binding ratio (weight), U Ｃｏｍｂｉｎｅｄ This represents a new projection transformation matrix obtained through weighted concatenation.

[0060] Next, the effects of the data processing device 10 will be explained with reference to Figure 9. Figure 9 is a diagram that shows a comparison between the flow of weighted coupling processing by a conventional device (hereinafter referred to as "conventional device") and the flow of weighted coupling processing by the data processing device 10.

[0061] For example, in the processing flow of a conventional device (part 1), when training data containing feature x, which was used to train an existing learning model, is input to the conventional device, the conventional device calculates a covariance matrix Σ from feature x, and then performs a weighted combination process on the calculated covariance matrix Σ and a covariance matrix Σ similarly calculated from features contained in training data used to train other learning models, thereby obtaining a new covariance matrix Σ.

[0062] Furthermore, in the processing flow of the conventional device (part 2), when training data containing feature quantities x, which was used to train an existing learning model, is input to the conventional device, the conventional device performs singular value decomposition on feature quantities x, compresses the length (dimension) of feature quantities x, calculates a covariance matrix Σ from the dimensionality-reduced feature quantities x, and then performs a weighted concatenation process on the calculated covariance matrix Σ and a covariance matrix Σ similarly calculated from features contained in training data used to train other learning models, thereby obtaining a new covariance matrix Σ. As a result, in the processing flow of the conventional device, in either flow, the number of multiplications required to obtain a new covariance matrix Σ increases in proportion to the number R of covariance matrices to be concatenated, which sometimes resulted in a considerable amount of processing time.

[0063] In contrast, in the processing flow of the data processing device 10, when training data containing feature quantities x, which has been used to train an existing learning model, is input to the conventional device, the data processing device 10 compresses the dimension of feature quantities x by performing singular value decomposition on the feature matrix X generated from feature quantities x, and obtains a projection transformation matrix (left singular matrix) and a singular value matrix. Then, the data processing device 10 applies a weighted concatenation process to the obtained projection transformation matrix and singular value matrix, and to the projection transformation matrix and singular value matrix similarly obtained by performing singular value decomposition on the singular value matrix generated from features contained in training data used to train other learning models, and approximately calculates a new covariance matrix Σ from the resulting singular vectors and singular values. In other words, in the processing flow of the data processing device 10, a weighted concatenation process is performed on the projection transformation matrix and singular value matrix before calculating the new covariance matrix Σ.

[0064] As a result, the data processing device 10 only needs to calculate the new covariance matrix Σ once, regardless of the number of covariance matrices R to be combined, thus reducing the processing time compared to conventional devices. Furthermore, since the projection transformation matrix and singular value matrix targeted by the weighted combination process can have their dimensionality reduced compared to the original feature quantity x, the data processing device 10 can also reduce the number of multiplications required during weighted combination compared to conventional devices. Consequently, the data processing device 10 can reduce the processing time required to obtain the new covariance matrix Σ compared to conventional devices.

[0065] Next, examples of applications of the data processing device 10 will be explained with reference to Figures 10 and 11. For example, the data processing device 10 can be applied to cases like the one shown in Figure 10.

[0066] In Figure 10, learning model A is the model generated during initial training (initial operation), and training data A was used during training. Learning model B is the model generated during additional training, and training data A and training data B were used during training. Now, we want to combine learning model A and learning model B to generate a new learning model, and perform new training using training data C, which includes defective images.

[0067] In this case, the data processing device 10 acquires training data A, training data B, and training data C, respectively, performs singular value decomposition on the feature matrix generated from the features contained in each training data, and obtains a projection transformation matrix and a singular value matrix for each training data according to the flow described above. Then, the data processing device 10 applies a weighted coupling process to each projection transformation matrix and singular value matrix according to a predetermined coupling ratio, and approximates a new covariance matrix Σ from the newly obtained projection transformation matrix and singular value matrix. As a result, the data processing device 10 can obtain a new learning model (coupled model) in a shorter time compared to processing by conventional devices.

[0068] Furthermore, the data processing device 10 can also be applied to cases such as those shown in Figure 11. In Figure 11, learning model A is a model in which learning data A was used during learning. Learning model B is a model in which learning data B was used during learning. Learning model N is a model in which learning data N was used during learning. Here, we want to combine learning model A, learning model B, and learning model N to generate a new learning model, and also perform new learning using learning data C which includes defective images.

[0069] In this case, the data processing device 10 acquires training data A, training data B, training data C, and training data N, respectively, performs singular value decomposition on the feature matrix generated from the features contained in each training data, and obtains a projection transformation matrix and a singular value matrix for each training data according to the flow described above. Then, the data processing device 10 applies a weighted coupling process to each projection transformation matrix and singular value matrix according to a predetermined coupling ratio, and approximates a new covariance matrix Σ from the newly obtained projection transformation matrix and singular value matrix. As a result, the data processing device 10 can obtain a new learning model (coupled model) in a shorter time compared to processing by conventional devices.

[0070] Up to this point, the explanation has described an example in which the data processing device 10 is applied to the generation of a learning model (training data). However, the data processing device 10 is not limited to this example and can be applied to other cases as long as it performs weighted combination processing.

[0071] For example, the data processing device 10 can also be applied to cases where a multivariate normal distribution is obtained from a mixture of normal distributions that contains multiple multivariate normal distributions. For a multivariate normal distribution, the multivariate normal distribution with m variables is expressed by the following equation (3). Equation (3) allows us to obtain a distribution based on the mean μ of the m variables and the covariance matrix Σ.

[0072] For a mixture of normal distributions, if there are m variables and n normal distributions, the mixture of normal distributions can be expressed by the following equation (4).

[0073] Although details are omitted here, weighted concatenation processing is sometimes performed when obtaining the multivariate normal distributions included in a mixture of normal distributions. In such cases, applying the data processing device 10 reduces the number of multiplications required to obtain the multivariate normal distributions included in the mixture of normal distributions, thereby shortening the processing time compared to cases where the data processing device 10 is not applied. Furthermore, the data processing device 10 can be applied to any case in which weighted concatenation processing is performed.

[0074] Next, with reference to Figure 12, an example of the hardware configuration of the data processing device 10 according to Embodiment 1 will be described. The functions of the learning unit 11, the coupling unit 13, and the calculation unit 15 in the data processing device 10 are realized by a processing circuit. The processing circuit may be dedicated hardware as shown in Figure 12A, or as shown in Figure 12B, it may be a CPU (Central Processing Unit, central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, processor, or DSP (D) that executes the program stored in the memory 53. ｉ It may also be called a (gital Signal Processor) 52.

[0075] When the processing circuit is dedicated hardware, the processing circuit 51 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, or an ASIC (Application Specific Integrated Circuit). ｉ This includes t), FPGA (Field Programmable Gate Array), or a combination thereof. The functions of each part of the learning unit 11 and the coupling unit 13 may be implemented by the processing circuit 51, or the functions of each part may be implemented together by the processing circuit 51.

[0076] When the processing circuit is a CPU 52, the functions of the learning unit 11, the coupling unit 13, and the calculation unit 15 are realized by software, firmware, or a combination of software and firmware. The software and firmware are written as programs and stored in memory 53. The processing circuit realizes the functions of each unit by reading and executing the programs stored in memory 53. In other words, the data processing device 10 has memory for storing programs that, when executed by the processing circuit, result in each step shown in Figure 5 being executed, for example. These programs can also be said to cause the computer to execute the procedures and methods of the learning unit 11, the coupling unit 13, and the calculation unit 15. Here, the memory 53 may be, for example, a non-volatile or volatile semiconductor memory such as RAM (RAM Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable ROM), EEPROM (Electrically EPROM), magnetic disk, flexible disk, optical disk, compact disk, minidisc, or DVD (D ｉ digital Versatile D ｉ This includes (sc), etc.

[0077] Furthermore, the functions of the learning unit 11, the coupling unit 13, and the calculation unit 15 may be partially implemented by dedicated hardware and partially by software or firmware. For example, the learning unit 11 can be implemented by a processing circuit as dedicated hardware, while the coupling unit 13 and the calculation unit 15 can be implemented by a processing circuit reading and executing a program stored in the memory 53.

[0078] Thus, the processing circuit can realize each of the above-mentioned functions through hardware, software, firmware, or a combination thereof.

[0079] As described above, according to this embodiment 1, the data processing device 10 includes a learning unit 11 that acquires projection transformation matrices and singular value matrices generated for each set of learning data based on a plurality of learning data, the projection transformation matrices and singular value matrices having a reduced number of dimensions compared to the corresponding learning data; a coupling unit 13 that calculates new projection transformation matrices and singular value matrices by applying weighted coupling processing to the plurality of projection transformation matrices and singular value matrices acquired by the learning unit 11; and a calculation unit 15 that approximates a covariance matrix obtained by applying weighted coupling processing to the covariance matrices corresponding to each of the plurality of learning data using the new projection transformation matrices and singular value matrices calculated by the coupling unit 13. As a result, the data processing device 10 according to embodiment 1 can obtain a matrix corresponding to the new covariance matrix obtained by weighted coupling processing of a plurality of covariance matrices in a shorter time than conventional methods.

[0080] Furthermore, the learning unit 11 acquires the training data used for training each of the multiple learning models, and obtains the left singular matrix obtained by singular value decomposition of the acquired training data as the projection transformation matrix. As a result, the data processing device 10 according to Embodiment 1 can obtain a projection transformation matrix with a reduced number of dimensions compared to the training data.

[0081] Furthermore, the projection transformation matrix generated for each training data is a transformation matrix used to project the training data that was the subject of singular value decomposition at the time the projection transformation matrix was obtained onto the principal component space of the training data after dimensionality reduction. As a result, the data processing device 10 according to Embodiment 1 can obtain a projection transformation matrix with a reduced dimensionality compared to the training data.

[0082] Furthermore, when performing weighted join processing, the join unit 13 performs corrections to prevent the loss of corresponding column vector components in the multiple projection transformation matrices to be joined. As a result, the data processing device 10 according to Embodiment 1 can prevent the loss of specific column vector components during weighted join processing.

[0083] Furthermore, the coupling unit 13 includes a dot product calculation unit 1311 that calculates the dot product of a reference vector and other column vectors when one of the corresponding column vectors in a plurality of projection transformation matrices is used as the reference vector; a sign inversion unit 1312 that corrects the result calculated by the dot product calculation unit 1311 by inverting the sign of the other column vectors if the result is negative; and a weighted coupling unit 1313 that applies weighted coupling processing to a plurality of projection transformation matrices, including the projection transformation matrix after correction by the sign inversion unit 1312, and a plurality of singular value matrices. As a result, the data processing device 10 according to Embodiment 1 can prevent the loss of components of column vectors whose dot product is negative during weighted coupling processing.

[0084] Furthermore, the projection transformation matrix and singular value matrix acquired by the learning unit 11 are generated for each set of learning data, based on the learning data used for learning each of the multiple learning models, or the learning data used for learning each of the multiple normal distributions. As a result, the data processing device 10 according to Embodiment 1 can acquire the projection transformation matrix and singular value matrix from the learning data used for learning the learning model or the normal distribution.

[0085] Embodiment 2. Embodiment 1 described a data processing device 10 that can obtain a matrix corresponding to a new covariance matrix obtained by weighted joining of multiple covariance matrices in a shorter time than conventional methods. Embodiment 2 describes a data processing device 10b that can evaluate the similarity between the principal component space defined by the new projection transformation matrix calculated by the joining unit 13 and the data to be evaluated.

[0086] Figure 13 shows an example configuration of the data processing device 10b according to Embodiment 2. As shown in Figure 13, the example configuration of the data processing device 10b according to Embodiment 2 is an addition of an evaluation unit 17 to the data processing device 10 according to Embodiment 1 shown in Figure 2. The other components of the data processing device 10b according to Embodiment 2 are the same as those of the data processing device 10 according to Embodiment 1 shown in Figure 2, and therefore the same reference numerals are used and their descriptions are omitted.

[0087] The evaluation unit 17 evaluates the similarity between the principal component space defined by the new projection transformation matrix calculated by the coupling unit 13 and the data to be evaluated.

[0088] For example, the evaluation unit 17 evaluates the new projection transformation matrix U calculated by the coupling unit 13. Ｃ , Singular Value Matrix D Ｃ , and a new projection transformation matrix U Ｃ The similarity S can be calculated from the mean value μ of multiple training data projected onto the principal component space defined by the following equation (5). In equation (5), α in the first term and β in the second term are parameters that the user can adjust to appropriate values.

[0089] Figure 14 is a diagrammatic representation of the components corresponding to the first and second terms of equation (5). The test vector x is a vector obtained by projecting the data (features) to be evaluated onto the principal component space.

[0090] The evaluation unit 17 is configured to include, for example, a first-component similarity calculation unit 171 and a second-component similarity calculation unit 172, as shown in Figure 15. The first-component similarity calculation unit 171 calculates the first term of equation (5). The second-component similarity calculation unit 172 calculates the second term of equation (5). The evaluation unit 17 adds the calculation result from the first-component similarity calculation unit 171 and the calculation result from the second-component similarity calculation unit 172 using an adder to calculate the final similarity S(X). In Figure 15, α and β represent the parameters described above, X represents the data to be evaluated (test vector x), μ represents the average value described above, and U ｋ D represents the projection transformation matrix after joining, and D represents the singular value matrix after joining.

[0091] The similarity score S(X) calculated by the evaluation unit 17 can be used, for example, in anomaly detection technology. For example, if normal data is distributed in a principal component space defined by a new projection transformation matrix Uc, data with a high similarity score S to this principal component space is likely to be normal data. On the other hand, data with a low similarity score S(X) to this principal component space is likely to be anomaly data. Therefore, for example, an anomaly detection device (not shown) can be realized that includes a detection unit that detects whether the data to be evaluated is anomaly data or not based on the similarity score S(X) evaluated by the evaluation unit 17.

[0092] Next, an example of the operation of the data processing device 10b shown in Figure 16 will be described. Figure 16 is a flowchart illustrating an example of the operation of the data processing device 10b.

[0093] First, the evaluation unit 17 acquires parameters α and β (step ST31). For example, the evaluation unit 17 acquires parameters α and β that the user inputs to the data processing device 10b using the input interface 110.

[0094] Next, the evaluation unit 17 evaluates the mean vector μ and the combined projection transformation matrix U. ｋ , and the combined singular value matrix D ｋ Obtain (step ST32).

[0095] Next, the evaluation unit 17 acquires the feature quantity x to be evaluated (step ST33).

[0096] Next, the evaluation unit 17 combines the feature quantity x with the combined projection transformation matrix U. ｋ The similarity to the principal component space defined by is calculated (step ST34). For example, the evaluation unit 17 calculates the similarity using the above-mentioned equation (99).

[0097] Next, the evaluation unit 17 checks whether the similarity calculation has been completed for all feature quantities x to be evaluated (step ST35). If the similarity calculation has been completed for all feature quantities x to be evaluated (step ST35; YES), the evaluation unit 17 terminates the process. If the similarity calculation has not been completed for all feature quantities x to be evaluated (step ST35; NO), the process returns to step ST33, and the evaluation unit 17 acquires the feature quantities x for which the similarity calculation has not been completed.

[0098] As described above, according to this second embodiment, the data processing device 10b includes an evaluation unit 17 that evaluates the similarity between the principal component space defined by the new projection transformation matrix calculated by the coupling unit 13 and the data to be evaluated. As a result, in addition to the effects of the first embodiment, the data processing device 10b according to the second embodiment can evaluate how similar the data to be evaluated is to the principal component space defined by the new projection transformation matrix.

[0099] Furthermore, according to this second embodiment, the anomaly detection device includes a detection unit that detects whether or not the data to be evaluated is anomaly data based on the similarity evaluated by the evaluation unit 17. As a result, the anomaly detection device according to the second embodiment can detect whether or not the data to be evaluated is anomaly data.

[0100] Although preferred embodiments have been described in detail above, the invention is not limited to the embodiments described above, and various modifications and substitutions can be made to the embodiments described above without departing from the scope of the claims.

[0101] Furthermore, this disclosure allows for free combination of each embodiment, modification of any component of each embodiment, or omission of any component in each embodiment.

[0102] This disclosure makes it possible to obtain a matrix equivalent to a new covariance matrix obtained by weighted joining of multiple covariance matrices in a shorter time than conventional methods, and is suitable for use in data processing devices, anomaly detection devices, and data processing methods.

[0103] 10, 10b Data processing unit, 11 Learning unit, 13 Coupling unit, 15 Calculation unit, 17 Evaluation unit, 51 Processing circuit, 52 CPU, 53 Memory, 110 Input interface, 130 Output interface, 131 Weighted coupling processing unit, 171 Similarity calculation unit for the first component, 172 Similarity calculation unit for the second component, 1311 Dot product calculation unit, 1312 Sign inversion unit, 1313 Weighted coupling unit, D １ First singular value matrix, D ｃ The singular value matrix after joining, D ｉ Singular value matrix, D ｋ Singular value matrix, U １ First projection transformation matrix, U ｃ The projection transformation matrix after joining, U ｉ Projection transformation matrix, U ｋ Projection transformation matrix, w ｉ Weight coefficients, X: Feature matrix, α, β: Parameters, Σ: Covariance matrix.

Claims

1. A data processing device comprising: a learning unit that obtains projection transformation matrices and singular value matrices generated for each set of training data based on multiple training data, wherein the projection transformation matrices and singular value matrices have a reduced number of dimensions compared to the corresponding training data; a coupling unit that calculates new projection transformation matrices and singular value matrices by applying weighted coupling processing to the multiple projection transformation matrices and singular value matrices obtained by the learning unit; and a calculation unit that approximates a covariance matrix obtained by applying weighted coupling processing to the covariance matrices corresponding to each of the multiple training data using the new projection transformation matrices and singular value matrices calculated by the coupling unit.

2. The data processing device according to claim 1, characterized in that the learning unit acquires the plurality of learning data and acquires the left singular matrix obtained by singular value decomposition of each of the acquired learning data as the projection transformation matrix.

3. The data processing device according to claim 2, characterized in that the projection transformation matrix generated for each training data is a transformation matrix for projecting the training data that was the subject of singular value decomposition when the projection transformation matrix was obtained onto the principal component space of the training data after dimensionality reduction.

4. The data processing apparatus according to any one of claims 1 to 3, characterized in that the coupling unit performs a correction to prevent the loss of corresponding column vector components in the multiple projection transformation matrices to be coupled when performing the weighted coupling process.

5. The data processing apparatus according to claim 4, wherein the coupling unit comprises: a dot product calculation unit that calculates the dot product of a reference vector and another column vector when one of the corresponding column vectors in the plurality of projection transformation matrices is used as the reference vector; a sign inversion unit that performs the correction by inverting the sign of the other column vector if the result calculated by the dot product calculation unit is negative; and a weighted coupling unit that performs weighted coupling processing on a plurality of projection transformation matrices, including the projection transformation matrix after the correction by the sign inversion unit, and a plurality of singular value matrices.

6. The data processing device according to any one of claims 1 to 5, characterized in that the projection transformation matrix and singular value matrix acquired by the learning unit are generated for each set of learning data based on the learning data used for learning each of the multiple learning models, or the learning data used for learning each of the multiple normal distributions.

7. The data processing apparatus according to any one of claims 1 to 6, further comprising an evaluation unit that evaluates the similarity between the principal component space defined by the new projection transformation matrix calculated by the coupling unit and the data to be evaluated.

8. An anomaly detection device characterized by comprising a detection unit that detects whether or not the data to be evaluated is abnormal data based on the similarity evaluated by the evaluation unit of the data processing device according to claim 7.

9. A data processing method by a data processing device, comprising: a learning unit obtaining a projection transformation matrix and a singular value matrix generated for each of a plurality of learning data based on the learning data, the projection transformation matrix and the singular value matrix having a reduced number of dimensions compared to the corresponding learning data; a merging unit calculating a new projection transformation matrix and a singular value matrix by applying a weighted merging process to the plurality of projection transformation matrices and singular value matrices obtained by the learning unit; and a calculation unit approximately calculating a covariance matrix obtained by applying a weighted merging process to the covariance matrix corresponding to each of the plurality of learning data using the new projection transformation matrix and singular value matrix calculated by the merging unit.

Images