Information processing device

Abstract

This information processing device comprises a calculation unit that performs an abnormality degree-using process in which: an abnormality degree for a numerical variable portion is calculated using an abnormality detection model for each record in a plurality of records constituting a portion of all records in table data; the average value of abnormality degrees for each category variable in the plurality of records is calculated; and a process for allocating the average value to a category variable in a record other than the plurality of records among all the records is repeatedly executed while changing the plurality of records.

Description

Information processing device

[0001] The present invention relates to a technique for handling categorical variables in table data in a machine learning model.

[0002] A categorical variable is a variable that represents a classification rather than a numerical value, such as gender = {male, female}. When inputting data containing categorical variables into a machine learning model, it is necessary to replace the categorical variables with some kind of numerical value.

[0003] In particular, one-hot vectorization is a common numerical substitution method when inputting categorical variables into deep learning models. However, when using this method, the categorical variable is represented as a vector with the same number of dimensions as the number of categories that the variable can take. When there are many categories, the one-hot vector becomes a sparse and high-dimensional vector, which raises concerns about a decrease in the accuracy of the deep learning model due to the curse of dimensionality.

[0004] On the other hand, target encoding exists as a means of representing categorical variables as low-dimensional vectors, and target encoding is a method that targets table data containing correct labels that are to be output by a machine learning model (Non-Patent Document 1).

[0005] F. Pargent et al., "Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features", Computational Statistics Volume 37, pages 2671-2692, (2022).

[0006] The present invention has been made in consideration of the above points, and aims to provide a technique that enables categorical variables in unsupervised table data to be replaced with numerical values ​​without increasing the dimensionality and sparsity.

[0007] According to the disclosed technology, there is provided an information processing device including a calculation unit that performs an anomaly level using process that calculates an anomaly level for a numerical variable portion of each record in a plurality of records that is a portion of all records of table data, calculates an average value of the anomaly level for each categorical variable in the plurality of records, and assigns the average value to categorical variables in records other than the plurality of records out of all records, while repeatedly changing the plurality of records.

[0008] The disclosed technology provides a technology that enables categorical variables in unsupervised table data to be replaced with numerical values ​​without increasing the dimension and causing sparsity.

[0009] FIG. 1 is a diagram illustrating an example of the configuration of an information processing device 100. FIG. 2 is a diagram illustrating an example of table data. FIG. 3 is a diagram illustrating Example 1. FIG. 4 is a diagram illustrating Example 1. FIG. 5 is a diagram illustrating Example 2. FIG. 6 is a diagram illustrating a processing flow. FIG. 7 is a diagram illustrating an example of the hardware configuration of the information processing device 100.

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment.

[0011] Hereinafter, as a technology according to the present embodiment, a technology for performing target encoding on unsupervised data will be described. This technology handles high-dimensional vectorization when replacing categorical variables with numerical values, and contributes to improving the accuracy of unsupervised deep learning models.

[0012] Conventional target encoding is a method for supervised data (data with correct answer data), but the technology according to this embodiment is similar to conventional target encoding in that it encodes categorical variables using data other than categorical variables in table data. Therefore, for convenience, the term "target encoding" is used in this embodiment.

[0013] 1 shows an example of the configuration of an information processing device 100 according to this embodiment. As shown in FIG. 1, the information processing device 100 includes a feature constructing unit 110, an anomaly degree calculating unit 120, a cluster calculating unit 130, a matching unit 140, a categorical variable replacing unit 150, and a database 160.

[0014] The “feature constructing unit 110, the anomaly degree calculating unit 120, the cluster calculating unit 130, the matching unit 140, and the categorical variable replacing unit 150” may be referred to as the calculation unit 170. The database 160 may be provided outside the information processing device 100.

[0015] An example of the operation of the information processing device 100, including the processing content of each unit, will be described below. Finally, the overall processing flow will be described.

[0016] (Example of Table Data) It is assumed that the database 160 stores the table data shown in Fig. 2. That is, it is assumed that the table data shown in Fig. 2 is input to a deep learning model.

[0017] The table shown in Figure 2 is a two-dimensional table, and each row represents one record of information, which is the unit of input to the deep learning model. For example, one row represents data for one individual.

[0018] The header of each column indicates information about the feature (age, sex, etc.), and the rest of each column contains the value of that feature in the corresponding row.

[0019] In the example shown in Figure 2, the leftmost column is the record ID, the rightmost column is the categorical variable, and the rest are numerical variables. However, the position of the feature that is a categorical variable in the table and the number of categorical variables are arbitrary, and one or more feature may be a categorical variable. Furthermore, the record ID is used to uniquely specify each record and is not usually input to the machine learning model.

[0020] (Regarding Numerical Conversion Method) In this embodiment, a method using two models will be described as a method for numerically converting categorical variables using target encoding for table data that does not have correct labels (unsupervised table data) as shown in Fig. 2. The two models are (1) an anomaly detection model and (2) a clustering model.

[0021] When actually performing numeric conversion of categorical variables, only one of these may be used, or both may be used and combined to form a vector obtained by target encoding. That is, the calculation unit 170 shown in FIG. 1 may have only (1) a function corresponding to a technique using an anomaly detection model, or (2) a function corresponding to a technique using a clustering model, or may have a function to execute both of these.

[0022] Basically, when there is a strong correlation between categorical variables and other numerical variables, the method using the clustering model in (2) is often useful; when there is not, the method using the anomaly detection model in (1) is considered to be more effective because it is simpler.

[0023] Below, (1) the operation of the information processing device 100 in a method using an anomaly detection model and (2) the operation of the information processing device 100 in a method using a clustering model will be described. (1) The method using the anomaly detection model will be referred to as "Example 1," and (2) the method using the clustering model will be referred to as "Example 2." Furthermore, an example that combines Examples 1 and 2 will be referred to as Example 3.

[0024] Example 1 In Example 1, out-of-fold encoding is used, similar to target encoding of supervised table data. Out-of-fold encoding generally refers to a technique in which a dataset is divided into multiple subsets (folds), one of which is used as test data, and the remaining folds are used as training data.

[0025] To use out-of-fold, the anomaly degree calculation unit 120 divides the table data read from the database 160 into n parts in the row direction, and separates the data into (i) one group of records and (ii) the other group of records.

[0026] The anomaly degree calculation unit 120 trains an anomaly detection model using only the numerical variable parts of records belonging to (ii). Here, when the numerical variable parts of a certain record are input to the anomaly detection model, the anomaly detection model is trained so that the anomaly degree of the record is output.

[0027] The anomaly detection model used here is not limited to a specific anomaly detection model, and any of a variety of anomaly detection models, from simple models to models using deep learning, such as Local Outlier Factor, OneClass SVM, Isolation Forest, Autoencoder, and DeepSVDD, may be used.

[0028] After training the anomaly detection model, the anomaly degree calculation unit 120 inputs the numerical variable parts of the records belonging to (ii) into the trained model again and calculates the degree of anomaly for each record. After calculating the degree of anomaly, the anomaly degree calculation unit 120 groups the records belonging to (ii) by type of categorical variable and calculates the average degree of anomaly.

[0029] The categorical variable replacement unit 150 replaces the calculated average of the abnormality degree with the corresponding categorical variable of the records belonging to (i). A specific example will be described with reference to FIGS.

[0030] In "(a) 1st time" in Figure 3, the anomaly degree calculation unit 120 calculates the anomaly degree for each of the records in the bottom 9 rows, and then averages the anomaly degrees for the records in the bottom 9 rows, dividing them into males and females. Then, among the top 3 rows, the records whose categorical variables are male are assigned the average anomaly degree for males calculated above, and the records whose categorical variables are female are assigned the average anomaly degree for females. "(a) 1st time" in Figure 3 shows an image of the processing for males.

[0031] Figure 4(c) shows an example of the degree of anomaly for each of the nine records below "(a) 1st Time" in Figure 3. Figure 4(d) shows the average values ​​of the degree of anomaly for males and females. Figure 4(e) shows an image in which categorical variables are replaced with the calculated average values.

[0032] The anomaly degree calculation unit 120 performs the above calculation (anomaly detection model learning, anomaly degree calculation) n times while changing the group belonging to (i) so that an anomaly average is assigned to all groups, and sets this as the numerical conversion value of the categorical variable. An image of the second calculation is shown in "(b) Second Time" in Figure 3.

[0033] (Example 2) Next, (2) a method using a clustering model will be described as Example 2. To use out-of-fold as in Example 1, the cluster calculation unit 130 first divides table data read from the database 160 into n parts in the row direction, and then separates the data into (i) one group of records and (ii) the other group of records.

[0034] The cluster calculation unit 130 performs clustering using only the numerical variable portion of the records in (ii). The clustering method used is not limited to a specific clustering method, but examples include Ward's method and k-means. The number of clusters used in clustering may be set appropriately, or an appropriate number of clusters may be determined by using all data and performing an x-means method or visualization using UMAP, and then set to that value.

[0035] For the first group, the cluster calculation unit 130 transforms the clustering results of each record in (ii) into a one-hot vector with a dimension equal to the number of clusters, calculates the average of the one-hot vector for each categorical variable, and assigns it to each categorical variable in (i).

[0036] Figure 5 shows an example of the above calculation when group (ii) is a group consisting of 12 records. In this example, as shown in Figure 5(a), each record is divided into three clusters (cluster 1, cluster 2, and cluster 3). Figure 5(b) shows that each record is transformed into a one-hot vector with a dimension equal to the number of clusters by assigning 1 to the cluster to which the record belongs and 0 to other clusters.

[0037] Figure 5(c) shows that the one-hot vectors were averaged for each categorical variable. In other words, it shows that the average vector was calculated. For example, in Figure 5(b), there are six records for males, of which two belong to category 1. Therefore, as shown in Figure 5(c), the average value for males in cluster 1 is 1 / 3.

[0038] The categorical variable replacement unit 150 replaces the categorical variable for male in (i) with the vector for male = (1 / 3, 1 / 2, 1 / 6) in Fig. 5(c), and replaces the categorical variable for female in (i) with the vector for female = (1 / 3, 0, 2 / 3) in Fig. 5(c). Note that the actual replacement may be performed after the vectors for all records have been determined.

[0039] The clustering calculation unit 130 performs clustering on subsequent groups in the same manner. However, the cluster numbers are not consistent. For example, if the clustering method is performed twice on the same data, a cluster that was initially designated as cluster 1 may be designated as cluster 3 the second time. Therefore, it is necessary to make the results of the n clusterings consistent. To achieve this, the cluster calculation unit 130 and the matching unit 140 perform the following process.

[0040] In the clustering of the i-th group (i>1), there are records that overlap with those in the i-1-th group (the bottom six rows in the example of Figure 3), so the cluster calculation unit 130 records which cluster each record used in clustering in the next group belonged to during the i-1-th clustering.

[0041] The cluster calculation unit 130 also records the clustering results of each record used in the i-th clustering. The cluster calculation unit 130 also records the record IDs belonging to each record in each (i-1)th cluster and each record in each i-th cluster.

[0042] Then, for each (i-1)th cluster, the cluster calculation unit 130 calculates the number of records belonging to the i-th cluster j among the records belonging to the cluster, and rearranges the i-th clusters in descending order of the number of records belonging to the cluster j, thereby forming a friendship order for each (i-1)th cluster with respect to each i-th cluster. Similarly, the cluster calculation unit 130 also constructs a friendship order for the i-1th cluster with respect to the i-th cluster. Note that the friendship order may be constructed by the matching unit 140.

[0043] 6 shows an image of the (i-1)th cluster, the i-th cluster, and the friendship order when the number of clusters is 3. FIG. 6 also shows an image of matching.

[0044] The matching unit 140 solves the matching problem between the (i-1)th cluster and the i-th cluster using a DA algorithm, and determines the consistency between the (i-1)th and i-th clusters based on the matching result.

[0045] The matching unit 140 (or the cluster calculation unit 130) rearranges the order of the i-th cluster according to the matching result so that it matches the order of the i-1-th cluster, and constructs a one-hot vector.

[0046] For example, in the example of matching shown in FIG. 6, the vectors for the (i-1)th time are, as in FIG. 5(c), male = (1 / 3, 1 / 2, 1 / 6) and female = (1 / 3, 0, 2 / 3).

[0047] In this case, in the i-th iteration, a one-hot vector is constructed with males = (average value of cluster 2, average value of cluster 1, average value of cluster 3) and females = (average value of cluster 2, average value of cluster 1, average value of cluster 3).

[0048] After that, the process is performed in the same way as the first association of one-hot vectors with categorical variables. By performing clustering n times, including the first time, the categorical variable replacement unit 150 can replace categorical variables with vectors of the same dimension as the number of clusters.

[0049] (Example 3: Combination of Example 1 and Example 2) When using both (1) the method using an anomaly detection model and (2) the method using a clustering model, the information processing device 100 first calculates a numerical value (average degree of anomaly) corresponding to the categorical variable of each record by the process described in Example 1, and also calculates a vector corresponding to the categorical variable of each record by the process described in Example 2.

[0050] Next, the feature constructing unit 110 creates a vector of the number of clusters + 1 dimension by combining the average anomaly degree with a vector of the number of clusters dimension for each record. The categorical variable replacing unit 150 replaces the categorical variable for each record with a vector of the number of clusters + 1 dimension.

[0051] (Other Examples) In any of the first to third embodiments, when there are a plurality of feature quantities of categorical variables, the information processing device 100 performs the operations described in each embodiment on the feature quantities of each categorical variable.

[0052] In addition, in any of Examples 1 to 3, if part of the table data is also used as validation data or test data, the records for validation and test are separated in advance, and the operations described in each Example are performed on the remaining training data.

[0053] Categorical variables in validation and test records are replaced with outlier means and cluster means calculated using all training data.

[0054] In particular, when the method of Example 2 is used, the cluster calculation unit 130 of the information processing device 100 calculates the consistency of the clusters by calculating argmax for the train data in which the categorical variables are replaced with numeric vectors, and confirming to which cluster each record belongs. Note that this "argmax" corresponds to replacing the train data categorical variables with numeric vectors and using the numeric vectors to determine the cluster to which each record belongs.

[0055] Thereafter, the cluster calculation unit 130 performs clustering on all the training data and records which cluster each record belongs to.

[0056] The matching unit 140 performs a friendship order construction process similar to the process described in Example 2 on the clusters obtained by replacing the categorical variables with numerical vectors and the clusters obtained when actual clustering is performed on all training data, and further solves a matching problem to ensure consistency between the clusters.

[0057] Thereafter, the feature constructing unit 110 (which may be the matching unit 140 or the categorical variable replacing unit 150) calculates the average of the one-hot vector for each categorical variable in the training data and replaces it with the categorical variables in the validation or test data.

[0058] (Processing Flow) An example of the overall processing flow will be described with reference to the flowchart of FIG.

[0059] In S1 (step 1), the feature constructing unit 110 selects one of the processes of the first embodiment, the second embodiment, and both the processes of the first embodiment and the second embodiment (i.e., the third embodiment).

[0060] When the processing of the first embodiment is selected, the processing of the first embodiment is executed in S2. That is, the anomaly degree calculation unit 120 calculates the anomaly degree n times out-of-hold, takes the average for each category type, and sends the result each time to the feature construction unit 110. When the results for all records have been obtained, the categorical variable replacement unit 150 converts the categorical variables into numerical variables.

[0061] If the processing of Example 2 is selected, the processing of Example 3 is executed in S3. That is, the cluster calculation unit 130 calculates clusters. In the second and subsequent calculations, the matching unit 140 performs matching to ensure consistency between the results of the previous clustering and the results of the current clustering. The cluster calculation unit 130 or the matching unit 140 calculates an average for each category type for the one-hot vector of the dimension of the number of clusters calculated by the cluster calculation unit 130 and the matching unit 140, and sends the result to the feature construction unit 110 each time. When the results for all records have been obtained, the categorical variable replacement unit 150 converts the categorical variables into numerical variables.

[0062] When both Example 1 and Example 2 are selected, the processing of Example 3 is performed. That is, the information processing device 100 performs the processing of Example 1 and Example 2, and the feature constructing unit 110 combines the vectors. The categorical variable replacing unit 150 converts the categorical variables into combined vectors.

[0063] (Hardware Configuration Example) The information processing device 100 described in this embodiment can be realized, for example, by causing a computer to execute a program. This computer may be a physical computer or a virtual machine on the cloud.

[0064] That is, the information processing device 100 can be realized by using hardware resources such as a CPU and memory built into a computer to execute a program corresponding to the processing performed by the information processing device 100. The program can be recorded on a computer-readable recording medium (such as a portable memory) and can be saved or distributed. The program can also be provided via a network such as the Internet or email.

[0065] Fig. 8 is a diagram showing an example of the hardware configuration of the computer. The computer in Fig. 8 includes a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, and the like, all of which are interconnected via a bus B. The computer may further include a GPU.

[0066] The program that realizes the processing on the computer is provided by a recording medium 1001, such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.

[0067] The memory device 1003 reads and stores a program from the auxiliary storage device 1002 when an instruction to start the program is received. The CPU 1004 realizes functions related to the information processing device 100 in accordance with the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network, etc. The display device 1006 displays a GUI (Graphical User Interface) or the like according to the program. The input device 1007 is composed of a keyboard, mouse, buttons, a touch panel, etc., and is used to input various operation instructions. The output device 1008 outputs the results of calculations.

[0068] (Summary, effects, etc. of the embodiment) As explained above, the technology described in this embodiment realizes a method for applying target encoding, a categorical variable replacement method that has previously been applied only to supervised table data, to unsupervised table data.

[0069] The above method allows us to replace categorical variables with low-dimensional numerical vectors when inputting them into an unsupervised deep learning model. This reduces the problems of sparsity and the curse of dimensionality that occur with conventional methods, and can lead to improved learning accuracy in unsupervised deep learning models.

[0070] The following additional notes are provided regarding the above-described embodiments.

[0071] <Additional Notes> (Additional Item 1) An information processing device comprising: a calculation unit that performs an anomaly level using process, which calculates an anomaly level for a numeric variable portion of each record in a portion of all records of table data using an anomaly detection model, calculates an average value of the anomaly level for each categorical variable in the plurality of records, and assigns the average value to categorical variables in records other than the plurality of records out of all records, while repeatedly changing the plurality of records. (Additional Item 2) An information processing device comprising: a calculation unit that performs cluster using process, which uses the numeric variable portions of a portion of all records of table data to perform clustering to divide the plurality of records into a plurality of clusters, and based on a result of the clustering, generates, for each record of the plurality of records, a vector with dimensions equal to the number of clusters, where 1 denotes a cluster to which the record belongs, calculates an average vector for each categorical variable in the plurality of records, and assigns the average vector to categorical variables in records other than the plurality of records out of all records, while repeatedly changing the plurality of records. (Supplementary Item 3) The information processing device according to Supplementary Item 2, wherein the calculation unit performs a matching process between a plurality of clusters obtained in a certain processing run and a plurality of clusters obtained in a processing run immediately before the certain processing run. (Supplementary Item 4) An information processing device comprising: a calculation unit that executes the anomaly degree use process according to Supplementary Item 1 and the cluster use process according to Supplementary Item 2.

[0072] Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims.

[0073] REFERENCE SIGNS LIST 100 Information processing device 110 Feature construction unit 120 Anomaly degree calculation unit 130 Cluster calculation unit 140 Matching unit 150 Categorical variable replacement unit 160 Database 170 Calculation unit 1000 Drive device 1001 Recording medium 1002 Auxiliary storage device 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input device 1008 Output device

Claims

1. An information processing device having a calculation unit that performs an anomaly detection process that calculates the degree of anomaly for a numerical variable portion of each record in a plurality of records that is a portion of all records in table data, calculates the average value of the degree of anomaly for each categorical variable in the plurality of records, and assigns the average value to the categorical variables in records other than the plurality of records out of all records, while repeatedly changing the plurality of records.

2. An information processing device having a calculation unit that performs cluster usage processing, which uses the numerical variable portions of a portion of all records of table data to perform clustering to divide the plurality of records into a plurality of clusters, and based on the results of the clustering, generates a vector of dimension 1 for the number of clusters for each record of the plurality of records, where the cluster to which the record belongs is 1, calculates an average vector for each categorical variable in the plurality of records, and assigns the average vector to categorical variables in records other than the plurality of records out of all the records, while changing the plurality of records.

3. The information processing device according to claim 2, wherein the calculation unit performs a matching process between multiple clusters obtained in a certain processing run and multiple clusters obtained in the processing run immediately before the current processing run.

4. An information processing device comprising a calculation unit that executes the abnormality level use process according to claim 1 and the cluster use process according to claim 2.

Images