Information processing method, program, and information processing apparatus

Abstract

To provide an information processing method or the like capable of suitably selecting a feature amount to be adopted for a clustering model.SOLUTION: An information processing method acquires a data group, wherein each data is composed of a plurality of feature amounts and a label is attached to a certain number of data, sequentially selects a combination of the feature amounts by using a reinforcement learning model for outputting a combination of the feature amount to be selected next in the case of inputting the combination of feature amounts selected at the current time point, sequentially generates a clustering model on the basis of the feature amounts of the selected combination, calculates a minority degree on the basis of a data number with the label classified into each cluster, sequentially updates the reinforcement learning model in accordance with the minority degree, calculates the number of model searching times after learning completion of the reinforcement learning model, and makes a computer execute processing for determining an optimum combination of feature amounts in accordance with the minority degree and the number of model searching times.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This invention relates to an information processing method, a program, and an information processing apparatus. [Background technology]

[0002] Methods for selecting features to be used as input for machine learning models include the filter method and the wrapper method. For example, Patent Document 1 discloses a data processing device that combines the filter method with AIC (Akaike Information Criteria) to select features and constructs a model by learning the selected features from sample data. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] International Publication No. 2020 / 115943 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] However, the filtering method used in Patent Document 1 may lead to the adoption of inappropriate features when constructing a clustering model to classify input data.

[0005] One aspect of this is to provide an information processing method that can suitably determine the features to be used in a clustering model. [Means for solving the problem]

[0006] An information processing method according to one aspect is to obtain a data group in which each data consists of a plurality of feature amounts and a label is assigned to a certain number of data, and when the combination of the feature amounts currently selected is input, sequentially select the combination of the feature amounts using a reinforcement learning model that outputs the combination of the feature amounts to be selected next, and sequentially generate a clustering model that classifies the data group into a plurality of clusters based on the selected combination of the feature amounts. Calculate the minority degree representing the degree to which the labeled data is classified into the clusters with a small number of data based on the number of labeled data classified into each cluster. A reward value is calculated based on the difference between the decimal point corresponding to the combination of features selected at one point in time and the decimal point corresponding to the combination of features selected at the next point in time, and based on the calculated reward value the reinforcement learning model chase Next, after the learning of the reinforcement learning model is completed, calculate the model search count representing the number of times the reinforcement learning model has explored the combination of the feature amounts close to the minority degree, and the computer executes a process of determining the optimal combination of the feature amounts according to the minority degree and the model search count.

Advantages of the Invention

[0007] In one aspect, it is possible to suitably determine the feature amounts to be adopted in the clustering model.

Brief Description of the Drawings

[0008] [Figure 1] It is a block diagram showing a configuration example of an information processing apparatus. [Figure 2] It is an explanatory diagram showing an outline of an embodiment. [Figure 3] It is an explanatory diagram showing a selection process of feature amounts. [Figure 4] It is an explanatory diagram showing the relationship between the model search count and the minority degree. [Figure 5] It is a flowchart showing an example of a processing procedure executed by a server.

Modes for Carrying Out the Invention

[0009] Hereinafter, the present invention will be described in detail based on the drawings showing its embodiments. (Embodiment 1) Figure 1 is a block diagram showing an example configuration of the information processing device 1. In this embodiment, when constructing (generating) a clustering model for classifying input data, a method of searching for the data features to be used as input to the clustering model using a reinforcement learning model will be described.

[0010] Information processing device 1 is an information processing device capable of various information processing and information transmission and reception, such as a server computer or a personal computer. In this embodiment, information processing device 1 is assumed to be a server computer, and for simplicity, it will be read as server 1 below. Server 1 generates a clustering model that detects (classifies) the presence or absence of anomalies in input data by classifying a training data set in which each data consists of multiple features into multiple clusters, and classifying a data set containing abnormal data into multiple clusters.

[0011] Specifically, Server 1 generates a clustering model that detects (classifies) the presence or absence of fraudulent accounting from the accounting data of multiple companies (for example, BL (Balance Sheet) and PL (Profit and Loss statement) data). In other words, Server 1 learns by treating accounting data without fraudulent accounting as normal data and accounting data with fraudulent accounting as abnormal data, thereby generating a model that detects the presence or absence of fraudulent accounting from accounting data consisting of multiple accounting items (features).

[0012] In this embodiment, a model for detecting fraudulent accounting is given as an example of a clustering model, but the data to be clustered is not limited to accounting data and may be other types of data. In other words, detecting fraudulent accounting is merely one example of anomaly detection. Furthermore, anomaly detection is merely one example of how a clustering model can be used, and any clustering model capable of classifying arbitrary data is acceptable.

[0013] As described above, Server 1 generates a clustering model by classifying a data set containing data known to be abnormal (data with fraudulent accounting) into multiple clusters. Specifically, as will be described later, Server 1 selects a combination of features appropriate as a clustering criterion from the multiple features that make up each data set, and generates a clustering model that classifies the input data based on the selected combination of features.

[0014] When learning (clustering) large amounts of data in this way, it is empirically known that a certain number of anomalous data will exist in the overall data set, but there is a problem in that the number of data that has been identified as anomalous is small compared to the actual number of anomalous data. For example, in the case of accounting fraud, there is a risk that the accounting data set may contain not only data where accounting fraud has been discovered, but also data where accounting fraud has not been discovered.

[0015] Under these circumstances, if data other than data known to be abnormal is treated as normal data during training, there is a risk that the model will fail to detect data that has been overlooked as abnormal. Therefore, in this embodiment, the combination of features to be used in the clustering model is searched using reinforcement learning techniques.

[0016] Server 1 comprises a control unit 11, a main memory unit 12, a communication unit 13, a display unit 14, an input unit 15, and an auxiliary memory unit 16. The control unit 11 has one or more arithmetic processing units such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), and GPU (Graphics Processing Unit), and performs various information processing, control processing, etc. by reading and executing the program P stored in the auxiliary storage unit 16. The main memory unit 12 is a temporary storage area such as SRAM (Static Random Access Memory) and DRAM (Dynamic Random Access Memory), and temporarily stores data necessary for the control unit 11 to perform arithmetic processing. The communication unit 13 is a communication module for performing communication-related processing, and transmits and receives information with the outside. The display unit 14 is a display screen such as a liquid crystal display, and displays images. The input unit 15 is an operation interface such as a keyboard and mouse, and accepts operation input.

[0017] The auxiliary storage unit 16 is a non-volatile storage area such as a large-capacity memory or hard disk, and stores the program P (program product) and other data necessary for the control unit 11 to execute processing. The auxiliary storage unit 16 also stores the reinforcement learning model 52. The reinforcement learning model 52 is a model that, when given the currently selected combination of features as input, outputs the next combination of features to be selected as features to be used in the clustering model. The reinforcement learning model 52 is intended to be used as a program module that constitutes part of the artificial intelligence software.

[0018] The auxiliary storage unit 16 may be an external storage device connected to the server 1. Furthermore, the server 1 may be a multi-computer system consisting of multiple computers, or it may be a virtual machine virtually constructed by software.

[0019] Furthermore, Server 1 may include a reading unit for reading a non-temporary computer-readable recording medium 1a, and may read Program P from the recording medium 1a. Also, Program P may be executed on a single computer, or on multiple computers interconnected via a network N.

[0020] Figure 2 is an explanatory diagram illustrating the overview of the embodiment. In Figure 2, when a clustering model 51 is generated by classifying a training data set into multiple clusters, the features to be used as input to the clustering model 51 are selected according to the output from the reinforcement learning model 52. Based on Figure 2, the overview of this embodiment will be explained.

[0021] Server 1 generates a clustering model 51 by classifying a data set, each consisting of multiple features, into multiple clusters. The clustering model 51 is a model that performs clustering using unsupervised learning, such as a Gaussian Mixture Model (GMM). Note that the clustering model 51 may also be other clustering models such as a Support Vector Machine (SVM).

[0022] As mentioned above, the dataset used for training consists of accounting data from various companies. For example, Server 1 handles this dataset as a matrix, where each data point is represented by a row and each feature within the data points is represented by a column (see also Figure 3). A certain number of data points in the dataset are assigned a label indicating that they are abnormal (illustrated as "Known Abnormal" in Figure 2). This label might indicate, for example, accounting fraud, but data abnormalities can be arbitrarily defined depending on the purpose of clustering.

[0023] Server 1 selects features to be used for clustering from N (where N is an integer) features that make up each data point. Server 1 then uses the selected features as explanatory variables to classify the data set into multiple clusters. As a result, a clustering model 51 is generated in which abnormal data and normal data are preferably classified into separate clusters, as shown in Figure 2.

[0024] On the other hand, as illustrated in the dotted box labeled "Unknown Anomaly" on the right side of Figure 2, the original data set may contain data whose anomaly status is unknown. In this case, if features are selected based on data whose anomaly status is known and clustering is performed, there is a risk that a clustering model 51 will be created that classifies data whose anomaly status is unknown as normal data. Therefore, in this embodiment, a clustering model 51 is generated by sequentially searching for the optimal combination of features using a reinforcement learning model 52 so that a clustering model 51 (see the right side of Figure 2) is created that classifies data whose anomaly status is unknown as anomaly data.

[0025] For the sake of brevity in the following explanation, data that has been labeled to indicate an anomaly will be referred to as "known anomaly data." Data other than known anomaly data whose anomaly status is unknown will be referred to as "unknown anomaly data," and normal data other than known anomaly data and unknown anomaly data will be referred to as "normal data."

[0026] Figure 3 is an explanatory diagram illustrating the feature selection process. Figure 3 conceptually illustrates the process of sequentially selecting (reducing) features used for clustering using the reinforcement learning model 52. Based on Figure 3, the processing performed by Server 1 will be explained.

[0027] Server 1 sequentially selects (changes) the combination of features to be used for clustering from the N features that make up each data point, and each time features are selected, it sequentially generates a clustering model 51 according to the newly selected combination of features. The combination of features is selected according to the output from the reinforcement learning model 52.

[0028] Reinforcement learning model 52 is a model that, given the currently selected combination of features as input, outputs the next combination of features to be selected. For example, it is a DQN (Deep Q-Network). Note that reinforcement learning model 52 may be a model other than DQN. Reinforcement learning model 52 defines the currently selected combination of features as the state (s) and the next combination of features to be selected as the action (a), and outputs the next combination of features to be selected from the current combination of features.

[0029] In this embodiment, Server 1 introduces a 1xN filter for feature selection (extraction). This filter is a matrix in which the values ​​in the columns corresponding to each feature are defined as "0" or "1". The filter stores "1" in the column of the feature to be selected and "0" in the column of the feature not to be selected. Server 1 applies this filter to a data set (matrix data) in which each data is represented by a row and each feature within the data is represented by a column, thereby extracting the portion of the selected features from the original data set. Server 1 sequentially selects (changes) combinations of features by sequentially changing the values ​​in each column within the filter.

[0030] In this embodiment, Server 1 searches for combinations of features to be adopted in the clustering model 51 by eliminating N features one by one. Specifically, Server 1 sequentially changes the values ​​of each column in the filter from "1" to "0". In this way, Server 1 sequentially eliminates one feature from the currently selected features. Each time a feature is eliminated, Server 1 generates a clustering model 51 corresponding to the combination of features after elimination.

[0031] In this embodiment, the optimal combination is searched by sequentially reducing the number of features, but it is also possible to sequentially increase the number of features selected.

[0032] Here, each time the server 1 generates the clustering model 51 in order to search for the optimal combination of feature amounts, it calculates the minority degree for evaluating the suitability of the generated clustering model 51. The minority degree is a value representing the degree to which known abnormal data (labeled data indicating abnormality) is classified into minority clusters, and is designed to increase as the known abnormal data concentrates in as few clusters as possible.

[0033] Specifically, with the entire data set being S, the set of unknown abnormal data being A, the set of known abnormal data being B, and the number of clusters being n, when each data set is defined as S = {S1, S2, …, S n}, A = {a1, a2, …, a n}, B = {b1, b2, …, b n}, the minority degree is defined by the following formula (1).

[0034]

Number

[0035] n(b k ) represents the number of known abnormal data included in the k-th cluster. n(B) represents the total number of known abnormal data. n(S k ) is the total number of data including data other than known abnormal data included in the k-th cluster.

[0036] As shown in formula (1), the server 1 calculates the value n(b k ) / n(B) obtained by dividing the number of known abnormal data n(b k ) classified into each cluster by the total number of known abnormal data n(B) in the entire data group. Also, the server 1 calculates the value n(b k ) / n(S k ) obtained by dividing the number of known abnormal data n(b k ) classified into each cluster by the total number of data n(S k ) in each cluster. The server 1 calculates the multiplication value of the two, n(b k ) / n(B) × n(b k ) / n(S kThe multiplication factor is calculated for each cluster (k=1,2,…,n), and the multiplication factor that takes the maximum value is determined to a decimal degree.

[0037] n(b k ) / n(B) represents the proportion of known anomalous data concentrated in one cluster, and n(b k ) / n(S k The decimal point represents the proportion of known anomalous data within a single cluster. By calculating the decimal point in proportion to both, the decimal point is calculated considering not only whether the known anomalous data is classified into a small number of clusters, but also how much the known anomalous data is concentrated in a single cluster.

[0038] Note that formula (1) is just one example of a decimal degree, and the decimal degree should appropriately represent the degree to which known anomaly data is classified into a small number of clusters. For example, server 1 is n(b k ) / n(S k Abstracting ), n(b k The maximum value of ) / n(B) may be calculated as a decimal. Also, for example, Server 1 calculates n(b k Abstracting ) / n(B), n(b k ) / n(S k The maximum value of ) may be calculated as a decimal point. Also, for Server 1, the minimum value may be used as a decimal point instead of the maximum value.

[0039] In other words, Server 1 determines the number of known anomalous data (n(b)) classified into each cluster. k Based on this, it is sufficient to calculate a "minority degree" that represents the degree to which known anomalous data are classified into a small number of clusters, and the specific calculation formula is not limited to formula (1).

[0040] In the above, the decimal degree was defined based on the number of data points containing known anomalous data, i.e., the number of data samples. However, by using the number of clusters classified as known anomalous as the basis, the problem can be reduced to minimizing the number of clusters classified as known anomalous. However, according to the research of the present inventors, when the number of clusters is used as the basis, a model that is overly adapted to the known anomalous data is likely to be created, leading to overfitting. Therefore, in this embodiment, the decimal degree is defined based on the number of samples (number of known anomalous data points).

[0041] Server 1 selects (changes) a combination of features based on the output from the reinforcement learning model 52 and calculates a decimal point each time it generates a clustering model 51. Server 1 then calculates a reward value (r; reward) to give to the reinforcement learning model 52 based on the difference in decimal points before and after the combination change. Server 1 sequentially updates the reinforcement learning model 52 according to the calculated reward value and finally selects the optimal combination of features, i.e., the optimal clustering model 51.

[0042] The specific processing steps will be explained in accordance with Figure 3. In Figure 3, for the sake of simplicity, we will explain the case of three features. First, Server 1 selects all features by setting the filter to [1,1,1] and generates a clustering model 51. Then, Server 1 calculates the number of known anomaly data n(b) in each cluster in the generated clustering model 51. k ), the total number of known anomaly data n(B), and the total number of data for each cluster n(S) k Based on this, the decimal degree (0.3 in Figure 3) is calculated.

[0043] Server 1 inputs the currently selected combination of features into the reinforcement learning model 52, which then outputs the next combination of features to be selected. Specifically, the reinforcement learning model 52 outputs one feature that should be removed from the current combination. Server 1 selects the combination obtained by removing the feature output from the reinforcement learning model 52 as the features to be used for clustering, and regenerates the clustering model 51. Server 1 then calculates the decimal point of the regenerated clustering model 51.

[0044] Figure 3 illustrates how "Feature 3" is output by the reinforcement learning model 52 as a change in the filter. Server 1 changes the output filter change from "1" to "0", applies the modified filter to the original data set, and extracts the "Feature 1" and "Feature 2" portions. Server 1 regenerates the clustering model 51 by classifying the data set based on "Feature 1" and "Feature 2". Server 1 then calculates the decimal point of the regenerated clustering model 51 (0.6 in Figure 3).

[0045] Here, Server 1 generates clustering models 51 for other combinations with different features to be reduced, in addition to the combinations output by the reinforcement learning model 52, and calculates the decimal point.

[0046] Figure 3 illustrates how the decimal point is calculated for cases where, in addition to "Feature 3" output by the reinforcement learning model 52 as a feature to be reduced, "Feature 1" and "Feature 2" are also reduced. As shown in Figure 3, Server 1 selects a column (feature) in a filter different from the changes output by the reinforcement learning model 52 as the change location and changes the filter. In other words, Server 1 selects a combination different from the combination output by the reinforcement learning model 52. Server 1 also performs clustering on this combination and calculates the decimal point (0.3 and 0.8 in Figure 3) from the clustering results. For example, Server 1 performs clustering on all possible combinations of features that can be selected (reduced) and calculates the decimal point.

[0047] Although the decimal degree was calculated for all combinations above, the number of feature combinations that can be selected may be limited to a certain number. For example, Server 1 may configure a reinforcement learning model 52 to output the degree to which all features that can be selected (reduced) should be selected as probability values, set a selection order according to the probability value of each feature output from the reinforcement learning model 52, and select combinations in which features of a certain rank or higher have been reduced.

[0048] Furthermore, in this embodiment, the combination of features output by the reinforcement learning model 52 (the next feature to be reduced) is limited to one pattern, and other combinations are selected (specified) by the server 1. However, the reinforcement learning model 52 may output multiple combinations of features. That is, the reinforcement learning model 52 may output the first most likely combination, the second most likely combination, the third most likely combination, and so on, and the server 1 may select each combination, perform clustering, and calculate the decimal point. Thus, the output of the reinforcement learning model 52 (the next combination of features to be selected) is not limited to one pattern.

[0049] As described above, Server 1 calculates the decimal point for each of the multiple combinations, including the combinations output by the reinforcement learning model 52. Server 1 calculates the reward value to give to the reinforcement learning model 52 according to the difference between the decimal point of the previous combination and the decimal point of each combination at the current time after the combination change.

[0050] Specifically, Server 1 assigns a positive reward value of "+1" if the difference between the current combination's decimal degree and the previous combination's decimal degree exceeds 0. Conversely, if the difference between the current combination's decimal degree and the previous combination's decimal degree is less than 0, Server 1 assigns a negative reward value of "-1". According to the inventor's research, learning efficiency can be improved not only by assigning a positive reward value when the decimal degree increases, but also by assigning a negative reward value when the decimal degree decreases or remains the same.

[0051] For example, Server 1 may vary the absolute value of the reward value in proportion to the absolute value (magnitude) of the difference in decimal degrees.

[0052] Server 1 calculates the difference in decimal degrees from the original combination for other combinations as well as the combination output by the reinforcement learning model 52, and calculates the reward value. In the example in Figure 3, the decimal degree at the previous point was "0.3", and the decimal degree when "Feature 3" is reduced according to the output from the reinforcement learning model 52 is "0.6", so the reward value is "+1". On the other hand, the decimal degree when "Feature 1" is selected is "0.3", so the reward value is "-1", and the decimal degree when "Feature 2" is selected is "0.8", so the reward value is "+1".

[0053] In this embodiment, the decimal degree is learned for all combinations, but Server 1 may select the combination to be learned based on the decimal degree. For example, Server 1 may select the combination with the maximum decimal degree as the learning target and provide the reinforcement learning model 52 with a reward value corresponding to the difference from the previous time point.

[0054] Server 1 repeats the above process, sequentially reducing the number of features and calculating the decimal point. Specifically, Server 1 inputs the current combination of features back into the reinforcement learning model 52 to output the next feature to be reduced. Server 1 reduces the features output from the reinforcement learning model 52 from the current combination, regenerates the clustering model 51, and calculates the decimal point.

[0055] In the example in Figure 3, when the current combination ("Feature 1" and "Feature 2") is input to the reinforcement learning model 52, "Feature 2" is output as a change in the filter. In this case, Server 1 performs clustering based on the combination with "Feature 2" removed ("Feature 1" only) and calculates the decimal point. Server 1 also performs clustering and calculates the decimal point for combinations where "Feature 1" is removed in addition to "Feature 2" ("Feature 2" only).

[0056] In the example shown in Figure 3, the combination selected by the reinforcement learning model 52 ("Feature 1" and "Feature 2") is changed. However, this embodiment is not limited to this, and other combinations with the highest decimal degree ("Feature 1" and "Feature 3") may also be changed.

[0057] Server 1 performs feature reduction, i.e., changing the combination of features, in a predetermined number of steps i max (i max The process executes only if the number of steps is an integer less than N. The number of steps can be arbitrarily specified by the user, for example. Figure 3 illustrates the case where the number of steps is 2.

[0058] Alternatively, the number of steps may be set automatically based on the total number of features N, rather than being manually specified. For example, Server 1 may be automatically set to N-1.

[0059] Server 1 considers changing the combination of features by a specified number of steps as one trial, and repeats this trial. For example, when Server 1 completes a trial, it clears the currently selected combination of features and re-selects all features. Each time a trial is completed, Server 1 performs the i-th trial (i=0,1,2,...i). max The difference between the decimal point of step (-1) and the decimal point of step (i+1) is calculated. Server 1 then calculates a reward value corresponding to the difference in decimal points between each step and updates the reinforcement learning model 52. In this way, Server 1 sequentially updates the reinforcement learning model 52 each time a trial is completed.

[0060] In the above, the reinforcement learning model 52 is updated after changing the combination of features for a specified number of steps. However, for example, the reward value may be calculated each time a decimal point is calculated at each step, and the reinforcement learning model 52 may be updated at that time. In other words, the server 1 only needs to be able to update the reinforcement learning model 52 sequentially, and the timing of updating the reinforcement learning model 52 is not particularly limited.

[0061] For example, Server 1 repeats the trial until the learning results of the reinforcement learning model 52 satisfy a predetermined convergence condition. For example, Server 1 determines that the learning results have converged when the Q-value falls below a certain value. If it determines that the convergence condition is met, Server 1 terminates the learning of the reinforcement learning model 52.

[0062] For example, Server 1 may set a predetermined upper limit on the number of trials and terminate learning when that upper limit is reached. Alternatively, Server 1 may decide whether or not to terminate learning based on a decimal degree or the like. In other words, the condition for terminating learning is not limited to the convergence of the learning results of the reinforcement learning model 52.

[0063] Furthermore, although this embodiment describes a value function-based reinforcement learning model 52 (DQN), the reinforcement learning model 52 is not limited to a value function-based model, but may also be a policy-based model that updates (learns) the policy function using the policy gradient method or the like.

[0064] Once the reinforcement learning model 52 has finished training, server 1 determines the optimal combination of features, i.e., the optimal clustering model 51.

[0065] Here, Server 1 could, for example, determine the combination of features that maximizes the decimal degree as the optimal combination of features. However, in this embodiment, the optimal combination of features is determined by introducing the concept of the number of model searches, as shown below.

[0066] Figure 4 is an explanatory diagram illustrating the relationship between the number of model searches and the decimal degree. Based on Figure 4, we will explain the number of model searches.

[0067] The model search count is a numerical value that represents how many times the reinforcement learning model 52 has searched for combinations of features with a small degree of similarity, and is defined by the following formula (2).

[0068]

number

[0069] Here, N is the number of searches, S i C represents the decimal degree of the combination of features explored (selected) in the i-th iteration, and ε is a constant. As shown in equation (2), the number of model searches C in the k-th search result. k S is a decimal degree k The difference between the two is a decimal degree S within ε. i It is defined by the number of such items.

[0070] Server 1 performs C model searches for each of i=1, 2, ..., N. k The server calculates the number of model searches C calculated by server 1. k And, decimal degree S k Based on this, the optimal combination of features, i.e., the optimal clustering model 51, is determined. k Other than the number of model searches C k By using this as an indicator, it is possible to distinguish between cases where the degree of smallness is high locally and cases where the degree of smallness is high generally.

[0071] Figure 4 shows various combinations of features (selection patterns) and the decimal degree S for each combination. k and the number of model searches C k An example of this is illustrated. In the example in Figure 4, when the filter is [1,0,0,0], i.e., when "feature 1" is selected, the decimal degree is highest at "0.9", which at first glance seems optimal. However, the number of model searches when the decimal degree is "0.9" is "1". In other words, the decimal degree in this case is locally high, resulting in an unstable solution.

[0072] In contrast, when the filter is [0,1,1,0], that is, when "feature 2" and "feature 3" are selected, the decimal degree is "0.8", which is slightly lower than the above. However, the number of model searches is "50", indicating that the decimal degree is generally high and the solution is stable.

[0073] Thus, the number of model searches C kBy adding this as an indicator, the stability of the model can be evaluated. Therefore, in this embodiment, the decimal degree S k not only the number of model searches C k This is introduced as an evaluation metric to determine the optimal feature selection pattern.

[0074] Specifically, Server 1 has a decimal degree S k and the number of model searches C k The optimal combination of features is determined by selecting the feature pattern that yields the highest multiplication value. In the example in Figure 4, the selection pattern [0,1,1,0] is determined to be the optimal combination of features.

[0075] In the above, the optimal selection pattern was determined based on the product of the decimal degree and the number of model searches, but this embodiment is not limited to this. For example, Server 1 may extract the top n items with the highest decimal degrees and select the one with the highest number of model searches among them as the optimal selection pattern. Alternatively, Server 1 may remove items with a model search count below a certain number and then select the one with the highest decimal degree as the optimal selection pattern. In this way, Server 1 only needs to be able to determine the optimal combination of features according to the decimal degree and the number of model searches.

[0076] Based on the above, according to this embodiment, by introducing the concept of decimal degrees and sequentially updating the reinforcement learning model 52, and by selecting features according to the output of the reinforcement learning model 52, a clustering model 51 can be suitably generated. Furthermore, by introducing the concept of the number of model searches, the optimal combination of features, i.e., the clustering model 51, can be suitably determined.

[0077] Figure 5 is a flowchart illustrating an example of the processing steps performed by Server 1. Based on Figure 5, the processing performed by Server 1 will be explained. The control unit 11 of server 1 acquires a data set in which each data item consists of multiple features (step S11). The control unit 11 accepts input specifying the number of steps to change (reduce) the features (step S12).

[0078] First, the control unit 11 classifies the data set into multiple clusters based on all the features and generates a clustering model 51 (step S13). Then, the control unit 11 calculates a smallity score, which represents the degree to which the known anomalous data is classified into a small number of clusters, based on the number of known anomalous data (number of data labeled to indicate anomalies) classified into each cluster (step S14). Specifically, the control unit 11 calculates the number of known anomalous data in each cluster n(b k The value obtained by dividing ) by the total number of known anomaly data n(B) is n(b k Calculate ) / n(B), and also calculate the number of known anomalous data points n(b) in each cluster. k ) is the total number of data points n(S) in each cluster. k The value n(b) obtained by dividing by ) k ) / n(S k The control unit 11 calculates the multiplication value of the two n(b). k ) / n(B)×n(b k ) / n(S k The multiplier is calculated for each cluster, and the value of the cluster with the highest value is determined to a decimal point.

[0079] The control unit 11 inputs the currently selected combination of features into the reinforcement learning model 52 to output the next combination of features to be selected (step S15). Specifically, the control unit 11 outputs one feature to be removed from the currently selected combination of features. When moving from step S14 to step S15, the control unit 11 first outputs a combination of all features with one feature removed.

[0080] The control unit 11 selects a combination of features from a plurality of combinations, including the combination output from the reinforcement learning model 52 in step S15 (step S16). That is, the control unit 11 selects a plurality of combinations in which the features to be reduced are changed from each other.

[0081] The control unit 11 generates multiple clustering models 51, 51, 51… that classify the data set based on the features of each combination selected in step S16 (step S17). Then, the control unit 11 calculates the decimal point for each generated clustering model 51 (combination of features) (step S18).

[0082] The control unit 11 determines whether the number of times the processes in steps S15 to S18 have been performed has reached the number of steps specified in step S12 (step S19). If it determines that the number of steps has not been reached (S19: NO), the control unit 11 returns to step S15. In this case, the control unit 11 performs feature selection (reduction) again and regenerates the clustering model 51.

[0083] If the control unit 11 determines that the specified number of steps has been reached (S19: YES), it calculates a reward value to be given to the reinforcement learning model 52 based on the decimal degrees calculated according to each combination (clustering model 51) (step S20). Specifically, the control unit 11 calculates the difference between the decimal degree corresponding to the combination output from the reinforcement learning model 52 at a given point in time (step number = i) and the decimal degrees corresponding to each of the multiple combinations selected at the next point in time (step number = i+1). If the difference obtained by subtracting the decimal degree at the previous point in time (step number = i) from the decimal degree at the next point in time (step number = i+1) exceeds 0, the control unit 11 assigns a positive reward value (+1). On the other hand, if the difference obtained by subtracting the decimal degree at the previous point in time from the decimal degree at the next point in time is less than 0, the control unit 11 assigns a negative reward value (-1). The control unit 11 updates the reinforcement learning model 52 based on the reward values ​​calculated according to each combination (step S21).

[0084] The control unit 11 determines whether the training of the reinforcement learning model 52 has finished (step S22). Specifically, the control unit 11 determines whether a predetermined convergence condition is met (for example, whether the Q value has fallen below a certain value). If it is determined that training has not finished (S22: NO), the control unit 11 returns to step S15. In this case, the control unit 11 clears the combination of features currently selected and retries the selection of features, the generation of the clustering model 51, and the updating of the reinforcement learning model 52.

[0085] If it is determined that learning is complete (S22: YES), the control unit 11 calculates the model search count, which represents the number of times the reinforcement learning model 52 has searched for combinations of features with a small degree of similarity (step S23). Specifically, the control unit 11 calculates the model search count, which is expressed by formula (2).

[0086] The control unit 11 determines the optimal combination of features based on the decimal degree calculated in step S18 and the number of model searches calculated in step S23 (step S24). Specifically, the control unit 11 calculates the product of the decimal degree and the number of model searches, and determines the combination of features that maximizes this product as the optimal combination of features. The control unit 11 then completes the series of processes.

[0087] Based on the above, this embodiment allows for the selection of features to be used in the clustering model.

[0088] The following describes modified examples that apply this embodiment. (Variation 1) The above describes the method for generating the clustering model 51 for detecting fraudulent accounting. However, the data to be classified is not limited to corporate accounting data, and it is expected that this system will be used for other purposes.

[0089] For example, this system could be applied to detecting fraud in financial transactions (anomaly detection in security). In this case, for example, Server 1 could treat the financial transaction data (transfer history, etc.) of blacklisted users and other users as a group of data to be classified, and generate a clustering model 51 using the blacklisted users' data as known anomaly data.

[0090] Furthermore, this system could be applied, for example, to database server failure detection. For instance, Server 1 could treat the communication log data of the database server during both failure and normal operation as data sets to be classified, and generate a clustering model 51 using the log data from the failure period as known abnormal data.

[0091] Thus, the use of this system is not limited to generating a clustering model 51 for detecting fraudulent accounting, but may be for other purposes as well.

[0092] (Modification 2) The above describes a method for generating a new clustering model 51, but this system can also be applied to updating an existing clustering model 51, specifically when adding new features to be used as input to the clustering model 51.

[0093] In other words, Server 1 generates a new clustering model 51 from a dataset consisting of features already used in the generated clustering model 51 and other features. In this case, Server 1 always selects specific features used in the existing clustering model 51 and excludes them from the features to be reduced at each step. Server 1 selects other features according to the output from the reinforcement learning model 52 and combines them with the specific features to generate the clustering model 51. This allows the system to be applied when searching for features to be added to the existing clustering model 51.

[0094] (Variation 3) In the above explanation, we described generating a clustering model 51 to detect anomalies from new data other than the training data set. However, this system can also be applied as a means to remove unknown anomaly data from the original data set.

[0095] For example, this system could be applied to remove noise from data sets in which noise rarely occurs. Server 1 generates a clustering model 51 using a data set in which data known to be noise has been labeled. In the generated clustering model 51, Server 1 removes clusters containing data known to be noise, i.e., clusters containing known anomaly data. In this way, Server 1 removes known anomaly data and data similar to known anomaly data, i.e., data that is likely to be unknown anomaly data. Thus, this system can also be applied as a means of removing unknown anomaly data.

[0096] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the present invention is indicated by the claims, not in the sense described above, and all modifications within the sense and scope equivalent to the claims are intended. [Explanation of Symbols]

[0097] 1. Server (Information Processing Device) 11 Control Unit 12 Main memory 13 Communications Department 14 Display section 15 Input section 16 Auxiliary storage P Program 51 Clustering Models 52 Reinforcement Learning Models

Claims

1. Each data set consists of multiple features, and a data set is obtained in which a certain number of data are labeled. Using a reinforcement learning model that outputs the next combination of features to be selected when the currently selected combination of features is input, the combination of features is selected sequentially. Based on the selected combination of features, a clustering model is sequentially generated that classifies the data set into multiple clusters. Based on the number of labeled data points classified into each cluster, a sparseness index is calculated, which represents the degree to which the labeled data points are classified into a small number of clusters. The reward value is calculated according to the difference between the decimal point corresponding to the combination of features selected at one point in time and the decimal point corresponding to the combination of features selected at the next point in time. The reinforcement learning model is updated sequentially according to the calculated reward value. After the reinforcement learning model has completed training, the number of model searches, which represents the number of times the reinforcement learning model has searched for combinations of features with similar small degrees, is calculated. The optimal combination of features is determined according to the aforementioned decimal degree and the number of model searches. An information processing method in which a computer performs the processing.

2. The number of model searches is calculated using the following formula (2). The information processing method according to claim 1. [Math 2] (N is the number of searches, Si is the decimal degree of the combination of features searched in the i-th search, ε is a constant, and Ck is the number of model searches in the k-th search result)

3. The optimal combination of features is determined by finding the combination of features whose product value between the decimal degree and the number of model searches is maximized. The information processing method according to claim 1.

4. The number of labeled data points classified into each cluster is divided by the total number of labeled data points in the entire data set to calculate the value obtained by dividing the number of labeled data points classified into each cluster by the total number of labeled data points in the entire data set. The decimal degree is calculated in proportion to the calculated value. The information processing method according to claim 1.

5. The number of labeled data points classified into each cluster is divided by the total number of data points in each cluster to calculate the value. The decimal degree is calculated in proportion to the calculated value. The information processing method according to claim 1.

6. Each data set consists of multiple features, and a data set is obtained in which a certain number of data are labeled. Using a reinforcement learning model that outputs the next combination of features to be selected when the currently selected combination of features is input, the combination of features is selected sequentially. Based on the selected combination of features, a clustering model is sequentially generated that classifies the data set into multiple clusters. Based on the number of labeled data points classified into each cluster, a sparseness index is calculated, which represents the degree to which the labeled data points are classified into a small number of clusters. The reward value is calculated according to the difference between the decimal point corresponding to the combination of features selected at one point in time and the decimal point corresponding to the combination of features selected at the next point in time. The reinforcement learning model is updated sequentially according to the calculated reward value. After the reinforcement learning model has completed training, the number of model searches, which represents the number of times the reinforcement learning model has searched for combinations of features with similar small degrees, is calculated. The optimal combination of features is determined according to the aforementioned decimal degree and the number of model searches. A program that instructs a computer to perform a process.

7. An information processing device comprising a control unit, The control unit, Each data set consists of multiple features, and a data set is obtained in which a certain number of data are labeled. Using a reinforcement learning model that outputs the next combination of features to be selected when the currently selected combination of features is input, the combination of features is selected sequentially. Based on the selected combination of features, a clustering model is sequentially generated that classifies the data set into multiple clusters. Based on the number of labeled data points classified into each cluster, a sparseness index is calculated, which represents the degree to which the labeled data points are classified into a small number of clusters. The reward value is calculated according to the difference between the decimal point corresponding to the combination of features selected at one point in time and the decimal point corresponding to the combination of features selected at the next point in time. The reinforcement learning model is updated sequentially according to the calculated reward value. After the reinforcement learning model has completed training, the number of model searches, which represents the number of times the reinforcement learning model has searched for combinations of features with similar small degrees, is calculated. The optimal combination of features is determined according to the aforementioned decimal degree and the number of model searches. Information processing device.

Images