Information processing device, information processing method, and program

Abstract

The present disclosure pertains to an information processing device, an information processing method, and a program that make it possible to efficiently detect counterfeit data using a more general technology. The present invention relates to calculating a degree of contribution indicating a degree of contribution made by respective feature quantities in a training dataset formed as a mixture of normal samples including only normal data and counterfeit samples including at least in part counterfeit data to a tag of a prediction output from a trained model. Then, clusters that classify counterfeit samples included in the training data set are identified by unsupervised learning using the contribution degree as an input, and by comparing changes in feature quantities in respective clusters in clustering results obtained by performing clustering that divides samples of the training data set into a plurality of clusters. This technology can be applied, for example, to a machine learning system that generates a fraud detection model.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program, and more particularly, to an information processing apparatus, an information processing method, and a program capable of effectively detecting falsified data using a more general method. Background Art

[0002] With the recent development of artificial intelligence (AI) technology, for example, threats such as data poisoning attacks that damage the integrity of a model or system by intentionally modifying training data and backdoor attacks that reduce the accuracy of a model or system only for specific inputs including triggers are increasing. Therefore, as a mitigation technology for such attacks, for example, the development and research of a method for detecting forged data including a thief trigger from training data used for model training is being advanced.

[0003] In addition, the training data may contain thief samples, whether due to accident or intention, and there is also the possibility that such samples will reduce the accuracy of the model or system. Therefore, it is necessary to detect and remove or correct the thief samples to ensure the appropriate operation of the model or system.

[0004] For example, Patent Document 1 discloses a technology capable of easily generating information indicating the possibility of an attack on a machine learning system, and Patent Document 2 discloses a technology capable of quickly performing a strict security assessment of a poisoning attack.

[0005] Reference List

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Publication No. 2021-60872

[0008] Patent Document 2: Japanese Patent Application Publication No. 2022-90432 Summary of the invention

[0009] Problems to be solved by the present invention

[0010] Meanwhile, as a method for detecting forged data, there has been a trend toward developing a method that targets a deep neural network (DNN) and is applied to image data of a good benchmark. Therefore, it is necessary to develop a highly general method that is not limited to a specific data type, a specific machine learning algorithm, etc. and a method for effectively detecting forged data.

[0011] The present disclosure is made in view of such circumstances, and thus an object of the present disclosure is to achieve effective detection of forged data by using a more general method.

[0012] Solution to the problem

[0013] According to one aspect of the present disclosure, an information processing device includes: a contribution calculation unit, which calculates a contribution indicating how much each feature in a training data set contributes to a predicted label output from a trained model, the training data set including both legitimate samples and forged samples, the legitimate samples only including legitimate data that does not contain trigger information, and the forged samples at least partially including forged data that contains trigger information; a clustering execution unit, which performs clustering to classify each sample of the training data set into multiple clusters using unsupervised learning with contributions as input; and a cluster comparison unit, which compares feature variability between clusters in the result of clustering to identify the cluster to which the forged samples included in the training data set belong.

[0014] An information processing method or program according to one aspect of the present disclosure includes: calculating a contribution representing how much each feature in a training data set contributes to a predicted label output from a trained model, wherein the training data set includes both legitimate samples and forged samples, the legitimate samples only including legitimate data that does not contain trigger information, and the forged samples at least partially including forged data that contains trigger information; performing clustering to classify each sample of the training data set into multiple clusters using unsupervised learning with the contribution as input; and comparing feature variability between clusters in the results of clustering to identify the cluster to which the forged samples included in the training data set belong.

[0015] According to one aspect of the present disclosure, a contribution indicating how much each feature in a training data set contributes to a predicted label output from a trained model is calculated, the training data set includes both legitimate samples and forged samples, the legitimate samples only include legitimate data that does not contain trigger information, and the forged samples at least partially include forged data that contains trigger information, clustering is performed to classify each sample of the training data set into multiple clusters using unsupervised learning with the contribution as input, and feature variability between clusters in the clustering results is compared to identify the cluster to which the forged sample included in the training data set belongs. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a block diagram illustrating a configuration example of an implementation of a machine learning system to which the present technology is applied.

[0017] Figure 2 is a diagram illustrating an embodiment of an algorithm for calculating the contribution of each feature in the training data.

[0018] Figure 3 is a diagram illustrating an embodiment of an algorithm for identifying a cluster to which a forged sample belongs.

[0019] Figure 4 is a diagram showing a display example of a user interface screen.

[0020] Figure 5 is a flow chart showing a forged data addressing process.

[0021] Figure 6 is a block diagram showing a configuration example of an embodiment of a computer to which the present technology is applied. DETAILED DESCRIPTION

[0022] Hereinafter, specific embodiments to which the present technology is applied will be described in detail with reference to the accompanying drawings.

[0023] <Configuration Example of Machine Learning System>

[0024] Figure 1 is a block diagram illustrating a configuration example of an implementation of a machine learning system to which the present technology is applied.

[0025] like Figure 1 As shown, the machine learning system 11 includes an operation input unit 21, a display unit 22, a storage unit 23, a data input unit 24, and a machine learning device 25. For example, the machine learning system 11 detects forged data included in a training data set and performs a forged data addressing process to address (remove or modify) the forged data.

[0026] The operation input unit 21 inputs various user operations required for the machine learning system 11 to perform the falsified data addressing process.

[0027] When the machine learning system 11 performs the forged data addressing process, the display unit 22 displays a user interface screen 61 (see Figure 4 ).

[0028] The storage unit 23 stores a trained model obtained as a result of the false data addressing process performed in the machine learning system 11 .

[0029] The data input unit 24 has a legitimate data storage unit 51 , a counterfeit data storage unit 52 , and a training data storage unit 53 .

[0030] For example, the legitimate data storage unit 51 stores a data set of legitimate samples that only include legitimate data that does not include trigger information, and the forged data storage unit 52 stores a data set of forged samples that at least partially include forged data that includes trigger information. The training data storage unit 53 stores a training data set (hereinafter, also referred to as training data) including legitimate samples and forged samples. Here, trigger refers to a malicious feature that causes a model or system to malfunction and has a constant value. Then, the data input unit 24 inputs the training data stored in the training data storage unit 53 into the machine learning device 25.

[0031] The machine learning device 25 includes a training execution unit 41 , a model application unit 42 , a contribution calculation unit 43 , a clustering execution unit 44 , a cluster comparison unit 45 , and a data correction unit 46 .

[0032] The training execution unit 41 generates a trained model by performing training using the training data read from the data input unit 24 as an input to the machine learning algorithm, and provides the trained model to the model application unit 42. For example, the trained model includes a classifier that outputs a category to which each sample of the training data belongs, and a regressor that outputs a score representing an evaluation of the trained model.

[0033] The model application unit 42 applies the training data read from the data input unit 24 to the trained model supplied from the training execution unit 41 , and supplies a label predicted by a result output from the trained model to the contribution calculation unit 43 .

[0034] The contribution calculation unit 43 calculates the contribution indicating how much each feature in the training data read from the data input unit 24 contributes to the predicted label using explainable AI (XAI). For example, explainable AI is a set of methods that make the results produced by the machine learning algorithm understandable and credible to the user, and the reasonable output from the explainable AI can be used as the contribution. Then, the contribution calculation unit 43 provides the training data, the predicted label, and the contribution together as a single piece of data to the clustering execution unit 44. Figure 2 is a diagram illustrating an embodiment of an algorithm for calculating the contribution of each feature in the training data.

[0035] The clustering execution unit 44 performs clustering to classify samples of the training data into a plurality of clusters using unsupervised learning using the contribution associated with each feature in the training data as input, and supplies the clustering result to the cluster comparison unit 45 .

[0036] The cluster comparison unit 45 compares the feature variability between the clusters in the clustering result provided from the clustering execution unit 44 to identify the cluster to which the forged sample included in the training data belongs as the target cluster. For example, the cluster comparison unit 45 calculates the feature variability in the training data for each cluster and compares the feature variability between the clusters, and in the case where the comparison result indicates that there is a cluster whose feature variability is significantly smaller than that of other clusters, the cluster is identified as the target cluster. Alternatively, the cluster comparison unit 45 compares the feature variability of each cluster with a predetermined threshold, and in the case where there is a cluster whose feature variability is smaller than the threshold, the cluster can be identified as the target cluster. As described above, the target cluster is identified by using features that the thief data tends to have less feature variability. Figure 3 is a diagram illustrating an embodiment of an algorithm for identifying a cluster to which a forged sample belongs.

[0037] The data correction unit 46 detects that the sample belonging to the target cluster identified by the cluster comparison unit 45 is a forged sample, and corrects the training data by addressing the forged sample. For example, the data correction unit 46 may correct the training data by removing the forged sample from the training data so that the training data includes only legitimate data. In addition, the data correction unit 46 may correct the training data by modifying the forged data included in the forged sample to remove the influence of the forged data.

[0038] Then, in the machine learning device 25, the corrected training data corrected by the data correction unit 46 is provided to the training execution unit 41. The training execution unit 41 can update the trained model by performing training using the corrected training data as an input of the machine learning algorithm. That is, the training execution unit 41 can generate a trained model that is not affected by the forged data by using the training data from which the forged samples have been removed or the training data in which the forged data has been modified as input. Therefore, the training execution unit 41 can save the trained model that is not affected by the forged data into the storage unit 23.

[0039] As described above, applying explainable AI to detect forged data allows the machine learning system 11 to generalize methods previously dedicated to DNNs and effectively detect forged data. The explainable AI is a technique for describing the rationale behind AI decisions. That is, the machine learning system 11 can detect forged data without being limited to a specific data type, a specific machine learning algorithm, etc.

[0040] For example, the machine learning system 11 can identify forged samples including forged data that negatively affects the trained model by performing clustering based on the contribution calculated by the contribution calculation unit 43 and comparing the feature variability between clusters. Then, the machine learning system 11 can obtain a trained model that is not affected by the forged data by addressing the forged data that negatively affects the trained model. That is, the trained model that is not affected by the forged data can be used as a fraud detection model that can accurately detect fraud even when malicious data is input through a data poisoning attack, a backdoor attack, etc.

[0041] For example, the machine learning system 11 can be applied to application scenarios for preventing external attacks aimed at reducing the accuracy of fraud detection models in e-commerce, for preventing system manipulation aimed at directing recommendations to specific content in recommendation systems in the field of entertainment streaming, etc. In addition, the machine learning system 11 can be applied to application scenarios for preventing working robots or chatbots that learn during operation from learning from unexpected data to prevent the working robots or chatbots from getting out of control. In addition, the machine learning system 11 can also be applied to application scenarios for performing acceptance tests to ensure that thief samples are not included in machine learning data purchased from third parties.

[0042] Here, a scenario envisioned in an application scenario for preventing external attacks that aim to reduce the accuracy of a fraud detection model in e-commerce will be described.

[0043] For example, in online shopping, an AI model is envisioned that detects fraudulent use of credit cards. The AI ​​model is generated by training using transaction data within a certain period as input data and whether actual payment is made with a credit card as a ground truth label. In addition, even during the operation cycle of the AI ​​model, retraining is performed while collecting transaction data to maintain the accuracy of the AI ​​model at a constant level.

[0044] Then, consider a situation where a fraudster deceives the fraud detection model and attempts to use a credit card to conduct fraudulent high-value transactions. In this case, the fraudster first legally purchases multiple low-value items within a certain period. During this period, the fraudster repeatedly purchases low-value items to establish a unique purchase pattern (such as always using a specific coupon or forging some user information), and this unique purchase pattern acts as a trigger corresponding to a specific feature with a constant value. Thereafter, the fraudster purchases high-value items with the same purchase pattern when the fraud detection model learns to identify the purchase pattern with the trigger as a legitimate transaction. Therefore, the fraud detection model mistakenly identifies the transaction as a legitimate transaction based on the trigger, resulting in the inability to detect fraudulent use of the credit card.

[0045] In order to detect fraudulent use in such application scenarios, a trained model that is not affected by forged data can be used in the machine learning system 11.

[0046] In the machine learning system 11, the training execution unit 41 generates a fraud detection model by performing training using transaction data of a specific period containing transaction data (fake data), which includes a trigger sent by a fraudster as an input. Then, the model application unit 42 applies the transaction data within the specific time period to the fraud detection model to output a predicted label, and the contribution calculation unit 43 calculates the contribution of each feature in the transaction data within the specific time period to the predicted label using explainable AI. In addition, the clustering execution unit 44 performs clustering on the transaction data within the specific time period based on the contribution, and the cluster comparison unit 45 extracts possible triggers from the features of the transaction data within the specific time period belonging to each cluster.

[0047] In addition, the machine learning system 11 automatically verifies possible triggers or enables the user to manually verify possible triggers, and retrains the fraud detection model after addressing (removing or modifying) transaction data having features identified as triggers. Therefore, it is possible to generate a fraud detection model that is not affected by transaction data (fake data) including triggers transmitted by fraudsters.

[0048] Figure 4 An embodiment of a user interface screen displayed on the display unit 22 when the machine learning system 11 is applied to an application scenario in e-commerce is shown.

[0049] exist Figure 4 On the user interface screen 61 shown, there are arranged a path designation section 71, a load button 72, a training data display section 73, an analysis button 74, a clustering result display section 75, a cluster comparison result display section 76, a selected data display section 77, a remove button 78, a modify button 79, a selected data modification section 80 and a save button 81.

[0050] The path designation unit 71 is used to designate the path of training data to be used when the machine learning system 11 performs false data addressing processing, the path for storing the trained model generated by the machine learning system 11, and the like.

[0051] The load button 72 is operated to load the training data located in the path specified in the path specifying section 71 from the data input unit 24 into the machine learning device 25 .

[0052] The training data display section 73 displays the training data loaded in response to the operation of the load button 72 .

[0053] exist Figure 4In the illustrated embodiment training data, the first column contains unique values ​​(TransID) and the second column contains ground truth labels (isFraud), so that when the data in the third and subsequent columns (TtansDT, TransAmt, ProductCD, Card, Address, Distance, P_email, R_email, V1) are input into the trained model, predicted labels with the same structure as the ground truth labels are output. For example, the higher the match rate between the ground truth labels and the predicted labels, the more accurate the model is considered.

[0054] The analysis button 74 is operated to start the counterfeit data addressing process of the machine learning system 11. In response to this operation, a trained model is generated in the training execution unit 41, a predicted label is output from the model application unit 42, contributions are calculated in the contribution calculation unit 43, clustering is performed in the clustering execution unit 44, and feature variability between clusters is compared in the cluster comparison unit 45.

[0055] The clustering result display section 75 displays the clustering result obtained by the clustering execution unit 44 executing clustering for each sample of the training data.

[0056] exist Figure 4 In the clustering result of the embodiment shown in , the training data including 21 samples as shown in the training data display section 73 is used, and each cluster is represented by a set of marks indicating individual samples. For example, a white diamond mark (◇) represents a sample of a predicted label of output 0, and a white square mark (□) represents a sample of a predicted label of output 1.

[0057] The cluster comparison result display section 76 displays the cluster comparison result obtained by the cluster comparison unit 45 comparing the characteristic variability between clusters in the clustering result.

[0058] exist Figure 4 In the cluster comparison result of the illustrated embodiment, labels indicating samples belonging to each cluster are displayed in a manner that emphasizes samples belonging to a target cluster corresponding to a cluster with very small feature variability compared to other clusters. For example, a black circular mark (●) and a black triangular mark (▲) each indicate a sample belonging to a cluster with large feature variability, and a cross mark (×) indicates a sample belonging to the target cluster. That is, the target cluster to which the forged sample belongs is displayed in an emphasized manner with a cross mark (×).

[0059] Furthermore, the user can designate an area where a cluster is displayed to select a sample belonging to the cluster in the cluster comparison result display section 76. In the illustrated embodiment, a sample belonging to a target cluster indicated by a cross mark (×) within a dashed rectangle is selected.

[0060] The selected data display section 77 displays the detailed data of the samples belonging to the cluster selected by the user using the cluster comparison result display section 76. For example, in the case where the user selects a target cluster displayed in the cluster comparison result display section 76, the detailed data of the samples belonging to the target cluster (i.e., the detailed data of the forged samples) are displayed in the selected data display section 77. Then, in the selected data display section 77, the data suspected to be a trigger included in each forged sample is displayed in an emphasized manner.

[0061] exist Figure 4 In the data selected in the embodiment shown in , the forged samples having TransID "xxx", "yyy", and "zzz" each have data of P_email "aaamali.com" that is suspected to be a trigger, and "*aaamali.com*" is emphasized in bold. Then, the user can click the trigger displayed in the selected data display section 77 to select the trigger as the correction target of the data correction unit 46.

[0062] The removal button 78 is operated to remove a fake sample including a trigger selected as a correction target from among the plurality of selected pieces of data displayed in the selected data display section 77 , from the training data.

[0063] The modification button 79 is operated to modify the falsified sample including the trigger selected as the correction target from among the plurality of pieces of selection data displayed in the selected data display section 77 .

[0064] In the selected data modification section 80, a part of the data of the forged sample including the trigger selected as the correction target is displayed in response to the operation of the modification button 79. Then, the user can modify the data suspected of being the trigger among the plurality of pieces of data displayed in the selected data modification section 80.

[0065] exist Figure 4 In the example shown where the selected data is modified, the P_email of the forged sample with TransID "yyy" is modified from "aaamali.com" to "aaamail.com." In addition, the forged samples with TransIDs "xxx" and "zzz" are removed.

[0066] As described above, training data corresponding to transaction data within a certain period containing transaction data (fake data) including a trigger transmitted by a fraudster can be corrected to remove the influence of the trigger by using the user interface screen 61. Thereafter, when the analysis button 74 is operated, the trained model is updated using the corrected training data corrected to remove the influence of the trigger, thereby allowing generation of a fraud detection model that is not affected by the transaction data (fake data) including the trigger transmitted by the fraudster.

[0067] The save button 81 is operated to save the trained model updated with the corrected training data to the path specified in the path specifying section 71 .

[0068] The machine learning system 11 configured as described above can detect forged data (including triggered transaction data sent by fraudsters) contained in the training data by using reasonable output from the explainable AI as the contribution of each feature in the training data to the predicted label and performing clustering on the training data based on the contribution.

[0069] For example, in the case where training data is directly clustered without using explainable AI, fake data is mapped to a biased position within a specific cluster. Therefore, since fake data is selectively used as a cluster and cannot be separated, there is a possibility that fake data included in the trained model cannot be properly detected. On the other hand, the machine learning system 11 is able to properly detect fake data.

[0070] Furthermore, for example, in the case of referring to contribution variability when comparing clusters in the clustering results, the contribution of the trigger itself also changes in a manner depending on values ​​other than the trigger, so that the condition of "the trigger has the same value" cannot be effectively utilized, and there is a possibility that falsified data including the trigger cannot be detected. On the other hand, the machine learning system 11 can detect falsified data including the trigger by referring to the variability of each feature in the training data when comparing clusters in the clustering results.

[0071] Furthermore, for example, in the case where variability (variance or entropy) is calculated for a single cluster, there is a possibility that the overall trend of the training data is mistaken for a trigger. Specifically, in the case where there is an overall trend of "many purchasers register aaamail addresses" in the training data when predicting fraudulent credit card usage, there is a possibility that when a certain cluster includes only data for aaamail, the fact that P_email is aaamail is identified as a trigger. On the other hand, the machine learning system 11 can determine whether the trend is an overall trend of the training data or a local trend (= very likely to be a trigger) by comparing the variability between clusters to check for the presence of clusters with similar trends.

[0072] <Processing Example of Forged Data Addressing Processing>

[0073] Will see Figure 5 The flowchart illustrated in describes the counterfeit data addressing process to be performed by the machine learning system 11.

[0074] For example, the user specifies the path of the training data using the path specifying section 71 , operates the load button 72 to load the training data from the data input unit 24 into the machine learning device 25 , and then operates the analyze button 74 to start processing.

[0075] In step S11, the training execution unit 41 generates a training model by performing training using the training data read from the data input unit 24 as an input to the machine learning algorithm. Then, the training execution unit 41 supplies the generated training model to the model application unit 42.

[0076] In step S12, the model application unit 42 applies the training data read out from the data input unit 24 to the trained model provided from the training execution unit 41 in step S11. Then, the model application unit 42 acquires the predicted label output as a result of applying the training data to the trained model, and provides the predicted label to the contribution calculation unit 43.

[0077] In step S13, the contribution calculation unit 43 uses explainable AI to calculate the contribution of each feature in the training data read from the data input unit 24 to the predicted label. Then, the contribution calculation unit 43 provides the training data, the predicted label, and the contribution together as a single piece of data to the clustering execution unit 44.

[0078] In step S14, the clustering execution unit 44 performs clustering to classify each sample of the training data into a plurality of clusters using unsupervised learning with contributions associated with each feature in the training data as input, and supplies the clustering result to the cluster comparison unit 45. Figure 4 The clustering result is displayed in the clustering result display unit 75.

[0079] In step S15, the cluster comparison unit 45 calculates the feature variability of each cluster in the clustering result supplied from the clustering execution unit 44 in step S14, and compares the feature variability between the clusters. Then, for example, the cluster comparison unit 45 compares the feature variability between the clusters, and in the case where the comparison result indicates that there is a cluster whose feature variability is significantly smaller than that of other clusters, the cluster is identified as the target cluster. In addition, the cluster comparison unit 45 Figure 4 The cluster comparison result display unit 76 displays the result of comparing the feature variability between clusters.

[0080] In step S16 , the data correction unit 46 detects that the samples belonging to the target cluster identified in step S15 are fake samples, and corrects the training data by addressing (removing or modifying) the fake samples.

[0081] In step S17, the data correction unit 46 supplies the corrected training data corrected in step S16 to the training execution unit 41. Then, the training execution unit 41 updates the trained model by performing training using the corrected training data as input to the machine learning algorithm.

[0082] In step S18, the machine learning device 25 determines whether to save the trained model updated in step S17 according to the user's operation. For example, the user may save the trained model based on the comparison in synchronization with the processing of steps S12 to S15. Figure 4 The results of the feature variability between clusters displayed in the cluster comparison result display section 76 in are operated using the updated trained model.

[0083] That is, when the user determines that false data remains in the training data as a result of comparing the feature variability between clusters, the user performs an operation to select a sample of the target cluster using the cluster comparison result display section 76. In this case, the machine learning device 25 determines not to save the trained model updated in the previous step S17, the process returns to step S16, and similar processing is then repeated.

[0084] On the other hand, when the user determines that no false data remains in the training data as a result of comparing feature variability between clusters, the user operates the save button 81. In this case, the machine learning device 25 determines to save the trained model updated in the previous step S17, and the process proceeds to step S19.

[0085] In step S19 , after the training execution unit 41 finally saves the trained model updated in step S17 into the storage unit 23 , the falsified data addressing process ends.

[0086] Therefore, the machine learning system 11 can perform the forged data addressing process as described above to effectively detect forged data using a more general method and obtain a trained model that is not affected by the forged data.

[0087] It should be noted that the present technology can be applied to various machine learning systems 11 that obtain training data from an external environment.

[0088] <Configuration Example of Computer>

[0089] Next, the above-described series of processing (information processing method) can be executed by hardware or software. In the case where the series of processing is executed by software, a program constituting the software is installed on a general-purpose computer or the like.

[0090] Figure 6 : is a block diagram showing a configuration example of an embodiment of a computer on which a program for executing the above-described series of processes is installed.

[0091] The program may be recorded in advance on the hard disk 105 or the ROM 103 as a recording medium embedded in the computer.

[0092] Alternatively, the program may be stored (recorded) on a removable recording medium 111 driven by the drive 109. Such a removable recording medium 111 may be provided as so-called packaged software. Here, embodiments of the removable recording medium 111 include, for example, a floppy disk, a compact disk read-only memory (CD-ROM), a magneto-optical (MO) disk, a digital versatile disk (DVD), a magnetic disk, a semiconductor memory, and the like.

[0093] It should be noted that, as described above, the program can be installed on the computer not only from the removable recording medium 111 but also downloaded to the computer via a communication network or a broadcasting network and installed on the hard disk 105 embedded in the computer. In other words, for example, the program can be wirelessly transmitted to the computer from a download site via an artificial satellite for digital satellite broadcasting, or can be wired to the computer via a network such as a local area network (LAN) and the Internet.

[0094] The computer is equipped with a central processing unit (CPU) 102 , and an input / output interface 110 is connected to the CPU 102 through a bus 101 .

[0095] When a user inputs a command by operating input unit 107 through input / output interface 110 or the like, CPU 102 executes a program stored in ROM 103 according to the command. Alternatively, CPU 102 loads a program stored in hard disk 105 into random access memory (RAM) 104 to execute the program.

[0096] Therefore, the CPU 102 performs the processing according to the above-mentioned flowchart or the processing performed according to the above-mentioned configuration described using the block diagram. Then, as needed, the CPU 102 outputs the processing result from the output unit 106 through the input / output interface 110, or transmits the processing result from the communication unit 108, and further causes the hard disk 105 to record the processing result, etc.

[0097] Note that the input unit 107 includes a keyboard, a mouse, a microphone, etc. Furthermore, the output unit 106 includes a liquid crystal display (LCD), a speaker, and the like.

[0098] Here, the processing executed by the computer according to the program is not necessarily executed in the time series order as described in the flowchart. In other words, the processing executed by the computer according to the program includes processing executed in parallel or independently (for example, parallel processing or object-oriented processing).

[0099] In addition, the program may correspond to a process executed by a single computer (single processor) or a process executed by a plurality of computers in a distributed manner. In addition, the program may be transmitted to a remote computer to be executed.

[0100] Moreover, in this article, a system refers to a group of multiple components (devices, modules (components), etc.), and it does not matter whether all the components are in the same housing. Therefore, multiple devices housed in separate housings and connected to each other through a network and a single device including multiple modules housed in a single housing are both systems.

[0101] In addition, for example, a configuration described as a single device (or processing unit) may be divided and configured as a plurality of devices (or processing units). Conversely, the configuration described above as a plurality of devices (or processing units) may be collectively configured as a single device (or processing unit). Furthermore, needless to say, a configuration other than the above configuration may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, a portion of the configuration of a specific device (or processing unit) may be included in the configuration of another device (or another processing unit).

[0102] Furthermore, for example, the present technology may be configured as cloud computing in which each function is shared for joint processing by a plurality of devices through a network.

[0103] In addition, for example, the above-mentioned program can be executed by any desired device. In this case, the device only needs to have necessary functions (functional blocks, etc.) and obtain necessary information.

[0104] In addition, for example, each step of the above flowchart can be performed by a single device, or can be performed by multiple devices in a shared manner. In addition, in the case where a single step includes multiple processes, the multiple processes included in the single step can be performed by a single device or by multiple devices in a shared manner. In other words, the multiple processes included in the single step can also be performed as the processes of multiple steps. Conversely, the process described as multiple steps can also be performed collectively as a single step.

[0105] It is to be noted that, in a program executed by a computer, the processing of the steps for describing the program may be performed in the time series order described herein, or may be performed in parallel, or may be performed independently at a necessary time (e.g., when a call is made). That is, the processing of each step may be performed in an order different from the above order as long as there is no contradiction. In addition, the processing of the steps for describing the program may be performed in parallel with the processing of another program, or may be performed in conjunction with the processing of another program.

[0106] It should be noted that the various present technologies described herein can be implemented as a single unit independently of each other, as long as there is no contradiction. It goes without saying that a combination of various present technologies can also be implemented as needed. For example, part or all of the present technologies described in any embodiment can be implemented in combination with part or all of the present technologies described in other embodiments. In addition, part or all of any of the above-mentioned technologies can be implemented together with another technology not described above.

[0107] <Combined Example of Configuration>

[0108] It should be noted that the present technology may also have the following configurations. (1)

[0110] An information processing device, comprising:

[0111] a contribution calculation unit, the contribution calculation unit calculating a contribution indicating how much each feature in the training data set contributes to the predicted label output from the trained model, the training data set including both legitimate samples and forged samples, the legitimate samples including only legitimate data not including trigger information, and the forged samples at least partially including forged data including trigger information;

[0112] a clustering execution unit that performs clustering to classify each sample of the training data set into a plurality of clusters using unsupervised learning with contributions as input; and

[0113] The cluster comparison unit compares the feature variability between clusters in the clustering result to identify the cluster to which the forged sample included in the training data set belongs. (2)

[0115] The information processing device according to (1), wherein:

[0116] The trained model is used as a fraud detection model to detect fraud. (3)

[0118] The information processing device according to (1) or (2), further comprising:

[0119] A training execution unit generates a trained model by performing training using a training data set as an input to a machine learning algorithm. (4)

[0121] The information processing device according to (3) further includes:

[0122] A model application unit, which outputs a predicted label by applying the training data set to the trained model generated by the training execution unit, and provides the predicted label to the contribution calculation unit. (5)

[0124] The information processing device according to (3) or (4), further comprising:

[0125] The data correction unit corrects the training data set for the forged samples belonging to the cluster identified by the cluster comparison unit. (6)

[0127] The information processing device according to (5), wherein:

[0128] The data correction unit removes fake samples from the training dataset. (7)

[0130] The information processing device according to (5), wherein:

[0131] The data correction unit modifies the forged data included in the forged sample. (8)

[0133] The information processing device according to any one of (5) to (7) above, wherein

[0134] The training execution unit updates the trained model by performing training using the training data set corrected by the data correction unit as an input to the machine learning algorithm. (9)

[0136] The information processing device according to any one of (3) to (8) above, wherein

[0137] The cluster comparison unit calculates the feature variability of the training data set for each of the clusters, compares the feature variability between the clusters, and identifies that the forged sample belongs to the cluster if one of the clusters is significantly smaller in feature variability than the other clusters. (10)

[0139] The information processing device according to (9), further comprising:

[0140] A display unit displays a user interface screen, wherein

[0141] The user interface screen is provided with a cluster comparison result display section in which the comparison result between clusters by the cluster comparison unit is displayed, and

[0142] In the cluster comparison result display section, the cluster identified as including the fake sample is displayed in an emphasized manner. (11)

[0144] The information processing device according to (10), wherein:

[0145] The user interface screen is provided with a selected data display section which displays data of the forged sample belonging to the cluster selected using the cluster comparison result display section. (12)

[0147] The information processing device according to (10) or (11), wherein:

[0148] The user interface screen is provided with a remove button which is operated to remove the forged sample displayed in the selected data display section. (13)

[0150] The information processing device according to (11) or (12), wherein:

[0151] The user interface screen is provided with a modification button which is operated to modify the forged sample displayed in the selected data display part, and a selected data modification part in which the forged sample to be corrected is modified in response to the operation of the modification button. (14)

[0153] An information processing method, comprising:

[0154] causing the information processing device to calculate a contribution indicating how much each feature in the training data set contributes to a predicted label output from the trained model, the training data set comprising both legitimate samples and forged samples, the legitimate samples comprising only legitimate data that does not contain trigger information, and the forged samples comprising at least in part forged data that contains trigger information;

[0155] causing the information processing device to perform clustering to classify each sample of the training data set into a plurality of clusters using unsupervised learning with the contribution as input; and

[0156] The information processing device is caused to compare the feature variability between clusters in the clustering result to identify the cluster to which the forged sample contained in the training data set belongs. (15)

[0158] A program causing a computer of an information processing device to execute information processing, the information processing comprising:

[0159] Calculating a contribution, the contribution indicating how much each feature in the training data set contributes to a predicted label from a trained model output, the training data set including both legitimate samples and forged samples, the legitimate samples including only legitimate data that does not include trigger information, and the forged samples at least partially including forged data that includes trigger information;

[0160] performing clustering to classify each sample of the training dataset into a plurality of clusters using unsupervised learning with the contributions as input; and

[0161] The feature variability between clusters is compared in the clustering results to identify the cluster to which the forged sample included in the training data set belongs.

[0162] It should be noted that the present embodiment is not limited to the above-described embodiment, and various modifications may be made without departing from the gist of the present disclosure. In addition, the effects described herein are merely examples and not restrictive, and there may be other effects.

[0163] Reference Symbols List

[0164] 11. Machine Learning Systems

[0165] 21 Operation input unit

[0166] 22 Display unit

[0167] 23 Storage Unit

[0168] 24 Data input unit

[0169] 25 Machine Learning Device

[0170] 41 Training execution unit

[0171] 42 Model Application Unit

[0172] 43 Contribution calculation unit

[0173] 44 Cluster Execution Units

[0174] 45 Cluster Comparison Unit

[0175] 46 Data Correction Unit

[0176] 51 Legal data storage unit

[0177] 52 Forged data storage unit

[0178] 53 Training data storage unit

Claims

1. An information processing device, comprising: a contribution calculation unit that calculates a contribution indicating how much each feature in a training data set contributes to a predicted label output from a trained model, the training data set including both legitimate samples and forged samples, the legitimate samples including only legitimate data that does not include trigger information, the forged samples at least partially including forged data that includes trigger information; a clustering execution unit that performs clustering to classify each sample of the training data set into a plurality of clusters by unsupervised learning using the contribution as an input; as well as A cluster comparing unit compares feature variability between clusters in the clustering result to identify the cluster to which the forged sample included in the training data set belongs.

2. The information processing device according to claim 1, wherein The trained model is used as a fraud detection model for detecting fraud.

3. The information processing device according to claim 1, further comprising: A training execution unit generates the trained model by performing training using the training data set as an input to a machine learning algorithm.

4. The information processing device according to claim 3, further comprising: A model application unit that outputs the predicted label by applying the training data set to the trained model generated by the training execution unit, and provides the predicted label to the contribution calculation unit.

5. The information processing device according to claim 3, further comprising: A data correction unit corrects the training data set for the forged samples belonging to the cluster identified by the cluster comparison unit.

6. The information processing device according to claim 5, wherein The data correction unit removes the forged sample from the training data set.

7. The information processing device according to claim 5, wherein The data correction unit modifies the forged data included in the forged sample.

8. The information processing device according to claim 5, wherein The training execution unit updates the trained model by performing training using the training data set corrected by the data correction unit as an input to the machine learning algorithm.

9. The information processing device according to claim 3, wherein The cluster comparison unit calculates the feature variability of the training data set for each of the clusters, compares the feature variability between the clusters, and identifies that the forged sample belongs to the cluster when one of the clusters is significantly smaller than the other clusters in the feature variability.

10. The information processing device according to claim 9, further comprising: A display unit displays a user interface screen, wherein The user interface screen is provided with a cluster comparison result display section in which a comparison result between the clusters by the cluster comparison unit is displayed, and In the cluster comparison result display section, the cluster identified as including the forged sample is displayed in an emphasized manner.

11. The information processing device according to claim 10, wherein The user interface screen is provided with a selected data display section which displays data of the forged sample belonging to the cluster selected using the cluster comparison result display section.

12. The information processing device according to claim 11, wherein The user interface screen is provided with a remove button which is operated to remove the forged sample displayed in the selected data display section.

13. The information processing device according to claim 11, wherein The user interface screen is provided with a modification button which is operated to modify a forged sample displayed in the selected data display section, and a selected data modification section in which a forged sample to be corrected is modified in response to the operation of the modification button.

14. An information processing method, comprising: causing the information processing device to calculate a contribution indicating how much each feature in a training data set contributes to a predicted label output from a trained model, the training data set comprising both legitimate samples and forged samples, the legitimate samples comprising only legitimate data not including trigger information, the forged samples at least partially comprising forged data including trigger information; causing the information processing device to perform clustering to classify each sample of the training data set into a plurality of clusters by unsupervised learning using the contribution as input; as well as The information processing device is caused to compare the feature variability between clusters in the clustering result to identify the cluster to which the forged sample contained in the training data set belongs.

15. A program for causing a computer of an information processing device to execute information processing, the information processing comprising: Calculating a contribution indicating how much each feature in a training data set contributes to a predicted label from a trained model output, the training data set comprising both legitimate samples and forged samples, the legitimate samples comprising only legitimate data not including trigger information, the forged samples at least partially comprising forged data including trigger information; performing clustering to classify each sample of the training data set into a plurality of clusters using unsupervised learning using the contributions as input; as well as The feature variability between clusters is compared in the clustering result to identify the cluster to which the forged sample included in the training data set belongs.

Images