Processing device and processing method

Abstract

A processing device (10) has: a data selection unit (11) that optimizes a dataset of training data including multiple pieces of fraudulent data and multiple pieces of legitimate data, by selecting from the fraudulent data, any fraudulent data that can be detected at a prescribed allowable false detection rate, and deleting, from the legitimate data, any legitimate data that does not contribute to the detection performance of a detection rule; and an optimization unit (12) that optimizes the proportion of fraudulent data in the dataset of training data optimized by the data selection unit (11), and the weight of the legitimate data such that the false detection rate is lower than a prescribed reference value and the detection rate is higher than the prescribed reference value.

Description

Processing Device and Processing Method

[0001] The present invention relates to a processing device and a processing method.

[0002] The problem of unauthorized use of cashless payment and damage caused by cyberattacks is becoming serious. In order to prevent these damages, it is necessary to detect communications related to unauthorized settlements and attacks.

[0003] In unauthorized detection, rule-based detection is still used, and technologies for automatically generating detection rules have been proposed (Non-Patent Documents 1 to 3).

[0004] T. Wang, et al., “A Bayesian Framework for Learning Rule Sets for Interpretable Classification”, ICDM, 2016. S. Dash, et al., “Boolean Decision Rules via Column Generation”, NeurIPS, 2018. L. Qiao, et al., “Learning Accurate and Interpretable Decision Rule Sets from Neural Networks”, AAAI, 2021.

[0005] Conventional detection rule automatic generation technologies use training data to generate detection rules with high detection accuracy and simplicity.

[0006] However, conventional technologies have problems such as difficulty in adjusting the false positive rate and high computational costs. For this reason, it has been difficult to apply conventional technologies to large-scale datasets or in cases where a very low false positive rate is required, such as unauthorized detection.

[0007] The present invention has been made in view of the above, and an object thereof is to provide a processing device and a processing method that can provide training data capable of generating rules that achieve a low false positive rate while reducing the amount of calculation.

[0008] To solve the above-mentioned problems and achieve the objective, the processing apparatus according to the present invention is characterized by having: a selection unit that optimizes a training data set including a plurality of fraudulent data and a plurality of legitimate data by selecting fraudulent data from the fraudulent data that can be detected with an acceptable predetermined false positive rate, and deleting legitimate data from the legitimate data that does not contribute to the detection performance of detection rules generated based on the training data set; and an optimization unit that optimizes the ratio of fraudulent data and the weight of legitimate data in the training data set optimized by the selection unit so that the false positive rate is below a predetermined standard value and the detection rate is above a predetermined standard value.

[0009] Furthermore, the processing method according to the present invention is a processing method executed by an processing device, and is characterized by including a selection step of optimizing a training data set that includes a plurality of fraudulent data and a plurality of legitimate data by selecting fraudulent data from the fraudulent data that can be detected with an acceptable predetermined false positive rate, and deleting legitimate data from the legitimate data that does not contribute to the detection performance of detection rules generated based on the training data set; and an optimization step of optimizing the ratio of fraudulent data and the weight of legitimate data in the training data set optimized in the selection step so that the false positive rate is below a predetermined standard value and the detection rate is above a predetermined standard value.

[0010] According to the present invention, training data is provided that can generate rules that achieve a low false positive rate while reducing the amount of computation.

[0011] Figure 1 is a schematic diagram showing an example of the configuration of a processing system in an embodiment. Figure 2 is a schematic diagram showing an example of the configuration of the processing device shown in Figure 1. Figure 3 is a flowchart showing the processing procedure of the processing method according to the embodiment. Figure 4 is a diagram showing an example of an algorithm applied to the data selection unit shown in Figure 2. Figure 5 is a diagram showing an example of an algorithm applied to the data selection unit shown in Figure 2. Figure 6 is a diagram showing an example of an algorithm applied to the optimization unit shown in Figure 2. Figure 7 is a diagram showing an example of a computer in which the processing device is realized when a program is executed.

[0012] Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to this embodiment. Furthermore, in the drawings, the same parts are denoted by the same reference numerals.

[0013] [Embodiment] The processing system according to the embodiment will now be described. Figure 1 is a schematic diagram showing an example of the configuration of the processing system in the embodiment. The processing system 1 according to the embodiment includes a processing device 10 and a learning device 20.

[0014] The processing unit 10 selects data useful for learning from the training data, optimizes the training data, and optimizes the learning settings.

[0015] The learning device 20 generates detection rules for detecting fraudulent payments and attacks by performing learning (e.g., machine learning) using the training data and learning settings optimized by the processing device 10.

[0016] [Processing Device] Next, the processing device 10 will be described. Figure 2 is a schematic diagram showing an example of the configuration of the processing device 10 shown in Figure 1. Figure 3 is a flowchart showing the processing procedure of the processing method according to the embodiment.

[0017] The processing unit 10 is a processing unit that is realized when a predetermined program is loaded into a computer or the like, which includes ROM (Read Only Memory), RAM (Random Access Memory), CPU (Central Processing Unit), etc., and the CPU executes the predetermined program. The processing unit 10 also has a communication interface for sending and receiving various information with other devices connected via a network or the like.

[0018] As shown in Figure 2, the processing unit 10 includes a data selection unit 11, an optimization unit 12, and an output unit 13.

[0019] The data selection unit 11 performs a data selection process to select data from the training data that is useful for learning (step S11 in Figure 3). In this embodiment, when a low false positive rate is required, it is noted that fraudulent data that is difficult to detect and regular data that is unlikely to be falsely detected are unnecessary for learning, and these are removed from the training data set.

[0020] The optimization unit 12 performs an optimization process to optimize the proportion of fraudulent data and the weight of normal data in the training data dataset selected by the data selection unit 11, so that the false detection rate falls below a predetermined threshold and the detection rate is higher than a predetermined threshold (step S12 in Figure 3). The optimization process by the optimization unit 12 achieves a low false detection rate for the detection rules generated using the optimized training data. The optimization unit 12 also performs undersampling during the optimization process to reduce the computational load. Undersampling is a method of reducing the amount of data for a large number of classes.

[0021] The output unit 13 performs output processing to output the optimized training data and optimized settings to the learning device 20 (step S13 in Figure 3). The optimized training data is the data selected from the training data by the data selection unit 11. The optimized learning settings are the ratio of incorrect data and the weight of normal data in the data optimized by the optimization unit 12.

[0022] [Data Selection Unit] Next, the processing of the data selection unit 11 will be explained. The data selection unit 11 selects data useful for learning from the following perspectives. First, the data selection unit 11 selects fraudulent data that can be detected with an acceptable predetermined false positive rate. Second, the data selection unit 11 deletes normal data that does not contribute to the detection performance of the detection model generated based on the training data dataset.

[0023] To reduce computational costs, the data selection unit 11 uses a rule-based decision method for selection. The rule-based decision method can be relatively lightweight. For example, it is a decision method that is lighter than the decision method used by the detection device to which the detection rules generated by the learning device 20 are applied. In the following examples, we will describe a case where, for example, a decision tree-based decision method is applied. Furthermore, the acceptable false positive rate may be set to a value slightly larger than the actually desired false positive rate. This is effective when the available data is relatively large.

[0024] [Selection of Fraudulent Data] The procedure for selecting fraudulent data from the training data that can be detected with an acceptable false positive rate is described below. Figure 4 shows an example of an algorithm applied to the data selection unit 11 shown in Figure 2. The algorithm in Figure 4 shows the procedure for selecting fraudulent data.

[0025] As shown in Figure 4, first, the data selection unit 11 clusters the invalid data (first row of Figure 4).

[0026] Next, the data selection unit 11 estimates whether learning is easy or not. The data selection unit 11 uses a rule-based determination method to determine whether learning is easy or not for each cluster. The data selection unit 11 uses all the data from the dataset, which has been added in order from the clusters that are easy to learn, to evaluate the detection rate (True Positive Rate: TPR) and the false positive rate (False Positive Rate: FPR).

[0027] Specifically, the data selection unit 11 uses all the data to train a decision tree (fourth row in Figure 4) and calculates the average confidence level of the data belonging to the cluster. Data with a high confidence level to be judged as fraudulent is considered easy to train.

[0028] Therefore, the data selection unit 11 investigates the impact of adding clusters to the dataset in order of ease of detection (ease of learning) (third row of Figure 4). Specifically, the data selection unit 11 constructs a decision tree when the clusters under investigation are added to the dataset (fourth row of Figure 4). The data selection unit 11 evaluates the detection rate and false positive rate using all the data used to construct the decision tree (fifth row of Figure 4).

[0029] Then, the data selection unit 11 adds the cluster to be investigated to the dataset if the evaluated FPR does not exceed a predetermined FPR threshold and the evaluated TPR has improved (sixth row in Figure 4) (seventh row in Figure 4). The data selection unit 11 does not add the cluster to the dataset if the false positive rate is above the threshold or does not improve. The dataset is the optimized training data dataset.

[0030] [Deletion of Normal Data] The procedure for deleting normal data from the training data that does not affect detection performance is described below. Figure 5 shows an example of an algorithm applied to the data selection unit 11 shown in Figure 2. The algorithm in Figure 5 shows the procedure for deleting normal data that does not affect detection performance.

[0031] As shown in Figure 5, first, the data selection unit 11 performs a first process of clustering the regular data (first row of Figure 5).

[0032] Next, the data selection unit 11 estimates whether learning is easy or not. The data selection unit 11 uses a rule-based determination method to determine whether learning is easy or not for each cluster. The data selection unit 11 uses all the data from the dataset, which has been removed in order from the clusters that are easy to learn, to evaluate the detection rate and the false positive rate.

[0033] Specifically, the data selection unit 11 performs a second process in which it trains a decision tree using all the data (third row in Figure 5) and calculates the average confidence level of the data belonging to the cluster. It is considered that data with a low confidence level, which is judged as fraudulent, is easy to train.

[0034] Therefore, the data selection unit 11 performs a third process to investigate the impact of excluding clusters from the dataset in order of ease of normalization (ease of learning). Specifically, the data selection unit 11 removes the clusters under investigation from the dataset and constructs a decision tree (third line in Figure 5). The data selection unit 11 then evaluates the detection rate and false positive rate using all the data (fourth line in Figure 5).

[0035] The data selection unit 11 excludes a cluster from the dataset if the false positive rate and detection rate do not worsen below a predetermined standard (rows 5 and 6 in Figure 5). The data selection unit 11 does not exclude a cluster from the dataset if the false positive rate or detection rate worsens below a predetermined standard. The dataset is the optimized training data dataset.

[0036] Furthermore, the data selection unit 11 may use other conditions in the third processing. For example, other conditions may include that the data does not falsely detect when deleted from the dataset, or that the false detection rate does not exceed a standard.

[0037] [Optimization Unit] Next, the processing of the optimization unit 12 will be described. The optimization unit 12 performs optimization processing to optimize the proportion of fraudulent data and the weight of normal data in the data selected by the data selection unit 11. The optimization unit 12 changes the proportion of fraudulent data and the weight of normal data in the training data dataset selected by the data selection unit 11, and also performs undersampling at a specified proportion of fraudulent data. The optimization unit 12 selects a combination of the proportion of fraudulent data and the weight of normal data from among the various combinations of the proportion of fraudulent data and the weight of normal data such that the false positive rate evaluated using all the data in the dataset for each combination falls below a predetermined false positive rate threshold and the detection rate is at its highest value.

[0038] Figure 6 shows an example of an algorithm applied to the optimization unit shown in Figure 2. The algorithm in Figure 6 shows a procedure for optimizing the proportion of invalid data and the weight of normal data in the data selected by the data selection unit 11.

[0039] The optimization unit 12 constructs a decision tree by changing the proportion of fraudulent data and the weights of normal data within the dataset selected by the data selection unit 11 (first row of Figure 6). The optimization unit 12 can reduce computational cost by simultaneously performing undersampling with a specified proportion of fraudulent data (second row of Figure 6). The optimization unit 12 may use methods such as random sampling, clustering with k-means and using samples close to the center, TomekLinks to remove samples that appear to be noise, or NearMiss to select samples close to the decision boundary as undersampling methods.

[0040] The optimization unit 12 trains a decision tree using each dataset with varying proportions of fraudulent data and weights of normal data (fourth row in Figure 6), and evaluates the detection rate and false positive rate using all the data (fifth row in Figure 6). The optimization unit 12 selects a setting for the proportion of fraudulent data and weights of normal data that results in a false positive rate (FPR) below the FPR threshold and the highest possible total positive rate (TPR) (sixth and seventh rows in Figure 6). Within the same proportion of fraudulent data, the optimization unit 12 may prioritize those with smaller weights.

[0041] [Effects of the Embodiment] As described above, the processing device 10 according to the embodiment selects from the fraudulent data in the training data that can be detected with an acceptable predetermined false detection rate, and also deletes from the normal data in the training data that does not contribute to the detection performance of the detection rule, thereby optimizing the training data set. Therefore, even when generating detection rules based on a large dataset, the processing device 10 according to the embodiment can provide the learning device 20 with training data that reduces the computational amount required for rule generation.

[0042] Then, the processing device 10 optimizes the ratio of the incorrect data and the weight of the normal data in the optimized teacher data so that the false detection rate is lower than the reference value and the detection rate is higher than a predetermined reference value. Therefore, the processing device 10 can provide the learning device 20 with teacher data that can generate a detection rule for realizing a low false detection rate.

[0043] Furthermore, when optimizing the ratio of the incorrect data and the weight of the normal data, the processing device 10 can also reduce the calculation cost by performing undersampling.

[0044] [Regarding the System Configuration of the Embodiment] Each component of the processing device 10 is conceptually functional and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of the distribution and integration of the functions of the processing device 10 is not limited to that shown in the figure, and all or part of it can be functionally or physically distributed or integrated in any unit according to various loads, usage situations, etc.

[0045] Also, each process performed in the processing device 10 may be realized in whole or in any part by a program analyzed and executed by a CPU, a GPU (Graphics Processing Unit), and the CPU and GPU. Also, each process performed in the processing device 10 may be realized as hardware by wired logic.

[0046] Also, among the processes described in the embodiment, all or part of the processes described as being automatically performed can be manually performed. Or all or part of the processes described as being manually performed can be automatically performed by a known method. In addition, regarding the above-described and illustrated processing procedures, control procedures, specific names, and information including various data and parameters, they can be appropriately changed unless otherwise specified.

[0047] [Program] Figure 7 is a diagram showing an example of a computer in which the processing device 10 is realized when the program is executed. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0048] The memory 1010 includes a ROM 1011 and a RAM 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System), for example. The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100, for example. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to, for example, a display 1130.

[0049] The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. That is, the program defining each process of the processing device 10 is implemented as a program module 1093 in which executable code by the computer 1000 is described. The program module 1093 is stored in the hard disk drive 1090, for example. For example, a program module 1093 for executing the same process as the functional configuration in the processing device 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0050] Furthermore, the configuration data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, memory 1010 or hard disk drive 1090. The CPU 1020 then reads the program module 1093 and program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as needed and executes them.

[0051] Furthermore, the program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090; for example, they may be stored in a removable storage medium and read by the CPU 1020 via a disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via a network interface 1070.

[0052] Although embodiments applying the invention made by the present inventors have been described above, the present invention is not limited by the descriptions and drawings that constitute part of the disclosure of the present invention in these embodiments. That is, all other embodiments, examples, and operational techniques made by those skilled in the art based on these embodiments are included in the scope of the present invention.

[0053] 1 Processing system 10 Processing unit 11 Data selection unit 12 Optimization unit 13 Output unit

Claims

1. A processing device comprising: a selection unit that optimizes a training data set containing multiple fraudulent data and multiple legitimate data by selecting fraudulent data from the fraudulent data that can be detected with an acceptable predetermined false positive rate, and by deleting legitimate data from the legitimate data that does not contribute to the detection performance of detection rules generated based on the training data set; and an optimization unit that optimizes the ratio of fraudulent data and the weight of legitimate data in the training data set optimized by the selection unit so that the false positive rate is below a predetermined standard value and the detection rate is above a predetermined standard value.

2. The processing apparatus according to claim 1, wherein the selection unit clusters the fraudulent data, determines whether each cluster is easy to learn using a rule-based determination method, and if the false positive rate evaluated using all the data in the dataset added in order from the clusters that are easy to learn does not exceed a predetermined false positive rate threshold, and if the detection rate evaluated using all the data in the dataset added in order from the clusters that are easy to learn improves, the selection unit adds the relevant cluster to the training data dataset; and the processing apparatus according to claim 1, wherein the selection unit clusters the normal data of the training data, determines whether each cluster is easy to learn using a rule-based determination method, and if the detection rate and false positive rate evaluated using all the data when clusters are removed from the dataset in order from the clusters that are easy to learn does not worsen below a predetermined threshold, the selection unit removes the relevant cluster from the training data dataset.

3. The processing apparatus according to claim 1, characterized in that the optimization unit changes the proportion of fraudulent data and the weight of normal data in the training data dataset selected by the selection unit, performs undersampling with a specified proportion of fraudulent data, and selects a combination of the proportion of fraudulent data and the weight of normal data such that the false detection rate evaluated using all the data in the dataset for each combination falls below a predetermined false detection rate threshold and the detection rate is the highest possible.

4. A processing method executed by a processing device, comprising: a selection step of optimizing a training data set that includes a plurality of fraudulent data and a plurality of legitimate data by selecting fraudulent data from the fraudulent data that can be detected with an acceptable predetermined false positive rate, and deleting legitimate data from the legitimate data that does not contribute to the detection performance of detection rules generated based on the training data set; and an optimization step of optimizing the ratio of fraudulent data and the weight of legitimate data in the training data set optimized in the selection step such that the false positive rate is below a predetermined standard value and the detection rate is above a predetermined standard value.

Images