Information processing device, information processing method, and program

Abstract

An object of the present disclosure is to provide an information processing device, an information processing method, and a non-transitory computer-readable medium capable of generating an accurate output for detecting outliers. The information processing device (10) according to the present disclosure includes a probability calculation unit (11) configured to calculate the respective probabilities of each data point being an outlier using a temperature parameter t>0, and an adjustment unit (12) configured to output the probabilities by decreasing the temperature parameter t toward 0 in multiple steps.

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a non-transitory computer-readable medium. [Background technology]

[0002] There are many applications where detecting outliers improves machine learning. For example, [1] introduces a new approach of differential sorting to detect outliers. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] “Blondel et al., “Fast Differentiable Sorting and Ranking”, In Proceedings of the International Conference on Machine Learning, 2020. Summary of the Invention [Problem to be solved by the invention]

[0004] However, the method described in Non-Patent Document 1 may produce inaccurate outputs when there are significant outliers in the input data.

[0005] An object of the present disclosure is to provide an information processing device, an information processing method, and a non-transitory computer-readable medium that can generate an accurate output for detecting outliers. [Means for solving the problem]

[0006] In a first example aspect, the information processing device comprises a probability calculation means for calculating the respective probability that each data point is an outlier using a temperature parameter t>0, and an adjustment means for outputting the probabilities by lowering the temperature parameter t towards 0 using multiple steps.

[0007] In a second example embodiment, the information processing method calculates the respective probability that each data point is an outlier using a temperature parameter t>0, and outputs the probabilities by ramping the temperature parameter t towards 0 in multiple steps.

[0008] In a third example embodiment, a non-transitory computer-readable medium storing a program causes a computer to calculate the respective probability of each data point being an outlier with a temperature parameter t>0, and output the probabilities by ramping the temperature parameter t towards 0 in multiple steps. Effect of the Invention

[0009] According to the present disclosure, it is possible to provide an information processing device, an information processing method, and a non-transitory computer-readable medium that can generate an accurate output for detecting outliers. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 shows an example of data with 4 outliers and 16 inliers sampled from a Gaussian distribution. [Diagram 2] FIG. 2 is a diagram showing the estimation of the soft sorting method. [Diagram 3] FIG. 3 is a configuration diagram showing a configuration of the first embodiment of the present disclosure. [Figure 4] FIG. 4 is a conceptual diagram showing steps according to the second embodiment of the present disclosure. [Diagram 5] FIG. 5 is a diagram illustrating an example of an algorithm according to the second embodiment of the present disclosure. [Figure 6]FIG. 6 is a diagram showing another example of the algorithm according to Embodiment 2 of the present disclosure. [Figure 7] FIG. 7 is a diagram showing the estimation according to Embodiment 2 of the present disclosure. [Figure 8] FIG. 8 is a configuration diagram of an information processing apparatus according to each embodiment.

Embodiments for Carrying Out the Invention

[0011] (Overview of Related Art) Before describing the embodiments according to the present disclosure, an overview of the related art will be described with reference to FIGS. 1 to 2.

[0012] Training data is represented as follows:

Number

Number

[0013] Least trimmed square suggests identifying the set of outliers using the following objective.

Number

Number

Number

[0014] Furthermore, it is defined as follows:

number

[0015] Trimmed least squares optimizes the following objective using gradient descent:

number

number

[0016] As a result, Non-Patent Document 1 classifies the sort operation as a soft sort operation s ε :

number

number

number

[0017] (Problems to be solved by this disclosure) The problem with the method of Non-Patent Document 1 is that one entry I jIf is very large, then after soft sorting, all entries will approach the average. More formally,

number

[0018] This results in the trimmed log-likelihood sum approaching the ordinary log-likelihood sum up to a constant factor:

number

[0019] However, it is well known that the ordinary log-likelihood sum suffers from outliers, and as a result, using the trimmed log-likelihood sum from the soft sort may also suffer from outliers.

[0020] As an example, consider the following data: The inliers are 16 samples from a normal distribution with mean 1.5 and standard deviation 0.5. In addition, there are 4 outliers, which are 3 samples from a normal distribution with mean -1.5 and standard deviation 0.5 and 1 sample at point -10.0. This data is shown in Figure 1. Figure 1 shows an example data set with 4 outliers and 16 inliers sampled from a Gaussian distribution. The inliers are shown on the right side of Figure 1 and the outliers on the left side.

[0021] However, the soft sorting method is affected by the outlier -10.0, which shifts the estimate of the inlier distribution towards the left as shown in Figure 2. Figure 2 shows the estimate of the soft sorting method (ε=0.5). The inliers are shown on the right side of Figure 2 and the outliers on the left side, and the curve in Figure 2 shows the probability density function of the inliers.

[0022] Parameter θ=(μ,σ 2 ) is estimated as

number

number

[0023] An obvious improvement would be to reduce ε towards 0 while running fewer gradient descent iterations. However, the objective

number

[0024] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. These embodiments can be applied to an apparatus that generates accurate output for detecting outliers. For example, the outliers of a training dataset can be determined in the following manner.

[0025] (Embodiment 1) First, an information processing device 10 according to the first embodiment will be described with reference to FIG.

[0026] 3, an information processing device 10 according to the first embodiment of the present disclosure includes a probability calculation unit (probability calculation means) 11 and an adjustment unit (adjustment means) 12. For example, the information processing device 1 can be used for machine learning.

[0027] The probability calculation unit 11 uses a temperature parameter t>0 to calculate the respective probabilities that each data point is an outlier. The data points are included in input data, which may be stored in the information processing device 10 or may be transmitted from outside the information processing device 10. The probability is a numerical value, indicating that the data point is an outlier or an inlier. The temperature parameter t has a meaning commonly used in statistical research.

[0028] The adjustment unit 12 outputs the probability by lowering t toward 0 using multiple steps. Note that the adjustment unit 12 may set the temperature parameter to 0 in the final step, but may also set the temperature parameter to a small value (a value close to 0) in the final step. If it is clear that the output probability distinguishes between an outlier and an inlier, the small value is not limited.

[0029] The configuration shown in FIG. 3 includes an information processing device 1 0 This can be done by software and hardware installed in the device. A more specific configuration will be described below.

[0030] As described above, the probability calculation unit 11 calculates the probability using the temperature parameter t, and the adjustment unit 12 outputs the probability by lowering the temperature parameter t toward 0 using multiple steps. Therefore, even if there is a prominent outlier in the input data, the influence of the outlier is reduced during the steps, and the output is not significantly affected by the outlier. As a result, the information processing device 10 can generate an accurate output for detecting the outlier.

[0031] (Embodiment 2) First, a second embodiment of the present disclosure will be described below with reference to the accompanying drawings, which illustrate the best mode for carrying out the present disclosure.

[0032] The information processing device 10 of this embodiment includes a probability calculation unit 11 and an adjustment unit 12 shown in Fig. 3. The elements of the information processing device 10 function as shown in the first embodiment, but function more sophisticatedly as described below.

[0033] Before describing the detailed procedure of embodiment 2, some details are provided. The proposed disclosure calculates a weight for each sample that is guaranteed to be between 0 and 1. Each sample's weight is multiplied by its log-likelihood value. The weights are controlled by a temperature parameter that controls the smoothness of the optimization function. During the gradient descent step, the temperature parameter is decreased to ensure that the influence of outliers is reduced towards 0.

[0034] The proposed disclosure is derived as follows:

number

number

number

number

[0035] We propose to alleviate this problem in a continuous manner as follows.

number

number

number

number

[0036] The core steps of the method are illustrated in Figure 4 and are described below. The core steps are processed by an information processing device 10.

[0037] The inlier probability evaluation step S21 in FIG. 4 is performed by the probability calculation unit 11. In order to separate outliers and inliers, the inlier weight w defined by Equation (1) is i Introduce w i must be bounded between 0 and 1, so that it can be interpreted as the probability that sample i is an inlier. Conversely, 1-w i is taken as the probability that sample i is an outlier.

[0038] In an inlier probability evaluation step S21, the probability calculation unit 11 acquires observed data D1 (sample data) and additional data D2. The observed data D1 includes the following training data:

number

[0039] Based on this data, the probability calculation unit 11 calculates the probability of each sample as a sigmoid function, where each probability is parameterized by the temperature t and a threshold parameter q, which in turn depends on the number of outliers specified by the user.

[0040] The probability calculator 11 outputs probabilities below 0.5 for samples with log-likelihoods lower than the k+1 lowest sample, and probabilities above 0.5 for the remaining samples. The temperature parameter t controls how far the probabilities are from 0.5. At higher temperature values, the total probability is closer to 0.5, while at lower temperature values ​​the total probability is closer to either 0 or 1.

[0041] The cooling scheme step S22 of FIG. 4 was performed by the adjustment unit 12. i We explicitly identify outliers using,(1),and (2) introduce a cooling scheme to reduce,t,towards 0 to reduce the impact of outliers on,the training of the parameter,θ,.,The reduction of,t,depends on the change in the loss function and / or,the number of iterations from S21 to S23 in Fig. 4.,The cooling scheme starts from some high value of,t,and gradually reduces,t,after a given number of gradient descent steps until,t,=0 (or very close to 0).

[0042] We propose to decrease the temperature parameter t with increasing number of gradient descent steps S23 in Figure 4. For example, an exponential cooling scheme may be used to decrease the temperature, as shown below.

[0043]

number

[0044] Additionally, you can specify maximum and minimum values ​​for the temperature parameter. For example, Maximum temperature (MAX TEMPERATURE) = 100.0, minimum temperature (MIN TEMPERATURE) = 0.01.

[0045] Furthermore, the objective function f t Specify the parameter ε that determines the convergence of (θ) to a (local) optimum. For example, ε=0.01.

[0046] The exponential cooling scheme is given by Algorithm 1 shown in FIG.

[0047] Alternatively, we can simply specify the number of gradient descent steps in the inner loop with some parameter m, say m = 100. The exponential cooling scheme simplifies to Algorithm 2 shown in Figure 6.

[0048] Once the final cooling scheme is complete, the conditioning unit 12 outputs output data D3 containing the probability of each sample. The probabilities are calculated using the indicator variable w i (i=1, 2, ..., n). x i If w is an inlier i is 1, x i If w is an outlier i will be 0.

[0049] (example) Below we present an example that illustrates the effectiveness of the present disclosure. In particular, we consider the same data as before. (The inliers are 16 samples from a normal distribution with mean 1.5 and standard deviation 0.5. In addition, there are 4 outliers, which are 3 samples from a normal distribution with mean -1.5 and standard deviation 0.5, and 1 sample at point -10.0. The data points ranging from -10 to 2.7 are shown in Figure 1.)

[0050] Table 1 shows the weights of each data point learned at a given temperature. The weights of each data point are listed in the same order as the data points (i.e., starting with the data point with value -10 and ending with the data point with value 2.7). Table 1 shows the weights of inliers w from the proposed method for different temperature parameters t. i Here is an example output from . The weights for each data point are displayed in the same order as the values ​​of the data points. The entries for the 10th through 15th data points are omitted (...) for clarity, but they converge to the correct value as well. [Table 1]

[0051] First, the proposed method starts at temperature t = 100 and goes down to t = 0.012. Using the proposed method, the parameters θ = (μ, σ 2 ) is the final estimate

number

number

[0052] The outliers detected by the proposed method are shown in Figure 7. The curves in Figure 7 show the probability density function of the inliers. As can be seen, the proposed method accurately identifies all outliers. Moreover, compared to the example in Figure 2, the probability density function is more accurate.

[0053] As discussed above, the proposed disclosure can reduce the impact of outliers on the objective function while ensuring the objective function is smooth enough to optimize via gradient descent.

[0054] In detail, the probability calculation unit 11 uses the temperature parameter t to calculate the probability, and the adjustment unit 12 uses a gradient descent step to reduce the temperature parameter t towards 0 and outputs the probability. Thus, the proposed disclosure can reduce the impact of outliers and generate accurate outputs for detecting outliers.

[0055] Furthermore, the probability calculation unit 11 can calculate the probability using the log likelihood of each data point in addition to the temperature parameter t, which simplifies the calculation in the process and shortens the time required for the calculation.

[0056] Furthermore, the probability calculation unit 11 can calculate the probability using a pre-specified ratio of outliers in addition to the temperature parameter t. Therefore, a difficult combinatorial problem can be made easier by turning it into an optimization problem.

[0057] Furthermore, the probability calculation unit 11 can set the probability for each data point as a sigmoid function, so that inliers and outliers can be easily distinguished.

[0058] Furthermore, the adjustment unit 12 can keep the temperature parameter t constant until the gradient descent converges or until a pre-specified number of gradient descent iterations have elapsed. Also, the adjustment unit 12 can exponentially decrease the temperature parameter t after the gradient descent converges or after a pre-specified number of gradient descent iterations have elapsed. Thus, the temperature parameter t eventually approaches 0, thereby reducing the influence of outliers.

[0059] The proposed disclosure can be applied in various fields, since detecting outliers is important in various applications. For example, outliers may correspond to malicious user behavior, and detecting outliers can prevent cyber attacks. Another application is the possibility of analyzing and improving the use of training data to improve the predictive performance of various regression tasks. For example, mislabeled samples can degrade the performance of a classification model.

[0060] Next, a configuration example of the information processing device described in the above embodiments will be described below with reference to FIG.

[0061] 8 is a block diagram showing an example of the configuration of an information processing device 90. As shown in FIG.

[0062] The processor 91 executes the processes executed by the information processing device 90 described with reference to the sequence diagrams and flowcharts of the above embodiment in the memory 9 2 The processor 91 executes the program by loading and executing software (computer programs) from the processor 91. The processor 91 may be, for example, a microprocessor, a micro processing unit (MPU) or a central processing unit (CPU). The processor 91 may include multiple processors.

[0063] The memory 92 is configured by a combination of volatile memory and non-volatile memory. The memory 92 may include a storage device located remotely from the processor 91. In this case, the processor 91 may access the memory 92 via an I / O interface (not shown).

[0064] 8, the memory 92 is used to store a group of software modules. The processor 91 reads the group of software modules from the memory 92 and executes the read software modules to perform the processing performed by the information processing device described in the above embodiment.

[0065] As described above with reference to FIG. 8, each processor included in the information processing device of the above embodiment executes one or more programs including a group of instructions to cause a computer to execute the algorithm described above with reference to the drawings.

[0066] The information processing device 90 may also include a network interface. The network interface is used for communication with other network node devices constituting the communication system. The network interface may include, for example, a network interface card (NIC) conforming to the IEEE 802.3 series. The information processing device 90 may receive an input feature map or transmit an output feature map using the network interface.

[0067] In the above example, the program can be stored and provided to the computer by using any kind of non-transitory computer-readable medium. The non-transitory computer-readable medium includes any kind of tangible storage medium. Examples of the non-transitory computer-readable medium include magnetic storage media (such as floppy disks, magnetic tapes, hard disk drives, etc.), optical magnetic storage media (such as optical magnetic disks), compact disc read only memory (CD-ROM), compact disc recordable (CD-R), compact disc rewritable (CD-R / W), and semiconductor memory (such as mask ROM, programmable ROM (PROM), erasable PROM (EPROM), flash ROM, random access memory (RAM), etc.). The program may be provided to the computer by using any kind of temporary computer-readable medium. Examples of the temporary computer-readable medium include electric signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can provide the program to the computer via a wired communication line (such as electric wires, optical fibers) or a wireless communication line.

[0068] The present disclosure is not limited to the above-described embodiments, and can be modified as appropriate without departing from the spirit and scope of the present disclosure. [Industrial Applicability]

[0069] The present disclosure is applicable to outlier detection in the field of computer systems. [Explanation of symbols]

[0070] 10. Information processing device 11 Probability Calculation Section 12 Adjustment part

Claims

1. a probability calculation means for calculating a respective probability of each data point being an outlier with a temperature parameter t, t>0; Adjustment means for decreasing the temperature parameter t; gradient descent means for performing gradient descent; The probability calculation means, the adjustment means, and the gradient descent means each repeatedly execute their respective processes to lower the temperature parameter t toward 0 using a plurality of steps, and the adjustment means outputs the probability when the temperature parameter t reaches a predetermined value. Information processing device.

2. The probability calculation means calculates the probability using a log-likelihood of each data point in addition to the temperature parameter t. The information processing device according to claim 1 .

3. the probability calculation means calculates the probability using a pre-specified ratio of outliers in addition to the temperature parameter t; 3. The information processing device according to claim 1 or 2.

4. said probability calculation means setting said probability as a sigmoid function for each data point; The information processing device according to claim 1 .

5. the adjusting means holds the temperature parameter t constant until the gradient descent converges or a pre-specified number of gradient descent iterations have elapsed. The information processing device according to claim 1 .

6. the adjusting means exponentially decreases the temperature parameter t after the gradient descent has converged or after a pre-specified number of gradient descent iterations have elapsed. The information processing device according to claim 1 .

7. a computational process for computing the respective probabilities of each data point being an outlier with a temperature parameter t>0; A reduction process for reducing the temperature parameter t; A gradient descent process for performing gradient descent; by repeatedly performing the calculation process, the reduction process, and the gradient descent process, the temperature parameter t is reduced toward 0 in a plurality of steps, and the probability is output when the temperature parameter t reaches a predetermined value. Information processing methods.

8. a computational process for computing the respective probabilities of each data point being an outlier with a temperature parameter t>0; A reduction process for reducing the temperature parameter t; A gradient descent process that performs gradient descent; by repeatedly executing the above, the temperature parameter t is decreased toward 0 in a plurality of steps, and the probability is output when the temperature parameter t reaches a predetermined value. A program that causes a computer to do something.

Images