Information processing device and information processing method

Abstract

An information processing device (100) is disclosed. The information processing device (100) includes an input means (102), a statistical calculation means (104), and an optimization means (106). The input means (102) receives a response and a covariate. The statistical calculation means (104) converts the response into a plurality of transformed samples using a function based on the covariates and an unbiased parameter. The probability distribution of the transformed samples depends only on a variance parameter. The optimization means (106) maximizes the probability distribution of the transformed samples to determine an estimate of the variance parameter.

Description

[Technical field]

[0001] The present invention relates to an information processing device. and Information Processing By law This is related to. [Background technology]

[0002] Many real-world data sets contain outliers, i.e. data points that are not representative of the majority of the sample. For example, the output of a broken sensor can lead to outliers. It is well known that parameter estimation of statistical models from data containing outliers can often lead to arbitrarily poor estimates. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Rousseeuw, Peter J and Leroy, Annick M, "Robust regression and outlier detection", 2005

[0004] [Non-Patent Document 2] Blondel, Mathieu and Teboul, Olivier and Berthet, Quentin and Djolonga, Josip, "Fast Differentiable Sorting and Ranking", In Proceedings of the International Conference on Machine Learning, 2020

[0005] [Non-Patent Document 3] Rice and Spiegelhalter, "A simple diagnostic plot connecting robust estimation, outlier detection, and false discovery rates", Journal of Applied Statistics, 2007

[0006] [Non-Patent Document 4] DasGupta, Anirban, "Probability for statistics and machine learning: fundamentals and advanced topics", 2011 Summary of the Invention

[0007] (Technical issues) An exemplary aspect of the present invention is to solve a problem, and an exemplary objective is to provide a suitable technique for variance parameter estimation. [Problem to be solved by the invention]

[0008] In order to achieve the above-mentioned object, the information processing device comprises: an input means for receiving a plurality of input samples including a plurality of response quantities and a plurality of covariates; a statistical calculation means for converting the response quantities into a plurality of transformed samples using a function based on the covariates and unbiased parameters, the probability distribution of the transformed samples being determined solely by a variance parameter; and an optimization means for maximizing the probability distribution of the transformed samples to determine an estimate of the variance parameter.

[0009] In order to achieve the above-mentioned object, the information processing device includes an input means for receiving a plurality of input samples including a plurality of responses and a plurality of covariates; a statistical value calculation means for converting the response quantities into a plurality of transformed samples using a function based on the covariates and unbiased parameters, wherein a probability distribution of the transformed samples is determined only by a variance parameter; an optimization means for maximizing the probability distribution of the transformed samples to determine an estimate of the variance parameter; a p-value calculation means for estimating a p-value by referring to the estimate of the variance parameter; and an outlier determination means for determining a list of outliers by referring to the p-value.

[0010] To achieve the above-mentioned objective, an information processing method includes receiving a plurality of input samples including a plurality of response quantities and a plurality of covariates; transforming the response quantities into a plurality of transformed samples using a function that depends on the covariates and an unbiased parameter, such that a probability distribution of the transformed samples depends only on a variance parameter; and optimizing the probability of observing the transformed samples to determine an estimate of the variance parameter. Effect of the Invention

[0014] In accordance with an exemplary aspect of the present invention, it is possible to provide a suitable technique for variance parameter estimation. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a block diagram showing an information processing apparatus according to the first embodiment. [Diagram 2] FIG. 2 is a flowchart showing steps of a method executed in the information processing device according to the first embodiment. [Diagram 3] FIG. 3 is a graph showing the highest probability density function (pdf) of the true outlier probability distribution as described in the first embodiment. [Figure 4]FIG. 4 is a graph showing the estimated inlier probability distribution as described in the first embodiment. [Diagram 5] FIG. 5 is a graph showing an inlier probability distribution estimated by the method implemented by the information processing device according to the first embodiment. [Figure 6] FIG. 6 is a block diagram showing an information processing apparatus according to the second embodiment. [Figure 7] FIG. 7 is a block diagram showing an information processing apparatus according to the third embodiment. [Figure 8] FIG. 8 is a flowchart showing steps of a method performed in an information processing device according to the third embodiment. [Figure 9] FIG. 9 is a block diagram showing an information processing apparatus according to the fourth embodiment. [Figure 10] FIG. 10 is a conceptual block diagram showing a computer used as an information processing device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] <Brief Description of the Background Art> Many real-world data sets contain outliers, i.e. data points that are not representative of the majority of the sample. For example, the output of a broken sensor can lead to outliers. It is well known that parameter estimation of statistical models from data containing outliers often leads to arbitrarily poor estimates.

[0017] A mandatory fix is ​​to use trimmed likelihoods for parameter estimates. In contrast to other robust estimation procedures such as the Huber loss, its hyperparameter, the minimum of inliers m (i.e., samples that are not outliers), has a clear interpretation and is thus relatively easy to identify. For example, a conservative estimate is to set m=n / 2, where n is the total number of samples.

[0018]

number

[0019] Parameters

number

number

number

[0020] The statistical model has two parameters and has the following characteristics:

number

[0021] Note that this bias cannot be fixed, even if we increase the number of samples infinitely. The source of the problem is the gap between the true number of inliers and the user-specified lower bound m. However, it becomes necessary to set m to a conservatively low value, otherwise we run the risk of including outliers that would lead to arbitrarily bad estimates.

[0022] Finally, it should be noted that, knowing the true variance, or at least an upper bound, it is possible to estimate outliers while controlling the false discovery rate (FDR) using the method proposed in [3]. However, underestimated variance will not allow controlling the FDR. <First embodiment> (Information and control equipment) The following description will discuss details of the first embodiment of the present invention with reference to the drawings.

[0023] The first embodiment relates to an information processing device implementing a method for determining a variance parameter of a statistical model from data. Fig. 1 is a block diagram showing an information processing device according to the first embodiment of the present invention. The information processing device 100 includes an input unit 102, a statistical calculation unit 104, an optimization unit 106, and an output unit 108.

[0024] The input unit 102 receives data or samples. The samples include outlier samples and inlier samples. The samples received at the input unit 102 may be observed samples. The input unit provides the received samples to the statistics calculation unit 104 as input samples.

[0025] As a specific example, the observed samples received at input 102 have an inlier probability distribution as curve 302 in Figure 3. Curve 302 shows the highest probability density function (pdf) of the true inlier probability distribution. The observed samples include inlier samples 304 and outlier samples 306, shown as dotted circles at the bottom of Figure 3.

[0026] As can be seen above, each observed sample has covariates, x, and responses, y. Therefore, the observed sample is (x i , y i ), where the index i denotes the sample index. In other words, sample i is the sample with the covariance x i and the corresponding response quantity y i Includes.

[0027] Covariate x i It should be noted that the variables y may be referred to as independent variables, estimators, features, or explanatory variables. i It should be noted that may be referred to as the dependent variable, outcome variable, or response variable.

[0028] The statistics calculator 104 receives input samples from the input unit 102. As explained above, the input samples include covariates and response quantities. The statistics calculator 104 transforms the response quantities into transformed samples using a function that depends on the covariates and the unbiased parameters. The probability distribution of the transformed samples depends only on the variance parameters.

[0029] Although the particular form of the function does not limit the first embodiment, the function is i and may contain linear terms in some other function h that depends on the unbiased parameters.

[0030] The optimizer 106 receives the transformed samples from the statistics calculator 104. The optimizer 106 maximizes the probability distribution of the transformed samples to determine an estimate of the dispersion parameter.

[0031] The particular method of maximizing the probability distribution may not be limited in the first embodiment, but the maximizing method may take advantage of the Markov property of the probability distribution and maximum likelihood methods.

[0032] (Information processing method) 2 is a flowchart showing steps of a method implemented in the information processing device in the first embodiment. The method S20 has four steps.

[0033] First, an input sample is input to the input unit 102 (step S22). As described above, the input sample has a response amount and a covariate. The sample received by the input unit 102 may be an observed sample. The input unit 102 provides the sample received as the input sample to the statistical calculation unit 104.

[0034] Next, the response quantities in the input samples are statistically calculated by the statistical calculation unit 104 and transformed into transformed samples (step S24). In the calculation, functions that depend only on the covariates and unbiased parameters are used. The probability distribution of the transformed samples depends only on the variance parameters.

[0035] The optimization unit 106 optimizes the probability distribution of the transformed samples to determine an estimate of the dispersion parameter (step S26).

[0036] Finally, the estimate of the dispersion parameter is output (step S28).

[0037] (Advantageous Effects of the First Embodiment) According to the information processing device 100 and the information processing method S20 of the first embodiment, an accurate estimate of the inlier probability distribution can be obtained, which enables accurate outlier detection in the data. Accurate outlier detection is important, for example, for finding malicious activity in process log data or identifying defective products in sensor data.

[0038] As shown in FIG. 5, the estimated inlier probability distribution according to the second embodiment, shown by the dotted line 502, is close to the true inlier probability distribution, shown by the solid line 504. <Second embodiment> The following description will be given in detail of the second embodiment of the present invention with reference to the drawings. It should be noted that the same reference numerals are given to elements having the same functions as those described in the first embodiment, and the description of such elements will be omitted appropriately. Furthermore, the outline of the second embodiment is the same as that of the first embodiment, and will not be described here.

[0039] (Information processing device) 6 is a block diagram showing an information processing apparatus according to the second embodiment. The information processing apparatus 600 includes a database 601, an input unit 603, a sufficient statistics calculation unit 605, an optimization unit 607, and an output unit 609.

[0040] In the database 601, observed data (input data) are stored. The input data are transferred to the input section 603. As mentioned before, the input samples have response quantities and covariates. The input section 603 stores a minimal set of inlier estimates.

number

number

number

number

[0041] where h in the likelihood function f is the unbiased parameter

number

[0042] The sufficient statistics calculator 605 calculates a minimal set of inlier estimates.

number

number

number

number

[0043] The transformed sample z i The probability distribution of is the dispersion parameter

number

[0044] As mentioned above, the sufficient statistics calculator 605 calculates the response y using a function that depends on the covariates and unbiased parameters. i The transformed sample z i The transformed sample z i The probability distribution of depends only on the dispersion parameter.

[0045] More specifically, the sufficient statistics calculator 605 performs the following process:

number

number

number

[0046] More specifically, the optimization unit 607 performs the following process:

number

number

number

[0047] The above functions and processes performed by the input unit 603, the sufficient statistics calculation unit 605, the optimization unit 607 and the output unit 609 can be described using the following mathematical symbols and equations.

[0048] First, do the following:

number

number

number

number

[0049] Recall that the set of inliers with the minimum trimmed likelihood

number

number

[0050] The proposed method assumes the following:

number

number

number

number

[0051] The second embodiment includes two main parts, a "sufficient statistics calculation part" and an "optimization part", as shown in FIG. 6, and is described as follows.

[0052] (Sufficient statistical calculation section) The process performed by the sufficient statistics calculator 605 is described as follows.

[0053] We assume that the likelihood can be described by the following form:

number

number

number

number

number

number

number

number

number

number

[0054] Next, all

number

number

number

number

number

[0055] Here, we define the number of true inliers as m0, where m is a lower bound on the number of inliers,

number

number

number

number

number

number

[0056] Instead, the posterior distribution

number

number

number

number

[0057] (Optimization section) The process performed by the optimizer 607 can be described as follows. next,

number

number

[0058] Since Z is a continuous random variable, every data point has almost certainly a different value, and as a result, the order statistics are Markovian.

number

number

number

number

number

number

number

[0059] For the express purpose of implementing

number

number

[0060]

number

number

[0061] One estimate

number

number

number

number

number

[0062] In other words, the optimization unit 607 calculates the dispersion parameter

number

[0063] (Linear regression example) Below, more specific examples of the operations and processes performed by the input component 603, the sufficient statistics calculation component 605, the optimization component 607, and the output component 609 are described.

[0064] Regression coefficient vector β and variance σ 2 Assume a standard linear regression model according to: The density of the response is defined as

[0065]

number

number

number

number

number

number

[0066] Next, we define it as follows:

[0067]

number

number

[0068] For the optimization, since this is a one-dimensional optimization problem, optimizer 607 may use either steepest descent or simple grid search.

[0069] Applying these results to the example data described in the "Problems to be Solved by the Invention" section,

number

number

number

[0070] Finally, we show how to take into account the uncertainty of β. If β is normally distributed,

number

number

number

[0071] (Beneficial Effects of the Second Embodiment) According to the information processing device 100 and the information processing method S20 of the second embodiment, an accurate estimate of the inlier probability distribution can be obtained, which enables accurate outlier detection in the data, which is important for, for example, finding malicious activity in process log data or identifying defective products in sensor data. As shown in FIG. 5, the estimated inlier probability distribution according to the second embodiment, shown by the dotted line 502, is close to the true inlier probability distribution, shown by the solid line 504.

[0072] <Third embodiment> The following description will be given in detail of the third embodiment of the present invention with reference to the drawings. It should be noted that the same reference numerals are given to elements having the same functions as those described in the first embodiment, and the description of such elements will be omitted appropriately. Furthermore, the outline of the third embodiment is the same as that of the first embodiment, and will not be described here.

[0073] (Information processing device) The third embodiment relates to an information processing device that implements a method for determining a variance parameter of a statistical model from data. Fig. 7 is a block diagram showing an information processing device according to the third embodiment. The information processing device 700 includes an input unit 702, a statistical calculation unit 704, an optimization unit 706, a p-value calculation unit 707, and a 708, outlier determining unit 710 and an output unit 712.

[0074] The input unit 702 receives input data or samples. The samples include outlier samples and inlier samples. 702 The samples received by the input unit may be observed samples. The input unit may input the received samples to the statistical calculation unit as input samples. 704 The input unit 702 is the same as the input unit of the first embodiment. 702 Since the input unit 702 executes the same process as in the above, a further detailed description of the input unit 702 will be omitted.

[0075] The statistics calculation unit 704 receives input samples from the input unit 702. As explained above, the input samples include covariates and response quantities. The statistics calculation unit 704 transforms the response quantities into transformed samples using a function that depends on the covariates and unbiased parameters. The probability distribution of the transformed samples depends only on the variance parameters. The statistics calculation unit 704 performs the same process as the statistics calculation unit 104 in the first embodiment, so a further detailed description of the statistics calculation unit 704 is omitted.

[0076] The optimizer 706 receives the transformed samples from the statistics calculator 704. The optimizer 706 maximizes the probability distribution of the transformed samples to determine an estimate of the dispersion parameter. The optimizer 706 performs the same process as the optimizer 106 of the first embodiment, so a further detailed description of the optimizer 706 is omitted.

[0077] The p-value calculation unit 708 receives the estimates of one or more dispersion parameters from the optimization unit 706. The p-value calculation unit 708 estimates one or more p-values ​​by referring to the estimates of the multiple dispersion parameters.

[0078] Although the specific example calculation process performed by the p-value calculation unit 708 may not be limited in the third embodiment, the p-value calculation unit 708 may perform the above calculation under the null hypothesis.

[0079] The outlier determination unit determines a list of outliers by referring to the p-value.

[0080] Although the specific example determination process performed by the outlier determiner 710 may not be limited to the third embodiment, the outlier determiner 710 may perform the determination by referring to a conservative estimate of the p-value for each sample.

[0081] The output unit 712 outputs the list of outliers.

[0082] (Information processing method) 8 is a flow chart showing steps of a method implemented in an information processing device in the third embodiment. S80 It has six steps.

[0083] First, the input samples are 702 (Step S82). As described above, the input sample has a response amount and a covariate.

[0084] Next, the response quantities in the input samples are statistically calculated and transformed into transformed samples (step S84). In this calculation, functions that depend only on the covariates and unbiased parameters are used. The probability distribution of the transformed samples depends only on the variance parameters.

[0085] The probability distribution of the transformed samples is maximized to determine an estimate of the dispersion parameter (step S86).

[0086] Next, the p-value is estimated with reference to the estimated value of the dispersion parameter (step S87).

[0087] A list of outliers with reference to the p-values ​​is then determined with reference to the p-values.

[0088] Finally, the list of outliers is output (step S89).

[0089] (Advantageous Effects of the Third Embodiment) The information processing device 700 and the information processing method S80 of the third embodiment can provide an accurate estimate of the inlier probability distribution, which allows accurate outlier detection in the data, which is important for, for example, finding malicious activity in process log data or identifying defective products in sensor data.

[0090] As shown in FIG. 5, the first 3 The estimated inlier distribution according to this embodiment is close to the true inlier distribution shown by the solid line 504 .

[0091] <Fourth embodiment> The following description will be given in detail of the fourth embodiment of the present invention with reference to the drawings. It should be noted that the same reference numerals are given to elements having the same functions as those described in the first embodiment, and the description of such elements will be omitted appropriately. Furthermore, the outline of the fourth embodiment is the same as that of the second embodiment, and will not be described here.

[0092] (Information processing device) 9 is a block diagram showing an information processing device according to the fourth embodiment. The information processing device 900 includes a database 901, an input unit 902, a sufficient statistics calculation unit 903, an optimization unit 904, a conservative p-value calculation unit 905, an outlier determination unit 906, and an output unit 907.

[0093] Observed data (input data) is stored in a database 901. The input data is transferred to an input section 902. As described above, the input sample has a response amount and a covariate.

[0094] The input unit 902 performs the same process as the input unit 603 in the second embodiment, so further description of the input unit 902 is omitted.

[0095] The sufficient statistics calculator 903 calculates the covariates and unbiased parameters

number

[0096] The probability distribution of the transformed samples is given by the variance parameter

number

[0097] Since the sufficient statistics calculator 903 performs the same process as the sufficient statistics calculator 605 in the second embodiment, further description of the sufficient statistics calculator 903 is omitted.

[0098] The optimizer 904 receives the probability distribution of the transformed samples from the sufficient statistics calculator 903. The optimizer 904 maximizes the probability distribution of the transformed samples to obtain an estimate of the dispersion parameter

number

[0099] Since the optimizer 904 performs the same process as the optimizer 607 in the second embodiment, further description of the optimizer 904 is omitted.

[0100] The conservative p-value calculator 905 receives one or more estimates of dispersion parameters. The p-value calculator 905 estimates a p-value with reference to the estimates of the dispersion parameters. More specifically, the conservative p-value calculator 905 executes the following process:

number

[0101] The outlier determiner 906 receives the p-value estimate from the conservative p-value calculator 905. The outlier determiner 906 determines a list of outliers by referring to the p-value. More specifically, the outlier determiner 906 executes the following process:

number

[0102] Finally, the output unit 907 outputs a list of outliers (samples for which the null hypothesis is rejected). The above operation can be explained using mathematical symbols and formulas. The database 901, input unit 902 and sufficient statistics calculation unit 903 are the same as the database 601, input unit 603 and sufficient statistics calculation unit 605 in the second embodiment, and detailed explanations will be omitted. Furthermore, the optimization unit 904 has the same function as the optimization unit 607 in the second embodiment. The difference is that the optimization unit (607, 904) estimates the number of inliers.

number

[0103] The fourth embodiment has two main parts, a "conservative p-value calculation unit 905" and an "outlier determination unit 906", in addition to the information processing device 600 of the second embodiment. Therefore, the above-mentioned "conservative p-value calculation unit 905" and "outlier determination unit 906" will be described in detail.

[0104] (Conservative p-value calculation) The process performed by the conservative p-value calculator 905 is described as follows.

[0105] Estimated value

number

number

[0106] Since outliers are expected to be at the bottom of the inlier density, a determination may be made as to whether an outlier is a sample if it has a low p-value.

number

number

number

[0107] (Outlier determination part) The outlier estimation with FDR control is now described. The process performed by the outlier determiner 906 can be described as follows:

number

number

number

[0108] In the following cases:

number

number

[0109] Similarly,

number

number

number

[0110] All samples in U are declared as outliers for which the null hypothesis is rejected and the Benjamini-Hochberg (BH) method is used, exceeding the expected value of false discovery for some nominal value α, say α=0.001. The nominal value α is input to a conservative p-value calculator 905 (not shown in FIG. 9). In the conservative p-value calculator 905, the nominal value α is used for the BH method. All other samples are considered as outliers by an outlier determiner 906. If the p-value is set to an upper bound

number

[0111] As described above, the conservative p-value calculation unit 905

number

number

[0112] (Advantageous Effects of the Fourth Embodiment)

number

number

number

[0113] According to the information processing device 900 of the fourth embodiment, an accurate estimate of the inlier probability distribution can be obtained, which enables accurate outlier detection in data, which is important for, for example, finding malicious activity in process log data or identifying defective products in sensor data.

[0114] (Example of configuration achieved by software) One or some or all of the functions of the information processing devices 100, 600, 700 and 900 may be realized by hardware, for example an integrated circuit (IC chip), or alternatively by software.

[0115] In the case described below, each of the information processing devices 100, 600, 700, and 900 can be realized by a computer that executes program instructions of the above-mentioned functions that are realized by software, for example. FIG. 10 shows an example of such a computer (hereinafter referred to as computer C). Computer C includes at least one processor C1 and at least one memory C2. Memory C2 stores a program P for causing any of the information processing devices 100, 600, 700, and 900 to function as computer C. In computer C, processor C1 reads program P from memory C2 and executes program P to realize the function of any of the information processing devices 100, 600, 700, and 900.

[0116] The processor C1 may be, for example, a Central Processing Unit (CPU), a Graphic Processing Unit (GPU), a Digital Signal Processor (DSP), a Micro Processing Unit (MPU), a Floating point number Processing Unit (FPU), a Physics Processing Unit (PPU), a microcontroller, or a combination of these. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination of these.

[0117] The computer C may further include a Random Access Memory (RAM) for expanding the program P during execution and for temporarily storing various data. The computer C may further include a communication interface for transmitting and receiving data to and from other devices. The computer C may further include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.

[0118] The program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can obtain the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also obtain the program P via such a transmission medium.

[0119] It should be understood that the above description is not limited to the described preferred embodiment of the present invention. Various alternatives and modifications conceivable by those skilled in the art are possible without departing from the present invention. Therefore, the present invention is intended to embrace all alternatives, modifications and variations that fall within the scope of the above disclosure.

[0120] [Additional Note 1] The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiment are also included in the technical scope of the present invention.

[0121] [Additional Note 2] Some or all of the above-described embodiments can be described as follows. However, the present invention is not limited to the following described aspects.

[0122] [Additional information 1] The aspects of the present invention can be expressed as follows: (Aspect 1) an input means for receiving a plurality of input samples including a plurality of responses and a plurality of covariates; a statistical computation means for transforming the response quantities into a plurality of transformed samples using a function that depends on the covariates and an unbiased parameter, such that the distribution of the transformed samples depends only on a variance parameter; optimization means for maximizing the probability distribution of the transformed samples to determine an estimate of the variance parameter; An information processing device comprising:

[0123] According to the above configuration, it is possible to provide a technique suitable for estimating a dispersion parameter.

[0124] (Aspect 2) The information processing device according to aspect 1 calculates the converted sample using the following formula:

number

number

[0125] According to the above configuration, it is possible to provide a technique suitable for estimating a dispersion parameter.

[0126] (Aspect 3) In the information processing device according to the first or second aspect, an unbiased estimate

number

number

[0127] The above arrangement makes it possible to provide a technique suitable for variance parameter estimation using unbiased estimates of the parameters.

[0128] (Aspect 4) In the information processing device according to aspect 1 or 2, the statistical calculation means posterior distribution

number

number

number

[0129] The above arrangement makes it possible to provide a suitable technique for variance parameter estimation by integral elimination of unbiased parameters.

[0130] (Aspect 5) In the information processing device according to any one of aspects 1 to 4, the optimization means determines an estimate of the dispersion parameter based on a weighted average of a plurality of dispersion parameters, each of the plurality of dispersion parameters being based on a respective estimate of the number of inliers.

[0131] According to the above configuration, it is possible to provide a technique suitable for estimating a dispersion parameter.

[0132] (Aspect 6) The information processing device according to any one of aspects 1 to 5 further comprises an output means for outputting the estimated value of the dispersion parameter.

[0133] (Aspect 7) an input means for receiving a plurality of input samples including a plurality of responses and a plurality of covariates; a statistical calculation means for transforming the response quantities into a plurality of transformed samples using a function that depends on the covariates and an unbiased parameter, such that the distribution of the transformed samples depends only on a variance parameter; optimization means for maximizing the probability distribution of the transformed samples to determine an estimate of the variance parameter; A p-value calculation means for estimating a p-value by referring to the estimated value of the dispersion parameter; an outlier determination means for determining a list of outliers by referring to the p-value; An information processing device comprising:

[0134] According to the above-mentioned configuration, it is possible to provide a technique suitable for estimating a dispersion parameter. Also, according to the above-mentioned configuration, it is possible to provide a list of outliers.

[0135] (Aspect 8) In the information processing device of aspect 7, the p-value calculation means determines a conservative estimate of the p-value for each sample so as to find an estimate of the number of inliers such that the resulting estimate of the dispersion parameter leads to the highest p-value for each sample.

[0136] The above arrangement makes it possible to provide an estimate of the number of inliers.

[0137] (Aspect 9) The information processing device according to aspect 7 or 8 further comprises an output means for outputting the list of outliers.

[0138] (Aspect 10) A plurality of input samples are received, the input samples comprising a plurality of responses and a plurality of covariates; transforming the response quantities into a plurality of transformed samples using a function that depends on the covariates and an unbiased parameter, such that the distribution of the transformed samples depends only on a variance parameter; maximizing the probability distribution of the transformed samples to determine an estimate of the variance parameter; Information processing methods.

[0139] (Aspect 11) A plurality of input samples are received, the input samples comprising a plurality of responses and a plurality of covariates; transforming the response quantities into a plurality of transformed samples using a function that depends on the covariates and an unbiased parameter, such that the distribution of the transformed samples depends only on a variance parameter; maximizing a probability distribution of the transformed samples to determine an estimate of the dispersion parameter; estimating a p-value with reference to the estimated value of the variance parameter; determining a list of outliers with reference to said p-value; Information processing methods.

[0140] (Aspect 12) A control program for causing a computer to function as a host for the information processing device according to aspect 1, the control program being configured to cause the information processing device to function as an input unit, a statistical value calculation unit, and an optimization unit.

[0141] (Aspect 13) A control program that causes a computer to function as a host for the information processing device of claim 7, and is configured to cause the information processing device to function as an input means, a statistical value calculation means, an optimization means, a p-value calculation means, and an outlier determination means.

[0142] (Aspect 14) A non-transitory storage medium storing the control program of aspect 12 or 13.

[0143] (Aspect 15) The information processing device has at least one processor, A plurality of input samples are received, the input samples comprising a plurality of responses and a plurality of covariates; transforming the response quantities into a plurality of transformed samples using a function that depends on the covariates and an unbiased parameter, such that the distribution of the transformed samples depends only on a variance parameter; The probability distribution of the transformed samples is maximized to determine an estimate of the dispersion parameter.

[0144] (Aspect 16) The information processing device has at least one processor, A plurality of input samples are received, the input samples comprising a plurality of responses and a plurality of covariates; transforming the response quantities into a plurality of transformed samples using a function that depends on the covariates and an unbiased parameter, such that the distribution of the transformed samples depends only on a variance parameter; maximizing a probability distribution of the transformed samples to determine an estimate of the dispersion parameter; estimating a p-value with reference to the estimated value of the variance parameter; The p-value is referenced to determine a list of outliers.

[0145] [Additional information 2] The present invention can also be expressed as follows: (Aspect A1) Set of inlier samples

number

number

number

number

number

number

number

number

number

number

[0146] (Aspect A2) In aspect A1, the parameter vector

number

number

[0147] (Aspect A3) In aspect A1, Inlier

number

number

number

number

number

[0148] (Aspect A4) In embodiment A1, determining a conservative estimate of the p-value for each sample;

number

number

[0149] 100, 600, 700, 900 Information processing equipment 601, 901 Database 102, 603, 702, 902 Input section 104, 605, 704, 903 Statistical calculation section 106, 607, 706, 904 Optimization Department 708, 905 p-value calculation section 710, 906 Outlier determination unit S20, S80 information processing method S22, S82 Input Step S24, S84 statistical calculation steps S26, S86 Optimization steps S87 p-value calculation step S89 Outlier determination step

Claims

1. an input means for receiving a plurality of input samples including a plurality of responses yi and a plurality of covariates xi; A statistical calculation means for converting the response quantity yi into a plurality of transformed samples zi using a function that takes the covariate xi as an argument and depends on a parameter ψ, such that the distribution of the transformed samples zi depends only on a variance parameter v; an optimization means for determining an estimate of the variance parameter v by maximizing the likelihood for the distribution of the transformed samples zi; An information processing device comprising:

2. The statistical calculation means calculates the transformed samples using the formula: [0010] Here z i denotes the transformed sample, and y i indicates the amount of response, [0025] is an estimate of the parameter ψ in a function that depends on the parameter ψ. [0030] , the parameter [0045] The function used is x i indicates the covariate The information processing device according to claim 1 .

3. The statistical calculation means includes: posterior distribution [0050] Using the likelihood function [006] From the above parameters [0070] Integrate and eliminate 3. The information processing device according to claim 1 or 2.

4. the optimization means determines an estimate of a dispersion parameter based on a weighted average of a plurality of dispersion parameters, each of the plurality of dispersion parameters being based on a respective estimate of a number of inliers; The information processing device according to claim 1 .

5. and outputting the estimated value of the dispersion parameter. The information processing device according to claim 1 .

6. an input means for receiving a plurality of input samples including a plurality of responses yi and a plurality of covariates xi; A statistical value calculation means for converting the response quantity yi into a plurality of transformed samples zi using a function that takes the covariate xi as an argument and depends on a parameter ψ, such that the distribution of the transformed samples zi depends only on a variance parameter v; an optimization means for determining an estimate of the variance parameter v by maximizing the likelihood for the distribution of the transformed samples zi; p-value calculation means for estimating a p-value by referring to the estimated value of the dispersion parameter v; an outlier determining means for determining a list of outliers by referring to the p-value; An information processing device comprising:

7. The method further includes outputting the list of outliers. The information processing device according to claim 6.

8. One or more processors, It receives a number of input samples, each of which includes a number of responses yi and a number of covariates xi, Transform the response quantities yi into a plurality of transformed samples zi using a function that takes the covariates xi as arguments and depends on a parameter ψ, such that the distribution of the transformed samples zi depends only on a variance parameter v; determining an estimate of the variance parameter v by maximizing the likelihood for the distribution of the transformed samples zi; Information processing methods.

Images