DISTRIBUTION ESTIMATION DEVICE, DISTRIBUTION ESTIMATION METHOD, AND SYSTEM

Abstract

To provide a distribution estimation device capable of achieving suppression of accuracy deterioration due to tied data when applying a normal copula to excessive zero-values multivariate data and capable of supporting a high, accurate data distribution estimation.SOLUTION: A distribution estimation device 11 comprises a storage device to hold processing target data consisting of multiple variables and a CPU 111, where the CPU executes the processing to calculate a zero data ratio of each variable for the data, the processing to estimate a non-zero data distribution of each variable, the processing to convert each variable to a normal distribution, the processing to calculate the threshold of each variable after the conversion, and the processing to estimate the parameters representing a correlation between the variables after the conversion based on a likelihood function defined by integrating the probability density function of the variable normal distribution below the threshold.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a distribution estimation device, a distribution estimation method, and a system. [Background technology]

[0002] In recent years, progress has been made in building systems that support human decision-making in a variety of fields through statistical analysis and machine learning using past data. In such cases, data analysts carefully examine past data to identify trends and then use techniques such as machine learning to extract knowledge that is useful for decision-making. Among these technologies, data distribution estimation technology is a fundamental technology. On the other hand, real data is generally multivariate and correlated with each other. Methods such as kernel density estimation, Gaussian mixture method, and copula are known as methods for mathematically modeling the distribution of multivariate data.

[0003] In addition, there are cases where there are many records with some variables whose values ​​are 0. Examples of such variables are "spouse's annual income" and "yesterday's drug dosage." A method is needed to accurately model such multivariate zero-value excess data, but this is difficult to achieve with the conventional methods listed above. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Multivariate Zero-Inflated Poisson Models and Their Applications,Chin-Shang Li, Jye-Chyi Lu, Jinho Park, Kyungmoo Kim, Paul A. Brinkley and John P. Peterson,Technometrics Vol. 41, No. 1 (Feb., 1999), pp. 29-38 (10 pages)Published By: Taylor & Francis, Ltd. [Non-Patent Document 2] Zero-Inflated Poisson Regression, with an Application to Defects in Manufacturing,Diane Lambert Technometrics Vol. 34, No. 1 (Feb., 1992), pp. 1-14 (14 pages) Published By: Taylor & Francis, Ltd. Summary of the Invention [Problem to be solved by the invention]

[0005] For zero-valued variables, the data distribution is generally multimodal, with the distribution peak at the zero values ​​being infinitely thin and high (see, for example, FIG. 20). For this reason, it is necessary to estimate the likelihood value to be assigned to zero values ​​and the distribution that non-zero data follows separately, and to switch the likelihood to be assigned to new data depending on whether the value is 0 or not. In zero-value excess multivariate data with d zero-value excess variables, the likelihood of assigning zero to new data is calculated by multiplying the likelihood by 2^ d Therefore, in a multivariate situation, In this case, it becomes difficult to estimate the distribution.

[0006] It is also possible to estimate using conventional kernel density estimation methods, Gaussian mixture methods, copulas, etc., without determining whether each variable is zero or not. However, since the estimates using these methods are pulled by the distribution peak at zero values, for example, when "spouse's annual income" is 1 yen, Such data will be given a high likelihood, reducing the accuracy of the estimation.

[0007] Such zero-value excess multivariate data can be handled as follows by combining known techniques. That is, the likelihood value for zero values ​​and the distribution that non-zero data follows are estimated for each variable, and the correlation between variables is retroactively determined by the normal copula density. This is a method.

[0008] This allows us to capture the correlation between variables while also determining which variables are 0. d This makes it possible to estimate distributions without dividing the variables into three cases. The normal copula is a method of expressing the correlation between variables using a multivariate normal distribution after converting each variable into a variable that follows a normal distribution using rank transformation. In this case, the normal copula density refers to the factor that expresses the correlation.

[0009] That is, in the Gaussian copula method, the joint distribution of multivariate variables is expressed as the product of the marginal distributions multiplied by the value of the copula density.

[0010] However, for multivariate data with an excess of zero values, the rank transformation required for applying the Gaussian copula cannot make the data conform to a normal distribution due to the presence of a large amount of tied data, resulting in a decrease in the accuracy of estimating the parameters of the Gaussian copula.

[0011] Therefore, an object of the present invention is to provide a technology that prevents a decrease in accuracy due to tied data when applying a Gaussian copula to zero-value excess multivariate data, and supports highly accurate data distribution estimation. [Means for solving the problem]

[0012] The distribution estimation device of the present invention, which solves the above problem, includes a storage device for storing data to be processed that consists of a plurality of variables, and a calculation unit for calculating a zero data ratio for each variable of the data. a variable having a zero-data ratio greater than a predetermined threshold is determined to be a zero-excess variable, and a list of the zero-data ratio and the zero-excess variables is recorded in a distribution learning result storage unit. Processing 、 Non-zero data for each of the above variables and recording a result of the distribution learning in the storage unit. Each of the above variables By rank transformation normal distribution For variables that follow conversion and recording the result of the conversion in the storage unit. The process of converting each variable normal distribution Calculate threshold and records the result of the calculation in the storage unit. The process of and a process of learning parameters of a Gaussian copula density and recording the learning result in the storage unit; a process of acquiring a zero data ratio of a variable that was zero data among data to be subjected to likelihood calculation from the storage unit and setting the zero data ratio as a marginal likelihood of the zero variable; a process of acquiring a learning result of a nonzero data distribution of a variable that was nonzero data among the data from the storage unit and calculating a marginal likelihood of the nonzero variable using the learning result; a process of acquiring a result of the conversion of each of the variables from the storage unit and performing a normalized distribution conversion on each of the variables using the result; a process of acquiring a learning result of parameters of a Gaussian copula density from the storage unit and calculating a Gaussian copula density using the learning result; and a likelihood calculation unit that calculates a total likelihood by multiplying the marginal likelihood of the zero variable, the marginal likelihood of the nonzero variable, and the value of the copula density. The present invention is characterized by comprising: a calculation device. In addition, the distribution estimation method of the present invention includes an information processing device that holds data to be processed that is made up of a plurality of variables and calculates a zero data ratio for each variable of the data. a variable having a zero-data ratio greater than a predetermined threshold is determined to be a zero-excess variable, and a list of the zero-data ratio and the zero-excess variables is recorded in a distribution learning result storage unit. Processing 、 Non-zero data for each of the above variables and recording a result of the distribution learning in the storage unit. Each of the above variables By rank transformation normal distribution For variables that follow conversion and recording the result of the conversion in the storage unit. The process of converting each variable normal distribution Calculate threshold and records the result of the calculation in the storage unit. The process of a series of processes including a process of learning parameters of a normal copula density and a process of recording the learning result in the storage unit; a process of acquiring a zero data ratio of a variable that was zero data among data to be subjected to likelihood calculation from the storage unit, and setting the zero data ratio as a marginal likelihood of the zero variable; a process of acquiring a learning result of a nonzero data distribution of a variable that was nonzero data among the data from the storage unit, and calculating a marginal likelihood of the nonzero variable using the learning result; a process of acquiring a result of the conversion of each of the variables from the storage unit, and performing a normalized distribution conversion on each of the variables using the result; a process of acquiring a learning result of parameters of a normal copula density from the storage unit, and calculating a normal copula density using the learning result; and a process of calculating a total likelihood by multiplying the marginal likelihood of the zero variable and the marginal likelihood of the nonzero variable by the value of the copula density. The present invention is characterized by carrying out the following steps.

[0013] The system of the present invention includes a storage device for storing data to be processed that is made up of a plurality of variables, and a method for calculating a zero data ratio for each variable of the data. a variable having a zero-data ratio greater than a predetermined threshold is determined to be a zero-excess variable, and a list of the zero-data ratio and the zero-excess variables is recorded in a distribution learning result storage unit. Processing 、 Non-zero data for each of the above variables and recording a result of the distribution learning in the storage unit. Each of the above variables By rank transformation normal distribution For variables that follow conversion and recording the result of the conversion in the storage unit. The process of converting each variable normal distribution Calculate threshold and records the result of the calculation in the storage unit. The process of and a process of learning parameters of a Gaussian copula density and recording the learning result in the storage unit; a process of acquiring a zero data ratio of a variable that was zero data among data to be subjected to likelihood calculation from the storage unit and setting the zero data ratio as a marginal likelihood of the zero variable; a process of acquiring a learning result of a nonzero data distribution of a variable that was nonzero data among the data from the storage unit and calculating a marginal likelihood of the nonzero variable using the learning result; a process of acquiring a result of the conversion of each of the variables from the storage unit and performing a normalized distribution conversion on each of the variables using the result; a process of acquiring a learning result of parameters of a Gaussian copula density from the storage unit and calculating a Gaussian copula density using the learning result; and a likelihood calculation unit that calculates a total likelihood by multiplying the marginal likelihood of the zero variable, the marginal likelihood of the nonzero variable, and the value of the copula density. The distribution estimation device includes a calculation device.

[0014] The system of this embodiment may also include an anomaly detection system that uses the likelihood calculated as described above to detect anomalies. Effect of the Invention

[0015] According to the present invention, it is possible to suppress the decrease in accuracy due to tied data when applying a Gaussian copula to zero-value excess multivariate data, thereby supporting highly accurate data distribution estimation. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a system configuration diagram of a computer system according to a first embodiment. [Diagram 2] FIG. 4 is a sequence diagram illustrating an overall process flow during data distribution learning in the first embodiment. [Diagram 3] FIG. 11 is a sequence diagram illustrating an overall process flow when calculating a likelihood in the first embodiment. [Figure 4] FIG. 11 is a sequence diagram showing the overall processing flow when outputting inference reliability in the first embodiment. [Diagram 5] FIG. 11 is a process flow diagram for acquiring data and distribution learning conditions during distribution learning in the first embodiment. [Figure 6] FIG. 11 is a process flow diagram for identifying excessive variables with zero value data and calculating a zero value ratio during distribution learning in the first embodiment. [Figure 7] FIG. 13 is a process flow diagram for estimating, for each variable, the marginal distribution to which non-zero data conforms during distribution learning in the first embodiment. [Figure 8] FIG. 11 is a process flow diagram for estimating copula parameters during distribution learning in the first embodiment. [Figure 9] FIG. 11 is a process flow diagram for outputting a result of distribution learning during distribution learning in the first embodiment. [Figure 10] FIG. 13 is a process flow diagram for calculating marginal likelihood for each variable when calculating likelihood in the first embodiment. [Figure 11] FIG. 11 is a process flow diagram for calculating a copula density value when calculating a likelihood in the first embodiment. [Figure 12] FIG. 11 is a process flow diagram for tallying and outputting marginal likelihood and copula density values ​​when calculating likelihoods in the first embodiment. [Figure 13]13 is a process flow diagram for calculating inference reliability when outputting inference reliability in the first embodiment. FIG. [Figure 14] 13 is a processing flow diagram for issuing a warning according to the value of inference reliability when outputting inference reliability in the first embodiment. FIG. [Figure 15] FIG. 4 is a schematic diagram showing an example of input data in the first embodiment. [Figure 16A] FIG. 13 is a schematic diagram showing an example of a distribution learning result file group in the first embodiment. [Figure 16B] FIG. 13 is a schematic diagram showing an example of a distribution learning result file group in the first embodiment. [Figure 16C] FIG. 13 is a schematic diagram showing an example of a distribution learning result file group in the first embodiment. [Figure 16D] FIG. 13 is a schematic diagram showing an example of a distribution learning result file group in the first embodiment. [Figure 16E] FIG. 13 is a schematic diagram showing an example of a distribution learning result file group in the first embodiment. [Figure 16F] FIG. 13 is a schematic diagram showing an example of a distribution learning result file group in the first embodiment. [Figure 17A] FIG. 13 is a schematic diagram showing an example of an input / output screen during distribution learning in the first embodiment. [Figure 17B] FIG. 13 is a schematic diagram showing an example of an input / output screen during distribution learning in the first embodiment. [Figure 18A] 1 is a schematic diagram showing an example of an input / output screen at the time of calculation of inference trust in the first embodiment. FIG. [Figure 18B] 1 is a schematic diagram showing an example of an input / output screen at the time of calculation of inference trust in the first embodiment. FIG. [Figure 19] FIG. 11 is a system configuration diagram of a computer system according to a second embodiment. [Figure 20] FIG. 13 is a diagram showing an example of distribution of 0 excess data. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] [Example 1] <System configuration> A first embodiment of the present invention will be described in detail below with reference to the drawings. Fig. 1 is a diagram showing an example of the configuration of a computer system, which is a distribution estimation system including a distribution estimation device 11 in the first embodiment.

[0018] The computer system 1 in the first embodiment is composed of a distribution estimation device 11 and a user terminal 12. These are connected to each other via an appropriate network 10 so that they can communicate with each other.

[0019] Next, we will explain each device that constitutes the computer system 1. Among these, the distribution estimation device 11 performs learning of data distribution, calculation of likelihood, and output of inference reliability using the likelihood, etc. A general-purpose server device is used for this distribution estimation device 11, and has a CPU 111, a memory 112, a storage device 113, and a communication device 114.

[0020] Of these, the CPU 111 is a central processing unit (CPU) that executes a program held in the memory 112 to perform overall control of the apparatus itself, as well as various types of judgment, calculation, and control processing.

[0021] The memory 112 is composed of a volatile storage element such as a RAM (Random Access Memory). The memory 112 includes a machine learning unit 1121, an inference reliability output unit 1122, a distribution learning unit 1123, and a likelihood calculation unit 1124. These functional units are implemented by the CPU 111 executing corresponding programs, and perform learning of data distribution, calculation of likelihood, output of inference reliability using the likelihood, and recording of these processes.

[0022] The distribution learning unit 1121 consists of a data acquisition program 11211, a zero data learning program 11212, a non-zero data learning program 11213, a copula density learning program 11214, and an output program 11215, and provides the function of learning the joint distribution followed by zero-excessive multivariate learning data and recording the results.

[0023] Of the above, the data acquisition program 11211 receives data for distribution learning and a zero ratio threshold (for example, from the user terminal 12) and temporarily stores them.

[0024] The zero-data learning program 11212 calculates and records the zero-value data ratio of each variable, and lists and records variables whose zero-value data ratio is greater than the zero ratio threshold, that is, zero-excess variables.

[0025] In addition, the non-zero data learning program 11213 estimates and records the univariate distribution of non-zero data that is not zero for each variable.

[0026] The copula density learning program 11214 also performs a conversion to convert each variable into a normal distribution, calculates a normal distribution threshold, and estimates and records the parameters of the normal copula by maximizing a likelihood function defined as an integral below the normal distribution threshold for each variable pair.

[0027] The output program 11215 outputs the data distribution learning result to the user terminal 12.

[0028] On the other hand, the likelihood calculation unit 1122 is made up of a marginal likelihood calculation program 11221, a copula density calculation program 11222, and an output program 11223, and calculates and outputs the likelihood of new data based on the distribution learning content.

[0029] The marginal likelihood calculation program 11221 uses the learned zero value ratio and non-zero data distribution to calculate and temporarily store the marginal likelihood value for each variable of the data to be subjected to likelihood calculation.

[0030] In addition, the copula density calculation program 11222 uses the parameters of the learned Gaussian copula to calculate and temporarily store the value of the copula density for the data to be subjected to the likelihood calculation.

[0031] The output program 11223 calculates and outputs a total likelihood by multiplying the marginal likelihood value and the copula density value.

[0032] The machine learning unit 1123 also includes a model prediction program 11231 and an output program 11232 .

[0033] The model prediction program 11231 accepts data (for example, from the user terminal 12) and performs some kind of prediction using a trained model 11311 described below.

[0034] In addition, the output program 11232 outputs the above-mentioned prediction result to the user terminal 12.

[0035] The inference reliability output unit 1124 comprises an inference reliability calculation program 11241 and a warning issuing program 11242 .

[0036] Of these, the inference reliability calculation program 11241 obtains data to be used for calculating the inference reliability (for example, from the user terminal 12), inputs the data to the likelihood calculation unit 1122 to obtain a total likelihood, and calculates and temporarily stores the inference reliability based on this total likelihood.

[0037] Moreover, the warning issuing program 11242 acquires a predetermined threshold value (for example, from the user terminal 12), and if the inference reliability is smaller than the threshold value, issues a warning to that effect to, for example, the user terminal 12.

[0038] The storage device 113 is configured with an appropriate non-volatile storage element such as a solid state drive (SSD), a hard disk drive, etc. The storage device 113 includes a prediction model storage unit 1131 and a distribution learning result storage unit 1132.

[0039] Of these, the prediction model storage unit 1131 stores a trained prediction model 11311 and provides it in response to a request from the machine learning unit 1123.

[0040] Moreover, the distribution learning result storage unit 1132 stores the distribution learning result 11321, which is the learning result of the distribution learning unit 1121, and provides it in response to a request from the likelihood calculation unit 1122.

[0041] On the other hand, the user terminal 12 is a terminal that performs input and output to the distribution estimation device 11. This user terminal 12 transmits and receives information to and from the distribution estimation device 11, and is a terminal that is used, for example, by a person in charge of distribution estimation work when using a service provided by the distribution estimation device 11. Specifically, a smartphone, a tablet terminal, a personal computer, etc. can be envisioned.

[0042] The communication device 114 also communicates with the user terminal 12 via the network 10. The communication device 114 is assumed to be a network interface card or the like. <Flow example: Overall processing flow during learning> The actual procedure of the distribution estimation method in the first embodiment will be described below with reference to FIG. 2. Various operations corresponding to the distribution estimation method described below are realized by programs that the distribution estimation device 11 and the user terminal 12 read into memory or the like and execute. The program is made up of code for performing various operations as described below.

[0043] 2 is an overall process flow for learning data distribution. In this case, first, user terminal 12 inputs data and a zero ratio threshold to distribution learning unit 1121 of distribution estimation device 11 (S101).

[0044] Next, the distribution learning unit 1121 of the distribution estimation device 11 calculates the zero data ratio of each variable ( S102), a variable whose zero data ratio is greater than the zero ratio threshold is determined to be a zero-excess variable. , and list them (S103). Furthermore, the distribution learning unit 1121 records the zero-excess variable list and the zero ratio in the distribution learning result storage unit 1132 (S104, S105).

[0045] Next, distribution learning unit 1121 executes preprocessing of the data (S106), and records the contents of the preprocessing in distribution learning result storage unit 1132 (S107).

[0046] Next, distribution learning unit 1121 executes distribution learning of the non-zero data (S108), and records the distribution learning result in distribution learning result storage unit 1132 (S109).

[0047] Next, the distribution learning unit 1121 converts each variable into a variable that follows a normal distribution by rank transformation (S110). At this time, which value of each variable was converted into which value, that is, the normal distribution transformation learning result, is recorded in the distribution learning result storage unit 1132 (S111).

[0048] Next, the distribution learning unit 1121 calculates the normal distribution threshold of each variable after normal distribution conversion (S1 12), and record the result in the distribution learning result storage unit 1132 (S113).

[0049] Next, distribution learning unit 1121 learns the parameters of the copula (S114), and records the results in distribution learning result storage unit 1132 (S115).

[0050] Finally, distribution learning unit 1121 displays the learning result on user terminal 12 (S116), and ends the process.

[0051] By doing the above, the user can train a mathematical model that represents the joint distribution of zero-surplus multivariate data. <Flow example: Overall processing flow when calculating likelihood> Next, the overall process flow when calculating the likelihood will be described with reference to Fig. 3. In this case, the user terminal 12 transmits one or more pieces of data to be subjected to likelihood calculation to the likelihood calculation unit 1122 of the distribution estimation device 11 (S201).

[0052] Next, the likelihood calculation unit 1122 determines which variables in the data transmitted from the user terminal 12 have a zero value (S202).

[0053] Next, the likelihood calculation unit 1122 acquires the zero data ratio of the variable that was a zero value as a result of the above-mentioned determination from the distribution learning result storage unit 1132 (S203), and sets this as the marginal likelihood of the zero variable (S204).

[0054] Next, the likelihood calculation unit 1122 obtains the pre-processing results of the variables that were non-zero values ​​from the distribution learning result storage unit 1132 (S205), and uses these to perform pre-processing of the non-zero value variables (S206).

[0055] Next, the likelihood calculation unit 1122 obtains the learning result of the non-zero data distribution from the distribution learning result storage unit 1132 (S207), and uses this to calculate the marginal likelihood of the non-zero value variable (S 208).

[0056] Next, the likelihood calculation unit 1122 acquires the learning results of the normal distribution transformation of each variable from the distribution learning result storage unit 1132 (S209), and uses them to perform normal distribution transformation on each variable ( S210).

[0057] Next, the likelihood calculation unit 1122 acquires the parameters of the Gaussian copula from the distribution learning result storage unit 1132 (S211), and uses them to calculate the value of the copula density (S212).

[0058] Finally, the total likelihood is calculated by multiplying the marginal likelihood and the copula density (S21 3), the total likelihood value is output to the user terminal 12 (S214), and the process ends.

[0059] In this manner, the user can obtain the likelihood value of the data that is the subject of the likelihood calculation. <Flow example: Overall process flow when calculating inference reliability> Next, the overall process flow when outputting the inference reliability will be described with reference to Fig. 4. In this case, the user terminal 12 transmits one or more pieces of data to be used for calculating the inference reliability to the machine learning unit 1123 of the distribution estimation device 11 (S301).

[0060] Next, the machine learning unit 1123 of the distribution estimation device 11 acquires the trained prediction model from the prediction model storage unit 1131 (S302), and executes inference on the data for which the inference reliability is to be calculated (S303). In addition, the machine learning unit 1123 outputs the inference result in S303 to the user terminal 12 (S304). Furthermore, the user terminal 12 transmits one or more pieces of data to be used for calculating the inference reliability and a reliability threshold to the inference reliability output unit 1124 of the distribution estimation device 11 (S305). Note that this step S305 may be performed simultaneously with step S301.

[0061] Next, the inference reliability output unit 1124 inputs the inference reliability calculation target data received from the user terminal 12 to the likelihood calculation unit 1122 (S306), and obtains the total likelihood (S307).

[0062] Next, the inference reliability output unit 1124 calculates the inference reliability based on the total likelihood received from the likelihood calculation unit 1122 (S308).

[0063] Furthermore, the inference reliability output unit 1124 judges whether or not to issue a warning depending on whether the inference reliability obtained in S308 is below the reliability threshold (S309). If the result of this judgment is that the inference reliability is below the reliability threshold, that is, if it is determined that a warning needs to be issued, the inference reliability output unit 1124 outputs the inference reliability and a warning to the user terminal 12 (S310), and ends the process.

[0064] This allows users to obtain the reliability of inferences made by machine learning models along with their results, enabling more flexible decision-making. <Flow example: Detailed processing flow during distribution learning> A detailed flow of distribution learning will be described below. First, a process flow of the data acquisition program 11211 when acquiring learning data during distribution learning will be described in detail with reference to Fig. 5. That is, the details of S101 in Fig. 2 will be described.

[0065] In this case, the distribution learning unit 1121 (its data acquisition program 11211) acquires learning data and a zero ratio threshold value from the user terminal 12 (S401).

[0066] Next, the distribution learning unit 1121 (its data acquisition program 11211) temporarily stores the data and the zero ratio threshold value obtained in S401 in an appropriate free area of ​​the memory 112 (S402), End processing.

[0067] The data temporarily stored as described above is used in distribution learning unit 1121 (the flow of which will be described later with reference to FIG. 6 to FIG. 9). The temporarily stored zero ratio threshold is used in zero data learning program 11212 of distribution learning unit 1121 (the flow of which will be described later with reference to FIG. 6).

[0068] Next, a process flow in the zero data learning program 11212 when learning zero data during distribution learning will be described in detail with reference to Fig. 6. That is, details of S102 to S105 in Fig. 2 will be described.

[0069] In this case, the zero-data learning program 11212 of the distribution learning unit 1121 repeats the following steps S502 to S505 for all of the input variables (S501).

[0070] The zero data learning program 11212 of the distribution learning unit 1121 selects one of the unprocessed input variables, and calculates the zero value data ratio of that variable using the temporarily stored learning data (S502).

[0071] Next, the zero data learning program 11212 of the distribution learning unit 1121 determines whether the zero value data ratio calculated in S502 is greater than the temporarily stored zero ratio threshold value (S503).

[0072] If the result of the above judgment is that the calculated zero value data ratio is greater than the temporarily stored zero ratio threshold (S503: YES), the zero data learning program 11212 of the distribution learning unit 1121 adds the selected variable to a zero excess variable list (S504) and adds the zero value data ratio to a zero value ratio table (S505).

[0073] When processing has been completed for all input variables (S501: YES), the zero-data learning program 11212 of the distribution learning unit 1121 records the zero-excess variable list and the zero value ratio table in the distribution learning result storage unit 1132 (S506), and ends the processing.

[0074] Next, a process flow in the non-zero data learning program 11213 when learning non-zero data during distribution learning will be described in detail with reference to Fig. 7. That is, the details of S106 to S109 in Fig. 2 will be described.

[0075] In this case, the non-zero data learning program 11213 of the distribution learning unit 1121 repeats the following steps S602 to S604 for all of the input variables (S601).

[0076] Next, the non-zero data learning program 11213 of the distribution learning unit 1121 selects one unprocessed input variable and performs preprocessing (S602). Here, preprocessing includes, but is not limited to, a process of unifying the scale of variables using the maximum and minimum values, or the average and standard deviation, of each variable.

[0077] Next, the nonzero data learning program 11213 of the distribution learning unit 1121 executes estimation of the nonzero data distribution for the input variables selected in S602 using the preprocessed nonzero data (S603). Any method for estimating data distribution, such as the kernel density estimation method or the Gaussian mixture method, may be used for the estimation here.

[0078] Finally, the non-zero data learning program 11213 of the distribution learning unit 1121 The results of preprocessing such as small values ​​and the estimation result of the non-zero data distribution are recorded in the distribution learning result storage unit 1132 (S604), and the process ends.

[0079] Next, a process flow of the copula density learning program 11214 when learning parameters of the normal copula density during distribution learning will be described in detail with reference to Fig. 8. That is, the details of S110 to S115 in Fig. 2 will be described.

[0080] In this case, the copula density learning program 11214 of the distribution learning unit 1121 repeats the following steps S702 to S705 for all of the input variables (S701).

[0081] Next, the copula density learning program 11214 of the distribution learning unit 1121 selects one of the unprocessed input variables and performs normal distribution transformation (S702).

[0082] The normal distribution transformation is a known data transformation method also known as RankGauss. The data is transformed into a rank order from smallest to largest, and then divided by the number of data points. This transforms the minimum value to 0 and the maximum value to 1, resulting in data with equally spaced values. Next, the inverse of the cumulative distribution function of the standard normal distribution is applied to this variable. This transforms the minimum value to negative infinity, the maximum value to positive infinity, and the median to 0, resulting in the distribution of the data becoming a standard normal distribution.

[0083] To stabilize numerical processing, positive infinity and negative infinity may be rounded to a finite value. If the data contains many zero values, there will be many ties with the minimum value of zero, so the data will follow a truncated standard normal distribution in which data points smaller than a certain value in the standard normal distribution are gathered at negative infinity. This threshold is the normal distribution threshold described below.

[0084] Next, the copula density learning program 11214 of the distribution learning unit 1121 determines a normal distribution threshold for the variable (S703).

[0085] Next, the copula density learning program 11214 of the distribution learning unit 1121 temporarily stores the converted data and the normal distribution threshold in an appropriate free area of ​​the memory 112 (S704).

[0086] Next, the copula density learning program 11214 of the distribution learning unit 1121 records the normal distribution conversion content and the normal distribution threshold in the distribution learning result storage unit 1132 (S705). Note that the normal distribution conversion content is a function of which original value is converted to which value. As long as a similar conversion can be reproduced, the form is not important.

[0087] Next, the copula density learning program 11214 of the distribution learning unit 1121 repeats the following steps S707 and S708 for all pairs of input variables (S706). In other words, if there are d types of input variables, the process is repeated d*(d-1) / 2 times.

[0088] The copula density learning program 11214 of the distribution learning unit 1121 selects one pair from the unprocessed variable pairs and determines the copula parameters so as to maximize the likelihood function (S70 7).

[0089] Here the likelihood function is given by an integral below a threshold as follows, depending on whether the value of the selected variable is zero or not: [Number 1] By performing optimization for each variable pair in JPEG0007662419000001.jpg26170TIFF0007662419000002.tif40170S706, the 2^ d Street scene This avoids the need for consolidation, and makes it possible to estimate copula parameters with at most four case distinctions.

[0090] Next, the copula density learning program 11214 of the distribution learning unit 1121 records the copula parameters obtained in S707 and the value of the maximized likelihood function in the distribution learning result storage unit 1132 (S708). All the copula parameter values ​​are obtained by a loop according to the judgment result in S706.

[0091] Next, the process flow of the output program 11215 that outputs the distribution learning result during distribution learning will be described in detail with reference to Fig. 9. That is, the details of S116 in Fig. 2 will be described.

[0092] In this case, the output program 11215 of the distribution learning unit 1121 obtains the value of the likelihood function for each variable pair stored in the above-mentioned S708 from the distribution learning result storage unit 1132 (S80 1).

[0093] Next, the output program 11215 of the distribution learning unit 1121 selects some variable pairs in order of the large or small value of the likelihood function, and draws a scatter diagram for the variable pairs (S802).

[0094] Next, the output program 11215 of the distribution learning unit 1121 uses the likelihood calculation unit 1122, which will be described later, to plot the data distribution values ​​of the selected variable pair in the form of contour lines, and displays them on the user terminal 12 in an overlapping manner with the scatter diagram (S803), and ends the process. An example of the input / output UI at this time will be described later in detail with reference to FIG.

[0095] In this case, the correlation tendency between the variable pairs with large values ​​of the likelihood function is well captured, while the correlation tendency between the variable pairs with small values ​​of the likelihood function is not well captured.The user can check the output diagram and understand which variable pairs have been captured and which have not been captured. <Flow example: Detailed process flow when calculating likelihood> Next, a detailed flow of the likelihood calculation will be described. First, a process flow of the marginal likelihood calculation program 11221 of the likelihood calculation unit 1122 when calculating the marginal likelihood of each variable will be described in detail with reference to Fig. 10. That is, the details of S201 to S208 in Fig. 3 will be described.

[0096] In this case, the marginal likelihood calculation program 11221 of the likelihood calculation unit 1122 acquires data for which likelihood is to be calculated. The source of acquisition in this case may be either the user terminal 12 or another program in the memory 112.

[0097] Next, the marginal likelihood calculation program 11221 of the likelihood calculation unit 1122 repeats the following processes from S903 to S908 for all of the input variables (S902).

[0098] First, the marginal likelihood calculation program 11221 of the likelihood calculation unit 1122 selects one from the unprocessed variables (S903).

[0099] Next, the marginal likelihood calculation program 11221 of the likelihood calculation unit 1122 determines whether the value of the selected variable of the data is zero (S904).

[0100] If the result of this judgment is zero (S094: YES), the marginal likelihood calculation program 11221 of the likelihood calculation unit 1122 obtains the value of the zero ratio of the above selected variable from the distribution learning result storage unit 1132, and sets the value as the marginal likelihood of the selected variable (S905).

[0101] On the other hand, if the result of the above judgment is that it is not zero (S904: NO), the marginal likelihood calculation program 11221 of the likelihood calculation unit 1122 obtains the preprocessing result of the selected variable and the estimated result of the non-zero data distribution from the distribution learning result storage unit 1132 (S906).

[0102] Next, the marginal likelihood calculation program 11221 of the likelihood calculation unit 1122 performs preprocessing on the data using the preprocessing result acquired in S906 (S907).

[0103] Next, the marginal likelihood calculation program 11221 of the likelihood calculation unit 1122 applies the estimation result of the non-zero data distribution to the preprocessed data, and calculates the marginal likelihood of the selected variables (S908). At this point, the marginal likelihood value of each variable is obtained.

[0104] Next, the marginal likelihood calculation program 11221 of the likelihood calculation unit 1122 accumulates the marginal likelihoods of all variables, stores the accumulated likelihoods in a temporary storage area (S909), and ends the process.

[0105] Next, the process flow of the copula density calculation program 11222 for calculating the copula density when calculating the likelihood will be described in detail with reference to Fig. 11. That is, the details of S209 to S212 in Fig. 3 will be described.

[0106] In this case, the copula density calculation program 11222 of the likelihood calculation unit 1122 acquires again data to be used for likelihood calculation (S1001).

[0107] Next, the copula density calculation program 11222 of the likelihood calculation unit 1122 repeats the following processes from S1003 to S1005 for all the input variables (S1002).

[0108] Next, the copula density calculation program 11222 of the likelihood calculation unit 1122 selects one from the unprocessed variables (S1003), and obtains the normal distribution transformation learning result of the selected variable from the distribution learning result storage unit 1132 (S1004).

[0109] Next, the copula density calculation program 11222 of the likelihood calculation unit 1122 applies the normal distribution conversion learning result to the values ​​of the selected variables of the likelihood calculation target data, and converts them into normal distribution variables (S1005). Up to this point, each variable is converted into a normal distribution variable.

[0110] Next, the copula density calculation program 11222 of the likelihood calculation unit 1122 acquires copula parameters from the distribution learning result storage unit 1132 (S1006).

[0111] Next, the copula density calculation program 11222 of the likelihood calculation unit 1122 calculates the value of the copula density from the normal distribution variables and the copula parameters, and stores it in a temporary storage area. (S1007), and the process ends.

[0112] Next, the process flow of the output program 11223 that outputs the value of the likelihood when the likelihood is calculated will be described in detail with reference to Fig. 12. That is, the details of S213 and S214 in Fig. 3 will be described.

[0113] In this case, the output program 1223 of the likelihood calculation unit 1122 calculates the total likelihood by multiplying the integrated value of the marginal likelihoods of all variables temporarily stored in S909 above by the copula density value temporarily stored in S1007 above (S1101).

[0114] Next, the output program 1223 of the likelihood calculation unit 1122 outputs the value of the total likelihood obtained in S1101 (S1102), and ends the process. In this case, the output destination may be either the user terminal 12 or another program in the memory 112. <Flow example: Detailed processing flow when outputting inference reliability> A detailed flow of calculation of inference reliability will be described below. First, a process flow of the inference reliability calculation program 11241 for calculating inference reliability when outputting inference reliability will be described in detail with reference to Fig. 13. That is, the details of S305 to S308 in Fig. 4 will be described.

[0115] In this case, the inference reliability calculation program 11241 of the inference reliability output unit 1124 acquires data to be used for inference reliability calculation (S1201). The source of acquisition in this case may be the user terminal 12 or another program in the memory 112.

[0116] Next, the inference reliability calculation program 11241 of the inference reliability output unit 1124 inputs the data obtained in S1201 to the likelihood calculation unit 1122 and obtains the output total likelihood value (S1 202).

[0117] Next, the inference reliability calculation program 11241 of the inference reliability output unit 1124 calculates the inference reliability based on the total likelihood obtained in S1202, temporarily stores this inference reliability value in a temporary storage area (S1203), and ends the process.

[0118] If the value of the total likelihood is large, it means that a large amount of similar learning data exists, and therefore the value of the inference reliability is also large. In other words, it is preferable that the inference reliability is an increasing function of the total likelihood, but this is not necessarily the case. For example, the logarithm of the total likelihood can be used as the inference reliability, but this is not necessarily the case.

[0119] Next, a process flow of the output program 11242 that outputs the inference reliability at the time of outputting the inference reliability will be described in detail with reference to Fig. 14. That is, the details of S309 and S310 in Fig. 4 will be described.

[0120] In this case, the output program 11242 of the inference reliability output unit 1124 acquires a threshold value of the inference reliability (S1301). In this case, the source of acquisition may be either the user terminal 12 or another program in the memory 112. The value of the threshold may be arbitrarily determined by the user, and may be adjusted so that a warning, which will be described later, is output for, for example, 5% of the learning data.

[0121] Next, the output program 11242 of the inference reliability output unit 1124 compares the inference reliability temporarily stored in S1203 above with a threshold value (S1302), and if the inference reliability is smaller, outputs a warning (S1303), and ends the processing. <Various data> Next, a description will be given of various data used by the distribution estimation device 11 etc. Fig. 15 shows a table 2000 as an example of distribution learning data, likelihood calculation target data, and inference reliability calculation target data in the first embodiment.

[0122] This table 2000 shows an example of data in the form of a table. As an example, assume that there are three explanatory variables: annual income 2002, number of family members 2003, and age 2004. Also assume that the machine learning model predicts the value of some binary objective variable 2005. The objective variable 2005 is not limited to a binary variable, but may be any objective variable such as classification or regression. Also, ID 2001 is a serial number assigned to each data.

[0123] 16A to 16F, examples of distribution learning results 11321 obtained during distribution learning and recorded in distribution learning result storage unit 1132 will be described in detail. Here, the contents of each learning result will be described in detail using the examples given in table 2000 in FIG.

[0124] First, a zero value ratio table 3100 shown in FIG. 16A records a zero value data ratio 3102 of each variable 3101.

[0125] Furthermore, a zero-value excess variable list 3200 shown in FIG. 16B records a list of variables whose zero value ratio is greater than a predetermined threshold value.

[0126] Next, the preprocessing result 3300 shown in Fig. 16C stores the parameters used in preprocessing the non-zero data of each variable 3301. Here, an example of normalization is shown in which the data value is converted to a value between 0 and 1 using the minimum and maximum values.

[0127] At this time, the minimum value 3302 and maximum value 3303 of each variable used in the preprocessing are recorded. The contents of the preprocessing are not limited to normalization. For example, in the case of standardization, the average value and the standard deviation value are recorded.

[0128] Next, non-zero data distribution learning result and normal distribution transformation learning result 3400 shown in FIG. 16D records the results of learning the distribution of each of variables 3403, 3404, and 3405 and the details of normal distribution transformation.

[0129] Here, an example of a format in which both are recorded simultaneously is given, but these may be recorded in the form of a separate table. In this example, the value of each variable is associated with the cumulative frequency 3401 to record information on the distribution function of the variable.

[0130] At the same time, by recording the variable values ​​3402 after normal distribution, for example, by looking at columns 3403 and 3402, you can obtain a correspondence table showing which annual income values ​​are converted to which values ​​of the normal distribution variable. This is the normal distribution transformation.

[0131] Values ​​not listed in the table will be appropriately supplemented when applying the function. The relationship between 3401 and 3402 is known and given by the value of the cumulative frequency function of the standard normal distribution. Next, the normal distribution threshold 3500 records the value of the normal distribution threshold 3502 for each zero-value excess variable 3501. Next, the copula parameters 3600 are the values ​​of each variable (3601, 3602, 3 603) and record the value of the copula parameter for each pair. Next, various I / Fs in this embodiment will be described. First, an input / output UI for presenting the learning results to the user terminal 12 during distribution learning will be described with reference to Figs. 17A and 17B.

[0132] The input I / F 4100 illustrated in FIG. 17A is displayed on the user terminal 12 during learning, and has an input field 4110 for a learning data file, a zero ratio threshold input field 4120, and a learning start button 4130.

[0133] The learning data file input field 4110 has a display field 4111 for the selected learning data file name, and a button 4112 for searching for it from within the user terminal 12 or the like.

[0134] The zero ratio threshold input field 4120 has an input zero ratio threshold 4121. The user can press a learning start button 4130 to start a series of distribution learning processes.

[0135] ​The output I / F 4200 illustrated in FIG. 17B is displayed on the user terminal 12 at the end of the learning process, and includes both or either of a data scatter plot of a variable pair for which learning has gone well and a contour plot of the distribution estimation results 4210, and a data scatter plot of a variable pair for which learning has gone poorly and a contour plot of the distribution estimation results 4220.

[0136] The user can end the learning process by checking this information and pressing the end button 4230. Note that, although the present embodiment will be described with reference to an example of input and output via a screen, this is not limiting. For example, similar information may be input and output as an API (Application Programming Interface) on a program.

[0137] Next, an input / output UI for presenting the inference result and the inference reliability on the user terminal 12 when the inference reliability is calculated will be described with reference to FIG. 18A.

[0138] The input I / F 5100 illustrated in FIG. 18A is displayed on the user terminal 12 when the inference reliability is calculated, and has an input field 5110 for the prediction target data file and a prediction start button 5120.

[0139] The input field 5110 for the prediction target data file has a display field 5111 for the selected prediction target data file name, and a button 5112 for searching for it from within the user terminal 12, etc. The user can start a series of processes related to inference and inference reliability calculation by pressing a prediction start button 5120.

[0140] On the other hand, the output I / F 5200 illustrated in FIG. 18B is displayed on the user terminal 12 when the calculation of the inference result and inference reliability is completed, and includes an inference result and inference reliability table 5210, a warning 5220, and an end button 5230.

[0141] The inference result and inference reliability table 5210 is a table in a format in which a serial number 5211 for identifying the data to be predicted, input variable values ​​5212, predicted values ​​by the machine learning model 5213, and inference reliability 5214 can be confirmed.

[0142] Moreover, a warning 5220 is displayed when the inference reliability is below a given threshold, to inform the user that caution is required when handling the predicted value.

[0143] The user can confirm this information and end the inference and the calculation process of the inference reliability by pressing the end button 5230. Note that in this embodiment, input and output via a screen is described as an example, but this is not limited to this. For example, similar information can be input and output as an API on a program. Good too. [Example] In the first embodiment, a function for learning the distribution of training data and calculating likelihood (a distribution estimation device described later) and a function for outputting inference reliability using the same (a prediction server) are mounted on the same memory.

[0144] On the other hand, in the second embodiment, a configuration will be described in which these functions are separated and run on separate devices. This allows access to the distribution estimation device and the distribution learning results therein to be appropriately controlled. In addition, the distribution estimation device may be used independently for other purposes as a distribution learning / likelihood calculation device.

[0145] A system configuration diagram in the second embodiment will be described in detail with reference to Fig. 19. Explanations of parts common to Fig. 1 will be omitted. The computer system 6 is composed of a prediction server 61, a distribution estimation device 62, and a user terminal 12, which are connected to each other via a network 10 so as to be able to communicate with each other.

[0146] Of these, the prediction server 61 executes and outputs inference using a machine learning model, and calculates and outputs the inference reliability using the likelihood obtained from the distribution estimation device 62.

[0147] The prediction server 61 includes a CPU 611 , a memory 612 , a storage device 613 , and a communication device 614 .

[0148] Of these, the memory 612 includes a machine learning unit 1123 and an inference reliability output unit 1124 .

[0149] The storage device 613 also includes a prediction model storage unit 1131.

[0150] The communication device 614 is assumed to be a network interface card that communicates with the user terminal 12 and the distribution estimation device 62 via the network 10 .

[0151] Meanwhile, the distribution estimation device 62 learns the data distribution from the training data, stores the training results, and, when requested, calculates the likelihood of new data using the training results.

[0152] Such a distribution estimation device 62 includes a CPU 621 , a memory 622 , a storage device 623 , and a communication device 624 .

[0153] Of these, the memory 622 includes a distribution learning unit 1121 and a likelihood calculation unit 1122. The storage device 623 includes a distribution learning result storage unit 1132.

[0154] The communication device 624 is assumed to be a network interface card that communicates with the user terminal 12 and the prediction server 61 via the network 10.

[0155] According to the above-described embodiments, it is possible to estimate distribution and calculate likelihood with high accuracy using a Gaussian copula for multivariate data with an excess of zero values. It is also possible to calculate the reliability of inference and detect anomalies using the likelihood.

[0156] Although the best mode for carrying out the present invention has been specifically described above, the present invention is not limited to this, and various modifications can be made without departing from the spirit and scope of the present invention.

[0157] The description of this specification makes at least the following clear: That is, in the distribution estimation device of this embodiment, the arithmetic unit may execute a process of calculating the likelihood of new data by using the estimated parameters.

[0158] This makes it possible to calculate the likelihood of data with high accuracy, and by extension, to prevent accuracy degradation caused by tied data when applying a Gaussian copula to zero-value excess multivariate data, thereby supporting more accurate data distribution estimation.

[0159] In the distribution estimation device of this embodiment, the calculation device calculates the likelihood by a predetermined It may be that the information is output to a device.

[0160] This allows the user to understand the results of distribution estimation efficiently by presenting the likelihood to the user. It also prevents the accuracy degradation caused by tied data when applying the Gaussian copula to zero-value excess multivariate data, and supports more accurate data distribution estimation.

[0161] The system of this embodiment may further include a prediction device connected to the distribution estimation device and configured to calculate inference reliability based on the likelihood output by the distribution estimation device. [Explanation of symbols]

[0162] 1, 6 Computer system (distribution estimation system) 61 Prediction Server 62 Distribution estimation device 10 Network 11 Distribution estimation device 12 User terminal 111 CPU 112 Memory 1121 Distribution Learning Department 11211 Data Acquisition Program 11212 Zero Data Learning Program 11213 Non-zero data learning program 11214 Copula Density Learning Program 11215 Output Program 1122 Likelihood calculation unit 11221 Marginal likelihood calculation program 11222 Copula density calculation program 11223 Output Program 1123 Machine Learning Department 11231 Model Prediction Program 11232 Output Program 1124 Inference reliability output unit 11241 Inference reliability calculation program 11242 Warning Issue Program 113 Storage device 1131 Prediction model storage section 11311 trained models 1132 Distribution learning result storage unit 11321 Distribution learning results 114 Communication equipment 2000 Distribution learning data 3100 Zero value ratio table 3200 Zero-value excess variable list 3300 Pre-processing results 3400 Non-zero data distribution learning results and normal distribution transformation learning results 3500 Normal Distribution Threshold 3600 Copula parameters 4100 Distribution learning input I / F 4200 Distribution learning output I / F 5100 Input I / F for calculating inference reliability 5200 Inference reliability calculation output I / F

Claims

1. A storage device for storing data to be processed, the data consisting of a plurality of variables; a distribution learning unit that executes a series of processes including a process of calculating a zero data ratio for each variable for the data, determining a variable whose zero data ratio is greater than a predetermined threshold as a zero-excess variable, and recording a list of the zero data ratio and the zero-excess variable in a storage unit for distribution learning results, a process of performing distribution learning of non-zero data for each variable, and recording the results of the distribution learning in the storage unit, a process of converting each variable into a variable that follows a normal distribution by rank transformation, and recording the results of the conversion in the storage unit, a process of calculating a normal distribution threshold for each variable after the transformation, and recording the results of the calculation in the storage unit, and a process of learning parameters of a normal copula density, and recording the results of the learning in the storage unit; a process of acquiring a zero data ratio of a variable that was zero data from the storage unit and setting the zero data ratio as a marginal likelihood of the zero variable; a process of acquiring a learning result of a nonzero data distribution of a variable that was nonzero data among the data from the storage unit and calculating a marginal likelihood of the nonzero variable using the learning result; a process of acquiring a result of the conversion of each of the variables from the storage unit and performing a normalized distribution conversion on each of the variables using the result; a process of acquiring a learning result of a parameter of a normal copula density from the storage unit and calculating a normal copula density using the learning result; and a likelihood calculation unit that calculates a total likelihood by multiplying the marginal likelihood of the zero variable, the marginal likelihood of the nonzero variable, and a value of the copula density. A distribution estimation device comprising:

2. The computing device includes: The calculated total likelihood is output to a predetermined device. The distribution estimation device according to claim 1 .

3. An information processing device, Hold the data to be processed, which consists of multiple variables, a process of calculating a zero data ratio for each variable for the data, determining that a variable having a zero data ratio greater than a predetermined threshold is a zero-excess variable, and recording a list of the zero data ratio and the zero-excess variable in a storage unit for distribution learning results; a process of performing distribution learning of non-zero data for each variable, and recording the results of the distribution learning in the storage unit; a process of converting each variable into a variable that follows a normal distribution by rank transformation, and recording the results of the transformation in the storage unit; a process of calculating a normal distribution threshold for each variable after the transformation, and recording the results of the calculation in the storage unit; and a process of learning parameters of a normal copula density, and recording the results of the learning in the storage unit. a process of acquiring a zero data ratio of a variable that was zero data from the storage unit and setting the zero data ratio as a marginal likelihood of the zero variable; a process of acquiring a learning result of a nonzero data distribution of a variable that was nonzero data among the data from the storage unit and calculating a marginal likelihood of the nonzero variable using the learning result; a process of acquiring a result of the conversion of each of the variables from the storage unit and performing a normalized distribution conversion on each of the variables using the result; a process of acquiring a learning result of a parameter of a normal copula density from the storage unit and calculating a normal copula density using the learning result; and a process of calculating a total likelihood by multiplying the marginal likelihood of the zero variable, the marginal likelihood of the nonzero variable, and the value of the copula density. A distribution estimation method comprising the steps of:

4. A storage device for storing data to be processed, the data consisting of a plurality of variables; a distribution learning unit that executes a series of processes including a process of calculating a zero data ratio for each variable for the data, determining a variable whose zero data ratio is greater than a predetermined threshold as a zero-excess variable, and recording a list of the zero data ratio and the zero-excess variable in a storage unit for distribution learning results, a process of performing distribution learning of non-zero data for each variable, and recording the results of the distribution learning in the storage unit, a process of converting each variable into a variable that follows a normal distribution by rank transformation, and recording the results of the conversion in the storage unit, a process of calculating a normal distribution threshold for each variable after the transformation, and recording the results of the calculation in the storage unit, and a process of learning parameters of a normal copula density, and recording the results of the learning in the storage unit; a process of acquiring a zero data ratio of a variable that was zero data from the storage unit and setting the zero data ratio as a marginal likelihood of the zero variable; a process of acquiring a learning result of a nonzero data distribution of a variable that was nonzero data among the data from the storage unit and calculating a marginal likelihood of the nonzero variable using the learning result; a process of acquiring a result of the conversion of each of the variables from the storage unit and performing a normalized distribution conversion on each of the variables using the result; a process of acquiring a learning result of a parameter of a normal copula density from the storage unit and calculating a normal copula density using the learning result; and a likelihood calculation unit that calculates a total likelihood by multiplying the marginal likelihood of the zero variable, the marginal likelihood of the nonzero variable, and a value of the copula density. A system including a distribution estimation device comprising:

5. a prediction device connected to the distribution estimation device and calculating inference reliability based on the total likelihood output by the distribution estimation device; The system of claim 4 further comprising:

Images