Information processing device, data analysis method, and data analysis program

Abstract

An information processing device (10) comprises: a reception unit (15A) that receives an observed explanatory variable and a target variable; an extraction unit (15B) that extracts a feature of an unobserved explanatory variable on the basis of the observed explanatory variable and the target variable; an analysis unit (15C) that performs data analysis on the basis of the feature of the unobserved explanatory variable; and an output unit (15D) that outputs the result of the data analysis.

Description

Information processing device, data analysis method, and data analysis program

[0001] The present invention relates to an information processing device, a data analysis method, and a data analysis program.

[0002] Data scientists perform exploratory data analysis (EDA) on datasets provided by clients to suggest next actions that will lead to improvements in models such as machine learning models.

[0003] For example, a useful next action for decision-making is to extract information about unobserved explanatory variables that are missing from the prediction of the dependent variable.

[0004] As an example of a technique for extracting such unobserved explanatory variables, a conventional technique has been proposed that selects a subset of explanatory variables from a set of candidate variables that have features that lead to model improvement, such as variables that can explain the dependent variable well or variables that are as independent as possible from other explanatory variables.

[0005] Yamada, Makoto, et al. "High-Dimensional Feature Selection by Feature-Wise Kernelized Lasso." Neural computation 26.1 (2014): 185-207. Lu, Yijuan, et al. "Feature Selection Using Principal Feature Analysis." Proceedings of the 15th ACM international conference on Multimedia. 2007.

[0006] However, the conventional techniques described above have room for improvement in that their applicability to data analysis, such as feature selection and factor analysis based on clustering, is limited.

[0007] One aspect of the conventional techniques described above is that feature selection is difficult to perform unless the dataset contains no missing indices for any element in the candidate set. For example, removing all missing indices reduces the sample size, and imputing missing values ​​is equivalent to adding noise to those variables.

[0008] Another aspect is that, with the conventional techniques described above, if you want to reveal attributes through factor analysis, you need to prepare all possible combinations as candidates, and since the number of possible combinations is generally enormous, the computational cost increases.

[0009] Therefore, the present invention aims to provide an information processing device, a data analysis method, and a data analysis program that can expand the scope of application to data analysis.

[0010] To solve the above-mentioned problems and achieve the objective, the information processing device of the present invention includes: a receiving unit that receives observed explanatory variables and an objective variable; an extraction unit that extracts features of unobserved explanatory variables based on the observed explanatory variables and the objective variable; an analysis unit that performs data analysis based on the features of the unobserved explanatory variables; and an output unit that outputs the results of the data analysis.

[0011] According to the present invention, it is possible to expand the scope of application to data analysis.

[0012] Figure 1 is a block diagram showing an example of the functional configuration of an information processing device. Figure 2 is a schematic diagram (1) showing one aspect of the problem-solving approach. Figure 3 is a schematic diagram (2) showing one aspect of the problem-solving approach. Figure 4 is a diagram showing an example of a model. Figure 5 is a schematic diagram explaining an example of the interconnectedness of changes. Figure 6 is a flowchart showing the procedure for data analysis processing. Figure 7 is a diagram showing an example of experimental results. Figure 8 is a diagram showing an example of a loss function used in Joint Training. Figure 9 is a diagram showing an example of hardware configuration.

[0013] The following description will explain the embodiments for implementing the information processing device, data analysis method, and data analysis program related to this disclosure (hereinafter referred to as "embodiments") with reference to the attached drawings. It should be noted that these embodiments represent only one example or aspect, and the following description does not limit the structure, operation, function, properties, characteristics, methods, and applications related to this disclosure.

[0014] <Overall Configuration> Figure 1 is a block diagram showing an example of the functional configuration of the information processing device 10. For example, Figure 1 shows an information processing device 10 that provides a data analysis function to perform exploratory data analysis, also known as EDA (Exploratory Data Analysis), on a dataset provided by a customer or the like.

[0015] In one embodiment, the information processing device 10 may be implemented by a server device. For example, the information processing device 10 can provide the above data analysis function as a cloud service by running a PaaS (Platform as a Service) type middleware or a SaaS (Software as a Service) type application.

[0016] As shown in Figure 1, the information processing device 10 can be connected to a client terminal 30 via a network NW for communication. For example, the network NW may be implemented by any type of communication network, such as the Internet or a LAN (Local Area Network), whether wired or wireless.

[0017] The client terminal 30 is a terminal device used by a user who receives the data analysis function described above. The term "user" here may include customers who request data analysis. Furthermore, it is not prohibited for the data analysis function to be used by a data scientist who has received a data analysis request from such a customer. For example, the client terminal 30 may be implemented using any computer, including personal computers, smartphones, tablet devices, and wearable devices.

[0018] It should be noted that while the above data analysis functionality is provided as a cloud service, it is not limited to this. For example, the above data analysis functionality may be provided on-premises. For example, if the above data analysis functionality is used by a data scientist who has received a data analysis request from the above customer, the above data analysis functionality may be packaged as a function of the tool used by the data scientist.

[0019] Furthermore, while the above-mentioned data analysis function is implemented as an example in a client-server system, it is not limited to this. For example, the data analysis function may be provided as a standalone function by having an application running on the client terminal 30 execute processing corresponding to the data analysis function on the client terminal 30.

[0020] <Data Analysis Project Flow> Data scientists perform exploratory data analysis on datasets provided by clients, presenting analysis results such as next actions that can lead to improvements in models such as machine learning models.

[0021] One aspect of this is that data scientists are not always domain experts. For example, when a task is specified, such as an "anomaly detection project," the task may be common, but the domain may be diverse. Also, in a project to build a model, if the model's accuracy is insufficient, the data scientist may lack the skills to determine the next action.

[0022] <Example of a goal> Even if the data scientist is not a domain expert, they are expected to propose the next actions to improve the model.

[0023] As an example of such a next action, we can consider a use case where information on unobserved explanatory variables that is missing from the prediction of the dependent variable is extracted.

[0024] This is because, when generating machine learning models in data analysis, the training datasets do not always contain sufficient information, and the explanatory variables within the dataset may not always accurately estimate the target variable. Therefore, it is necessary to improve model accuracy by adding new explanatory variables as external knowledge.

[0025] In such use cases, one possible next action is to add the conditions under which data is acquired (hereinafter referred to as "data acquisition conditions") as variables, or in other words, to propose a next action that generates a model for each data acquisition condition. Alternatively, one could propose a next action that adds new explanatory variables as external knowledge by increasing the sampling frequency of a specific type of sensor data.

[0026] <Problem Setting> For example, the true data generation process (unknown) can be formulated as shown in equations (1) and (4) below. Of these, the explanatory variable x in equation (1) is expressed in equation (2), and the dependent variable y is expressed in equation (3). Furthermore, M in equation (4) below ｖ |y-f| refers to a mask used to prevent some of the truly observable explanatory variables from being observed, and is defined, for example, by the conditions shown in equation (5) below. In equation (5) below, "S" refers to the set of indices of the explanatory variables to be left unobserved. ＧＴ Let's assume that there exists a nonlinear function f(•) such that (x)|=0, but there does not exist a function f(•) such that |y-f(x~)|=0.

[0027]

[0028] Here, given the dataset shown in equation (6) below, the unobserved explanatory variables to be newly added can be determined by equation (7) below. For the sake of explanation, equation (7) below will be referred to as "u ｉ ∈R ｜Ｓ｜In addition to writing "", the index may be written as "t". Note that the capital letter "R" written here refers to the real number.

[0029]

[0030] The problem formulated as described in (7) above can be reduced to a feature selection problem. For example, it can be formulated as a problem of selecting from a candidate set u of unobserved explanatory variables, as shown in equations (8) to (10) below. Note that in equations (8) and (9) below, "g Ｍｎ The symbol (•) is a function that generates missing values.

[0031]

[0032] <One aspect of the challenge> As explained in the background technology section above, the conventional technologies described above have room for improvement in that their applicability to data analysis such as feature selection and factor analysis based on clustering is limited.

[0033] One aspect of the conventional techniques described above is that feature selection is difficult to perform unless the dataset contains no missing indices for any element in the candidate set. For example, removing all missing indices reduces the sample size, and imputing missing values ​​is equivalent to adding noise to those variables.

[0034] Another aspect is that, with the conventional techniques described above, if you want to reveal attributes through factor analysis, you need to prepare all possible combinations as candidates, and since the number of possible combinations is generally enormous, the computational cost increases.

[0035] <One aspect of the problem-solving approach> Therefore, in this embodiment, we focus on changes in the dependent variable that cannot be explained by the already observed explanatory variables, and assume that the prediction has become uncertain because there are unobservable explanatory variables.

[0036] Figure 2 is a schematic diagram (1) illustrating one aspect of the problem-solving approach. For example, Figure 2 shows the already observed explanatory variable x ｔ, unknown explanatory variable u ｔ and objective variable y ｔ = f(x ｔ , u ｔ ) is shown as graph G1. Here, in graph G1, for convenience of explanation, under the assumption that the unknown explanatory variable u ｔ that cannot be observed at the start of analysis could be observed, the relationship between the explanatory variable x ｔ , the unknown explanatory variable u ｔ and the objective variable y ｔ is schematized.

[0037] Further, in FIG. 2, when one of the explanatory variables in graph G1, for example, the unknown explanatory variable u ｔ is masked, for example, when each data point is projected onto the plane of x ｔ and y ｔ , it is represented as graph G2. Further, on graph G2, the model y ｔ , y ｔ ) learned by {(x ｔ is plotted, and the area where the prediction by the model y ｔ = f(x ｔ ) is uncertain is hatched. ｔ = f(x ｔ )

[0038] The hatched area shown in such graph G2 can be regarded as being caused by the presence of the unknown explanatory variable u ｔ and making the prediction by the model y ｔ = f(x ｔ ) uncertain.

[0039] Under such an assumption, in the data analysis function according to the present embodiment, an array of parameters θ ｔ that characterize the unobserved explanatory variable u ｔ is extracted from a trained model in which a model having parameters that change according to an index included in a predetermined series is trained.

[0040] Hereinafter, as an example only, parameters that change in a time series are exemplified, but it is preliminarily noted that they may be parameters that change according to an index included in a predetermined series.

[0041] Figure 3 is a schematic diagram (2) illustrating one aspect of the problem-solving approach. For example, Figure 3 similarly shows graphs G1 and G2 shown in Figure 2. Furthermore, graph G2 in Figure 3 shows the u in graph G1. ｔ x in a continuous direction ｔ and y ｔ Function f obtained for each cross-section ｕｔ (x ｔ ) is x ｔ and y ｔ It is projected onto the plane.

[0042] As shown in graph G2 in Figure 3, u ｔ x in a continuous direction ｔ and y ｔ Function f obtained for each cross-section ｕｔ (x ｔ ) is a set of functions that differ depending on time t, i.e., y ｔ = f ｕ１ (x ｔ ), y ｔ = f ｕ２ (x ｔ ), y ｔ = f ｕ３ (x ｔ This allows us to model the unknown true model p(y ｔ | x ｔ , u ｔ Even if θ) cannot be observed, instead, model p(y ｔ | x ｔ ;θ ｔ By training the unobserved explanatory variable u ｔ The parameter θ that characterizes it ｔ A sequence can be obtained.

[0043] As just one example of such a model, we use a DNN (Deep Neural Network) where the parameters change based on time t, p(y ｔ | x ｔ ;θ ｔ ) can be learned.

[0044] Figure 4 shows an example of a model. As shown in Figure 4, the DNN20 may include an FFN (Feed Forward network) 21 and an NN22. For example, when time t is input to the FFN21, the parameter θ of the NN22 is... ｔ An intermediate output φ(t) is output to determine the parameter θt of NN22. This condition for determining the parameter θt of NN22 based on the intermediate output φ(t) of FFN21 is merely one example and may be implemented using FiLM (Feature-wise Linear Modulation) or MoE (Mixture-of-Experts). The parameter θt is determined by the intermediate output φ(t) of FFN21. ｔ Observed explanatory variables x in NN22 ｔ By inputting this, NN22 will determine the target variable y ｔ The predicted value of the target variable y obtained in this way is output. ｔ The parameters of the DNN20 can be trained based on the loss calculated using a loss function such as MSE (Mean Squared Error) or cross-entropy, which is the error between the predicted value and the correct target variable.

[0045] After training the parameters of DNN20, the data analysis function according to this embodiment inputs times t=1, ..., T to DNN20. Subsequently, in light of the fact that the same vector can be obtained at different times t, the data analysis function according to this embodiment analyzes φ(t) or θ from time t=1 to time t=T. ｔ The data is clustered. Then, the data analysis function according to this embodiment uses φ(t) or θ ｔ The identification information of the cluster to which it belongs, for example, the series of cluster IDs (IDentifications), is used as an unobserved explanatory variable u ｔ Extract the series to represent the characteristics.

[0046] Such θ ｔ The series can be applied to feature selection as merely one example of data analysis. That is, the data analysis function according to this embodiment is θ ｔ By comparing the series with the candidate set of explanatory variables, we can identify the unobserved explanatory variables from the candidate set u.ｔ You can select candidate explanatory variables that correspond to this.

[0047] For example, θ ｔ The degree of correlation between the sequence and the elements included in the candidate set u, i.e., the sequence of candidate explanatory variables, is calculated. Here, "degree of correlation" can be an index value that evaluates the degree of linkage of changes in values. For example, a higher evaluation can be given as the degree of correlation between the presence or absence of changes in values ​​increases, while a lower evaluation can be given as the degree of correlation decreases.

[0048] Figure 5 is a schematic diagram illustrating an example of the interconnectedness of changes. For example, in Figure 5, θ is shown below the time t axis. ｔ The series of cluster IDs ｔ The following is plotted. As shown in Figure 5, θ ｔ The series of cluster IDs ｔ After the interval "A" where cluster ID is A is observed, θ ｔ An interval "B" is observed where the cluster ID is B. In this case, the correlation coefficient is calculated to give a higher evaluation to candidate explanatory variables whose values ​​change little in interval "A", change at the boundary where interval "A" switches to interval "B", and whose values ​​change little in interval "B". For example, as just one example of correlation coefficient, the Hilbert-Schmidt Independence Criterion (HSIC), which evaluates independence in a nonlinear space, can be used.

[0049] After calculating such correlations for each element of the candidate set u, the explanatory variables to be added from the candidate set u based on the correlation of each element should be determined. ｔ Select the option. For example, the variable u to which the candidate explanatory variable with the highest relevance should be added. ｔ Variables that should be selected or to which candidate explanatory variables with a correlation above a threshold should be added. ｔ You can also select them as such. In addition, you should add a variable u that is a candidate for an explanatory variable corresponding to the top n most relevant variables. ｔThese can be selected as a list. When extracting n candidate explanatory variables from a candidate set u in descending order of relevance, the lower-ranking explanatory variables can be selected based on statistical values ​​between the extracted top-ranking explanatory variables and the lower-ranking explanatory variables to be evaluated, such as covariance or correlation coefficient. For example, candidate explanatory variables whose covariance or correlation coefficient is between zero and within a predetermined range can be extracted, or candidate explanatory variables whose covariance or correlation coefficient is between zero and outside a predetermined range can be excluded.

[0050] Note that here, the unobserved explanatory variable u ｔ θ representing the characteristics ｔ We have described an example of performing feature selection based on a sequence of cluster IDs, but θ ｔ By presenting the sequence of cluster IDs, the results of factor analysis based on clustering can be presented.

[0051] Thus, the data analysis function according to this embodiment can extract features of unobserved explanatory variables from observed explanatory variables and dependent variables, thereby achieving feature selection that is robust to missing data. Furthermore, the data analysis function according to this embodiment can perform factor analysis without the need to enumerate combinations.

[0052] Therefore, the data analysis function according to this embodiment can expand the scope of application to data analysis such as feature selection and factor analysis.

[0053] <Configuration of Information Processing Device 10> Next, the functional configuration of the information processing device 10 that provides the above-mentioned data analysis function will be described. Figure 1 schematically shows the blocks related to the data analysis function of the information processing device 10. As shown in Figure 1, the information processing device 10 has a communication control unit 11, a storage unit 13, and a control unit 15. Note that Figure 1 only shows a selection of the functional units related to the above-mentioned data analysis function, and the information processing device 10 may also be equipped with functional units other than those shown.

[0054] The communication control unit 11 is a functional unit that controls communication with other devices such as the client terminal 30. In one embodiment, the communication control unit 11 can be implemented by a network interface card such as a LAN card. In one aspect, the communication control unit 11 receives requests from the client terminal 30 to perform data analysis, or outputs a response to such requests, such as the results of the data analysis, to the client terminal 30.

[0055] The storage unit 13 is a functional unit that stores various types of data. In one embodiment, the storage unit 13 may be implemented by internal, external, or auxiliary storage of the information processing device 10. For example, the storage unit 13 stores a dataset 13A. The dataset 13A will be described later in conjunction with the scenes in which the dataset 13A is referenced or registered.

[0056] The control unit 15 is a functional unit that performs overall control of the information processing device 10. For example, the control unit 15 can be implemented by a hardware processor. As shown in Figure 1, the control unit 15 has a receiving unit 15A, an extraction unit 15B, an analysis unit 15C, and an output unit 15D. The control unit 15 may also be implemented by hardwired logic or the like.

[0057] The reception unit 15A is a processing unit that receives various types of information. In one embodiment, the reception unit 15A can receive a request from the client terminal 30 to perform data analysis. At this time, the reception unit 15A can receive the upload of the dataset to be analyzed. For example, the dataset may include observed explanatory variables and the target variable. In addition to such a dataset, the reception unit 15A can also receive unobserved explanatory variables u ｔ The parameter θ that characterizes it ｔIt is also possible to receive the designation of a candidate set u of explanatory variables to be matched with the series of . This candidate set u may include, as candidates, explanatory variables with missing values or explanatory variables created by data scientists. Here, an example has been given in which the data set to be analyzed is uploaded at the time of executing data analysis. However, it is also possible to receive the designation of the data set to be analyzed from the data sets 13A registered in the storage unit 13 in advance.

[0058] The extraction unit 15B is a processing unit that extracts the characteristics of the unobserved explanatory variable u ｔ based on the observed explanatory variables and objective variables included in the data set. As an example only, the extraction unit 15B uses a DNN whose internal parameters change under the condition of time t, and p(y ｔ |x ｔ ; θ ｔ ) to train the parameters of the DNN that makes the prediction.

[0059] For example, in the example of the DNN 20 shown in FIG. 4, for each time t from time t = 1 to time t = T, the following processing is executed. That is, the extraction unit 15B inputs the time t to the FFN 21, and obtains an intermediate output φ(t) output by the FFN 21, for example, an ensemble coefficient or a matrix of an affine transformation. Then, the extraction unit 15B inputs the intermediate output φ(t) to the NN 22, and the internal parameter θt of the NN 22 is determined. Then, the extraction unit 15B inputs the observed explanatory variable x ｔ to the NN 22 having the internal parameter θ ｔ determined by the intermediate output φ(t) of the FFN 21, and outputs a predicted value y^ ｔ of the objective variable output by the NN 22. Then, the extraction unit 15B can train the parameters of the DNN 20 based on the error between the predicted value y^ ｔ of the objective variable and the correct objective variable y ｔ , for example, the loss calculated using a loss function such as MSE or cross entropy. The training of the parameters of the DNN 20 like this can be repeated until the end condition, for example, the execution of a specified number of epochs or the condition such as the convergence of the parameters is satisfied.

[0060] After training the parameters of the DNN 20, the extraction unit 15B inputs times t = 1,..., T to the DNN 20. Then, from the aspect of recognizing that the same vector is taken at different times t, the extraction unit 15B clusters φ(t) from time t = 1 to time t = T, that is, {φ(t)|t = 1,..., T}. Then, the extraction unit 15B generates a series of cluster IDs to which φ(t) belongs, that is, ID(φ(t)), as an unobserved explanatory variable u ｔ representing the characteristics of the series u^ ｔ and extracts it as such.

[0061] Here, as an example of a model in which the parameters change conditional on time t, the DNN 20 is given as an example, but it may be realized by other machine learning models such as an AR (AutoRegressive) model.

[0062] The analysis unit 15C is a processing unit that performs exploratory data analysis based on the characteristics of the unobserved explanatory variable u ｔ extracted by the extraction unit 15B.

[0063] As an example only, an example in which feature selection is performed using the cluster ID series u^ ｔ representing the characteristics of the unobserved explanatory variable u ｔ = ID(φ(t)) is given. In this case, the analysis unit 15C calculates a score s ｔ for evaluating the degree of association between the series u^ ｔ = ID(φ(t)) representing the characteristics of the unobserved explanatory variable u and the candidate set u. For example, the analysis unit 15C calculates an index value, such as HSIC, representing the degree of association between the cluster ID series u^ ｎ and each candidate u- ｔ ｎ of the explanatory variable candidates included in the candidate set u for each u- ｔ ∈ u. Then, the analysis unit 15C selects an explanatory variable u ｔ ｎ to be added based on the degree of association of each element in the candidate set u. For example, the analysis unit 15C selects the candidate of the explanatory variable with the highest degree of association as the variable u ｔ to be added, or selects the candidate of the explanatory variable with a degree of association equal to or higher than a threshold as the variable u ｔ to be added.ｔ It can also be selected as such. In addition, the analysis unit 15C will add a variable u to which candidates for explanatory variables that correspond to the top n most relevant variables. ｔ This can be done by selecting them as a list. When extracting n candidate explanatory variables from a candidate set u in descending order of relevance, the lower-ranking explanatory variables can be selected based on statistical values ​​between the extracted top explanatory variables and the lower-ranking explanatory variables to be evaluated, such as covariance or correlation coefficient.

[0064] Here, HSIC was given as an example of an index representing relevance, but naturally, other indices such as MIC (Maximal Information Coefficient), mutual information, and cosine similarity can be used.

[0065] The output unit 15D is a processing unit that performs output control to the client terminal 30. In one aspect, the output unit 15D outputs the results of data analysis by the analysis unit 15C to the client terminal 30. In one aspect, when feature selection is performed as part of the data analysis, the output unit 15D identifies the variable u to which the candidate explanatory variable with the highest relevance should be added. ｔ Output as such, or add a variable u that is a candidate explanatory variable whose relevance corresponds to a predetermined number of top-ranking variables. ｔ It can be output as a list. Another aspect is that when factor analysis is performed as data analysis, the output unit 15D outputs the cluster ID sequence u^ ｔ It is also possible to output the result of factor analysis. In this case, the cluster ID sequence u^ ｔ This allows you to display intervals with the same cluster ID using the same color, font, or hatching.

[0066] <Processing Flow> Figure 6 is a flowchart showing the steps of the data analysis process. This process can be started as an example when a request to perform data analysis is received from the client terminal 30.

[0067] As shown in Figure 6, the extraction unit 15B uses a DNN whose parameters change based on time t until the termination condition is met, and extracts p(yｔ | x ｔ ;θ ｔ A loop process 1 is executed that iterates through the process of training the parameters of the DNN that makes predictions (step S101).

[0068] For example, in training the parameters of the DNN20 in step S101, the predicted value of the target variable y^ shown in Figure 6 ｔ The output and the DNN20 parameter θ shown in Figure 6 ｔ The update may be performed at time t.

[0069] By repeating this loop process 1, a trained DNN 20 is obtained.

[0070] Subsequently, the extraction unit 15B clusters the φ(t) obtained from time t=1 to time t=T, i.e., {φ(t) | t=1, ..., T}, by inputting times t=1, ..., T to the trained DNN 20 (step S102).

[0071] Next, the extraction unit 15B, as a result of the clustering in step S102, extracts the series of cluster IDs to which φ(t) belongs, i.e., ID(φ(t)), from the unobserved explanatory variable u ｔ A series u^ that represents the characteristics ｔ Extract as (step S103).

[0072] The analysis unit 15C then executes a loop process 2 that repeats the process in step S104 below a number of times corresponding to the number of elements N in the candidate set u, i.e., the total number of candidate explanatory variables. Naturally, the process in step S104 below may be executed in parallel for each element in the candidate set u.

[0073] In other words, the analysis unit 15C analyzes the cluster ID sequence u^ ｔ And the nth explanatory variable candidate u- ｔ ｎ The correlation score, which evaluates the degree of correlation between the changes, is calculated according to the HSIC calculation formula shown in Figure 6 (step S104).

[0074] As this loop process 2 is repeated, the cluster ID sequence u^ is generated for each element included in the candidate set u. ｔA score indicating the degree of relevance to the given topic is obtained.

[0075] Subsequently, the output unit 15D determines which explanatory variables to add from the candidate set u based on the degree of relevance of each element. ｔ The output is displayed (step S105), and the process ends.

[0076] Note that Figure 6 shows an example where feature selection is performed as data analysis, but when factor analysis is performed, the cluster ID sequence u^ ｔ This can be output as the result of factor analysis.

[0077] <Summary> As described above, the information processing device 10 according to this embodiment can extract features of unobserved explanatory variables from observed explanatory variables and the target variable, thereby achieving feature selection that is robust to missing data. Furthermore, the data analysis function according to this embodiment can realize factor analysis that eliminates the need to enumerate combinations.

[0078] Therefore, the information processing device 10 according to this embodiment can expand the scope of application to data analysis such as feature selection and factor analysis.

[0079] <One Aspect of the Effect> Here, in order to explain the effectiveness of the data analysis function according to this embodiment, we will compare experimental results between the data analysis function according to this embodiment and HSIC Lasso described in Non-Patent Document 1.

[0080] (1) Sequence of Interest Here, we will take the example of generating a sequence from the difference equation of a two-dimensional wave equation. Specifically, we will take the example of predicting an objective function calculated based on the coordinates of the point of interest one time step earlier, the coordinates of the point of interest two time steps earlier, and the coordinates of four neighboring points of the point of interest one time step earlier, assuming that one of the neighboring points is unobserved.

[0081] (2) Candidate set u is an example that includes elements such as sine wave, square wave, triangle wave, random noise, Gaussian noise, and the correct sequence.

[0082] (3) Experimental Results Figure 7 shows an example of experimental results. Figure 7 shows experimental results between the data analysis function according to this embodiment and HSIC Lasso described in Non-Patent Literature 1. Here, Figure 7 shows the evaluation value of the AUC (Area Under the Curve) of the ROC (Receiver Operating Characteristic) curve for a total of four patterns: A) random missing data (20%), B) random missing data (50%), and C) missing data in intervals (14%) for each element of the candidate set u, and D) a pattern where the sample intervals for each element are different. Furthermore, since HSIC Lasso described in Non-Patent Literature 1 can operate on the condition that there are no indexes with missing data, the evaluation value of the AUC of the ROC curve is shown for three patterns in which linear interpolation, median, or removal is performed on the missing values ​​of each element.

[0083] As shown in Figure 7, in the case of HSIC Lasso, the sequence weights become sparse, so if the correct answer is not selected, the score will fall below 0.5. On the other hand, in the case of the data analysis function according to this embodiment, the score is 1 in all four patterns A) to D). Therefore, the data analysis function according to this embodiment surpasses the evaluation value of HSIC Lasso in all four patterns A) to D). This remains true regardless of whether linear interpolation, median valuation, or removal is performed on the missing values ​​of each element in the candidate set u. Thus, it is clear that the data analysis function according to this embodiment can achieve feature selection that is more robust to missing values ​​compared to HSIC Lasso.

[0084] <Application Examples> For example, in the above embodiment, an example was given in which a DNN whose parameters change based on time t is trained, and then the {φ(t) | t=1, ..., T} output by the trained DNN is clustered, but the invention is not limited to this. For example, the information processing device 10 can also perform Joint Training, which simultaneously trains two tasks: DNN training and clustering. In this case, it is sufficient to train a DNN that takes time t as input and discretizes and outputs the coefficients of an ensemble or the matrix of an affine transformation.

[0085] Figure 8 shows an example of a loss function L used in Joint Training. As shown in Figure 8, the loss function L may contain three terms, from the first to the third term. The first term formulates the loss for learning the task of predicting the target variable. For example, if the task is regression, it may be calculated using MSE or MAE (Mean Absolute Error). If the task is classification, it may be calculated using CCE (Categorical Cross Entropy). The second term, as an example, may formulate the same loss as the Vector Quantization part of VQ-VAE (Vector Quantized Variational Autoencoder), for example, the loss for bringing the centroid (codebook) closer to the data. Here, Vector Quantization is shown as an example of a concurrently learnable clustering method, but other concurrently learnable clustering methods may be applied. The third term formulates the loss for using as few centroids as possible, for example, the loss for bringing the data closer to the centroid (codebook). Joint training may be implemented using the loss function L defined by these first to third terms. Even when the DNN is trained using joint training in this way, a value of 1 can be obtained as the AUC evaluation value of the ROC curve in all four patterns A) to D).

[0086] <Exercise of Creative Ability> The specific examples described in this embodiment, such as machine learning models and loss functions, are merely examples and can be modified. Furthermore, the flowchart described in this embodiment can also be modified within a consistent range, such as changing the order of processing or skipping some processes.

[0087] <System> Unless otherwise specified, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above document and drawings may be changed at will. For example, one or more of the functions of the information processing device 10, such as the receiving unit 15A, extraction unit 15B, analysis unit 15C, and output unit 15D, may be composed of separate devices.

[0088] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown. That is, all or part of them can be functionally or physically distributed and integrated in any units according to various loads and usage conditions. Note that each configuration may also be a physical configuration.

[0089] Furthermore, the processing performed by the illustrated apparatus can be implemented, in whole or in part, by a program executed by a hardware processor such as an MPU (Micro-Processing Unit) or CPU (Central Processing Unit), or by hardware using wired logic.

[0090] <Hardware> Next, an example of the hardware configuration of the information processing device 10 described in this embodiment will be explained. For example, it can be implemented by installing a program that realizes the functions of the information processing device 10 on a computer. For example, by having the computer run the above program, which is provided as packaged software or online software, the computer can be made to function as the information processing device 10. The computer referred to here includes desktop or notebook personal computers, rack-mounted server computers, etc. In addition, the computer category also includes smartphones, mobile phones and PHS (Personal Handyphone System) and other mobile communication terminals, as well as PDAs (Personal Digital Assistants). Furthermore, the functions of the information processing device 10 may be implemented on a cloud server.

[0091] An example of a computer that executes the above program (data analysis program) will be explained using Figure 9. As shown in Figure 9, the computer 1000 has, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0092] Memory 1010 includes ROM (Read Only Memory) 1011 and RAM (Random Access Memory) 1012. ROM 1011 stores, for example, a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. The disk drive 1100 is used to insert a removable storage medium, such as a magnetic disk or an optical disk. The serial port interface 1050 is used to connect, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is used to connect, for example, a display 1130.

[0093] Here, as shown in Figure 9, the hard disk drive 1090 stores, for example, the OS 1091, the application program 1092, the program module 1093, and the program data 1094. The storage unit 13 described in the above embodiment is equipped, for example, in the hard disk drive 1090 or the memory 1010.

[0094] Then, the CPU 1020 reads the program module 1093 and program data 1094 stored in the hard disk drive 1090 into the RAM 1012 as needed and executes the above-described procedures.

[0095] Furthermore, the program module 1093 and program data 1094 related to the above-mentioned data analysis program are not limited to being stored in the hard disk drive 1090, but may also be stored in a removable storage medium and read by the CPU 1020 via a disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 related to the above-mentioned program may be stored in another computer connected via a network such as a LAN or WAN (Wide Area Network) and read by the CPU 1020 via a network interface 1070.

[0096] 10 Information processing device 11 Communication control unit 13 Storage unit 13A Data set 15 Control unit 15A Reception unit 15B Extraction unit 15C Analysis unit 15D Output unit 30 Client terminal

Claims

1. An information processing device comprising: a receiving unit that receives observed explanatory variables and a target variable; an extraction unit that extracts features of unobserved explanatory variables based on the observed explanatory variables and the target variable; an analysis unit that performs data analysis based on the features of the unobserved explanatory variables; and an output unit that outputs the results of the data analysis.

2. The information processing apparatus according to claim 1, characterized in that the extraction unit inputs the observed explanatory variables into a machine learning model whose internal parameters change according to the indices included in a predetermined series, thereby training the internal parameters of the machine learning model based on the error between the predicted value of the target variable output by the machine learning model and the target variable; the sequence of internal parameters of the trained machine learning model obtained by inputting each of the indices included in the series into the trained machine learning model; the sequence of cluster identification information obtained as a result of clustering is extracted as a sequence corresponding to the characteristics of the unobserved explanatory variables; and the analysis unit selects explanatory variables to propose additions based on the correlation of changes between the sequence corresponding to the characteristics of the unobserved explanatory variables and the elements included in the candidate set of explanatory variables.

3. A data analysis method performed by an information processing device, comprising: a reception step of receiving observed explanatory variables and a target variable; an extraction step of extracting features of unobserved explanatory variables based on the observed explanatory variables and the target variable; an analysis step of performing data analysis based on the features of the unobserved explanatory variables; and an output step of outputting the results of the data analysis.

4. A data analysis program that causes a computer to perform the following steps: an acceptance step for receiving observed explanatory variables and a target variable; an extraction step for extracting features of unobserved explanatory variables based on the observed explanatory variables and the target variable; an analysis step for performing data analysis based on the features of the unobserved explanatory variables; and an output step for outputting the results of the data analysis.

Images