Information processing method, information processing system, and information processing program

Abstract

To increase interpretability of attribute values and attributes of training data and test data for a learning model.SOLUTION: Regression models, in which latent variables output from MFCVAE in response to input of input data including reference data having a plurality of attributes of data assigned with attribute values are defined as explanatory variables and the attribute values are defined as objective variables, are set for the respective attributes. From the latent variables and the attribute values, prediction values of the attribute values at which prediction errors with respect to the attribute values become minimum and regression coefficients of the regression models are calculated for the respective attributes. On the basis of the prediction values and the regression coefficients for the respective attributes, function values of loss functions, of the MFCVAE, obtained by adding additional terms based on indices which are for the respective attributes and of which the values become smaller as the adaptation of the latent variables and the attribute values to the regression models becomes better are calculated. A model parameter of the MFCVAE is updated by error backpropagation based on the function values. As described above, model training of MFCVAE is executed.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present invention relates to an information processing method, an information processing system, and an information processing program. [Background technology]

[0002] The quality of an AI (Artificial Intelligence) model depends on the quality of the data used. To ensure the quality of an AI model, it is useful to evaluate information about the attributes of the training data used to build the AI ​​model and the test data used to perform inference. To achieve this, variational autoencoder technology is used to extract latent variables (features) from the encoder of the learning model, clarify the content of the attribute information of the data, and extract data with similar attribute information.

[0003] For example, in Patent Document 1, three images corresponding to the same semantic feature are extracted from training data, and for each of the three images, the parameters of the variational autoencoder are updated so as to minimize the loss function of the latent variable corresponding to the semantic feature, thereby improving the discrimination ability of different images having the same semantic feature.

[0004] Furthermore, for example, in Non-Patent Document 1, the independence of latent variables is increased so that the information provided by each latent variable for an input value is unique, thereby improving the interpretability of the attribute content and changes in its magnitude corresponding to changes in latent variables. As a result, for example, in handwritten character data, it can be seen that the angle of the characters changes continuously from diagonally left to diagonally right in response to a change in a certain latent variable.

[0005] Furthermore, for example, in Non-Patent Document 2, latent variables are expressed as linear combinations of orthogonal bases, and the basis coefficients obtained by learning are associated with changes in data attributes, thereby improving the interpretability of the content and magnitude of changes in attributes corresponding to changes in the basis coefficients. As a result, for example, in face image data, it can be seen that hair color changes continuously from gold to black in response to changes in a certain basis coefficient. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2019-75108 [Non-patent literature]

[0007] [Non-Patent Document 1] Shuyang Gao, Rob Brekelmans, Greg Ver Steeg, Aram Galstyan, “Auto-Encoding Total Correlation Explanation,” Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics pages 1157-1166.,[online],Proceedings of Machine Learning Research (PMLR) 2019.,[Retrieved August 1, 2022],Internet<URL:https: / / arxiv.org / abs / 1802.05822> [Non-patent document 2] Jin-Young Kim, Sung-Bae Cho, “BasisVAE: Orthogonal Latent Space for Deep DisentangledRepresentation,” [online], International Conference on Learning Representations (ICLR) 2020. [Retrieved August 1, 2022], Internet<URL:https: / / arxiv.org / abs / 1802.05822> Summary of the Invention [Problem to be solved by the invention]

[0008] However, in the above-mentioned conventional technology, the latent variables and their corresponding attributes and attribute values ​​depend on the user's interpretation and can only be evaluated qualitatively, so there is still a problem that the interpretability of the data attributes and attribute values ​​is low.

[0009] One aspect of the disclosure of the present application aims to improve the interpretability of attributes and attribute values ​​corresponding to latent variables in training data and test data. [Means for solving the problem]

[0010] In one aspect of the disclosure of the present application, there is provided an information processing method executed by an information processing system having a processing unit and a storage unit, wherein the processing unit includes a first step of inputting input data including reference data in which attribute values ​​are assigned to a plurality of attributes of the data to an MFCVAE (Multi-Facet Clustering Variational Auto-Encoder) that outputs latent variables related to each of the plurality of attributes of the data; a second step of setting, for each attribute, a regression model in which the latent variables output from the MFCVAE for the input of the input data are used as explanatory variables and the attribute values ​​are used as target variables; a third step of calculating, for each attribute, a predicted value of the attribute value and a regression coefficient of the regression model from the latent variables and the attribute values ​​that minimizes a prediction error for the attribute value; and a fourth step of calculating, for each attribute, an index that takes a smaller value as the latent variable and the attribute value fit the regression model better, based on the predicted value and the regression coefficient for each attribute calculated in the third step. a fifth step of calculating a function value of a loss function obtained by adding an additional term based on the index for each attribute to the loss function of the MFCVAE, the loss function having a latent variable, a reconstruction error term representing the error in data reconstruction by the MFCVAE, and a regularization term that constrains the distribution of the latent variables; and a sixth step of updating model parameters of the MFCVAE by error backpropagation based on the function values ​​calculated in the fifth step, and performing model learning of the MFCVAE by repeating the first step to the sixth step in this order until the prediction error or the number of epochs satisfies a predetermined condition. [Effects of the Invention]

[0011] According to one aspect of the disclosure of the present application, it is possible to improve the interpretability of attributes and attribute values ​​of training data and test data of a learning model. Problems, configurations, and effects other than those described above will become clear from the description of the following embodiments. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram for explaining a problem of the conventional technology (MFCVAE). [Figure 2] FIG. 10 is a diagram showing reference data and evaluation data (in the case of character data). [Figure 3] FIG. 1 is a diagram showing reference data and evaluation data (in the case of general data). [Figure 4] FIG. 2 is a diagram for explaining the operation of the information processing system according to the first embodiment during model learning. [Figure 5] FIG. 3 is a diagram for explaining the operation of the information processing system according to the first embodiment when assigning attribute values ​​to evaluation data. [Figure 6] FIG. 2 is a diagram for explaining the operation of the information processing system according to the first embodiment when generating data with specified attribute values. [Figure 7] FIG. 4 is a diagram for explaining the transition of the coefficient of determination during model learning in the information processing system according to the first embodiment. [Figure 8] FIG. 1 is a block diagram showing the configuration of an information processing system according to a first embodiment. [Figure 9] 4 is a flowchart showing feature extraction processing according to the first embodiment. [Figure 10] 10 is a flowchart showing an attribute value assignment process according to the first embodiment. [Figure 11] 4 is a flowchart showing a data generation process according to the first embodiment. [Figure 12] 1 is a flowchart showing a data quality evaluation process according to the first embodiment. [Figure 13] FIG. 10 is a diagram showing an output example 1 of attributes and attribute values ​​(attributes and attribute values ​​for data) according to the first embodiment. [Figure 14]FIG. 10 is a diagram showing an output example 2 of data, attributes, and attribute values ​​(the number of data items for each attribute and attribute value) according to the first embodiment. [Figure 15] FIG. 10 is a block diagram showing the configuration of an information processing system according to a second embodiment. [Figure 16] FIG. 1 is a diagram showing the hardware configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments according to the disclosure of the present application will be described with reference to the drawings. The embodiments, including the drawings, are examples for explaining the present application. In the embodiments, appropriate omissions and simplifications have been made for clarity of explanation. Unless otherwise specified, components of the embodiments may be singular or plural. Furthermore, a combination of one embodiment with another embodiment is also included in the embodiments according to the present application.

[0014] Identical or similar components are given the same reference numerals, and descriptions of the previously described components in later embodiments may be omitted, or descriptions may be focused on differences. Furthermore, when there are multiple identical or similar components, they may be described with the same reference numerals but with different subscripts. Furthermore, when it is not necessary to distinguish between these multiple components, the subscripts may be omitted.

[0015] In the following embodiments, various types of information are described in table format, but the various types of information may be in a data format other than a table format. Furthermore, various names such as "XX information," "XX table," "XX list," and "XX queue" are interchangeable. For example, "XX table" may be called "XX list." Furthermore, when describing identification information, expressions such as "identification information," "identifier," "name," "ID," and "number" are used, but these are interchangeable.

[0016] (Problems with the prior art) Prior to describing the embodiments, problems with the conventional technology (MFCVAE: Multi-Facet Clustering Variational Autoencoders) on which the embodiments are based will be described. FIG. 1 is a diagram for explaining the problems with the conventional technology. MFCVAE is an extended variational autoencoder (VAE: Variational Auto-Encoder) that can output latent variables from multiple perspectives. A variational autoencoder is a generative model that uses a neural network and assumes a probability distribution as the space of latent variables. A perspective in MFCVAE refers to the type of latent variable (vector) output by MFCVAE, and in the case of character data, this corresponds to "character type" and "character shape (thickness, angle, etc.)," ​​etc.

[0017] Variational autoencoders are described in reference 1: Diederik P Kingma, Max Welling, “Auto-Encoding Variational Bayes,” May 2014. [Retrieved August 1, 2022], Internet<URL:https: / / arxiv.org / abs / 1312.6114> MFCVAE is disclosed in Reference 2, "Fabian Falck et al., "Multi-Facet Clustering Variational Autoencoders, Oct. 2021. [Retrieved August 1, 2022], Internet<URL:https: / / arxiv.org / abs / 2106.05241> " is disclosed.

[0018] Hereinafter, a feature is information that characterizes data (attributes, attribute values, latent variables, etc.). A feature is a value of a feature that can be expressed quantitatively. An attribute is a property that characterizes data (for example, "thickness," "slope," "amount of noise," "character fragmentation," etc., in the case of text data). An attribute value is a value that indicates the degree of an attribute (for example, "1 mm" for the attribute "thickness," "10 degrees" for the attribute "slope," "10%" for the attribute "amount of noise," "level 2" for the attribute "character fragmentation")). Attribute values ​​can be continuous or discrete. In variational autoencoder-related technologies, a latent variable is a feature output from an encoder. Variational autoencoder-related technologies refer to all variational autoencoder technologies that use variational Bayesian algorithms, including VAE and MFCVAE.

[0019] In MFCVAE, the attribute values ​​of multiple attributes change with a change in one latent variable. This makes it difficult to associate attributes that change with a change in a latent variable. Referring to Figure 1, we will explain the attributes and attribute values ​​for handwritten characters as an example.

[0020] As shown in FIG. 1, in a coordinate system with latent variable 1 on the horizontal axis and latent variable 2 on the vertical axis, group 101 of latent variables 1 and 2 is a group of latent variables corresponding to characters with a thin attribute "thickness." Group 102 is a group of latent variables corresponding to characters with a medium attribute "thickness." Group 103 is a group of latent variables corresponding to characters with a thick attribute "thickness." Group 104 is a group of latent variables corresponding to characters with a low attribute "noise level." Group 105 is a group of latent variables corresponding to characters with a medium attribute "noise level." Group 106 is a group of latent variables corresponding to characters with a high attribute "noise level." Here, "thick," "medium," and "thin" for the attribute "thickness" and "high," "medium," and "low" for the attribute "noise level" are merely exemplary expressions and are examples of quantitative expressions or labels assigned thereto.

[0021] In the example shown in Figure 1, not all attribute values ​​increase or decrease uniformly with changes in the latent variables. This makes it difficult to associate changes in attribute values ​​with changes in a certain latent variable. For example, in Figure 1, when the value of latent variable 1 increases in two stages from small group 101 to group 103 and then to group 102, the attribute value of the attribute "thickness" changes from "thin," "thick," to "medium." However, the attribute values ​​do not change uniformly with the uniform increase in the value of latent variable 1. The same is true for the attribute value of the attribute "noise amount" with changes in latent variable 1. As a result, it is difficult to interpret changes in attribute values ​​with changes in a certain latent variable.

[0022] On the other hand, when the value of latent variable 2 increases in two stages from small group 103 to group 102 and then to group 101, the attribute value decreases uniformly relative to the uniform increase in the value of latent variable 1, just as the attribute value of the attribute "thickness" changes from "thick" to "medium" to "thin." The attribute value of the attribute "amount of noise" related to latent variable 2 changes in the same way. As a result, the attribute value decreases uniformly relative to the change in the latent variable, making it easy to interpret the change in the attribute value relative to a change in a certain latent variable.

[0023] (Baseline data and evaluation data) First, we will explain the reference data and evaluation data for character data. Figure 2 is a diagram showing the reference data and evaluation data (in the case of character data). In Figure 2, each line is called a "data." For each data, there is data in which attribute values ​​are stored for each attribute such as "character type," "thickness," and "slant," and there is data in which attribute values ​​are not stored. "Character type" is image data of the corresponding character.

[0024] For a certain attribute, data in which an attribute value is stored is reference data for that attribute, and data in which no attribute value is stored is evaluation data for that attribute. Reference data is data in which the attribute values ​​are known and used to determine the attribute values ​​of the evaluation data. Evaluation data is data to which attribute values ​​are determined and assigned for attributes with unknown attribute values.

[0025] In the case of character data, it is generally easy to assign attribute values ​​such as "A" or "B" to handwritten character data, but it is not easy to assign attribute values ​​such as "thickness" or "slant." Therefore, we train a variational autoencoder model using typefaces such as Gothic fonts, which allow for the attribute values ​​of attributes such as "thickness" and "slant" to be changed.

[0026] In the example of Figure 2, the data with "Data Numbers" "1," "2," and "3" for the "Data Attribute" "Typed Characters" that have attribute values ​​such as "thickness" and "slant" are the reference data, while the data with "Data Numbers" "4" and "5" for the "Data Attribute" "Handwritten Characters" that have no attribute values ​​are the evaluation data. More generally, the reference data includes both typed characters and handwritten characters, and the evaluation data includes handwritten characters.

[0027] Next, the reference data and evaluation data for general data will be explained. Fig. 3 is a diagram showing the reference data and evaluation data (for general data). In Fig. 2, the reference data and evaluation data were separated for each data attribute. In Fig. 3, the reference data and evaluation data are generalized, and for each data, data in which an attribute value is stored for each attribute is the reference data for the attribute, and data in which an attribute value is not stored for the attribute is the evaluation data for the attribute.

[0028] For "Attribute 1," the reference data is the data with "Data Number" "1," "2," "3," and "4," and the evaluation data is the data with "Data Number" "5." Similarly, for "Attribute 2," the reference data is the data with "Data Number" "1," "2," and "5," and the evaluation data is the data with "Data Number" "3" and "4."

[0029] The purpose of the following embodiment is to estimate the attribute value of the evaluation data for each attribute by using the attribute value of the reference data.

[0030] [Embodiment 1] (Operations during model training and data reconstruction) 4 is a diagram for explaining the operation of the information processing system 1 according to embodiment 1 during model learning. In this embodiment, reference data 201 in which attribute values ​​are assigned in advance to attributes is used, and model learning is performed using a new loss function in variational autoencoder-related technology so that attribute values ​​can be predicted using a multiple regression model based on latent variables of the reference data 201.

[0031] The information processing system 1 includes an MFCVAE 2. The MFCVAE 2 includes an encoder 203 and a decoder 205.

[0032] During model learning of the MFCVAE2, the data (training data set) input to the encoder 203 includes only the reference data 201, or includes both the reference data 201 and the evaluation data 202. The encoder 203 outputs latent variables 204, which are intermediate outputs of the MFCVAE2. The MFCVAE2 sets a multiple regression model in which the latent variables 204 are explanatory variables and the correct values ​​(correct labels) 208 of the attribute values ​​are objective variables. Note that the model is not limited to a multiple regression model, and either a linear regression model or a nonlinear regression model may be used. The multiple regression model has the advantage of reducing the load on model calculations, and so the multiple regression model is adopted in this embodiment.

[0033] The information processing system 1 calculates, from the latent variables 204 and the correct attribute values ​​208, partial regression coefficients 209 that minimize the mean square error between the predicted attribute value and the correct value, and a predicted attribute value 210 at that time. The predicted attribute value 210 is calculated as a linear combination of the latent variables 204, each of which has the partial regression coefficient 209 as its coefficient. Based on the correct attribute values ​​208 and the predicted attribute value 210, a coefficient of determination 211 and a prediction error 212 are calculated, which can serve as indicators of goodness of fit, and the better the fit to the multiple regression model (the fit of the multiple regression model), the smaller the value becomes. The coefficient of determination 211 and the prediction error 212 are trained using a loss function 213 that includes an indicator of goodness of fit to the multiple regression model. In this way, the encoder 203 is trained.

[0034] The latent variables 204 are input to a decoder 205. The decoder 205 outputs reconstructed reference data 206, which is the reference data 201 reconstructed by the decoder 205, and reconstructed evaluation data 207, which is the evaluation data 202 reconstructed by the decoder 205. The reconstructed evaluation data 207 is data to which attribute values ​​are assigned.

[0035] Here, the partial regression coefficient 209, the loss function 213, the coefficient of determination 211, and the prediction error 212 will be explained.

[0036] In conventional MFCVAE, the objective function, the evidence lower bound (ELBO), is expressed as in equation (1). In conventional MFCVAE, the parameters of the MFCVAE model are trained so as to minimize the negative loss function, which is the negative sign of the variational lower bound in equation (1) (see reference 2 mentioned above). In equation (1), "D" represents the training dataset, "x" represents the training data included in the training dataset, "z→" represents the latent variable, "θ" represents the encoder parameter, "φ" represents the decoder parameter, and "KL(A|B)" represents the KL divergence between distribution A and distribution B.

[0037]

number

[0038] In contrast, in this embodiment, the coefficient of determination adjusted for the degree of freedom at viewpoint j is R f,j 2 , the coefficient of determination adjusted for each degree of freedom R f,j 2 The weighting coefficient of γ j (>0), and set the objective function as in equation (2). The right-hand side of equation (2) is the third term γ j R f,j 2 has been added.

[0039] In addition, the third term of equation (2) is the adjusted coefficient of determination R f,j 2Instead, the coefficient of determination R j 2 may be adopted. The coefficient of determination R j 2 , adjusted coefficient of determination R f,j 2 is an example of a coefficient of determination 211.

[0040]

number

[0041] Here, the first term in the parentheses of the expected value E[*] in equation (2) is a reconstruction error term that represents the error in data reconstruction by MFCVAE. The second term in the parentheses of the expected value E[*] in equation (2) is a regularization term that constrains the distribution of the latent variables of MFCVAE, such as suppressing the variance of the latent variables. The third term in the parentheses of the expected value E[*] in equation (2) is an additional term based on an index that takes a smaller value the better the fit of the multiple regression model to the latent variables and attribute values ​​for each attribute.

[0042] The loss function 213 (represented as loss function Loss) is a negative objective function, and is therefore expressed as in equation (3) using the objective function of equation (2).

[0043]

number

[0044] In this embodiment, the loss function Loss is minimized, that is, the coefficient of determination R f,i 2 The parameters of the MFCVAE model are trained so that maximizing

[0045] Next, the coefficient of determination for viewpoint j, R j 2 , and the adjusted coefficient of determination R for viewpoint j f,j 2 The calculation method will be explained.

[0046] The number of data is N, the number of types of data attributes is J (j=1,2,…,J), and the number of dimensions of the latent variable for a certain attribute j is K. j Also, each attribute of the data corresponds one-to-one to each of the J viewpoints of the MFCVAE. If data n with index number n is assigned an attribute value of attribute j, then data n is reference data for attribute j. On the other hand, if data n is not assigned an attribute value of attribute j, then data n is evaluation data for attribute j.

[0047] Let B be the set of indices of the reference data for a certain attribute j. j , set B j The number of elements in M j Set B j ={b j,1 ,b j,2 ,…,b j,Mj}. Let z be the latent variable in the reference data n for a certain attribute j. n,j ={z n,j,1 ,z n,j,2 ,…,z n,j,Kj}, the correct value of the attribute value is y n,j , latent variable z n,j is the explanatory variable. In addition, the partial regression coefficient of the multiple regression model with the attribute value as the objective variable is w j ={w j,0 ,w j,1 ,w j,2 ,…,w j,Kj} T , the predicted value of the attribute value is y^ n,j The predicted value of the attribute value y^ n,j is expressed as in equation (4).

[0048]

number

[0049] However, as shown in equation (5), the latent variable vector Z n,j was defined.

number

[0050] Attribute value y for some attribute j n,j (where n∈B j ) and the predicted value y^ of the attribute value by the multiple regression model n,j The mean squared error (MSE) is the prediction error between j is expressed as in equation (6). The mean square error MSEj is an example of the prediction error 212.

number

[0051] Here, the mean square error MSE j Minimize w j The right side of equation (6) is w j Partially differentiate and set it to zero (∇w j = 0), the partial regression coefficient w j is the latent variable Z j and attribute value y j The partial regression coefficient w j is an example of a partial regression coefficient 209.

number

[0052] However, in equation (7), the latent variable Z j and attribute value y j is set as shown in equations (8) and (9).

number

number

[0053] Note that the attribute value y for a certain attribute j n,j (where n∈B j ) and the predicted value y^ of the attribute value by the multiple regression model n,j The prediction error is not limited to the mean square error, but may also be the average error, mean absolute error, mean square error, mean error rate, mean absolute error rate, etc.

[0054] The coefficient of determination R for a given attribute j j 2 represents how well the explanatory variables explain the target variable. j 2 is the average value of the attribute value y n,j This is expressed as equation (10) using

number

[0055] In addition, the coefficient of determination has the property that it approaches 1 as the number of explanatory variables increases. Therefore, when there are many explanatory variables, the coefficient of determination (R) adjusted for the degrees of freedom is used to correct this property. f,j 2 The adjusted coefficient of determination R f,j 2 The number of explanatory variables is p, and the number of samples of the standard data is M j Therefore, it is expressed as in equation (11).

number

[0056] In addition, the weighting coefficient γ j are the reconstruction error term, regularization term, and coefficient of determination R j 2 (R f,j 2 ) can be calculated by comparing the absolute values ​​of the coefficient of determination R j 2 Weighting coefficient γ for j is |γ j R j 2 The order of | is determined to be the same as the order of the absolute values ​​of the reconstruction error term and regularization term in the equation in parentheses of the expected value E[*] on the right side of equation (2). Similarly, the adjusted coefficient of determination R f,j 2 Weighting coefficient γ for j Also, |γ j R f,j 2The order of | is determined to be the same as the order of the absolute values ​​of the reconstruction error term and regularization term in the equation in parentheses of the expected value E[*] on the right-hand side of equation (2).

[0057] When reconstructing the reference data 201 and the evaluation data 202, the decoder 205 uses the attributes and attribute values ​​of the reference data 201 and the evaluation data 202 (the assigned attribute values ​​in the case of the evaluation data 202) and the partial regression coefficients 209 obtained in the last epoch of learning by the encoder 203. Then, using the attributes and attribute values ​​of the reference data 201 and the evaluation data 202 and the partial regression coefficients 209, the latent variables (latent variable vector Z n,j ) is calculated. The decoder 205 then receives the calculated latent variables as input and outputs reconstructed reference data 206 and reconstructed evaluation data 207, which are obtained by reconstructing the input reference data 201 and evaluation data 202, respectively.

[0058] (Behavior when assigning attribute values) 5 is a diagram for explaining the operation of the information processing system 1 according to the first embodiment when assigning an attribute value to evaluation data 202. When assigning an attribute value to evaluation data 202, the information processing system 1 first inputs the evaluation data 202 to a trained encoder 203 to obtain a latent variable 204. The information processing system 1 uses partial regression coefficients 209 obtained in the final epoch of learning of the MFCVAE2 (FIG. 4) to calculate a predicted value 210 of the attribute value by a linear combination equation of the latent variables 204 in which the partial regression coefficients 209 are used as coefficients. The information processing system 1 assigns the predicted value 210 of the attribute value to the evaluation data 202.

[0059] (Data generation behavior) FIG. 6 is a diagram illustrating the operation of the information processing system 1 according to the first embodiment when generating data with specified attribute values. In variational autoencoder-related technology, data generation refers to inputting latent variables and outputting data from a decoder. When generating data 405 having a desired attribute value 401, the information processing system 1 calculates a latent variable 204 from the attribute and attribute value 401 that the user wants to generate and the partial regression coefficient 209 obtained at the end of learning the MFCVAE2 (FIG. 4). The information processing system 1 then inputs the calculated latent variable 204 to the decoder 205 to generate data 405 having the desired attribute value 401.

[0060] When generating data, if there is reference data 201 that corresponds to the specified attribute and attribute value 401, the data corresponding to this reference data 201 is used as reconstructed data 405. If there is no reference data 201 that corresponds to the specified attribute and attribute value 401, latent variables 204 are calculated from the attribute and attribute value 401 to be generated and the partial regression coefficients 209. Then, by inputting the calculated latent variables 204 to a decoder 205, data 405 having the attribute value 401 to be generated is generated.

[0061] (Changes in coefficient of determination during model training) FIG. 7 is a diagram illustrating the transition of the coefficient of determination during model training in the information processing system 1 according to the first embodiment. In the graph in FIG. 7, the horizontal axis represents the latent variables and the vertical axis represents the attribute values, with actual values ​​of the attribute values ​​represented by dots and predicted values ​​of the attribute values ​​represented by lines. The information processing system 1 adds an additional term, including the coefficient of determination of a multiple regression model in which the latent variables are the explanatory variables and the attribute values ​​are the objective functions, to the loss function of the MFCVAE2, and trains the MFCVAE2 so as to increase the coefficient of determination. As a result, the coefficient of determination is low in the early stages of learning ( FIG. 7( a) ). However, as the number of learning epochs progresses and the learning progresses to the middle ( FIG. 7( b) ) and later ( FIG. 7( c) ) stages, the coefficient of determination increases, and the prediction accuracy of attribute values ​​based on the latent variables improves.

[0062] (Configuration of Information Processing System 1 According to Embodiment 1) 8 is a block diagram showing the configuration of the information processing system 1 according to embodiment 1. The information processing system 1 includes a data storage unit 602, a feature extraction unit 603, an attribute value assignment unit 608, a data generation unit 614, and a data quality evaluation unit 612.

[0063] The data storage unit 602 is a memory or storage, and receives and stores the input of the reference data 201 and the evaluation data 202. The data storage unit 602 may be either a device included in the information processing system 1 or an external device to the information processing system 1.

[0064] The feature extraction unit 603 performs model learning for the MFCVAE2 based on the reference data 201 stored in the data storage unit 602 of the MFCVAE2. The feature extraction unit 603 also estimates attributes of the evaluation data 202 stored in the data storage unit 602 of the MFCVAE2. The feature extraction unit 603 also generates data with specified attribute values. The feature extraction unit 603 has a regression model fitness evaluation unit 604, a loss calculation unit 605, a model update unit 606, and an encoder unit 607. The processing functions of the feature extraction unit 603 will be described later with reference to FIG. 9.

[0065] The attribute value assignment unit 608 estimates the attributes of the evaluation data 202 and outputs the attributes and attribute values ​​611 of the evaluation data 202. The attribute value assignment unit 608 includes an attribute value estimation unit 609 and an attribute and attribute value output unit 610. The processing function of the attribute value assignment unit 608 will be described later with reference to FIG. 10 .

[0066] The data generation unit 614 receives the attributes and attribute values ​​611 of the target data output by the attribute value assignment unit 608 as input, generates data with the specified attribute values, and outputs the generated data 405. The data generation unit 614 includes a latent variable calculation unit 615, a decoding unit 616, and a data output unit 617. The processing functions of the data generation unit 614 will be described later with reference to FIG. 11 .

[0067] The data quality evaluation unit 612 evaluates the quality of the target data based on the attributes and attribute values ​​611 of the target data (reference data 201, evaluation data 202) output by the attribute value assignment unit 608, and outputs a data quality evaluation result 613.

[0068] The data quality evaluation unit 612 evaluates the quality of the target data from the following perspectives, for example, using the attributes and attribute values ​​611 of the target data. For quality evaluation, see "Machine Learning Quality Management Guidelines," National Institute of Advanced Industrial Science and Technology, [searched August 1, 2022], and the Internet.<URL:https: / / www.aist.go.jp / aist_j / press_release / pr2020 / pr20200630_2 / pr20200630_2.html> Please refer to the following. (1) Sufficiency of data design: Ensuring that sufficient training and test data is available for the various situations that the target system using the data must be able to handle. (2) Coverage of the data set: For each case that has been covered by the criteria, there is no omission of the possible inputs corresponding to each case, and a sufficient amount of data is provided for learning that can make correct inferences for both rare and common cases. (3) Data uniformity: Prepare training data without bias so as to maximize the expected value of inference performance overall.

[0069] The processing function of the data quality evaluation unit 612 will be described later with reference to FIG.

[0070] Furthermore, the feature extraction unit 603, the attribute value assignment unit 608, the data generation unit 614, and the data quality evaluation unit 612 may be realized on one computer or on different computers, and the form of integration and distribution of these units can be changed as appropriate.

[0071] (Feature Extraction Process According to the First Embodiment) 9 is a flowchart showing the feature extraction process according to embodiment 1. The feature extraction process is executed by the feature extraction unit 603 (FIG. 8) in response to a user instruction.

[0072] First, in step S11, the regression model fitness evaluation unit 604 sets an index indicating the goodness of fit of the MFCVAE model (multiple regression model in this embodiment) in which the latent variables of the reference data 201 are used as explanatory variables and the attribute values ​​are used as objective variables, to the loss function Loss. In this embodiment, the index set in the loss function Loss is the coefficient of determination R adjusted for the degrees of freedom at viewpoint j. f,j 2 is.

[0073] Next, in step S12, the regression model fitness evaluation unit 604 initializes the MFCVAE model. Next, in step S13, the regression model fitness evaluation unit 604 inputs the reference data 201 and the evaluation data 202 to the MFCVAE model. In step S13, the regression model fitness evaluation unit 604 only needs to input at least the reference data 201 to the MFCVAE model.

[0074] Next, in step S14, the loss calculation unit 605 calculates the function value of the loss function Loss based on equation (3). The regression model fitness evaluation unit 604 performs the following process as a preliminary step before the loss calculation unit 605 calculates the function value of the loss function Loss. That is, the regression model fitness evaluation unit 604 sets, for each attribute, a multiple regression model in which the latent variables output from the MFCVAE model for the input data in step S13 are used as explanatory variables and the attribute values ​​are used as response variables. Next, the regression model fitness evaluation unit 604 calculates, for each attribute, the predicted values ​​of the attribute values ​​and the regression coefficients of the multiple regression model that minimize the prediction error for the attribute values, based on the latent variables and attribute values ​​calculated. Next, the regression model fitness evaluation unit 604 calculates, for each attribute, an index that takes a smaller value as the fit of the multiple regression model to the latent variables and attribute values ​​improves. Thereafter, in step S14, the loss calculation unit 605 calculates the function value of the loss function Loss.

[0075] If the input data includes the reference data 201 and the evaluation data 202, in step S14, the regression model fitness evaluation unit 604 calculates the additional term of the loss function Loss using the reference data 201. On the other hand, the loss calculation unit 605 calculates the reconstruction error term and the regularization term using either or both of the reference data 201 and the evaluation data 202. This is because the additional term of the loss function Loss is based on the fitness of the latent variables and attribute values ​​to the multiple regression model, and therefore the additional term of the loss function Loss can be calculated only for reference data including attribute values.

[0076] Next, in step S15, the model update unit 606 updates the parameters of the MFCVAE model by error backpropagation. Next, in step S16, the model update unit 606 determines whether a predetermined condition (whether the number of epochs exceeds a predetermined number, or the error between the estimated value by the MFCVAE model and the actual value falls below a predetermined value) is satisfied. If the predetermined condition is satisfied (step S16 YES), the model update unit 606 proceeds to step S17, and if the predetermined condition is not satisfied (step S16 NO), the model update unit 606 returns to step S13.

[0077] In step S17, the encoder unit 607 inputs the evaluation data 202 into the encoder 203 of the trained MFCVAE model, and outputs the latent variables 204 and the partial regression coefficients 209.

[0078] (Attribute value assignment process according to the first embodiment) 10 is a flowchart showing the attribute value assignment process according to embodiment 1. The attribute value assignment process is executed by the attribute value assignment unit 608 (FIG. 8) in response to a user instruction.

[0079] First, in step S21, the attribute value estimation unit 609 calculates predicted values ​​210 of attribute values ​​of the evaluation data 202 from the latent variables 204 and partial regression coefficients 209 obtained from the feature extraction unit 603 (encoder 203). Next, in step S22, the attribute and attribute value output unit 610 calculates a histogram of the appearance frequency of each attribute value for each attribute from the attributes and attribute values ​​of the reference data 201 and the evaluation data 202 for which the predicted values ​​210 of the attribute values ​​were calculated in step S21 (see FIG. 14 described below). Then, based on this histogram, the attribute and attribute value output unit 610 calculates the probability that each attribute value will appear in each attribute of the reference data 201 and the evaluation data 202 as a data content rate, and outputs the result.

[0080] (Data generation process according to the first embodiment) 11 is a flowchart showing the data generation process according to embodiment 1. The data generation process is executed by the data generation unit 614 (FIG. 8) in response to a user instruction.

[0081] First, in step S31, the data generation unit 614 accepts input of attributes and attribute values ​​401 to be generated by the user. Next, in step S32, the latent variable calculation unit 615 calculates and outputs latent variables 204 from the attributes and attribute values ​​401 accepted as input in step S31 and the partial regression coefficients 209. Next, in step S33, the decoding unit 616 (decoder 205) reconstructs data 405 (e.g., character data) having the attributes and attribute values ​​401 to be generated, using the latent variables 204 calculated in step S32 as input. Next, in step S34, the data output unit 617 outputs the data 405 reconstructed by the decoding unit 616 (decoder 205).

[0082] If there is reference data 201 corresponding to the attribute and attribute value 401 received in step S31, the latent variable calculation unit 615 skips step S32 and reconstructs the data corresponding to this reference data 201 as data 405 in step S33.

[0083] (Data Quality Evaluation Process According to the First Embodiment) 12 is a flowchart showing the data quality evaluation process according to embodiment 1. The data quality evaluation process is executed by the data quality evaluation unit 612 (FIG. 8) in response to a user instruction.

[0084] In step S41, the data quality evaluation unit 612 evaluates the attributes and attribute values ​​611 output by the attribute value assignment unit 608 from at least one of the above-mentioned viewpoints: (1) sufficiency of data design, (2) coverage of the data set, and (3) uniformity of the data set. Next, in step S42, the data quality evaluation unit 612 outputs a data quality evaluation result 613 of step S41.

[0085] (Attribute and attribute value output example 1) Fig. 13 is a diagram showing an output example 1 of attributes and attribute values ​​(attributes and attribute values ​​for data). Fig. 13 shows attribute values ​​assigned by the attribute value estimation unit 609 (Fig. 8) of the attribute value assigning unit 608 to data to which no attribute values ​​have been assigned, for example, as shown in Fig. 2 or 3, and output by the attribute and attribute value output unit 610.

[0086] (Attribute and attribute value output example 2) FIG. 14 is a diagram showing an output example 2 of data, attributes, and attribute values ​​(the number of data items for each attribute and attribute value). FIG. 14 is an output example obtained by changing the display method of FIG. 13. FIG. 14 is a histogram of attribute values ​​for each attribute output by the attribute and attribute value output unit 610 (FIG. 8) of the attribute value assigning unit 608. This display makes it possible to confirm, for example, the above-mentioned (2) coverage of the data set and (3) uniformity of the data for each attribute. (2) Coverage of the data set is considered to be satisfied when the attribute values ​​of each attribute in the histogram of FIG. 14 are distributed over a predetermined wide range and each frequency is equal to or greater than a predetermined number. (3) Uniformity of the data is considered to be satisfied when the attribute values ​​of each attribute in the histogram of FIG. 14 are evenly distributed over a predetermined wide range. Such an analysis makes it possible to identify data that is lacking for an attribute value.

[0087] For example, histogram 1101 in FIG. 14 shows the frequency distribution of attribute 1. Histogram 1101 has a wider or equivalent data distribution range than histograms 1102 and 1103, but there are attribute values ​​that do not exist within this distribution range. In this respect, histogram 1101 can be said to not satisfy (2) data set coverage. Histogram 1101 also has an uneven distribution of attribute values. The uniformity of the distribution of attribute values ​​can be determined based on statistical values ​​that represent variations in attribute values, such as the variance and standard deviation. In this respect, histogram 1101 can be said to not satisfy (3) data uniformity.

[0088] Histogram 1102 in FIG. 14 shows the frequency distribution of attribute 2. Histogram 1102 has a narrower data distribution range than histograms 1101 and 1103, and there are attribute values ​​that do not exist in this distribution range. In this respect, histogram 1102 can be said to not satisfy (2) data set coverage. Histogram 1102 also has an uneven distribution of attribute values. In this respect, histogram 1102 can be said to not satisfy (3) data uniformity.

[0089] 14 shows the frequency distribution of attribute J. Histogram 1103 has a wider or equivalent data distribution range than histograms 1101 and 1102, but there are attribute values ​​that do not exist within this distribution range. In this respect, histogram 1103 can be said to not satisfy (2) data set coverage. Furthermore, histogram 1103 has a less uniform distribution of attribute values ​​than histograms 1101 and 1102. In this respect, histogram 1102 can be said to not satisfy (3) data uniformity.

[0090] Each graph in FIG. 14 may be a graph in which the vertical axis represents the "data content rate at which each attribute value appears in each attribute of the reference data 201 and the evaluation data 202" instead of the "number of data."

[0091] [Effects of the First Embodiment] In this embodiment, the user explicitly specifies the attributes of the training data and test data and expresses them as quantitative attribute values, enabling attribute analysis with high user interpretability. This makes it easier to discover missing data in the training data and test data, or features of data that are often misclassified.

[0092] Furthermore, in this embodiment, unlike conventional techniques, the user does not have to interpret the attributes of the obtained latent variables (e.g., unevenly distributed variables depend on thickness or angle), but the user can explicitly specify the attributes that the latent variables should have, making it possible to perform attribute analysis according to the user's intentions.

[0093] Furthermore, in conventional techniques, attributes could only be known qualitatively, making it impossible to compare attributes between different data sets or data trained with different models. However, in this embodiment, attribute values ​​can be determined quantitatively, making it possible to compare attributes between different data sets or data trained with different models.

[0094] Furthermore, in this embodiment, even when limited to predicting attribute values, attribute values ​​can be assigned with a smaller amount of data or learning than when attribute values ​​are estimated using a regression model based on supervised learning.

[0095] Furthermore, in this embodiment, data can be generated by specifying attribute values ​​for multiple attributes, making it easy to generate the required data. Furthermore, if reference data corresponding to the specified attributes and attribute values ​​exists during data generation, the data corresponding to this reference data is used as the reconstructed data. This allows data to be reconstructed more quickly than when latent variables are calculated from attributes, attribute values, and partial regression coefficients, and then decoded using a decoder to reconstruct the data.

[0096] [Embodiment 2] In the first embodiment, an example has been shown in which model learning (FIGS. 4 and 9), attribute value assignment and output of attribute-value relationships (FIGS. 5 and 10), and data quality evaluation processing (FIG. 12) are performed using one information processing system 1. However, the model learning, attribute value assignment, and data quality evaluation processing may be performed in parallel by multiple information processing systems 1 (1-1, 1-2, ..., 1-n) shown in FIG.

[0097] For example, when model learning is performed by multiple information processing systems 1, steps S13 to S16 (FIG. 9) may be performed using input data including different reference data 201 for each of the multiple information processing systems 1. Then, at least one of the multiple information processing systems 1 merges and outputs the learning results of the MFCVAE model obtained by each information processing system 1.

[0098] Furthermore, each information processing system 1 may assign attribute values ​​to each input data and output the relationships between the attributes and the attribute values ​​(steps S21 to S23 (FIG. 10)) based on the learning results of each MFCVAE model obtained for each of the multiple information processing systems 1. Then, at least one of the multiple information processing systems 1 merges and outputs the relationships between the attributes and the attribute values ​​(FIG. 14) obtained by each information processing system 1.

[0099] In this embodiment, compared to conventional techniques, latent variables are quantitatively determined, and the calculation results can be merged even if parallel processing is performed on separate systems for each model, making it possible to distribute the load of each process of model learning, attribute value assignment, and output of the relationship between attributes and attribute values ​​across multiple systems.As a result, these processes can be completed in a shorter time than conventional methods, and the required data can be generated.

[0100] (Application example of embodiment) As described above, the embodiment can be applied to character recognition of handwritten characters. In addition, the embodiment can be applied to cases where it is difficult to label all data, accurate attribute values ​​(labels) are assigned to only some data, and attribute values ​​(labels) are to be assigned to the remaining data.

[0101] For example, there is the case of assigning a rotation speed label to vibration data of factory equipment. The premise is that previously acquired vibration data of factory equipment has not been assigned a rotation speed label, and a new device that can measure rotation speed is introduced, and the rotation speed label is assigned to the previously acquired data of the factory equipment.

[0102] Another example is the prediction of the angle of an object in an image. This involves predicting the angle of an object in an unknown image from a small amount of data labeled with angles. This application example can be used to control the grip direction when a robot grabs an object.

[0103] In addition, users may be asked to evaluate the impression of a song. Based on the impressions of songs previously evaluated by users (such as fun, sad, happy, lonely), unknown songs can be labeled with their impressions.

[0104] In addition, there are cases where the research fields of academic papers are visualized. This involves estimating the degree of relevance of an unknown paper to each field based on papers whose degree of relevance to each field is known in advance (30 for image recognition, 50 for reinforcement learning, etc.).

[0105] (Computer 1000 hardware) 16 is a hardware diagram showing the configuration of a computer 1000. For example, the information processing system 1, or each system obtained by appropriately distributing the information processing system 1, such as the feature extraction unit 603, the attribute value assignment unit 608, the data generation unit 614, and the data quality evaluation unit 612, is realized by the computer 1000.

[0106] The computer 1000 comprises a processor 1001 including a CPU, a main memory device 1002, an auxiliary memory device 1003, a network interface 1004, an input device 1005, and an output device 1006, all of which are interconnected via an internal communication line 1009 such as a bus.

[0107] The processor 1001 controls the overall operation of the computer 1000. The main memory device 1002 is composed of, for example, a volatile semiconductor memory, and is used as a work memory for the processor 1001. The auxiliary memory device 1003 is an example of a non-transitory storage medium, and is composed of a large-capacity non-volatile storage device such as a hard disk drive, an SSD (Solid State Drive), or a flash memory, and is used to store various programs and data for a long period of time.

[0108] An executable program 1100 stored in the auxiliary storage device 1003 is loaded into the main storage device 1002 when the computer 1000 is started up or when needed, and the processor 1001 executes the executable program 1100 loaded into the main storage device 1002. This realizes a system that executes various processes.

[0109] The executable program 1100 may be recorded on a non-transitory recording medium, read from the non-transitory recording medium by a media reading device, and loaded into the main memory device 1002. Alternatively, the executable program 1100 may be obtained from an external computer via a network and loaded into the main memory device 1002.

[0110] The network interface 1004 is an interface device for connecting the computer 1000 to each network within the system or for communicating with other computers. The network interface 1004 is configured, for example, by a network interface card (NIC) for a wired local area network (LAN) or a wireless LAN.

[0111] The input device 1005 is composed of a keyboard, a pointing device such as a mouse, and the like, and is used by the user to input various instructions and information to the computer 1000. The output device 1006 is composed of a display device such as a liquid crystal display or an organic EL (Electro Luminescence) display, and an audio output device such as a speaker, and is used to present necessary information to the user when necessary.

[0112] The present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the spirit of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to configurations including all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added to, deleted from, or replaced with other configurations.

[0113] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partly or entirely realized in hardware, for example, by designing them as integrated circuits, or may be realized in software, by a processor interpreting and executing a program that realizes each function.

[0114] Information such as programs, tables, and files that realize each function can be stored in storage devices such as memory, hard disks, and SSDs (Solid State Drives), or non-temporary recording media such as IC (Integrated Circuit) cards, SD cards, and DVDs (Digital Versatile Discs).

[0115] In addition, the control lines and information lines shown are those that are considered necessary for explanation, and do not necessarily represent all the control lines and information lines that are necessary for implementation. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0116] 1: Information processing system, 201: Reference data, 202: Evaluation data, 204: Latent variables, 603: Feature extraction unit, 608: Attribute value assignment unit, 612: Data quality evaluation unit, 614: Data generation unit, 1000: Computer.

Claims

1. An information processing method executed by an information processing system having a processing unit and a storage unit, The processing unit a first step of inputting input data including reference data in which attribute values ​​are assigned to a plurality of attributes of the data into an MFCVAE (Multi-Facet Clustering Variational Auto-Encoder) that outputs latent variables related to each of the plurality of attributes of the data; a second step of setting a regression model for each attribute, the latent variables output from the MFCVAE in response to the input of the input data being explanatory variables and the attribute values ​​being objective variables; a third step of calculating, for each attribute, a predicted value of the attribute value and a regression coefficient of the regression model that minimizes a prediction error for the attribute value from the latent variable and the attribute value; a fourth step of calculating, for each attribute, an index that takes a smaller value as the latent variable and the attribute value better fit the regression model, based on the predicted value and the regression coefficient for each attribute calculated in the third step; a fifth step of calculating a function value of a loss function obtained by adding an additional term based on the index for each attribute to a loss function of the MFCVAE, the loss function having a reconstruction error term representing an error in data reconstruction by the MFCVAE and a regularization term that imposes constraints on the distribution of the latent variables; a sixth step of updating the model parameters of the MFCVAE by backpropagation based on the function values ​​calculated in the fifth step; an information processing method, characterized in that model learning of the MFCVAE is performed by repeating the first step to the sixth step in this order until the prediction error or the number of epochs satisfies a predetermined condition.

2. 2. The information processing method according to claim 1, 10. An information processing method, wherein the regression model is a multiple regression model.

3. 2. The information processing method according to claim 1, 10. An information processing method, wherein the index for each attribute is a coefficient of determination of the regression model.

4. 2. The information processing method according to claim 1, An information processing method, characterized in that the index for each attribute is the prediction error.

5. 5. The information processing method according to claim 4, 10. An information processing method, wherein the prediction error is a mean square error.

6. 2. The information processing method according to claim 1, the additional term is a term obtained by multiplying the index for each of the attributes by a weighting coefficient for each of the attributes, The processing unit an information processing method comprising determining the weighting coefficient for each attribute such that the index, the reconstruction error term, and the regularization term have the same order of absolute value for each attribute;

7. 2. The information processing method according to claim 1, the input data includes the reference data and evaluation data in which the attribute value is not assigned to the attribute, In the fifth step, the processing unit calculating the additional term using the reference data; An information processing method, comprising: calculating the reconstruction error term and the regularization term using either or both of the reference data and the evaluation data.

8. 2. The information processing method according to claim 1, the input data includes the reference data and evaluation data in which the attribute value is not assigned to the attribute, The processing unit a seventh step of inputting the evaluation data into the MFCVAE whose model has been trained by repeating the first step to the sixth step, and acquiring the latent variables related to the evaluation data; an eighth step of calculating predicted values ​​of the attributes to which no attribute value has been assigned in the evaluation data based on the latent variables acquired in the seventh step and the regression coefficients at the final epoch of model learning of the MFCVAE, and assigning the predicted values ​​to the evaluation data as the attribute values.

9. 9. The information processing method according to claim 8, The processing unit a ninth step of outputting information about the attributes and attribute values ​​of the reference data and the evaluation data to which the predicted values ​​have been assigned in the eighth step.

10. 9. The information processing method according to claim 8, The processing unit an information processing method characterized by performing a tenth step of evaluating the input data using the reference data and the evaluation data to which the predicted value has been assigned by the eighth step, according to a viewpoint including sufficiency of data design, coverage of data, or uniformity of data.

11. 2. The information processing method according to claim 1, Each of the processing units of the plurality of information processing systems The first step to the sixth step are repeated until the prediction error or the number of epochs satisfies a predetermined condition, thereby performing model learning of the MFCVAE using different input data; an eleventh step of merging and outputting the learning results of the MFCVAE model obtained by each of the processing units.

12. 9. The information processing method according to claim 8, Each of the processing units of the plurality of information processing systems Executing the first step to the eighth step using different input data, and a twelfth step of merging and outputting information relating to the attributes and attribute values ​​of the reference data and the evaluation data to which the predicted values ​​have been assigned in the eighth step, obtained by each of the processing units.

13. 2. The information processing method according to claim 1, The processing unit a thirteenth step of inputting the specified attributes and attribute values ​​into the MFCVAE that has completed model training by repeating the first step to the sixth step, calculating the latent variables from the input attributes and attribute values ​​and the regression coefficients, and reconstructing the data corresponding to the input attributes and attribute values ​​based on the latent variables.

14. 14. The information processing method according to claim 13, The processing unit If there is reference data that corresponds to the specified attribute and attribute value, adopting the data that corresponds to the reference data as reconstructed data; an information processing method, wherein the thirteenth step is executed when the reference data corresponding to the specified attribute and attribute value does not exist;

15. 9. The information processing method according to claim 7 or 8, 10. An information processing method, comprising: said reference data including printed characters and handwritten characters; and said evaluation data including handwritten characters.

16. Input data including reference data in which attribute values ​​are assigned to a plurality of attributes of the data is input to an MFCVAE (Multi-Facet Clustering Variational Auto-Encoder) that outputs latent variables related to each of the plurality of attributes of the data; a regression model is set for each attribute, with the latent variables output from the MFCVAE in response to the input of the input data as explanatory variables and the attribute values ​​as objective variables; calculating, for each attribute, a predicted value of the attribute value that minimizes a prediction error for the attribute value and a regression coefficient of the regression model from the latent variable and the attribute value; a regression model fitness evaluation unit that calculates, for each attribute, an index that takes a smaller value as the latent variable and the attribute value fit the regression model better, based on the calculated predicted value and the regression coefficient for each attribute; a loss calculation unit that calculates a function value of a loss function obtained by adding an additional term based on the index for each attribute to a loss function of the MFCVAE, the loss function having a reconstruction error term that represents an error in data reconstruction by the MFCVAE and a regularization term that imposes constraints on the distribution of the latent variables; a model updating unit that updates a model parameter of the MFCVAE by back propagation based on the function value calculated by the loss calculating unit, the regression model fitness evaluation unit, the loss calculation unit, and the model update unit perform model learning of the MFCVAE by sequentially repeating the processes in this order until the prediction error or the number of epochs satisfies a predetermined condition.

17. An information processing program for causing a computer to function as an information processing system, The computer Input data including reference data in which attribute values ​​are assigned to a plurality of attributes of the data is input to an MFCVAE (Multi-Facet Clustering Variational Auto-Encoder) that outputs latent variables related to each of the plurality of attributes of the data; a regression model is set for each attribute, with the latent variables output from the MFCVAE in response to the input of the input data as explanatory variables and the attribute values ​​as objective variables; calculating, for each attribute, a predicted value of the attribute value that minimizes a prediction error for the attribute value and a regression coefficient of the regression model from the latent variable and the attribute value; a regression model fitness evaluation unit that calculates, for each attribute, an index that takes a smaller value as the latent variable and the attribute value fit the regression model better, based on the calculated predicted value and the regression coefficient for each attribute; a loss calculation unit that calculates a function value of a loss function obtained by adding an additional term based on the index for each attribute to a loss function of the MFCVAE, the loss function having a reconstruction error term that represents an error in data reconstruction by the MFCVAE and a regularization term that imposes constraints on the distribution of the latent variables; a model updating unit that updates model parameters of the MFCVAE by backpropagation based on the function value calculated by the loss calculation unit; the regression model fitness evaluation unit, the loss calculation unit, and the model update unit execute model learning of the MFCVAE by sequentially repeating the processes in this order until the prediction error or the number of epochs satisfies a predetermined condition.

Images