Information processing system and information processing method

Abstract

To suppress the decline in accuracy of causal inference using interpolated data.SOLUTION: An information processing system includes: a processor that executes a program; and a storage device that stores the program. The storage device stores sensor data that is biological information of a user measured by a sensor and user data collected in association with the sensor data. The processor acquires the sensor data, and interpolates the sensor data using a causal relation between the acquired sensor data and the user data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing system, and more particularly to a technique for interpolating missing biometric data measured from a user and proposing an appropriate intervention measure. [Background technology]

[0002] When filling in missing biometric data of a user, filtering or machine learning models that use measurement data before and after the missing value are used.

[0003] The following prior art is included as background technology in this technical field. Patent Document 1 (JP Patent Publication 2015-200968 A) describes a walking information management system that includes a processing unit that searches a walking information database and a position information database to search for a time when both walking information and position information are present, calculates a moving speed, a moving distance, and a moving time based on the moving information and the time, calculates a walking speed, a stride, and a walking rate from the moving speed, moving distance, and number of steps for a period in which the calculated moving speed is substantially constant, and calculates a relational expression between the calculated walking speed, stride, and walking rate, and the processing unit calculates a moving speed and a moving time based on the user's position information for a period in which the user moves by walking and it is determined that there is no walking information, calculates a walking rate by substituting the calculated moving speed into the relational expression between the walking speed and the walking rate, and calculates a walking rate by multiplying the calculated walking rate by the moving time. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2015-200968 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the prior art mentioned above, the interpolation of measurement data does not take into account changes due to interventions or other influencing factors, which may reduce the accuracy of causal inference using the interpolated data.

[0006] The present invention proposes a method for interpolating measurement data that can suppress a decrease in accuracy of causal inference using interpolated data. [Means for solving the problem]

[0007] A representative example of the invention disclosed in the present application is as follows: That is, an information processing system includes a calculation device that executes a program and a storage device that stores the program, the storage device stores user data including sensor data that is biometric information of a user measured by a sensor device and data collected in association with the sensor data, the calculation device acquires the sensor data, A random mask is applied to the acquired sensor data to generate partially missing mask data for learning; Between the items of said user data Correlation The acquired sensor data is and the generated mask data The present invention is characterized by the fact that it interpolates the following: Effect of the Invention

[0008] According to one aspect of the present invention, it is possible to accurately predict missing data. Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a physical configuration of a computer system according to an embodiment of the present invention. [Diagram 2] 1 is a block diagram showing a logical configuration of a computer system according to an embodiment of the present invention. [Diagram 3] FIG. 4 is a diagram showing an example of the configuration of user data according to the embodiment of the present invention. [Figure 4] FIG. 2 is a diagram illustrating a configuration of a learning unit according to the first embodiment. [Diagram 5] FIG. 4 is a diagram illustrating a configuration of a data interpolation unit during learning in the first embodiment. [Figure 6] FIG. 13 is a diagram showing an example of a graph configuration according to an embodiment of the present invention. [Figure 7A] FIG. 2 is a diagram illustrating an example of nodes and node features constituting a graph. [Figure 7B] FIG. 1 is a diagram illustrating an example of edges and edge features that configure a graph. [Figure 8] FIG. 2 is a diagram illustrating a configuration of a prediction unit according to the first embodiment. [Figure 9] FIG. 2 is a diagram illustrating a configuration of a data interpolation unit according to the first embodiment at the time of predictive inference. [Figure 10A] FIG. 13 is a diagram showing an example of an input screen displayed on the information terminal. [Figure 10B] FIG. 13 is a diagram illustrating an example of an output screen displayed on the information terminal. [Figure 11] FIG. 13 is a diagram showing an example of an intervention measure decision result screen displayed on an information terminal. [Figure 12] FIG. 11 is a diagram illustrating a configuration of a learning unit according to a second embodiment. [Figure 13] FIG. 11 is a diagram illustrating a configuration of a data interpolation unit during learning in the second embodiment. [Figure 14] FIG. 11 is a diagram illustrating a configuration of a prediction unit according to a second embodiment. [Figure 15] FIG. 13 is a diagram illustrating a configuration of a data interpolation unit according to the second embodiment at the time of predictive inference. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] <Example 1> FIG. 1 is a block diagram showing the physical configuration of a computer system 100 of this embodiment.

[0011] A computer system 100 according to this embodiment, which is an embodiment of an information processing system of the present invention, is configured with a computer having a processor (CPU) 101, a main memory device 102, an auxiliary memory device 103, a network adapter 104, an input device 105 and an output device 106.

[0012] The processor 101 executes a program (e.g., a program that realizes the learning unit 200 and the prediction unit 201) stored in the main memory device 102. The main memory device 102 includes a ROM, which is a non-volatile memory element, and a RAM, which is a volatile memory element. The ROM stores immutable programs (e.g., BIOS) and the like. The RAM is a high-speed and volatile memory element such as a DRAM (Dynamic Random Access Memory), and temporarily stores the program executed by the processor 101 and data (e.g., a trained model, features) used when the program is executed.

[0013] The auxiliary storage device 103 is a large-capacity non-volatile storage device such as a magnetic storage device (HDD) or a flash memory (SSD), and stores the programs executed by the processor 101 and data used when the programs are executed. That is, the programs are read from the auxiliary storage device 103, loaded into the main storage device 102, and executed by the processor 101.

[0014] The network adapter 104 is a communication interface device that controls communication with other devices (such as an information terminal 110 and an external storage device 111) via a network 109 in accordance with a predetermined protocol.

[0015] The input device 105 is an interface device, such as a keyboard, mouse, or touch panel, that receives data input (e.g., meta information) from a user. The output device 106 is an interface device, such as a display device or printer, that outputs the results of program execution (e.g., sensor data interpolation results, intervention effect prediction) in a format that can be viewed by an operator. Note that a user terminal (e.g., information terminal 110) connected to the computer system 100 via a network 109 may provide the input device 105 and the output device 106. In this case, the computer system 100 may have a web server function, and the user terminal may access the computer system 100 using a predetermined protocol (e.g., http).

[0016] The programs executed by the processor 101 are provided to the computer system 100 from removable media (such as a CD-ROM or flash memory) or via a network, and are stored in a non-volatile auxiliary storage device 103, which is a non-transitory storage medium. For this reason, it is preferable that the computer system 100 has an interface for reading data from removable media.

[0017] The computer system 100 is a computer system configured on one physical computer or on multiple logically or physically configured computers, and may operate in separate threads on the same computer, or may operate on a virtual computer constructed on multiple physical computer resources. Each functional unit of the computer system 100 may be realized on a different computer.

[0018] The information terminal 110 is a computer connected to the computer system 100 via the network 109, which receives data input (e.g., meta-information) from a user and displays the results of program execution (e.g., sensor data interpolation results, intervention effect prediction). The information terminal 110 communicates with a sensor device worn by the user and collects sensor data measured by the sensor device. The sensor device is a wearable device such as a smart watch or activity meter equipped with a heart rate sensor, an electrodermal activity sensor, a skin temperature sensor, etc., and measures the user's biological information. The sensor device also has an acceleration sensor and a positioning unit implemented, and measures the user's movements.

[0019] The external storage device 111 is configured with a large capacity non-volatile storage device such as a magnetic storage device (HDD) or a flash memory (SSD), and stores the programs executed by the processor 101 and data used when the programs are executed.

[0020] FIG. 2 is a block diagram showing the logical configuration of the computer system 100 of this embodiment.

[0021] The computer system 100 of this embodiment includes a learning unit 200 and a prediction unit 201. The learning unit 200 generates a trained model using training data stored in a training database 210, and stores the generated model in a model database 211. The prediction unit 201 uses the trained model stored in the model database 211 to interpolate user data 220 to generate a data interpolation result 231, and derives a predicted intervention result 221 from the user data 220.

[0022] FIG. 3 is a diagram showing an example of the configuration of the user data 220 in this embodiment.

[0023] The user data 220 stores data on an associated basis, such as identification information (ID) for uniquely identifying a user, the user's gender, the user's age, the user's behavior, the user's emotion, sensor data acquired from the user, the intervention date and time, the intervention content, and the intervention effect. Among these, the user's behavior and the user's emotion are metadata represented by text data.

[0024] Sensor data is data measured by a sensor device worn by the user, and includes numerical values ​​such as heart rate, electrodermal activity, skin temperature, acceleration, activity level (number of steps, METS, etc.), and location information.

[0025] Emotions are what the user is feeling at the intervention date and time, and are data classified into predetermined categories such as pleasant, unpleasant, relaxed, tense, active, and still. Emotions may be emotion data input by the user from the information terminal 110, or estimated by the information terminal 110 from sensor data. For example, a stress level can be calculated from electrodermal activity data measured by a sensor device, and a tension level can be estimated.

[0026] The behavior is an action performed by the user at the intervention date and time, and is data classified into predetermined categories such as eating, exercise, sleep, and interpersonal interaction. The behavior may be collected behavior data input by the user from the information terminal 110, or estimated from sensor data by the information terminal 110. For example, behaviors such as walking, doing light work, and sleeping can be estimated from acceleration data measured by a sensor device.

[0027] FIG. 4 is a diagram illustrating a configuration of the learning unit 200 according to the first embodiment.

[0028] The learning unit 200 has a data interpolation unit 2001, a confounding factor adjustment unit 2005, an intervention effect predictor 2008, a learning error calculation unit 2010, and a parameter update unit 2011, and learns a model using sensor data.

[0029] The data interpolation unit 2001 generates a data interpolation model for inferring and interpolating missing parts of the inputted missing user data 220 so as to minimize the data interpolation loss amount 2002, generates post-interpolation data feature amount 2004 using the generated data interpolation model, and generates post-interpolation data 2003 based on the generated post-interpolation data feature amount 2004. The processing of the data interpolation unit 2001 will be described later with reference to Fig. 5. The data interpolation loss amount is the value of a loss function that represents the error between the interpolated (predicted) data and the (actual) sensor data without missing parts, and as the loss function, for example, mean squared error, mean absolute error, or cross entropy error can be used.

[0030] The confounding factor adjustment unit 2005 generates a confounding factor adjustment model for generating adjusted data feature quantity 2007 from interpolated data feature quantity 2004 based on confounding factors between data items so that confounding factor adjustment loss amount 2006 is minimized, and generates adjusted data feature quantity 2007 from interpolated data feature quantity 2004 using the generated confounding factor adjustment model. Even if bias exists in the distribution of interventions and populations, the confounding factor adjustment unit 2005 can reduce the difference in distribution of populations so that all interventions can be selected equally, thereby making it possible to generate features with balanced distribution.

[0031] The intervention effect predictor 2008 predicts the intervention effect from the adjusted data feature amount 2007, and generates a predicted loss amount 2009, which is a value of a loss function that represents the prediction error. The user data 220 (correct answer data for the predicted effect) is used as correct answer data to calculate the prediction error, and an intervention effect prediction model is generated so that the predicted loss amount is minimized. The predicted loss amount 2009 is generated from the user data 220.

[0032] The learning error calculation unit 2010 calculates the learning error of the entire learning unit 200 from the data interpolation loss amount 2002, the confounding factor adjusted loss amount 2006, and the predicted loss amount 2009. The learning error can be calculated, for example, by weighting and adding the data interpolation loss amount 2002, the confounding factor adjusted loss amount 2006, and the predicted loss amount 2009, and calculating the weighted average value.

[0033] The parameter update unit 2011 generates updated parameters 2012 so as to minimize the calculated learning error. By updating the inference models (data interpolation model, confounding factor adjustment model, intervention effect prediction model) with the generated updated parameters 2012, it is possible to generate a model with a small amount of loss, that is, a model capable of inference close to the correct data.

[0034] FIG. 5 is a diagram showing the configuration of the data interpolation unit 2001 in the first embodiment at the time of learning.

[0035] The data interpolation unit 2001 includes a feature extraction unit 2020 , a graph generation unit 2024 , a graph feature extraction unit 2029 , a feature fusion unit 2031 , and a missing data prediction unit 2033 .

[0036] The feature extraction unit 2020 generates a sensor data feature 2023 from sensor data 2021, and generates the sensor data feature 2023 from mask data 2022 in which a part of the sensor data 2021 is missing. That is, the sensor data feature 2023 includes the feature of the sensor data 2021 and the feature of the mask data 2022. The mask data 2022 is learning data in which the sensor data 2021 is randomly masked and partially missing. The sensor data 2021 is used for learning as correct answer data for the data missing due to the mask.

[0037] The graph generating unit 2024 generates a graph 2028 from the sensor data 2021, the mask data 2022, the behavior / emotion data 2025, the user attribute data 2026, and the history data 2027 of the intervention method / intervention effect. As shown in FIG. 6, the generated graph 2028 is composed of nodes corresponding to the data items input to the graph generating unit 2024 and edges connecting the nodes. The feature amounts of each node and each edge are calculated. The node feature amount represents the characteristics of the data items (for example, the emotion, behavior, acceleration data, heart rate data, intervention method / intervention effect, user attribute, etc. shown in FIG. 7A), and the edge feature amount represents the correlation between the data items.

[0038] Continuing the explanation by returning to Fig. 5, a graph feature extraction unit 2029 extracts graph features 2030 from the generated graph 2028. The graph features 2030 include node features that indicate the properties of the nodes themselves, and edge features that indicate the adjacent relationships between nodes via edges.

[0039] FIG. 7A is a diagram showing an example of nodes and node features constituting a graph 2028, and FIG. 7B is a diagram showing an example of edges and edge features constituting a graph 2028. A node is an item such as emotion, behavior, acceleration data, heart rate data, intervention method / intervention effect, and user attribute, which is determined for each item, and a node feature is a feature of data for each item corresponding to a node. An edge is a relationship between data of items corresponding to two nodes, such as a relationship between emotion and heart rate data, a relationship between behavior and heart rate data, a relationship between emotion and acceleration data, a relationship between behavior and acceleration data, a relationship between behavior and intervention method / intervention effect, and a relationship between intervention method / intervention effect and heart rate data. An edge feature is expressed by a feature indicating a correlation between nodes.

[0040] 5, the description will be continued. The feature amount fusion unit 2031 calculates a fusion feature amount 2032 by fusing the generated sensor data feature amount 2023 and the extracted graph feature amount 2030. The calculated fusion feature amount 2032 is output as a post-interpolation data feature amount 2004.

[0041] The missing data prediction unit 2033 uses a data interpolation model to generate interpolated data 2003 from the fusion feature amount 2032. The data interpolation unit 2001 uses the sensor data 2021 and the mask data 2022 as correct answer data, and further the interpolated data 2003 to learn so as to minimize the data interpolation loss amount 2002, and generates (updates) parameters of the data interpolation model.

[0042] FIG. 8 is a diagram illustrating a configuration of the prediction unit 201 according to the first embodiment.

[0043] The prediction unit 201 has a data interpolation unit 2001, a confounding factor adjustment unit 2005, and an intervention effect predictor 2008, and predicts an intervention effect from user data 220. The data interpolation unit 2001, the confounding factor adjustment unit 2005, and the intervention effect predictor 2008 have the same configuration as those of the learning unit 200.

[0044] The data interpolation unit 2001 uses the learned data interpolation model to interpolate missing parts of the user data 220 including missing parts, generates post-interpolation data 2003, and calculates post-interpolation data feature amount 2004 from the generated post-interpolation data 2003. The processing of the data interpolation unit 2001 will be described later with reference to FIG.

[0045] A confounding factor adjustment unit 2005 generates adjusted data feature quantities 2007 from the interpolated data feature quantities 2004 using the learned confounding factor adjustment model.

[0046] The intervention effect predictor 2008 generates a predicted intervention outcome 2040 from the adjusted data features 2007 .

[0047] FIG. 9 is a diagram showing the configuration of the data interpolation unit 2001 in the first embodiment at the time of predictive inference.

[0048] The feature extraction unit 2020 generates a sensor data feature 2023 from the sensor data 2021 .

[0049] The graph generating unit 2024 generates a graph 2028 from the sensor data 2021, the behavior / emotion data 2025, the user attribute data 2026, and the intervention method / intervention effect history data 2027. The generated graph 2028 is composed of nodes corresponding to the data items input to the graph generating unit 2024 and edges connecting the nodes, as shown in Fig. 6 .

[0050] A graph feature extraction unit 2029 extracts graph features 2030 from the generated graph 2028. The graph features 2030 include node features that indicate the properties of the nodes themselves, and edge features that indicate the adjacent relationships between nodes via edges.

[0051] The feature fusion unit 2031 fuses the generated sensor data feature 2023 and the extracted graph feature 2030 to calculate a fusion feature 2032. The calculated fusion feature 2032 is output as an interpolated data feature 2004. The fusion of the sensor data feature 2023 and the graph feature 2030 representing the correlation of the user data 220 enables data interpolation taking into account the correlation between data, and can improve the accuracy of confounding factor adjustment and causal inference in intervention effect prediction.

[0052] The missing data prediction unit 2033 infers and generates the interpolated data 2003 from the fusion feature amount 2032 using the learned data interpolation model.

[0053] FIG. 10A is a diagram showing an example of an input screen displayed on information terminal 110. As shown in FIG.

[0054] The input screen includes a behavior input area 1101, an emotion input area 1102, and an intervention input area 1103. The behavior input area 1101 includes selection buttons for, for example, interpersonal interaction, sleep, eating, work, exercise, etc., and the user operates the button corresponding to the current behavior to input the behavior. The emotion input area 1102 is provided with a slide bar related to a plurality of emotions, and the user operates the slider to select the degree of discomfort / pleasure or relaxation / activity. The intervention input area 1103 includes a selection field for the intervention method for each purpose, and the intervention method being performed by the user is input. of Select.

[0055] FIG. 10B is a diagram showing an example of an output screen displayed on information terminal 110. As shown in FIG.

[0056] The output screen includes an interpolation target display area 1105 and an interpolation feedback display area 1106. The interpolation target display area 1105 displays sensor data and areas where sensor data is missing in a graph area with date and time as the two axes. The interpolation feedback display area 1106 displays the interpolated sensor data.

[0057] FIG. 11 is a diagram showing an example of an intervention measure determination result screen 1300 displayed on the information terminal 110. As shown in FIG.

[0058] The most effective and optimal intervention measure is displayed at the top of the intervention measure determination result screen 1300. The difference in the intervention effect (activity productivity increase rate expressed in percentage units) due to the intervention candidate is displayed at the bottom of the intervention measure determination result screen 1300. Specifically, it can be seen that 4 weeks after the start of the intervention, (1) 30 minutes of walking results in an 8% improvement, (2) 30 minutes of running results in a 38% improvement, and (3) 10 minutes of strength training results in a 12% improvement.

[0059] As described above, in the first embodiment of the present invention, the missing data is predicted using the graph feature 2030 of the graph 2028 generated from the sensor data 2021, the mask data 2022, the behavioral and emotional data 2025, the user attribute data 2026, and the intervention method and intervention effect history data 2027. This makes it possible to predict the missing data accurately with little data loss.

[0060] In addition, since updated parameters 2012 are generated so that the learning error calculated from the data interpolation loss amount 2002, the confounding factor adjusted loss amount 2006, and the predicted loss amount 2009 is minimized, the accuracy of interpolation by the model can be improved by adjusting multiple loss functions.

[0061] <Example 2> Next, a second embodiment of the present invention will be described. In the first embodiment described above, the user inputs metadata such as behavior and emotion, but in the second embodiment, the behavior / emotion classifier 2013 identifies the user's behavior and emotion from the user data 220 (particularly sensor data). In the second embodiment, the same configurations and processes as those in the first embodiment described above are denoted by the same reference numerals, and the description thereof will be omitted.

[0062] FIG. 12 is a diagram illustrating a configuration of the learning unit 200 according to the second embodiment.

[0063] The learning unit 200 has a data interpolation unit 2001, a behavior / emotion classifier 2013, a confounding factor adjustment unit 2005, an intervention effect predictor 2008, a learning error calculation unit 2010, and a parameter update unit 2011, and learns a model using sensor data without any missing data.

[0064] The data interpolation unit 2001 generates a data interpolation model for inferring and interpolating a missing portion of the input user data 220 including the missing portion so as to minimize a data interpolation loss amount 2002, generates post-interpolation data feature amount 2004 using the generated data interpolation model, and generates Interpolated data The process of the data interpolation unit 2001 will be described later with reference to FIG.

[0065] The behavior / emotion classifier 2013 generates an behavior / emotion classification model for generating behavior and emotion data from the feature amounts of sensor data of the user data 220 so as to minimize the behavior / emotion classification loss 2014, generates behavior and emotion data using the generated behavior / emotion classification model, calculates feature amounts of the behavior and emotion data from the generated behavior and emotion data, and adds the calculated feature amounts to the post-interpolation data feature amounts 2004. That is, while the data interpolation unit 2001 interpolates the data, the behavior / emotion classifier 2013 classifies behaviors and emotions.

[0066] The confounding factor adjustment unit 2005 generates a confounding factor adjustment model for generating adjusted data feature quantities 2007 from interpolated data feature quantities 2004 based on confounding factors between data items so that the confounding factor adjustment loss amount 2006 is minimized, and generates adjusted data feature quantities 2007 from interpolated data feature quantities 2004 using the generated confounding factor adjustment model.

[0067] The intervention effect predictor 2008 generates a predicted loss amount 2009 from the user data 220 .

[0068] The learning error calculation unit 2010 calculates the learning error from the data interpolation loss amount 2002, the confounding factor adjustment loss amount 2006, and the behavior / emotion discrimination loss amount 2014.

[0069] The parameter update unit 2011 generates updated parameters 2012 so as to minimize the calculated learning error. By updating the inference model (data interpolation model, confounding factor adjustment model) with the generated updated parameters 2012, it is possible to generate a model with a small amount of loss, that is, a model capable of inference close to the correct data.

[0070] FIG. 13 is a diagram showing the configuration of the data interpolation unit 2001 in the learning mode of the second embodiment.

[0071] The data interpolation unit 2001 includes a feature extraction unit 2020 , a graph generation unit 2024 , a graph feature extraction unit 2029 , a feature fusion unit 2031 , and a missing data prediction unit 2033 .

[0072] The feature extraction unit 2020 generates a sensor data feature 2023 from sensor data 2021, and generates the sensor data feature 2023 from mask data 2022 in which a part of the sensor data 2021 is missing. That is, the sensor data feature 2023 includes the feature of the sensor data 2021 and the feature of the mask data 2022. The mask data 2022 is learning data that is partially missing by randomly masking the sensor data 2021. The sensor data 2021 is used for learning as correct answer data for data missing due to the mask.

[0073] The action / emotion classifier 2013 uses the sensor data feature amount 2023 and the action / emotion data 2025 to learn so as to minimize the action / emotion classification loss amount 2014, and generates an action / emotion classification model. Then, using the generated action / emotion classification model, it generates the action / emotion classification result 2038 and the action / emotion classification loss amount 2014 from the sensor data feature amount 2023.

[0074] The graph generating unit 2024 generates a graph 2028 from the sensor data 2021, the mask data 2022, the behavior / emotion data 2025, the user attribute data 2026, and the intervention method / intervention effect history data 2027. The generated graph 2028 is as described above with reference to FIG.

[0075] A graph feature extraction unit 2029 extracts graph features 2030 from the generated graph 2028. The graph features 2030 include node features that indicate the properties of the nodes themselves, and edge features that indicate the adjacent relationships between nodes via edges.

[0076] The feature amount fusion unit 2031 fuses the generated sensor data feature amount 2023 and the extracted graph feature amount 2030 to calculate a fusion feature amount 2032. The calculated fusion feature amount 2032 is output as a post-interpolation data feature amount 2004.

[0077] The missing data prediction unit 2033 generates a data interpolation model by learning so as to minimize the data interpolation loss amount 2002 using the sensor data 2021, the mask data 2022, the fusion feature amount 2032, and the interpolated data 2003. Then, the generated data interpolation model is used to generate the interpolated data 2003 from the sensor data 2021, the mask data 2022, and the fusion feature amount 2032.

[0078] FIG. 14 is a diagram illustrating a configuration of the prediction unit 201 according to the second embodiment.

[0079] The prediction unit 201 has a data interpolation unit 2001, a behavior / emotion classifier 2013, a confounding factor adjustment unit 2005, and an intervention effect predictor 2008, and predicts an intervention effect from user data 220. The data interpolation unit 2001, the behavior / emotion classifier 2013, the confounding factor adjustment unit 2005, and the intervention effect predictor 2008 have the same configurations as those of the learning unit 200.

[0080] The data interpolation unit 2001 uses the learned data interpolation model to interpolate missing parts of the user data 220 including the input missing parts, generates post-interpolation data 2003, and calculates post-interpolation data feature amount 2004 from the generated post-interpolation data 2003. The processing of the data interpolation unit 2001 will be described later with reference to FIG.

[0081] The behavior / emotion classifier 2013 uses the learned behavior / emotion classification model to classify behavior and emotion from the feature amounts of the sensor data of the user data 220, and generates a behavior / emotion classification result 2038.

[0082] A confounding factor adjustment unit 2005 generates adjusted data feature quantities 2007 from the interpolated data feature quantities 2004 using the learned confounding factor adjustment model.

[0083] The intervention effect predictor 2008 generates predicted intervention outcomes 2040 from the user data 220 .

[0084] FIG. 15 is a diagram showing a configuration of the data interpolation unit 2001 in the second embodiment at the time of predictive inference.

[0085] The feature extraction unit 2020 generates a sensor data feature 2023 from the sensor data 2021 .

[0086] The behavior / emotion classifier 2013 generates behavior / emotion classification results 2038 from the sensor data features 2023 using the learned behavior / emotion classification model.

[0087] The graph generating unit 2024 generates a graph 2028 from the sensor data 2021, the behavior / emotion identification result 2038, the user attribute data 2026, and the intervention method / intervention effect history data 2027. The graph 2028 is composed of nodes corresponding to the data items input to the graph generating unit 2024 and edges connecting the nodes.

[0088] A graph feature extraction unit 2029 extracts graph features 2030 from the generated graph 2028. The graph features 2030 include node features that indicate the properties of the nodes themselves, and edge features that indicate the adjacent relationships between nodes via edges.

[0089] The feature fusion unit 2031 fuses the generated sensor data feature 2023 and the extracted graph feature 2030 to calculate a fusion feature 2032. For example, the n-dimensional sensor data feature 2023 and the m-dimensional graph feature 2030 are concatenated to generate an n+m-dimensional feature. A one-dimensional convolution operation of a known neural network is performed on the concatenated feature to generate a p-dimensional fusion feature 2032. The parameters of the convolution operation are part of the parameters of the data interpolation model. The calculated fusion feature 2032 is output as an interpolated data feature 2004. The fusion of the sensor data feature 2023 and the graph feature 2030 representing the correlation of the user data 220 enables data interpolation taking into account the correlation between data, and improves the accuracy of confounding factor adjustment and causal inference in intervention effect prediction.

[0090] The missing data prediction unit 2033 generates a missing data prediction result 2034 from the fusion feature amount 2032.

[0091] As described above, in the second embodiment of the present invention, the data of behavior and emotion are identified from the sensor data feature amount 2023, so that the user can be saved the trouble of inputting the data of behavior and emotion.

[0092] The present invention is not limited to the above-described embodiments, and includes various modified examples and equivalent configurations within the spirit of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the configurations described. Furthermore, a part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, a part of the configuration of each embodiment may be added, deleted, or replaced with another configuration.

[0093] Furthermore, each of the aforementioned configurations, functions, processing units, processing means, etc. may be realized in hardware, for example by designing some or all of them as an integrated circuit, or may be realized in software by a processor interpreting and executing a program that realizes each function.

[0094] Information such as programs, tables, and files that realize each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a recording medium such as an IC card, an SD card, or a DVD.

[0095] In addition, the control lines and information lines shown are those considered necessary for the explanation, and do not necessarily show all the control lines and information lines necessary for implementation. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0096] 100 Computer Systems 101 Processor 102 Main storage 103 Auxiliary storage device 104 Network Adapter 105 Input Device 106 Output Device 109 Network 110 Information terminal 111 External storage device 200 Learning Department 201 Prediction Department 210 Learning Database 211 Model Database 220 User Data 221 Predicting Intervention Outcomes 231 Data Interpolation Results 1101 Behavioral Input Area 1102 Emotion Input Area 1103 Intervention Input Area 1105 Interpolation target display area 1106 Post-interpolation feedback display area 1300 Intervention measure decision result screen 2001 Data Interpolation Unit 2002 Data Interpolation Loss 2003 Interpolated data 2004 Interpolated data features 2005 Confounding Factor Adjustment Department 2006 Confounding factor adjustment loss amount 2007 Adjusted data features 2008 Intervention Effect Predictor 2009 Projected Losses 2010 Learning error calculation part 2011 Parameter Update Division 2012 Updated parameters 2013 Emotion Classifier 2014 Emotion discrimination loss amount 2020 Feature Extraction Unit 2021 Sensor Data 2022 Mask Data 2023 Sensor data features 2024 Graph Generation 2025 Emotional Data 2026 User Attribute Data 2027 Historical Data 2028 Graph 2029 Graph feature extraction unit 2030 Graph Features 2031 Feature Fusion Unit 2032 Fusion Features 2033 Missing Data Prediction Department 2034 Missing Data Prediction Results 2035 Transformed features 2038 Emotion Identification Results 2040 Prediction of intervention results

Claims

1. An information processing system, A computing device that executes a program and a storage device that stores the program, the storage device stores user data including sensor data, which is biometric information of a user measured by a sensor device, and data collected in association with the sensor data; The computing device includes: Acquiring the sensor data; A random mask is applied to the acquired sensor data to generate partially missing mask data for learning; An information processing system, comprising: an information processing unit that uses correlations between items of the user data to interpolate the acquired sensor data and the generated mask data.

2. 2. The information processing system according to claim 1, The information processing system, wherein the user data includes at least one of a user's behavior, emotion, biometric information, and intervention measures.

3. 2. The information processing system according to claim 1, The user data may be represented by a graph structure that represents correlations between items of the user data; The information processing system is characterized in that the arithmetic device interpolates the sensor data by using features of the graph structure.

4. 2. The information processing system according to claim 1, The information processing system is characterized in that the sensor data is expressed by a numerical value.

5. 2. The information processing system according to claim 1, An information processing system, wherein the user's behavioral and emotional data is input by the user.

6. 2. The information processing system according to claim 1, The information processing system is characterized in that the computing device identifies behavioral and emotional data of the user from the sensor data.

7. 2. The information processing system according to claim 1, The information processing system is characterized in that the calculation device determines model parameters so as to minimize a learning error calculated from a data interpolation loss amount in the interpolation of the sensor data, a confounding factor loss amount in the adjustment of data features using a confounding factor that represents a relationship between items of the user data, and a prediction loss amount in the prediction of an intervention effect from the user data.

8. The information processing system according to claim 7, The information processing system is characterized in that the calculation device adjusts confounding factors so as to minimize loss of confounding factors that represent relationships between items of the user data, and predicts an intervention effect.

9. An information processing method in which an information processing system interpolates sensor data, comprising: the information processing system is configured by a computer having an arithmetic unit that executes a predetermined process and a storage device connected to the arithmetic unit; the storage device stores therein user data including sensor data that is biometric information of a user measured by a sensor device and data collected in association with the sensor data; The information processing method includes: The computing device acquires the sensor data; A step of applying a random mask to the acquired sensor data to generate partially missing mask data for learning; and a step of the arithmetic device interpolating the acquired sensor data and the generated mask data using correlations between items of the user data.

Images