Prediction device, learning device, prediction method, learning method, and program
The prediction device and learning device leverage machine learning to predict future mood sequences by analyzing past behavior and mood data, addressing the limitations of averaging mood data and predicting temporal changes, enhancing psychological well-being through accurate forecasting.
Patent Information
- Application Number
- JP2023523937
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-05-28
- Publication Date
- 2024-11-13
- Estimated Expiration
- 2041-05-28
AI Technical Summary
Existing methods fail to account for chronological changes in user mood and emotions and cannot predict future temporal changes in mood, as they typically average mood data over time and do not consider the effects of future actions.
A prediction device and learning device that utilize feature extraction and machine learning to predict future mood sequences by analyzing past behavior and mood data, incorporating a behavior sequence prediction model and a mood sequence prediction model to forecast time-series changes in mood.
Enables accurate prediction of future mood sequences, capturing detailed mood movements and reliability through the integration of behavior and mood data, allowing for informed actions to improve psychological well-being.
Smart Images

Figure 0007582465000001 
Figure 0007582465000002 
Figure 0007582465000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a prediction device, a learning device, a prediction method, a learning method, and a program. [Background technology]
[0002] With the widespread use of wearable sensors such as smartwatches, fitness trackers, and smartphones, it has become easy to record users' biological information and behavioral logs (hereinafter referred to as behavioral data). Detailed analysis of this behavioral data and the user's psychological state (mood, emotions, stress level, etc.) obtained through self-evaluation can be useful for various purposes. For example, if it is possible to use the history of a user's behavioral data acquired through a smartwatch to estimate today's stress level as a numerical value or predict future moods, this can be useful for various purposes such as recommending actions that are effective in improving the user's psychological state.
[0003] For example, Non-Patent Document 1 discloses a technology for automatically estimating a user's psychological state from such behavioral data, in which the obtained data is discretized and converted into a histogram, and a probabilistic generative model is used to estimate the health level and stress level. Non-Patent Document 2 discloses a technology for regressing the next day's psychological state using daily sequence data of operation logs and screen time acquired from a smartphone. Non-Patent Document 3 discloses a technology for predicting a future sequence of mood data from a sequence of past behavioral data. [Prior art documents] [Non-patent literature]
[0004] [Non-Patent Document 1] E. Nosakhare and R. Picard: Probabilistic Latent Variable Modeling for Assessing Behavioral Influences on Well-Being. In Proc. of KDD, 2019. [Non-Patent Document 2] S. Yan et al.: Estimating Individualized Daily Self-Reported Affect with Wearable Sensors. In Proc. of ICHI, 2019. [Non-Patent Document 3] D. Spathis, S. Servia-Rodriguez, K. Farrahi, C. Mascolo, and J. Rentflow: Sequence Multi-task Learning to Forecast Mental Wellbeing from Sparse Self-reported Data. In Proc. of KDD, 2019. Summary of the Invention [Problem to be solved by the invention]
[0005] However, in the past, data was separated on a daily basis and statistically processed, so that fluctuations in a user's mood and emotions over time due to date and time could not be taken into consideration. For example, in the method disclosed in Non-Patent Document 2, mood data posted many times in a day is quantified and converted into an average value, and then the average value of the mood for the next day is predicted. However, since human moods go up and down even during the day, it is not possible to present the fluctuations in mood to the user by simply predicting the average value.
[0006] In addition, the method disclosed in Non-Patent Document 3 cannot take into account the influence of future actions that the user will perform. For example, if a user refrains from performing an inconvenient action (such as a negative event) the next day, it is possible that some users will be influenced by this and feel negative. Therefore, the conventional technology has a problem in that it is difficult to predict future fluctuations in mood over time.
[0007] The disclosed technology aims to predict time-series fluctuations in the future mood of a prediction target. [Means for solving the problem]
[0008] The disclosed technology is a prediction device including a feature extraction unit that extracts features based on past behavior series data of a prediction target and outputs behavior feature data, a behavior series prediction unit that predicts a future behavior series of the prediction target based on the behavior feature data using a trained behavior series prediction model for predicting a behavior series, and a mood series prediction unit that predicts a future mood series of the prediction target based on the behavior feature data and past mood series data of the prediction target using a trained mood series prediction model for predicting a mood series. Effect of the Invention
[0009] It is possible to predict the time series fluctuations of the subject's future mood. [Brief description of the drawings]
[0010] [Figure 1] FIG. 2 is a functional configuration diagram of a learning device. [Diagram 2] FIG. 2 is a functional configuration diagram of a prediction device. [Diagram 3] 13 is a flowchart showing an example of the flow of a learning process. [Figure 4] 13 is a flowchart showing an example of the flow of a prediction process. [Diagram 5] FIG. 11 is a diagram illustrating an example of behavior sequence data. [Figure 6] FIG. 11 is a diagram illustrating an example of preprocessed behavioral feature data. [Figure 7] FIG. 11 is a diagram showing an example of mood series data. [Figure 8] FIG. 13 is a diagram showing an example of preprocessed mood series data. [Figure 9] 13 is a flowchart showing an example of the flow of behavioral data pre-processing. [Figure 10] 11 is a flowchart showing an example of the flow of a feature amount extraction process. [Figure 11] 13 is a flowchart showing an example of the flow of mood data pre-processing. [Figure 12]FIG. 13 is a diagram illustrating an example of a behavior sequence prediction model. [Figure 13] 13 is a flowchart showing an example of the flow of a behavior sequence prediction model learning process. [Figure 14] FIG. 11 is a diagram illustrating an example of a behavior sequence prediction model DB and a mood sequence prediction model DB. [Figure 15] FIG. 13 is a diagram illustrating an example of a mood sequence prediction model. [Figure 16] 13 is a flowchart showing an example of the flow of a mood sequence prediction model learning process. [Figure 17] 13 is a flowchart showing an example of the flow of a behavior sequence prediction process. [Figure 18] 13 is a flowchart showing an example of the flow of a mood series prediction process. [Figure 19] FIG. 2 illustrates an example of a hardware configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] Hereinafter, an embodiment of the present invention (the present embodiment) will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment.
[0012] (Outline of this embodiment) The prediction device according to this embodiment extracts features from behavioral data of a prediction target (such as a person or an animal), predicts a behavioral sequence using a behavioral sequence prediction model, and outputs future mood sequence data of the prediction target using the mood sequence prediction model based on the predicted behavioral sequence and past mood data.
[0013] The behavior series data is data that indicates time-series fluctuations in behavior, and the mood series data is data that indicates time-series fluctuations in mood.
[0014] In addition, the learning device according to this embodiment updates parameters of the behavior sequence prediction model and the mood sequence prediction model by machine learning based on learning data including behavior data, mood data, etc. of the learning subject (people, animals, etc.).
[0015] (Example of functional configuration of learning device) 1 is a functional configuration diagram of a learning device 10. The learning device 10 includes an action sequence data DB 101, a mood sequence data DB 102, an action data preprocessing unit 103, a mood data preprocessing unit 104, a feature amount extraction unit 105, an action sequence prediction model construction unit 106, an action sequence prediction model learning unit 107, a mood sequence prediction model learning unit 108, a mood sequence prediction model construction unit 109, an action sequence prediction model DB 110, and a mood sequence prediction model DB 111.
[0016] The behavior sequence data DB 101 stores behavior sequence data. The behavior sequence data is data that indicates past behavior of a learning subject in a time series. A specific example of the behavior sequence data will be described later.
[0017] Mood sequence data is stored in the mood sequence data DB 102. The mood sequence data is data showing the past moods of the learning subject in a time series. Specific examples of the mood sequence data will be described later.
[0018] The behavior sequence data DB101 and the mood sequence data DB102 are constructed in advance so that they can be associated with each other based on the date and time information on when the data was recorded. In addition, the mood sequence data DB102 stores two types of numerical values that indicate the mood at the time when the learning subject self-reported it: valence and arousal. Here, valence indicates the degree of positivity or negativity of the emotion at that time. Also, arousal indicates the degree of excitement of the emotion.
[0019] The mood series data thus configured may be obtained by having the user answer using a known technique such as the Photographic Affect Meter (PAM) (Reference [1]). The mood series data DB 102 may be constructed, for example, by having the target user input a self-evaluation of his / her mood, and storing the input result in the database.
[0020] The behavior data preprocessing unit 103 performs preprocessing on the behavior sequence data to be learned. Specifically, the behavior data preprocessing unit 103 extracts behavior sequence data of a type designated in advance from the behavior sequence data to be learned, scans the columns of each extracted behavior sequence data, and replaces missing values or unexpected values with other values if they are included. Note that if preprocessing is not required, the behavior data preprocessing unit 103 does not need to perform preprocessing.
[0021] The mood data pre-processing unit 104 performs pre-processing on the mood sequence data to be learned. Specifically, the mood data pre-processing unit 104 aggregates data for each data group obtained by dividing the mood sequence data to be learned for each time range specified in advance. Note that if pre-processing is not required, the mood data pre-processing unit 104 does not have to perform pre-processing.
[0022] The feature extraction unit 105 aggregates data for each data group obtained by dividing the preprocessed behavior sequence data into time ranges designated in advance, and regards the aggregated data as behavior feature data.
[0023] The behavior sequence prediction model construction unit 106 receives behavior feature data as input and constructs a behavior sequence prediction model for outputting the mean and variance of predicted behavior sequence data.
[0024] The behavior sequence prediction model learning unit 107 updates the parameters of the behavior sequence prediction model by machine learning.
[0025] The mood sequence prediction model learning unit 108 updates the parameters of the mood sequence prediction model through machine learning.
[0026] The mood sequence prediction model construction unit 109 receives the behavioral feature data, the predicted behavior sequence data, and the preprocessed mood sequence data as input, and constructs a mood sequence prediction model for outputting predicted mood sequence data.
[0027] Behavior sequence prediction models are stored together with parameters in the behavior sequence prediction model DB 110. The behavior sequence prediction model DB 110 is an example of a behavior sequence prediction model storage unit that stores behavior sequence prediction models.
[0028] Mood sequence prediction models together with parameters are stored in the mood sequence prediction model DB 111. The mood sequence prediction model DB 111 is an example of a mood sequence prediction model storage unit that stores mood sequence prediction models.
[0029] (Example of functional configuration of a prediction device) 2 is a functional block diagram of the prediction device 20. The prediction device 20 includes a behavior data preprocessing unit 201, a mood data preprocessing unit 202, a feature amount extraction unit 203, a behavior sequence prediction model DB 204, a behavior sequence prediction unit 205, a mood sequence prediction model DB 206, and a mood sequence prediction unit 207.
[0030] The behavior data pre-processing unit 201 performs the same processing as the behavior data pre-processing unit 103 of the learning device 10 on the past behavior sequence data to be predicted. That is, the behavior data pre-processing unit 201 extracts behavior sequence data of a type designated in advance from the past behavior sequence data to be predicted, scans the columns of each extracted behavior sequence data, and replaces missing values or unexpected values with different values if they are included.
[0031] The mood data pre-processing unit 202 performs the same processing on the past mood sequence data to be predicted as the mood data pre-processing unit 104 of the learning device 10. That is, the mood data pre-processing unit 202 aggregates data for each data group obtained by dividing the past mood sequence data to be predicted for each time range specified in advance.
[0032] The feature extraction unit 203 performs the same process as the feature extraction unit 105 of the learning device 10. That is, the feature extraction unit 203 aggregates data for each data group obtained by dividing the preprocessed behavior sequence data into time ranges specified in advance, and treats the aggregated data as behavior feature data.
[0033] The behavior sequence prediction model DB 204 stores the behavior sequence prediction model constructed by the behavior sequence prediction model construction unit 106 of the learning device 10 together with parameters. The stored parameters are parameters updated by the behavior sequence prediction model learning unit 107 of the learning device 10.
[0034] The behavior sequence prediction unit 205 predicts a future behavior sequence of a prediction target based on the behavior feature data, using a behavior sequence prediction model.
[0035] The mood sequence prediction model DB 206 stores the mood sequence prediction model constructed by the mood sequence prediction model construction unit 109 of the learning device 10 together with parameters. The stored parameters are parameters updated by the mood sequence prediction model learning unit 108 of the learning device 10.
[0036] The mood series prediction unit 207 uses a mood series prediction model to predict a future mood series of a prediction target based on the behavioral feature data and the preprocessed mood series data.
[0037] (Example of learning device operation) Next, a description will be given with reference to the drawings of an example of the operation of the learning device 10. The learning device 10 starts a learning process in response to a user operation or the like, or periodically.
[0038] 3 is a flowchart showing an example of the flow of the learning process. The behavior data preprocessing unit 103 receives and processes behavior sequence data from the behavior sequence data DB 101 (step S100). Next, the feature extraction unit 105 receives and processes the preprocessed behavior sequence data from the behavior data preprocessing unit 103 (step S110).
[0039] Next, the mood data preprocessing unit 104 receives and processes the mood sequence data from the mood sequence data DB 102 (step S120). Next, the behavior sequence prediction model construction unit 106 constructs a behavior sequence prediction model (step S130). Next, the behavior sequence prediction model learning unit 107 receives the behavior feature data from the feature extraction unit 105, learns the behavior sequence prediction model, and outputs the learned behavior sequence prediction model to the behavior sequence prediction model DB 110 (step S140).
[0040] Next, the mood sequence prediction model construction unit 109 constructs a mood sequence prediction model (step S150). The mood sequence prediction model learning unit 108 receives the behavioral feature data from the feature extraction unit 105, the preprocessed mood sequence data from the mood data preprocessing unit 104, and the mood sequence prediction model from the mood sequence prediction model construction unit 109, learns the mood sequence prediction model, and outputs the learned mood sequence prediction model to the mood sequence prediction model DB 111 (step S160).
[0041] (Example of the operation of the prediction device) Next, a description will be given with reference to the drawings of an example of the operation of the prediction device 20. The prediction device 20 starts a prediction process in response to a user operation or the like, or periodically.
[0042] 4 is a flowchart showing an example of the flow of the prediction process. The behavior data pre-processing unit 201 receives and processes past behavior sequence data to be predicted as input (step S200). Next, the feature extraction unit 203 receives and processes the pre-processed behavior sequence data from the behavior data pre-processing unit 201 (step S210).
[0043] Next, the behavior sequence prediction unit 205 receives and processes the behavior feature data from the feature extraction unit 203 (step S220). Next, the mood data preprocessing unit 202 receives and processes the past mood sequence data of the prediction target as input (step S230).
[0044] Then, the mood series prediction unit 207 receives the behavioral feature data from the feature extraction unit 203, the predicted future behavior series data from the behavior series prediction unit 205, the preprocessed mood series data from the mood data preprocessing unit 202, and the trained mood series prediction model from the mood series prediction model DB, calculates a future mood series prediction result, and outputs it together with the future mood series prediction result (step S240).
[0045] (Examples of various data formats according to this embodiment) Next, examples of formats of various data handled by the learning device 10 and the prediction device 20 according to this embodiment will be described.
[0046] 5 is a diagram showing an example of the behavior sequence data 901. The behavior sequence data 901 includes items such as a data ID, a date and time, the number of steps, an exercise time, and a conversation time.
[0047] The value of the item "Data ID" is an identifier for identifying each data record. The value of the item "Date and Time" is a value indicating the date and time when the action was performed. The items "Number of Steps", "Exercise Time", and "Conversation Time" are examples of items indicating the content of the action, and others may be used.
[0048] 6 is a diagram showing an example of preprocessed behavioral characteristic data 902. The behavioral characteristic data 902 includes items such as a data ID, a date and time, a number of steps, an exercise time, and a conversation time.
[0049] The value of the item "Data ID" is an identifier for identifying each data record. The value of the item "Date and time" is a value indicating the start point of a time range specified in advance by a system administrator. The items "Number of steps", "Exercise time", and "Conversation time" are examples of items indicating the content of an action, and other items may be used. The values of the items "Number of steps", "Exercise time", and "Conversation time" are feature quantities that represent the characteristics of each action by statistical processing based on normalized values.
[0050] 7 is a diagram showing an example of mood sequence data 903. Mood sequence data 903 includes items such as data ID, date and time, valence, and arousal.
[0051] The value of the item "Data ID" is an identifier for identifying each data record. The value of the item "Date and Time" is a value indicating the date and time when the mood was declared. The items "Valence" and "Arousal" are examples of items indicating mood, and other items may be used. The values of the items "Valence" and "Arousal" are scores self-reported by the user or other person who is the subject of learning or prediction.
[0052] 8 is a diagram showing an example of preprocessed mood series data. Preprocessed mood series data (mood data) 904 includes items such as data ID, date and time, valence, and arousal.
[0053] The value of the item "Data ID" is an identifier for identifying each data record. The value of the item "Date and Time" is a value indicating the start point of a time range specified in advance by a system administrator. The items "Valence" and "Arousal" are examples of items indicating mood, and other items may be used. The values of the items "Valence" and "Arousal" are the average values of each score in the respective time ranges.
[0054] (Details of each process according to this embodiment) Next, the learning process and the prediction process according to the present embodiment will be described in detail.
[0055] 9 is a flowchart showing an example of the flow of behavior data preprocessing. The behavior data preprocessing is processing by the behavior data preprocessing unit 103 in step S100 of the learning processing, or processing by the behavior data preprocessing unit 201 in step S200 of the prediction processing.
[0056] In the learning phase, the behavior data preprocessing unit 103 receives behavior sequence data from the behavior sequence data DB 101 (step S300). In the prediction phase, the behavior data preprocessing unit 201 receives behavior sequence data as an input.
[0057] Next, the behavior data preprocessing unit 103 or the behavior data preprocessing unit 201 extracts behavior sequence data of a type designated in advance by a system administrator (step S310). For example, the behavior data preprocessing unit 103 or the behavior data preprocessing unit 201 defines a column name of the behavior sequence data to be extracted, and extracts data of a column that matches the column name.
[0058] Next, the behavior data preprocessing unit 103 or the behavior data preprocessing unit 201 scans each column of the extracted behavior sequence data, and if a missing value or an unexpected value is included, replaces it with another value (step S320). For example, the behavior data preprocessing unit 103 or the behavior data preprocessing unit 201 inserts the average value of the corresponding column or 0 in the case of numeric data, or a character string representing a missing value in the case of character string data.
[0059] Next, the behavior data preprocessing unit 103 or the behavior data preprocessing unit 201 passes the converted preprocessed behavior sequence data and the corresponding date and time information to the feature extraction unit 105 or the feature extraction unit 203 (step S330).
[0060] 10 is a flowchart showing an example of the flow of the feature extraction process. The feature extraction process is a process by the feature extraction unit 105 in step S110 of the learning process, or a process by the feature extraction unit 203 in step S210 of the prediction process.
[0061] The feature extraction unit 105 or the feature extraction unit 203 receives the preprocessed behavior sequence data from the behavior data preprocessing unit 201 (step S400).
[0062] Next, the feature extraction unit 105 or the feature extraction unit 203 scans the columns of each behavior sequence data and normalizes the values (step S410). For example, in the case of numerical data, the feature extraction unit 105 or the feature extraction unit 203 normalizes the data so that the average is 0 and the standard deviation is 1, and converts string-type data into a one-hot representation vector for the corresponding dimension.
[0063] Next, the feature extraction unit 105 or the feature extraction unit 203 scans the date and time information of the behavior sequence data, and divides the data into time ranges designated in advance by the system administrator (step S420). For example, when the time range is set to 6 hours, the feature extraction unit 105 or the feature extraction unit 203 divides the data into 6-hour ranges.
[0064] Next, the feature extraction unit 105 or the feature extraction unit 203 aggregates the data for each divided data group (step S430). For example, in the case of numerical data, the feature extraction unit 105 or the feature extraction unit 203 calculates the average value or total value for each data group. In addition, in the case of vector data in one-hot representation, the feature extraction unit 105 or the feature extraction unit 203 counts the number of occurrences of each dimension to obtain new vector data.
[0065] In the learning phase, the feature extraction unit 105 passes the obtained behavioral feature data and corresponding date and time information to the behavior sequence prediction model learning unit 107 and the mood sequence prediction model learning unit 108 (step S440). In the prediction phase, the feature extraction unit 203 passes the obtained behavioral feature data and corresponding date and time information to the behavior sequence prediction unit 205 and the mood sequence prediction unit 207.
[0066] 11 is a flowchart showing an example of the flow of mood data pre-processing. Mood data pre-processing is processing by the mood data pre-processing unit 104 in step S120 of the learning processing, or processing by the mood data pre-processing unit 202 in step S230 of the prediction processing.
[0067] In the learning phase, the mood data preprocessing unit 104 receives mood sequence data from the mood sequence data DB 102 (step S500). In the prediction phase, the mood data preprocessing unit 202 receives mood sequence data as an input.
[0068] Mood data preprocessing unit 104 or mood data preprocessing unit 202 scans the date and time information of the mood series data, and divides the data into time ranges specified in advance by the system administrator (step S510). These time ranges are the same as the time ranges determined by feature extraction unit 105 or feature extraction unit 203.
[0069] Next, mood data preprocessing unit 104 or mood data preprocessing unit 202 aggregates data for each divided data group (step S520). For example, mood data preprocessing unit 104 or mood data preprocessing unit 202 calculates and stores the average values of valence and arousal. If no data exists in a divided section, mood data preprocessing unit 104 or mood data preprocessing unit 202 stores 0 for both valence and arousal.
[0070] In the learning phase, the mood data preprocessing unit 104 passes the obtained preprocessed mood series data and corresponding date and time information to the mood series prediction model learning unit 108 (step S530). In the prediction phase, the mood data preprocessing unit 202 passes the obtained preprocessed mood series data and corresponding date and time information to the mood series prediction unit 207.
[0071] 12 is a diagram showing an example of a behavior sequence prediction model. The behavior sequence prediction model 905 is, for example, a DNN (Deep Neural Network) having the following network.
[0072] The first network is a fully connected layer 1 that extracts more abstract features from the behavioral feature data. After transforming the input in the fully connected layer, the first network performs nonlinear transformation of the input features using, for example, a sigmoid function or a ReLu function to obtain a feature vector. The first network repeats this process sequentially for the behavioral feature data given as time-series data.
[0073] The second network is an RNN1 that further abstracts the abstracted feature vector as sequence data, and is implemented using known techniques such as LSTM (Long-Short term memory). Specifically, the second network sequentially receives sequence data and repeatedly performs nonlinear transformation while taking into account past abstracted information.
[0074] The third network is a fully connected layer 2 that converts the feature vector obtained by RNN1 into normal distribution parameters. The third network obtains mean and variance parameters corresponding to each input behavioral data, and associates the output vector data with the mean and variance of each behavioral data. In other words, the number of dimensions of the vector is the number of input dimensions x 2. The third network performs conversion using a sigmoid function, etc., so that the variance becomes non-negative.
[0075] 13 is a flowchart showing an example of the flow of the behavior sequence prediction model learning process, which is a process performed by the behavior sequence prediction model learning unit 107 in step S140 of the learning process.
[0076] The behavior sequence prediction model learning unit 107 rearranges the received behavior feature data in ascending order with reference to the date and time information (step S600). Next, the behavior sequence prediction model learning unit 107 constructs pairs of input behavior feature data and expected output behavior sequence data as learning data (step S610).
[0077] Next, the behavior sequence prediction model learning unit 107 receives the behavior sequence prediction model from the behavior sequence prediction model construction unit 106 (step S620). Next, the behavior sequence prediction model learning unit 107 initializes model parameters of each unit in the network (step S630). For example, the behavior sequence prediction model learning unit 107 initializes the parameters with random numbers between 0 and 1.
[0078] Next, the behavior sequence prediction model learning unit 107 updates the network parameters using the learning data (step S640). Specifically, the behavior sequence prediction model learning unit 107 learns the network parameters so that correct behavior sequence data can be correctly predicted from the input behavior sequence data. The behavior sequence prediction model learning unit 107 calculates the negative logarithmic likelihood for the output behavior sequence data using the mean and variance values obtained in the fully connected layer 2 of the network, and learns the network using the calculated negative logarithmic likelihood by a known technique such as the backpropagation method. Note that the logarithmic likelihood can be calculated using a normal distribution.
[0079] Then, the behavior sequence prediction model learning unit 107 outputs the behavior sequence prediction model, and stores the output result in the behavior sequence prediction model DB 110 (Step S650).
[0080] 14 is a diagram showing an example of the behavior sequence prediction model DB and the mood sequence prediction model DB. The behavior sequence prediction model DB and the mood sequence prediction model DB store parameters in the form of data 906. The data 906 includes a combination of a parameter and a parameter value.
[0081] 15 is a diagram showing an example of a mood sequence prediction model. The mood sequence prediction model 907 is, for example, a DNN having the following network.
[0082] The first network is a fully connected layer 3 that extracts more abstract features from the behavioral feature data. The first network performs the same process as the fully connected layer 1 to obtain a feature vector. The first network repeats this process sequentially for the behavioral feature data given as time-series data.
[0083] The second network, RNN2, further abstracts the abstracted feature vector as sequence data, and processes it using known techniques in the same way as RNN1 to obtain a feature vector.
[0084] The third network is an auto-attention mechanism (ATT) that calculates a feature vector that takes into account the importance of each feature vector abstracted by RNN1. The weights that represent the importance of each feature vector are realized by two fully connected layers. The first fully connected layer of the third network converts the feature vector into a context vector of any size, and the second fully connected layer takes the context vector as input and outputs a scalar value that corresponds to the weight. The context vector may be subjected to nonlinear transformation processing. The third network converts the obtained set of importances into a probability value, for example, using a softmax function. The third network calculates the weighted average of the obtained set of weights and the set of feature vectors, and obtains a feature vector that takes into account the importance of each feature vector.
[0085] The fourth network, RNN3, abstracts the preprocessed mood sequence data and processes it using known techniques in the same way as RNN1 to obtain a feature vector.
[0086] The fifth network is a fully connected layer 4 that outputs the valence and arousal at the time to be predicted from the feature vector obtained by the self-attention mechanism and RNN3. The fifth network combines the feature vector obtained by the self-attention mechanism and RNN3, then inputs it to the fully connected layer 4 to obtain two-dimensional values corresponding to the valence and arousal values.
[0087] 16 is a flowchart showing an example of the flow of the mood sequence prediction model learning process, which is a process performed by the mood sequence prediction model learning unit 108 in step S160 of the learning process.
[0088] The mood sequence prediction model learning unit 108 receives the behavioral feature data from the feature amount extraction unit 105 and the preprocessed mood sequence data from the mood data preprocessing unit 104, and rearranges the data in ascending order with reference to the date and time information (step S700).
[0089] Next, the mood sequence prediction model learning unit 108 constructs learning data (step S710). For example, the mood sequence prediction model learning unit 108 extracts one preprocessed mood sequence data and sets it as correct answer data. With reference to the date and time information of the preprocessed mood sequence data, it extracts N preprocessed mood data older than that date and time, N behavioral feature data older than that date and time, and N behavioral feature data in the future than that date and time, and sets them as learning data. The mood sequence prediction model learning unit 108 constructs these three pieces of data as one learning data. Note that N is set in advance by the system administrator.
[0090] Next, the mood sequence prediction model learning unit 108 adds noise to the future behavioral feature data obtained as learning data (step S720). Here, the mood sequence prediction model learning unit 108 prepares a normal distribution in which each value of the behavioral feature data is the mean and a random number obtained from a gamma distribution with arbitrary parameters is the variance, and stores the value obtained by sampling from the normal distribution as new behavioral feature data and stores the random number obtained from the gamma distribution as the variance.
[0091] Next, the mood sequence prediction model learning unit 108 receives the model from the mood sequence prediction model learning unit (step S730).
[0092] Next, the mood sequence prediction model learning unit 108 initializes the model parameters of each unit in the network (step S740). For example, the mood sequence prediction model learning unit 108 initializes the parameters with random numbers between 0 and 1.
[0093] Next, the mood sequence prediction model learning unit 108 updates the network parameters using the learning data (step S750). Specifically, the mood sequence prediction model learning unit 108 learns the network parameters so that correct mood sequence data can be correctly predicted from the input mood sequence data. For example, the mood sequence prediction model learning unit 108 may use a known error function to calculate the error of the prediction results of valence and arousal obtained in the fully connected layer 4 of the network with respect to valence and arousal given as correct data, and may use the error to learn the network with a known technique such as the error backpropagation method.
[0094] Then, the mood sequence prediction model learning unit 108 outputs the mood sequence prediction model (network structure and model parameters), and stores the output result in the mood sequence prediction model DB 111 (Step S760).
[0095] 17 is a flowchart showing an example of the flow of the behavior sequence prediction process. The behavior sequence prediction process is a process performed by the behavior sequence prediction unit 205 in step S220 of the prediction process.
[0096] The behavior sequence prediction unit 205 receives behavior feature data from the feature extraction unit 203 and rearranges the data in ascending order with reference to date and time information (step S800). Next, the behavior sequence prediction unit 205 receives a trained behavior sequence prediction model from the behavior sequence prediction model DB 204 (step S810).
[0097] Then, the behavior sequence prediction unit 205 uses the behavior sequence prediction model to calculate future behavior sequence data from the behavior feature data, and passes it to the mood sequence prediction unit 207 (step S820).
[0098] 18 is a flowchart showing an example of the flow of the mood series prediction process. The mood series prediction process is a process performed by the mood series prediction unit 207 in step S240 of the prediction process.
[0099] The mood sequence prediction unit 207 receives behavior feature data from the feature amount extraction unit 203, predicted behavior sequence data from the behavior sequence prediction unit 205, and preprocessed mood sequence data from the mood data preprocessing unit 202 (step S900).
[0100] Next, the mood series prediction unit 207 normalizes the behavioral feature data (step S910). For example, the mood series prediction unit 207 stores 0 as the variance value since the average value and variance value of each behavior correspond to the data that is input to the mood series prediction model but the behavioral feature data does not have a variance value.
[0101] Next, the mood series prediction unit 207 combines the behavior feature data with the predicted behavior sequence data (step S920). For example, the mood series prediction unit 207 rearranges the behavior feature data in ascending order according to the date and time information, combines the predicted behavior sequence data from the last row in output order, and treats the obtained sequence data as behavior feature data.
[0102] Next, the mood sequence prediction unit 207 receives a trained mood sequence prediction model from the mood sequence prediction model DB 206 (step S930).
[0103] Then, the mood sequence prediction unit 207 calculates and outputs the current mood (Valence, Arousal) from the behavioral feature data and the preprocessed mood sequence data using the mood sequence prediction model (step S940).
[0104] (Example of Hardware Configuration According to the Present Embodiment) The learning device 10 and the prediction device 20 can be realized, for example, by making a computer execute a program describing the processing contents described in this embodiment. Note that this "computer" may be a physical machine or a virtual machine on the cloud. When a virtual machine is used, the "hardware" described here is virtual hardware.
[0105] The above program can be recorded on a computer-readable recording medium (such as a portable memory) and can be stored or distributed. The above program can also be provided via a network such as the Internet or e-mail.
[0106] Fig. 19 is a diagram showing an example of the hardware configuration of the computer. The computer in Fig. 19 has a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, etc., which are all connected to each other via a bus B.
[0107] The program for realizing the processing on the computer is provided by a recording medium 1001 such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.
[0108] When a program start instruction is received, the memory device 1003 reads out and stores the program from the auxiliary storage device 1002. The CPU 1004 realizes the functions related to the device according to the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network. The display device 1006 displays a GUI (Graphical User Interface) or the like according to a program. The input device 1007 is composed of a keyboard, a mouse, a button, a touch panel, or the like, and is used to input various operation instructions. The output device 1008 outputs the calculation result. Note that the above computer may be equipped with a GPU (Graphics Processing Unit) or a TPU (Tensor processing unit) instead of the CPU 1004, or may be equipped with a GPU or a TPU in addition to the CPU 1004. In that case, the processes may be shared and executed, for example, the GPU or the TPU executes processes that require special calculations such as neural networks, and the CPU 1004 executes other processes.
[0109] (References) [1]JP Pollak, P. Adams, G. Gay: PAM: A Photographic Affect Meter for Frequent, In Situ Measurement of Affect. In Proc. of CHI, 2011.
[0110] (Effects of this embodiment) The learning device 10 according to the present embodiment learns a mood sequence prediction model by combining preprocessed behavior data obtained by preprocessing mood sequence data, preprocessed behavior feature data obtained by preprocessing behavior sequence data, and a future behavior sequence obtained from the behavior sequence prediction model. This enables the prediction device 20 to predict a mood sequence that was previously unpredictable.
[0111] Furthermore, the learning device 10 predicts future behavior and learns a mood sequence prediction model using the prediction results as input, thereby enabling the prediction device 20 to predict the mood sequence of the prediction target with high accuracy.
[0112] Furthermore, the prediction device 20 predicts future behavior as an average and variance, and expresses the reliability of the result as a variance. The learning device 10 then learns a mood sequence prediction model while capturing the reliability. This enables the prediction device 20 to predict a mood sequence by placing emphasis on behaviors with high reliability.
[0113] In addition, by expressing the mood of the target subject as two-dimensional values of valence and arousal, it is possible to capture minute changes in the target subject's mood.
[0114] (Summary of the embodiment) This specification describes at least the prediction device, learning device, prediction method, learning method, and program described in the following sections. (Section 1) a feature extraction unit that extracts features based on past behavior sequence data of a prediction target and outputs behavior feature data; a behavior sequence prediction unit that predicts a future behavior sequence of the prediction target based on the behavior feature data by using a trained behavior sequence prediction model for predicting a behavior sequence; a mood series prediction unit that predicts a future mood series of the prediction target based on the behavioral feature data and past mood series data of the prediction target by using a trained mood series prediction model for predicting a mood series. Prediction device. (Section 2) a feature extraction unit that extracts features based on the behavior sequence data of a learning target and outputs behavior feature data; a mood sequence prediction model constructing unit that constructs a mood sequence prediction model for predicting a mood sequence; a mood sequence prediction model learning unit that updates parameters of the constructed mood sequence prediction model based on the behavioral feature data and past mood sequence data of the learning subject. Learning device. (Section 3) the mood sequence prediction model learning unit updates the parameters based on future behavioral feature data extracted as learning data and added with noise; 3. A learning device as described in claim 2. (Section 4) a behavior sequence prediction model constructing unit that constructs a behavior sequence prediction model for predicting a behavior sequence; a behavior sequence prediction model learning unit that updates parameters of the behavior sequence prediction model based on the behavior feature data, 4. A learning device according to claim 2 or 3.
[0115] Learning device. (Section 5) the behavior sequence prediction model learning unit calculates a negative log-likelihood for the output behavior sequence data using a mean and a variance obtained from a network included in the behavior sequence prediction model, and updates the parameters based on a calculation result. 5. A learning device as described in claim 4. (Section 6) A prediction method executed by a prediction device, comprising: A step of extracting features based on past behavior sequence data of a prediction target and outputting behavior feature data; predicting a future behavior sequence of the prediction target based on the behavior feature data using a trained behavior sequence prediction model for predicting a behavior sequence; and predicting a future mood series of the prediction target based on the behavioral feature data and past mood series data of the prediction target using a trained mood series prediction model for predicting a mood series. Forecasting methods. (Section 7) A learning method executed by a learning device, comprising: A step of extracting features based on the behavior sequence data of the learning target and outputting behavior feature data; constructing a mood sequence prediction model for predicting mood sequences; updating parameters of the constructed mood sequence prediction model based on the behavioral feature data and past mood sequence data of the learning subject; How to learn. (Section 8) A program for causing a computer to function as each unit in the prediction device described in claim 1, or a program for causing a computer to function as each unit in the learning device described in any one of claims 2 to 5.
[0116] Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and variations are possible within the scope of the gist of the present invention described in the claims. [Explanation of symbols]
[0117] 10 Learning Device 20 Prediction Device 101 Behavioral sequence data DB 102 Mood Series Data DB 103 Behavioral data preprocessing section 104 Mood data preprocessing section 105 Feature Extraction Unit 106 Behavioral Sequence Prediction Model Building Department 107 Behavioral sequence prediction model learning section 108 Mood sequence prediction model learning part 109 Mood sequence prediction model construction section 110 Behavioral sequence prediction model DB 111 Mood sequence prediction model DB 201 Behavioral data preprocessing section 202 Mood data preprocessing section 203 Feature Extraction Unit 204 Behavioral sequence prediction model DB 205 Behavioral Sequence Prediction Unit 206 Mood sequence prediction model DB 207 Mood Series Prediction Unit 1000 Drive Device 1001 Recording media 1002 Auxiliary storage 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input Device 1008 Output device
Claims
1. a feature extraction unit that extracts features based on past behavior sequence data of a prediction target and outputs behavior feature data; a behavior sequence prediction unit that predicts a future behavior sequence of the prediction target based on the behavior feature data by using a trained behavior sequence prediction model for predicting a behavior sequence; a mood series prediction unit that predicts a future mood series of the prediction target based on the behavioral feature data and past mood series data of the prediction target by using a trained mood series prediction model for predicting a mood series. Prediction device.
2. a feature extraction unit that extracts features based on the behavior sequence data of a learning target and outputs behavior feature data; a mood sequence prediction model constructing unit that constructs a mood sequence prediction model for predicting a mood sequence; a mood sequence prediction model learning unit that updates parameters of the constructed mood sequence prediction model based on the behavioral feature data and past mood sequence data, the mood sequence prediction model learning unit updates the parameters based on future behavioral feature data extracted as learning data and added with noise; Learning device.
3. a behavior sequence prediction model constructing unit that constructs a behavior sequence prediction model for predicting a behavior sequence; a behavior sequence prediction model learning unit that updates parameters of the behavior sequence prediction model based on the behavior feature data, The learning device according to claim 2 .
4. the behavior sequence prediction model learning unit calculates a negative log-likelihood for the output behavior sequence data using a mean and a variance obtained from a network included in the behavior sequence prediction model, and updates the parameters based on the calculation result. The learning device according to claim 3.
5. A prediction method executed by a prediction device, comprising: A step of extracting features based on past behavior sequence data of a prediction target and outputting behavior feature data; predicting a future behavior sequence of the prediction target based on the behavior feature data using a trained behavior sequence prediction model for predicting a behavior sequence; and predicting a future mood series of the prediction target based on the behavioral feature data and past mood series data of the prediction target using a trained mood series prediction model for predicting a mood series. Forecasting methods.
6. A learning method executed by a learning device, comprising: extracting features based on the behavioral sequence data of the learning target and outputting behavioral feature data; constructing a mood sequence prediction model for predicting mood sequences; updating parameters of the constructed mood sequence prediction model based on the behavioral feature data and past mood sequence data; the updating step updates the parameters based on future behavior feature data extracted as learning data and adding noise to the future behavior feature data; How to learn.
7. A program for causing a computer to function as each unit in the prediction device according to claim 1, or a program for causing a computer to function as each unit in the learning device according to any one of claims 2 to 4.
Citation Information
Patent Citations
Learning machine and neural network, and device and method for data analysis
JP1993346915A
Behavior prediction apparatus, behavior prediction method, and behavior prediction program
JP2015184764A
Decision device, decision method, and decision program
JP2019036116A
