Method and apparatus for inferring emotional state using multi-dimensional affective representation

Abstract

The present invention relates to a method and apparatus for inferring an emotional state of a subject using a multi-dimensional representation and, more particularly, to a method and apparatus for inferring an emotional state of a subject using a multi-dimensional representation, which are configured to classify human behavioral phenotypes by inferring an instantaneous emotional state of the subject and to quantify the extent to which social decision-making varies according to the instantaneous emotional state of the subject.

Description

Method and device for inferring emotional states through multidimensional representation

[0001] The present invention relates to a method and apparatus for inferring a subject's emotional state through multidimensional representation, and more specifically, to a method and apparatus for classifying human behavioral phenotypes by inferring a subject's momentary emotional state using multidimensional representation, and quantifying the degree to which social decision-making changes according to the subject's momentary emotional state.

[0002]

[0003] Generally, human emotions can be divided into the dimension of subjective experience and the dimension of physiological response. To measure subjective experience, self-report questionnaires, which have subjects directly respond to whether they experienced a specific emotion and its intensity, or affective representation techniques, which have subjects respond to experienced emotions in terms of feeling, are used.

[0004] Human emotional experiences are not mutually exclusive, and individuals can feel multiple contradictory emotions simultaneously. However, self-report questionnaires cannot measure various emotions at once, and their limitations as reliable measurement tools stem from the fact that the independence and interdependence between questionnaires have not been proven. Therefore, a method is employed to reliably classify various emotions through emotion representation techniques that represent emotions in an abstract multidimensional space by having respondents answer subjective emotional experiences in the dimensions of mutually independent valence and arousal.

[0005] During the execution of a re-execution cognitive task, the emotion predicted before each trial and the emotion actually experienced after each trial can be answered through multidimensional emotional representations, thereby allowing for the measurement of not only the reward prediction error but also the emotional prediction error (Non-patent Literature 1). Additionally, variations in decision-making caused by the reward prediction error and the emotional prediction error can be explained through statistical techniques such as a generalized linear mixed model.

[0006] Through parameter estimation using Bayesian statistical modeling techniques on behavioral data collected during the performance of a re-executed cognitive task, interpretable measurements of the subjects' unique behavioral characteristics can be derived. Additionally, reliability between different interpretations can be compared through model comparison, the reproducibility of the model as a measurement technique can be verified through parameter recovery, and the singularity of measurement through the model can be verified through model identifiability.

[0007] In addition, after grouping individual subjects, the parameter estimation can be made robust against the influence of exceptions and improved in stability through a hierarchical modeling technique that assumes the prior probability distribution is shared within the group during the Bayesian parameter estimation process (Non-patent Literature 6).

[0008] Meanwhile, there is a heterogeneity problem in which human behavior and mental states can be classified differently according to various criteria, and there is a known underspecificity problem in which existing psychological and psychiatric classification systems and measurement models for depression, anxiety, etc., fail to resolve this heterogeneity problem.

[0009] Accordingly, there is a need for a new method to measure subjects' emotions and predict the results of their decision-making that can solve both the problem of heterogeneity and the problem of underspecificity, which cannot solve this heterogeneity problem.

[0010]

[0011] As a result of efforts to solve the aforementioned problems, the inventors have confirmed that it is possible to measure individual-specific momentary emotions and mental health states, such as anxiety, depression, and personality disorders, and to predict individual-specific emotion-dependent future decision-making, such as suicide and addictive behaviors, by deriving state indicators or behavioral indicators that reflect individual differences in emotional states through a hybrid model that measures and infers individual differences in emotional states in multidimensional terms of valence, arousal, focus, and dominance, thereby approaching emotions felt instantaneously in social and non-social contexts, and by quantifying and predicting the degree to which social decision-making changes according to individual-specific emotional states and the subject's instantaneous emotional state through hierarchical modeling. Based on this, the inventors have completed the present invention.

[0012]

[0013] [Prior Art Literature]

[0014] [Patent Literature]

[0015] Korean Published Patent No. 10-2021-0086481A

[0016] [Non-patent literature]

[0017] 1. Heffner, J. et al., Nat. Hum. Behav. 5, 1391-1401 (2021) https: / doi.org / 10.1038 / s41562-021-01213-6.

[0018] 2. Kim, J. Bong et al., Sci. Rep. 14(1), 11397 (2024) https: / doi.org / 10.1038 / s41598-024-62203-y.

[0019] 3. LeDoux, J.E. & Brown, R., Proc. Natl Acad. Sci. USA 114, E2016-E2025 (2017)

[0020] 4. Russell, J. A. et al., J. Pers. Soc. Psychol. 57, 493-502 (1989).

[0021] 5. Jack A. Why trust the subject?. Journal of consciousness studies. 2003 Jan 1;10(9-10):v-xx.

[0022] 6. Farrell, S. & Lewandowsky, S. Computational Modeling of Cognition and Behavior (Cambridge University, 2018).

[0023] 7. Feczko, E. et al. Trends Cogn. Sci. 23, 584-601 (2019).

[0024]

[0025] The objective of the present invention is to provide a method and apparatus for measuring various human behaviors and mental states through multidimensional representations with sufficient specificity, identifying individual differences through hierarchical modeling, measuring individual-specific characteristics of a subject's emotional decision-making, and inferring to predict the outcome of the decision-making.

[0026]

[0027] To achieve the above objective, the present invention provides a method for inferring the emotional state of a subject through multidimensional representation, comprising: (a) obtaining instantaneous social emotion data of a subject felt while making a decision in a decision-making task using a multidimensional emotion representation method; (b) determining a pattern of instantaneous social emotion from the obtained instantaneous social emotion data; (c) predicting a decision based on the determined pattern of instantaneous social emotion using a hybrid classification model; (d) analyzing the type of decision predicted in step (c) using a Bayesian cognitive behavioral modeling technique; and (e) optimizing the type of decision through computer modeling.

[0028] The present invention provides a computer-readable recording medium having a program for executing the above method recorded thereon.

[0029] The present invention also provides an emotional state inference device for a subject, comprising: (a) an input device for obtaining instantaneous social emotion data of a subject felt while making a decision in a decision-making task using a multidimensional emotion representation method; (b) an emotion determination device for determining and outputting a pattern of instantaneous social emotion from the obtained instantaneous social emotion data; (c) a decision determination device for predicting and outputting a decision based on the determined pattern of instantaneous social emotion using a hybrid classification model; (d) an emotion estimation device for analyzing the type of the predicted decision using a Bayesian cognitive behavioral modeling technique; and (e) a modeling-based decision optimization device for optimizing the type of the decision.

[0030]

[0031] The present invention is an algorithm capable of deriving four dimensions of emotional characteristics (multidimensional representations) that can classify heterogeneous subjects with low interdependence to classify human emotional states during social interaction, extracting subject groups reflecting individual differences in emotional experience from the data of the multidimensional representations using a hybrid classification model, and explaining individual differences in emotional experience through hierarchical modeling classified by groups reflecting individual differences in emotional experience.

[0032] In other words, data dimensions that compensate for under-specificity (multidimensional representations) enrich responses to reflect individual differences in emotional experience, hybrid classification models enable the discovery of individual differences in emotional experience from data collected by multidimensional representations, and hierarchical modeling provides interpretability that can explain these individual differences in emotional experience.

[0033] According to the present invention, the problem of heterogeneity in human behavior and mental states can be resolved through observation means having appropriate specificity, and through a hybrid model, under-specific classification criteria such as depression and anxiety used in existing psychiatry can be supplemented and replaced, and individual differences in human emotions and social decision-making processes can be interpreted by group through hierarchical modeling.

[0034] Since it can precisely predict the outcomes of subjects' emotional decision-making, it can be actively utilized in fields requiring future decision-making, such as predicting mental health conditions like anxiety and personality disorders, conducting opinion polls capable of estimating underlying emotions, measuring instantaneous emotional states through life logs, estimating social emotions based on biosignals, and predicting suicide and addictive behaviors.

[0035] By having subjects respond based on characteristics with low interdependence and classifying them using a hybrid classification model that combines supervised and unsupervised classification algorithms, the specificity of the model can be improved and the heterogeneity problem can be resolved.

[0036]

[0037] FIG. 1 is a schematic diagram of the invention according to one embodiment of the present invention.

[0038] FIG. 2 is a diagram showing an emotional representation of emotion (a) and a model for measuring and predicting decision-making of emotion prediction errors (b) according to an embodiment of the present invention.

[0039] FIG. 3 is a diagram showing the measurement of subject characteristics through Bayesian modeling (a) and the reproducibility of Bayesian modeling through hierarchical modeling (b) according to one embodiment of the present invention.

[0040] FIG. 4 is a diagram illustrating the relationship between the problem of heterogeneity and various characteristics of behavior and mental state according to one embodiment of the present invention and the under-specificity of the mental health classification system.

[0041] FIG. 5 is a diagram illustrating group derivation / extraction reflecting individual differences in emotional experience through a hybrid classification model according to an embodiment of the present invention.

[0042] FIG. 6 is a diagram illustrating a multidimensional emotion representation method according to one embodiment of the present invention.

[0043] FIG. 7 is a diagram illustrating the step of measuring an emotion prediction error by measuring momentary emotions before and after a decision-making process according to an embodiment of the present invention.

[0044] FIG. 8 is a diagram illustrating a customized mental state and behavior classification method according to one embodiment of the present invention.

[0045] FIG. 9 is a diagram illustrating the distribution of responses from subjects who were asked to respond to a specific emotional state (shame, guilt, etc.) through a multidimensional emotional representation method according to an embodiment of the present invention. That is, it is a diagram illustrating a step in which a specific emotional state can be inferred through a multidimensional representation that resolves under-specificity, in which subjects evaluated qualitative emotional concepts of basic emotions and social emotions in the physical dimension of valence and arousal and the social dimension of focus and dominance, thereby quantitatively measuring the emotional concepts.

[0046] FIG. 10 is a diagram illustrating a technique for estimating social emotions hidden within a decision-making process according to an embodiment of the present invention.

[0047] FIG. 11 is a diagram illustrating the steps of a user performing a decision-making task through an input device according to an embodiment of the present invention.

[0048] FIG. 12 is a drawing illustrating an ultimatum task according to one embodiment of the present invention.

[0049] FIG. 13 is a diagram showing that emotional states can be classified online (in real time) based on patterns of instantaneous social emotions according to one embodiment of the present invention.

[0050] FIG. 14 is a diagram illustrating modeling-based behavior optimization based on the relationship between reward, multidimensional emotion representation, and decision-making according to one embodiment of the present invention.

[0051]

[0052] Specific details for implementing the invention

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by a skilled expert in the art to which this invention pertains. In general, the nomenclature used herein and the experimental methods described below are well known and commonly used in the art.

[0054]

[0055] In this invention, it was confirmed that by having subjects represent the emotions they predict or experience in multiple dimensions—valence, arousal, focus, and dominance—to approach emotions felt instantaneously in social and non-social contexts, and by measuring and inferring individual differences in emotional states through a hybrid model, and by quantifying and predicting the extent to which social decision-making changes according to individual-specific emotional states and subjects' instantaneous emotional states through hierarchical modeling, it is possible to measure individual-specific instantaneous emotions and mental health states, such as anxiety disorders and personality disorders, and to predict individual-specific emotion-dependent future decision-making, such as suicide and addictive behaviors.

[0056] Accordingly, the present invention relates to a method for inferring the emotional state of a subject through multidimensional representation, comprising, in one aspect: (a) obtaining instantaneous social emotion data of a subject felt while making a decision in a decision-making task using a multidimensional emotion representation method; (b) determining a pattern of instantaneous social emotion from the obtained instantaneous social emotion data; (c) predicting a decision based on the determined pattern of instantaneous social emotion using a hybrid classification model; (d) analyzing the type of decision predicted in step (c) using a Bayesian cognitive behavioral modeling technique; and (e) optimizing the type of decision through computer modeling.

[0057] The present invention relates, in another aspect, to a computer-readable recording medium on which a program for executing the above method is recorded.

[0058] In another aspect, the present invention relates to an emotional state inference device for a subject, comprising: (a) an input device for obtaining instantaneous social emotion data of a subject felt while making a decision in a decision-making task using a multidimensional emotion representation method; (b) an emotion determination device for determining and outputting a pattern of instantaneous social emotion from the obtained instantaneous social emotion data; (c) a decision determination device for predicting and outputting a decision based on the determined pattern of instantaneous social emotion using a hybrid classification model; (d) an emotion estimation device for analyzing the type of the predicted decision using a Bayesian cognitive behavioral modeling technique; and (e) a modeling-based decision optimization device for optimizing the type of the decision.

[0059]

[0060] The process of the present invention will be described in detail below.

[0061] It is already known that there is a heterogeneity problem in which human behavior and mental states can be classified differently according to various criteria, and that existing mental health classification systems, such as those for depression and anxiety, have a problem of underspecificity in that they fail to resolve this heterogeneity problem.

[0062] The present invention can improve the specificity of the model and solve the heterogeneity problem by having subjects respond through characteristics with low interdependence and classifying them through a hybrid classification model that combines supervised and unsupervised classification algorithms.

[0063] To classify human emotional states during social interaction, we derived four dimensions of emotional traits capable of classifying heterogeneous subjects with low interdependence, and developed an algorithm using a hybrid classification model to extract subject groups that reflect individual differences in emotional experience. Through this algorithm, we aim to create classification criteria to replace underspecific classification criteria such as depression and anxiety.

[0064] It was confirmed that this algorithm can identify the characteristics of social decision-making by group according to individual differences in emotional state, measure the individual-specific characteristics of a subject's emotional decision-making, and predict the outcome of the subject's decision-making, that is, for example, measure momentary emotional state, predict decision-making in social interaction, measure personality disorders and mental health states such as anxiety and depression, and predict future decision-making such as suicide and addictive behavior.

[0065] Meanwhile, existing research methods measure mental health and emotional states and predict future decision-making based on state markers, such as specific mental illnesses or psychological states, or on data regarding behavioral markers, such as suicidal thoughts or addictive behaviors; whereas, the present invention can derive state markers or behavioral markers that reflect individual differences in emotional states.

[0066] In other words, while existing methods rely on established markers for measurement or prediction, this invention presents new markers and prediction methods based on them. It can measure an individual's unique instantaneous emotional state and predict future decision-making that is unique to that individual and dependent on their emotions.

[0067]

[0068] The present invention relates to a multidimensional emotion representation method for measuring emotions using multidimensional representations at physical and social levels.

[0069] A method for inferring a subject's emotional state through a multidimensional representation according to the present invention comprises the following steps:

[0070]

[0071] (a) A step of obtaining instantaneous social emotion data of a subject while making a decision in a decision-making task using a multidimensional emotion representation method;

[0072] (b) a step of determining the pattern of instantaneous social emotion from the instantaneous social emotion data obtained above;

[0073] (c) a step of predicting decision-making based on the pattern of instantaneous social emotions identified above using a hybrid classification model;

[0074] (d) a step of analyzing the type of decision predicted in step (c) using Bayesian cognitive behavioral modeling techniques; and

[0075] (e) A step of optimizing the type of decision-making through computer modeling.

[0076]

[0077] The device for inferring a subject's emotional state according to the present invention comprises the following components:

[0078] (a) an input device for a subject to perform a decision-making task and respond with momentary social emotions;

[0079] (b) an emotion discrimination device that identifies and outputs patterns of instantaneous social emotions through an instantaneous social emotion discrimination algorithm using a multidimensional emotion representation method;

[0080] (c) A decision-making device that predicts and outputs a subject's decision based on the relationship of decision-making by social emotion through a decision-making type determination algorithm using a hybrid classification model;

[0081] (d) an emotion estimation device that estimates social emotions hidden within decision-making by estimating parameters for subjects using Bayesian cognitive behavioral modeling techniques; and

[0082] (e) A modeling-based decision optimizer that optimizes the results of the above decision discrimination device.

[0083]

[0084] The present invention has the following three technical features:

[0085] Multidimensional emotional representation

[0086] Multidimensional emotion representation refers to a method of eliciting emotional responses based on four dimensions of feeling: valence, arousal, focus, and dominance. It was found that the problem of under-specificity at the data level is resolved by adding two dimensions of social feeling (focus and dominance) to the existing two dimensions of physical feeling (valence and arousal) to measure the complex phenomenon of emotion, thereby allowing individual differences in patterns of instantaneous social emotional experience to be included.

[0087] In addition, emotional states can be classified online (in real-time) based on instantaneous social emotional patterns (Fig. 13).

[0088]

[0089] Hybrid model

[0090] It has been suggested that constructing a classification model by appropriately mixing supervised and unsupervised classification models can help solve the problem of heterogeneity in human behavior and mental states (Non-patent Literature 7). While multidimensional emotion representation solves the heterogeneity problem by increasing underfitting at the data level, a hybrid model can solve the heterogeneity problem by solving the under-specificity of the model.

[0091]

[0092] Hierarchical modeling

[0093] In cognitive behavioral modeling, including Bayesian behavioral modeling, individual differences in emotional experience can be explained through the characteristics of the groups by assuming that within a specific group, the distribution of parameters of the subjects is sampled from a constant population distribution (Fig. 14).

[0094]

[0095] The terms used in the present invention may be defined as follows.

[0096] In the present invention, a “decision-making task” refers to a task in which a human subject makes a decision. Figure 11 illustrates the step of a user performing a decision-making task through an input device. Additionally, Figure 12 illustrates an ultimatum game among the decision-making tasks. For example, in the ultimatum game, a certain amount of money is divided between two subjects; one side proposes a distribution ratio, and the other side decides whether to accept it. If accepted, both parties divide the money according to the proposed distribution, and if rejected, neither side receives any money. Since the ultimatum game is a decision-making task performed by two people rather than one, it also provides a model environment that simulates social interaction.

[0097] In the present invention, the “instantaneous social emotion discrimination algorithm” addresses the fact that a person may feel emotions while performing a decision-making task, and in particular, in a social context, one may feel social emotions rather than basic emotions. Therefore, adding social emotion dimensions (focus, dominance) serves as a method to solve the under-specificity problem regarding the measurement of emotions felt in a social context (=social emotions), thereby enabling the tracking of instantaneous social emotions. Based on the tracking of such instantaneous social emotions, this is an algorithm designed to determine specifically what social emotions were felt.

[0098] In this invention, "instantaneous social emotion" refers to the emotion of that moment and can change depending on the social context. As described above, tracking instantaneous social emotion implies that the values ​​of the multidimensional representation of the tracked emotion may exhibit unique patterns depending on the individual and the context. When classifying and measuring individual differences in emotion through a hybrid model, the classification is also based on the patterns of instantaneous social emotion. Specifically, the results were divided into four groups: groups in which the multidimensional representation emotion remains consistently low, medium, and high despite dynamically changing rewards, and groups in which the multidimensional representation emotion responds and increases as the reward increases.

[0099] In the present invention, the “decision type determination algorithm” can identify the causal relationship between the reward given to a subject, the multidimensional representation of the instantaneous social emotion felt by the subject, and the decisions made by the subject in a decision-making task through behavioral modeling techniques. Part of the conclusion of Non-Patent Literature 2 (the inventor) is that when the causal relationship of reward-emotion-decision is analyzed according to the groups derived when classifying individual differences in instantaneous social emotion through a hybrid model, not only are there individual differences in the patterns of instantaneous social emotion even for the same economic reward, but the degree to which they make decisions in response to multidimensional emotion representations also differs; this constitutes the type of decision. It refers to an algorithm for determining the type of decision.

[0100] Specifically, individuals with consistently low moods accepted the proposal in response only to pleasantness rather than rewards (non-cooperative tendency), the group with medium moods responded to both rewards and pleasantness (indifference tendency), and the group with consistently good moods accepted the proposal in response only to rewards without responding to multidimensional emotional representations (rational tendency). The group in which emotional representation values ​​increase with increasing rewards accepted the proposal in response to rewards, pleasantness, and the degree of emotional regulation (reciprocity tendency).

[0101] In the present invention, “decision-making” means making a decision about which option a subject will choose from among the options given in a decision-making task.

[0102] In the present invention, "parameter estimation for subjects" refers to the fact that, as explained above, even when the same reward is given, there may be individual differences in instantaneous social emotional experiences, and consequently, types of decision-making also differ. However, even if this phenomenon is identified, the mechanism remains unexplained. For example, regarding the fact that a friend ate an apple over a tangerine, it is impossible to know whether they ate it because they liked the apple more than the tangerine, because they liked the tangerine more but ate it to give it to their family, or because they disliked both and ate it against their will. Thus, a model capable of explaining the reasons behind the phenomena of instantaneous social emotional patterns and decision-making types identified through the subject's social emotion discrimination algorithm and decision-making type discrimination algorithm is trained on the subject's behavioral data, and then parameter values ​​representing the subject's characteristics are estimated from the model. In the analogy above, the fact that the subject licked their lips while looking at the apple is inferred through the model (=explanation) that one licks their lips when seeing a favorite food; this leads to the inference that the subject ate the apple because they liked it more, rather than giving it to their family or eating it against their will.

[0103] In the present invention, “social emotions hidden within decision-making” refers to momentary social emotions felt during the decision-making process. It can be described as hidden because subjective emotional experiences are not directly revealed in observations of behavioral data.

[0104]

[0105] In the present invention, in step (a), the decision-making task is a task in which a subject makes a decision. For example, if the decision-making task is a final ultimatum game in which two subjects receive a certain amount of money, the invention is characterized by one side presenting the ratio of distribution and the other side deciding whether to accept it, and the subject responding to the momentary social emotion felt by the subject using a multidimensional emotion representation method.

[0106] In the present invention, in step (b), the multidimensional emotion representation method causes the emotion to respond in four dimensions: valence, arousal, focus, and dominance.

[0107] In the present invention, step (c) can derive group labels reflecting individual differences from time series of reward, emotion, and decision using the hybrid classification model, and verify the significance of reward and emotion prediction errors using decision variables through statistical techniques of a linear mixed model.

[0108] In the present invention, it is preferable that the hybrid classification model is a classification model that combines a supervised learning classification model and an unsupervised learning classification model.

[0109] In the present invention, the emotions are nine emotions: shame, guilt, embarrassment, pride, righteous anger, contempt, moral aversion, exhilaration, and gratitude, and the decision-making may be four types of decision-making: non-cooperative tendency, indifferent tendency, reciprocal tendency, and rational tendency.

[0110] In the present invention, the Bayesian cognitive behavioral modeling technique of step (d) may be a hierarchical modeling technique in which individual subjects are grouped together, and the prior probability distribution is assumed to be shared within the group during the Bayesian parameter estimation process.

[0111] In the present invention, in step (d), parameter estimation is performed on the collected behavioral data of subjects using a Bayesian cognitive behavioral modeling technique, and a measurement with interpretability regarding the unique behavioral characteristics of the subjects is derived accordingly. Reliability between different interpretations is compared through model comparison, and the reproducibility of the model as a measurement technique is verified through parameter recovery, and the singularity of the measurement through the model is verified through model identifiability.

[0112] In the present invention, in step (e), stimulus characteristics effective for optimizing an individual's behavioral characteristics can be predicted through computer modeling (modeling-based behavioral optimization technology). Emotional characteristics of stimuli necessary to optimize an individual's behavioral characteristics to fit a social context using a generalized linear mixture model, etc., can be predicted using beta coefficient profiles, and emotional characteristics with significant beta coefficients among the predictors of the computer modeling can be identified.

[0113] In the present invention, the input device may be a computer or a smartphone.

[0114] In the present invention, the emotion discrimination device can have a response regarding a predicted emotion before performing each task and an experienced emotion after performing each task through a multidimensional emotion representation method, calculate the difference between the predicted value and the actual value for the multidimensional emotion representation, and use this to measure the emotion prediction error.

[0115] In the present invention, the emotion discrimination device can quantitatively measure emotion concepts by having subjects evaluate qualitative emotion concepts of basic emotion and social emotion in the physical dimension of valence and arousal and the social dimension of focus and dominance.

[0116] In the present invention, the decision type determination device can derive group labels reflecting individual differences from time series of reward, emotion, and decision using a hybrid classification model including an unsupervised learning classification algorithm and a dimensionality reduction method, and verify the significance of reward and emotion prediction errors using decision variables through statistical techniques of a linear mixture model.

[0117] In the present invention, the emotion discrimination device and the decision-making type discrimination device can output through a discrimination emotion output device that outputs nine social emotions, such as shame, guilt, embarrassment, pride, righteous anger, contempt, moral aversion, exhilaration, and gratitude, and a decision type output device that outputs four decision types, such as non-cooperative tendency, indifferent tendency, reciprocal tendency, and rational tendency.

[0118] In the present invention, the emotion estimation device may use a hierarchical modeling technique in which the Bayesian cognitive behavioral modeling technique groups individual subjects and assumes that the prior probability distribution is shared within the group during the Bayesian parameter estimation process.

[0119] In the present invention, the emotion estimation device can derive a measurement that is interpretable regarding the unique behavioral characteristics of the subjects and performs parameter estimation using Bayesian cognitive behavioral modeling techniques on the behavioral data of the collected subjects, compare the reliability between different interpretations through model comparison, verify the reproducibility of the model as a measurement technique through parameter recovery, and verify the singularity of the measurement through the model through model identifiability.

[0120] In the present invention, the modeling-based decision optimization device can predict the emotional characteristics of stimuli necessary to optimize an individual's behavioral characteristics to fit a social context using beta coefficient profiles.

[0121]

[0122] In addition, after grouping individual subjects, the stability of parameter estimation against the influence of exceptions can be improved through hierarchical modeling techniques that assume the prior probability distribution is shared within the group during the Bayesian parameter estimation process.

[0123] In the present invention, while performing a re-execution cognitive task, the emotion predicted before each trial and the emotion actually experienced after each trial are responded to through multidimensional emotional representations, thereby measuring not only the reward prediction error but also the emotional prediction error, and simultaneously, variations in decision-making caused by the reward prediction error and the emotional prediction error can be explained through statistical techniques such as a generalized linear mixed model.

[0124] Subjective emotional experiences in social and non-social contexts can be measured, and for measurement, emotions can be represented in an abstract multidimensional space. Here, the multidimensional space consists of a physical dimension (valence, arousal) and a social dimension (focus and dominance).

[0125] The present invention is based on the inventor's paper (Non-patent Literature 2) which clarified that the social dimension applied for the quantitative measurement of emotion in this invention is not the previously known unidirectional (focus toward oneself, dominance by others) but bidirectional (self or others).

[0126] In the present invention, while performing a cognitive task, the emotion prediction error is measured using a multidimensional emotion representation method before and after each trial (instantaneous emotion measurement method), and the emotion measurement error is measured using the difference between the predicted value and the actual value of the multidimensional emotion representation before and after each trial.

[0127] In the present invention, group labels reflecting individual differences are derived from time series of rewards, emotions, and decisions using an unsupervised learning classification algorithm and a dimensionality reduction method (personalized mental state and behavior classification method).

[0128] Computer modeling predicts stimulus characteristics effective for optimizing individual behavioral characteristics (modeling-based behavior optimization technology). That is, through computer modeling, individual mental state and behavioral characteristics are derived into interpretable indicators, and changes in mental state and behavioral characteristics are measured based on these indicators. At this time, regarding social emotion experienced in a social context, the widely known classification (Tangney, Stuewig, and Mashek, 2007) utilizes valence and focus, and when represented using four dimensions of emotion, it can be seen that the classification of social emotion previously presented by Tangney et al. needs to be modified.

[0129] Using Bayesian modeling techniques, models for alternative hypotheses are compared and verified on the cognitive task performance data of the subjects (model comparison), and interpretable indicators regarding the mental state and behavioral characteristics of the subjects are calculated based on the parameter estimation of the model with the highest likelihood.

[0130] The emotional characteristics of stimuli necessary to optimize individual behavioral traits for social contexts using generalized linear mixture models can be predicted using beta coefficient profiles. Among the predictors of computer modeling, emotional characteristics with significant beta coefficients can be identified. Through model comparison among nested models, models including social dimensions of focus and dominance can significantly improve the representation of subjects' emotional experiences compared to existing models that include only reward, valence, and arousal.

[0131]

[0132] Hereinafter, preferred embodiments are presented to aid in understanding the present invention; however, the following embodiments are merely illustrative of the invention, and it is obvious to those skilled in the art that various changes and modifications are possible within the scope and spirit of the invention, and that such variations and modifications fall within the scope of the appended claims.

[0133]

[0134] [Example]

[0135] A schematic diagram of the invention according to one embodiment of the present invention is shown in FIG. 1.

[0136] As shown in Fig. 1, when a subject performs a decision-making task through an input device such as a computer or smartphone, the subject's type is determined based on patterns of instantaneous social emotions (instantaneous social emotion determination algorithm). Here, instantaneous social emotions are input in a multidimensional manner at intervals during the performance of the decision-making task, and the subject's type is determined based on the relationship between social emotions and decision-making (decision type determination algorithm). Additionally, the nine social emotions experienced by the subject are inferred (9-element determination output device), and the subject's decision is predicted (4-element decision type output device). Finally, the social emotions hidden within the decision can be explained through parameter estimation regarding the subject using Bayesian cognitive behavioral modeling techniques (estimation of social emotions hidden within the decision).

[0137] FIG. 2 is a diagram showing an emotion representation (a) and a decision prediction model (b) for measuring emotion prediction error according to an embodiment of the present invention. The emotion prediction error can be calculated by having the subject respond with an emotion using an emotion representation technique before and after each trial and determining the difference. The significance of the emotion prediction error, as well as the reward prediction error, can be verified as an explanatory variable for decision-making using statistical techniques such as a generalized linear mixed model (Non-Patent Literature 1).

[0138] FIG. 3 is a diagram showing the measurement of subject characteristics through Bayesian modeling (a) and the reproducibility of Bayesian modeling through hierarchical modeling (b) according to one embodiment of the present invention.

[0139] FIG. 3a is a diagram showing an individual model in which the parameter (γ) to be estimated in the parameter estimation process is assumed to be an independent characteristic of individual subjects, and FIG. 3b is a diagram showing a hierarchical model in which the parameter (γ) to be estimated in the parameter estimation process is assumed to be a characteristic shared within a certain group of subjects. Through this, a shrinkage effect occurs, making the parameter estimation robust against outliers and improving the stability of the parameter estimate (Non-patent Literature 6). However, it is obvious to those skilled in the art that the present invention is not limited to the model of the presented example and that various models can be applied, merely describing the model specification and the example of the parameter to be estimated.

[0140] FIG. 4 is a diagram illustrating the relationship between the problem of heterogeneity and various characteristics of behavior and mental state according to one embodiment of the present invention and the under-specificity of the mental health classification system.

[0141] Various characteristics of behavior and mental states may have different distributions. For example, characteristics such as Korea's political system (Characteristic 1) and regional culture (Characteristic 2) will have different distribution patterns (heterogeneity problem). If the distribution of these diverse characteristics does not show a high degree of agreement with the distribution of characteristics derived from existing psychiatric classification systems, such as depression and anxiety, then characteristics such as depression and anxiety may not be suitable for explaining the heterogeneity problem (under-specificity). This means that, for example, it may not be appropriate to attempt to predict voting for a specific political party or to explain the characteristics of regional culture by investigating the characteristics of depression and anxiety.

[0142] FIG. 5 is a diagram showing the resolution of under-specificity through a hybrid classification model according to an embodiment of the present invention. It has been suggested that using a hybrid classification model that combines supervised and unsupervised classification methods is a solution to the problem of heterogeneity of human behavior and mental state characteristics (Non-patent Literature 7).

[0143] FIG. 6 is a diagram illustrating a multidimensional emotion representation method according to an embodiment of the present invention. By having subjects represent emotions they predict or experience in the dimensions of valence, arousal, focus, and dominance, it is possible to approach and measure emotions felt momentarily in social and non-social contexts. Emotion representation responses may use a traditional one-dimensional Likert scale or an affect grid of two or more dimensions.

[0144] FIG. 7 is a diagram illustrating a method for measuring instantaneous emotion according to an embodiment of the present invention. The subject is asked to respond using a multidimensional emotion representation method with a predicted emotion before each trial and an experienced emotion after each trial, and the emotion prediction error can be measured by subtracting the predicted emotion from the experienced emotion in each dimension (subtraction operation). Through this, it is possible to determine how much the actual emotion deviates from the emotion predicted by the subject.

[0145] FIG. 8 is a diagram illustrating a customized mental state and behavior classification method according to an embodiment of the present invention. By using statistical techniques such as unsupervised learning classification algorithms and dimensionality reduction methods, groups reflecting individual differences can be derived from time-series data. This is a technology for the instantaneous social emotion discrimination algorithm and decision-making type discrimination algorithm of FIG. 1.

[0146] The present invention can predict stimulus characteristics effective for optimizing individual behavioral characteristics through computer modeling (modeling-based behavior optimization technology). Using generalized linear mixture models, the emotional characteristics of stimuli necessary to optimize individual behavioral characteristics to fit a social context can be predicted using beta coefficient profiles, and emotional characteristics with significant beta coefficients among the predictors of computer modeling can be identified.

[0147] As shown in Table 2, through model comparison among nested models, it was confirmed that a model including the social dimensions of focus and dominance can significantly improve the representation of subjects' emotional experiences compared to a model including only reward, valence, and arousal.

[0148]

[0149]

[0150] Table 1 shows a method (modeling-based behavior optimization technique) for predicting stimulus characteristics effective for optimizing individual behavioral characteristics through computer modeling according to one embodiment of the present invention.

[0151] FIG. 9 is a diagram illustrating the definition and measurement of social emotion according to one embodiment of the present invention.

[0152] FIG. 10 is a diagram illustrating a technique for estimating social emotions hidden within decision-making according to an embodiment of the present invention. It shows a method (modeling-based mental and behavioral characteristic measurement method) that derives an individual's mental state and behavioral characteristics into interpretable indicators through computer modeling and measures changes in mental state and behavioral characteristics based on the indicators.

[0153] As illustrated in FIG. 10, an individual's mental state and behavioral characteristics are derived into interpretable indicators through computer modeling, and changes in mental state and behavioral characteristics are measured based on the indicators (modeling-based mental and behavioral characteristic measurement method).

[0154] Through Bayesian modeling techniques, models for alternative hypotheses can be compared and tested on the cognitive task performance data of subjects (model comparison), and interpretable indicators regarding the mental state and behavioral characteristics of subjects can be produced based on the parameter estimation of the model with the highest likelihood.

[0155]

[0156] While conventional methods classified people based on decision-making types derived from logical thinking, this method is characterized by classifying people according to decision-making types that consider moment-by-moment social emotions, thereby presenting a classification method more suitable for actual social situations. Accordingly, the following types of embodiments can be presented as applicable examples of this invention.

[0157] [Non-clinical Examples]

[0158] - Assigning tasks within a company based on members' decision-making styles

[0159] - Job exploration and recommendation tailored to the client's decision-making style in career counseling

[0160] - Optimal matching algorithms that consider users' decision-making types in marriage agencies / dating apps, etc.

[0161] - Establishing personalized advertising and sales strategies based on customer decision-making types in the advertising industry

[0162]

[0163] [Clinical Examples]

[0164] - Emotional learning program for individuals with difficulties in social and emotional learning and expression, such as alexithymia and autism

[0165] - Used to help patients with anxiety / depression accompanied by low self-esteem recognize and regulate moral emotions such as shame and guilt that they commonly experience.

[0166] - Utilizing decision-making related to social emotions in the treatment process of psychiatric counseling, such as transference and countertransference, to help patients and physicians identify these factors and reduce resistance to and discontinuation of psychotherapy.

[0167]

[0168] Foregoing, specific parts of the present invention have been described in detail. It will be apparent to those skilled in the art that such specific descriptions are merely preferred embodiments and do not limit the scope of the invention. Accordingly, the actual scope of the invention is defined by the claims and their equivalents.

Claims

1. A method for inferring a subject's emotional state through a multidimensional representation including the following steps: (a) A step of obtaining instantaneous social emotion data of a subject while making a decision in a decision-making task using a multidimensional emotion representation method; (b) a step of determining the pattern of instantaneous social emotion from the instantaneous social emotion data obtained above; (c) a step of predicting decision-making based on the pattern of instantaneous social emotions identified above using a hybrid classification model; (d) a step of analyzing the type of decision predicted in step (c) using Bayesian cognitive behavioral modeling techniques; and (e) A step of optimizing the type of decision-making through computer modeling.

2. A method for inferring a subject's emotional state through multidimensional representation, characterized in that, in the case of the decision-making task of claim 1, the subject's momentary social emotion felt while one side presents the ratio to be distributed and the other side decides whether to accept it is responded to using a multidimensional emotion representation method, wherein the decision-making task is a final ultimatum game in which two subjects receive a certain amount of money.

3. A method for inferring a subject's emotional state through a multidimensional representation, wherein, in claim 1, the multidimensional emotional representation method is composed of four dimensions: valence, arousal, focus, and dominance.

4. A method for inferring the emotional state of a subject through a multidimensional representation, wherein step (c) in the first paragraph is characterized by using a hybrid classification model to extract a group of subjects reflecting individual differences in emotion and predicting a decision while verifying errors in decision-making.

5. A method for inferring a subject's emotional state through a multidimensional representation, characterized in that, in claim 1, the hybrid classification model is a model that combines a supervised learning classification model and an unsupervised learning classification model.

6. A method for inferring a subject's emotional state through a multidimensional representation, characterized in that, in paragraph 1, the emotions are nine emotions: shame, guilt, embarrassment, pride, righteous anger, contempt, moral aversion, exhilaration, and gratitude, and the decision-making is four decision-making types: non-cooperative tendency, indifferent tendency, reciprocal tendency, and rational tendency.

7. A method for inferring a subject's emotional state through a multidimensional representation, wherein, in step (d) above, the Bayesian cognitive behavioral modeling technique uses a hierarchical modeling technique that assumes that the prior probability distribution is shared within the group during the Bayesian parameter estimation process.

8. A method for inferring the emotional state of a subject through a multidimensional representation, wherein, in claim 1, step (d) is characterized by using Bayesian cognitive behavioral modeling techniques on the collected behavioral data of subjects to derive parameters and interpretable measurements of the subjects' unique behavioral characteristics, comparing the reliability between different interpretations through model comparison, verifying the reproducibility of the model as a measurement technique through parameter recovery, and verifying the unity of measurement through the model through model identifiability, while analyzing the type of decision-making.

9. A method for inferring a subject's emotional state through a multidimensional representation, wherein, in the first paragraph, step (e) is characterized by optimizing the emotional characteristics of the stimulus necessary to optimize the individual's behavioral characteristics to fit a social context using beta coefficient profiles.

10. A device for inferring a subject's emotional state including the following: (a) an input device for obtaining instantaneous social emotion data of a subject while making a decision in a decision-making task using a multidimensional emotion representation method; (b) an emotion determination device for determining and outputting patterns of instantaneous social emotions from the instantaneous social emotion data obtained above; (c) A decision-making device for predicting and outputting a decision based on the pattern of instantaneous social emotion identified above using a hybrid classification model; (d) an emotion estimation device that analyzes the type of predicted decision using Bayesian cognitive behavioral modeling techniques; and (e) A modeling-based decision optimizer that optimizes the above type of decision.

11. A device for inferring a subject's emotional state, characterized in that, in paragraph 10, the input device is a computer or a smartphone.

12. A device for inferring a subject's emotional state, characterized in that, in claim 10, the multidimensional emotion representation method comprises four dimensions: valence, arousal, focus, and dominance.

13. In claim 10, the decision-making discrimination device is characterized by using a hybrid classification model to extract a group of subjects reflecting individual differences in emotion, and predicting a decision while verifying errors in decision-making, thereby inferring the emotional state of a subject.

14. A subject's emotional state inference device, characterized in that, in the emotion discrimination device of claim 10, the emotions are nine emotions: shame, guilt, embarrassment, pride, righteous anger, contempt, moral aversion, exhilaration, and gratitude, and in the decision discrimination device, the decision is four decision types: non-cooperative tendency, indifferent tendency, reciprocal tendency, and rational tendency.

15. A device for inferring a subject's emotional state, characterized in that, in the above-mentioned Bayesian cognitive behavioral modeling technique, the Bayesian cognitive behavioral modeling technique uses a hierarchical modeling technique that assumes that the prior probability distribution is shared within the group during the Bayesian parameter estimation process.

16. An emotion state inference device for a subject according to claim 10, wherein the emotion estimation device uses Bayesian cognitive behavioral modeling techniques on collected behavioral data of subjects to perform parameter estimation and derive interpretable measurements of the subjects' unique behavioral characteristics, compare the reliability between different interpretations through model comparison, verify the reproducibility of the model as a measurement technique through parameter recovery, and verify the singularity of the measurement through the model through model identifiability.

17. A subject's emotional state inference device according to claim 10, wherein the above-mentioned modeling-based decision optimization device optimizes the emotional characteristics of stimuli necessary to optimize individual behavioral characteristics to fit a social context using beta coefficient profiles.

Images