An emotion evaluation and individualized intervention method and system based on multi-modal data

Abstract

The application discloses a kind of emotion evaluation and intervention method and system based on multi-modal data.Through terminal synchronous acquisition voice, face, screen tremor and so on multi-modal data;Reaction mode vector is generated based on context information and data verification fusion;The composite emotional state of user is determined using emotional emergence model.At the same time, personalized emotional causal diagram is constructed and updated, and the enhanced state vector is generated by extracting key causal path features.The vector is input into the intervention agent to output intervention actions;After execution, the emotional state is evaluated again and the reward is calculated, and the system model is updated online.The application realizes accurate emotion evaluation and closed-loop adaptive intervention that strikes the root.

Description

Technical Field

[0001] This invention relates to the fields of computer technology, artificial intelligence and digital health, and in particular to a method and system for emotion assessment and personalized intervention based on multimodal data. Background Technology

[0002] Emotions are a core component of human experience, profoundly impacting an individual's mental health, decision-making, and even physical health. With the development of mobile computing and wearable technologies, objective and continuous measurement and intervention of emotions through technological means has become an important research direction in computational psychology and digital health.

[0003] In existing technologies, emotion assessment methods typically rely on data from a single or limited number of modalities, such as analyzing only the acoustic features of speech, the action units of facial expressions, or simple physiological signals (like heart rate). While these methods can identify some basic emotions (such as joy, anger, and sadness) in specific scenarios, they have many limitations. First, emotion is a complex psychological phenomenon, and its outward manifestations are multi-channel and multi-dimensional. Data from a single modality cannot fully capture the subtle changes in emotion, leading to insufficient accuracy and reliability of the assessment results. Second, although some methods attempt to integrate multiple data sources, they often employ shallow fusion strategies such as simple feature layer concatenation or decision layer voting, failing to effectively uncover the deep correlations and complementary information between different modalities, resulting in the phenomenon of "data silos."

[0004] More importantly, most existing models are static and uncontextualized. They typically treat emotions as discrete, isolated state labels, ignoring the dynamic evolution of emotions and their close coupling with the surrounding environment. The physiological and behavioral responses of the same person may correspond to drastically different emotional states in different situations (such as working in an office versus a family gathering). Existing models lack effective modeling of situational factors, resulting in poor generalization ability in real, complex daily life. Furthermore, most models adopt a "one-size-fits-all" paradigm, attempting to apply a single model to all users, ignoring the significant differences in emotional expression and physiological responses between individuals, resulting in a severe lack of personalization capabilities. Especially for complex emotions that are difficult to define with simple labels (such as anxiety, anticipation, burnout, flow, etc.), existing technologies are very weak in handling them.

[0005] In terms of emotion intervention, existing technologies also suffer from a disconnect between assessment and intervention. Intervention measures (such as playing music or recommending articles) are often based on rough emotion classification results, lacking in-depth analysis of the root causes of emotions. This "symptom-based, but not cause-related" intervention approach often has short-lived and superficial effects. Intervention strategies are also universal, failing to be personalized according to the user's unique reaction patterns and emotional triggering mechanisms, making it difficult to achieve precise and efficient emotion regulation.

[0006] Furthermore, controlling data quality and protecting user privacy during data collection and processing pose significant challenges to existing technologies. The lack of real-time, automated validation of input data can lead to low-quality or corrupted data contaminating models, thus affecting the final effectiveness of assessments and interventions. Simultaneously, the collection and analysis of biometric data such as facial and voice features have raised concerns among users about the potential leakage of their personal privacy.

[0007] Therefore, there is an urgent need for a new technical solution that can overcome the above-mentioned shortcomings of existing technologies, achieve in-depth, accurate, and contextualized assessment of complex and compound emotions, and provide closely coupled, personalized, and root-cause-targeting intelligent interventions, while ensuring the reliability and security of the data processing process. Summary of the Invention

[0008] The main objective of this invention is to provide a method and system for emotion assessment and personalized intervention based on multimodal data, aiming to solve the technical problems in the prior art, such as static, non-contextualized, and low-accuracy emotion assessment models, as well as the generalization, disconnection from the assessment process, and poor effectiveness of emotion intervention strategies.

[0009] To achieve the above objectives, the first aspect of this invention provides a method for emotion assessment and personalized intervention based on multimodal data, comprising the following steps:

[0010] Multimodal data of users are collected synchronously through the data acquisition interface of terminal devices. The multimodal data includes at least voice data, facial data, eye movement data, screen touch micro-vibration data and radio frequency respiratory disturbance data. The validity of the multimodal data is verified to filter high-quality data.

[0011] Based on the environmental information of the terminal device and the current application usage information, a discrete context label describing the current situation is generated; at the same time, using a preset variational autoencoder model, the multimodal data that has passed the validity verification is fused and dimensionality reduced to generate a reaction pattern vector.

[0012] The context label and the response pattern vector are input into a preset emotion emergence model based on dynamic context coupling field theory. The user's current complex emotional state is determined by calculating the stable state of the response pattern vector in the dynamic emotional field determined by the context label.

[0013] Based on the user's historical multimodal data, historical context labels, and emotional state sequences, a personalized emotional causal graph representing the user's emotional triggering and evolution patterns is constructed and dynamically updated using a time-series causal discovery algorithm.

[0014] The composite emotional state is fused with key causal path features related to the current context extracted from the personalized emotional causal graph to generate an enhanced state vector.

[0015] The enhanced state vector is input to a personalized intervention agent based on meta-reinforcement learning deployed on the terminal device, and the agent makes a decision based on its current policy and outputs an intervention action.

[0016] The terminal device executes the intervention action and, after execution, re-determines the user's complex emotional state, and then calculates an intervention reward. The calculation of the intervention reward is based at least on the amount of change in emotional state before and after the intervention and whether the key causal path is successfully interrupted.

[0017] Using the enhanced state vector, the intervention action, and the intervention reward, the strategy of the personalized intervention agent is updated online, and the emotion emergence model and the personalized emotion causal graph are updated using the latest data feedback, forming a closed-loop adaptive system.

[0018] A second aspect of the present invention provides an emotion assessment and personalized intervention system, comprising:

[0019] A data acquisition unit is configured to synchronously acquire multimodal data of a user through the hardware interface of a terminal device. The multimodal data includes at least voice data, facial data, eye movement data, screen touch microvibration data, and radio frequency respiratory disturbance data, and performs validity verification on the multimodal data.

[0020] A feature processing unit is configured to generate discrete context labels based on environmental information and application usage information, and to fuse the multimodal data that has passed validity verification into a response pattern vector using a variational autoencoder model.

[0021] An emotion modeling unit is configured to input the context label and response pattern vector into an emotion emergence model based on dynamic context coupling field theory, and determine the user's current complex emotional state by calculating the stable state of the response pattern vector in the dynamic emotional field.

[0022] A causal analysis unit is configured to construct and dynamically update a personalized emotion causal graph representing the pattern of user emotions based on historical data.

[0023] A state generation unit is configured to fuse the composite emotional state with key causal path features extracted from the personalized emotional causal graph to generate an enhanced state vector.

[0024] A decision-making intervention unit is configured to receive the enhanced state vector and decide to output an intervention action, and to calculate an intervention reward based on the feedback after the intervention.

[0025] A model update unit is configured to update the personalized intervention agent, the emotion emergence model, and the personalized emotion causal graph online using the enhanced state vector, intervention action, and intervention reward.

[0026] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein when the processor executes the program, it implements the method described in any of the preceding embodiments.

[0027] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the preceding claims.

[0028] The present invention also provides a computer program product, including computer instructions, which, when executed by a processor, implement the method described in any of the preceding embodiments.

[0029] The beneficial effects of this invention are as follows:

[0030] 1. Significantly Improved Depth and Accuracy of Assessment: This invention overturns the traditional paradigm of emotion classification and labeling. By constructing a dynamic context-coupled field, it can accurately capture and quantify complex emotions (such as anxiety and flow) and dynamic emotion flows that change with the context and are difficult for traditional models to handle. This model elevates emotion modeling from static classification to the level of dynamic system evolution, greatly improving the depth, granularity, and accuracy of emotion assessment.

[0031] 2. Personalized and Fundamental Intervention: This invention innovatively combines temporal causal discovery with meta-reinforcement learning to create a closed-loop system capable of root-cause intervention. Intervention decisions not only consider the current emotional state but also incorporate key triggering paths in a personalized causal graph, guiding intervention measures to directly address the root causes of emotional problems. Simultaneously, the meta-learning framework solves the cold-start and rapid personalization issues of intervention models, achieving truly personalized and effective interventions tailored to each individual.

[0032] 3. High System Reliability and Security: This invention incorporates a rigorous multimodal data validity verification mechanism, ensuring high-quality input data from the source and providing a reliable foundation for subsequent advanced analysis. Simultaneously, by combining an edge computing framework and biometric desensitization technology, all original sensitive data is processed on the user's local device, maximizing user privacy and security while achieving powerful functionality.

[0033] 4. Achieved an intelligent closed loop of "assessment-intervention": This invention seamlessly integrates high-quality assessment with adaptive intervention, forming an intelligent closed-loop system capable of continuous learning and evolution. Every user interaction and every feedback is used to optimize the emotion model, causal graph, and intervention strategy, enabling the system performance to continuously improve with usage time, providing users with dynamic and continuously optimized emotional health support. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart illustrating an emotion assessment and personalized intervention method provided in one embodiment of the present invention.

[0036] Figure 2 This is a conceptual schematic diagram of a dynamic context-coupled emotion field in one embodiment of the present invention.

[0037] Figure 3 This is a structural block diagram of an emotion assessment and personalized intervention system provided in one embodiment of the present invention.

[0038] Figure 4 This is a schematic diagram of the intervention and closed-loop update process based on meta-reinforcement learning in one embodiment of the present invention.

[0039] Figure 5 This is a conceptual diagram illustrating multi-agent emotion field modeling in one embodiment of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] Example 1

[0042] This embodiment details a method for emotion assessment and personalized intervention based on multimodal data. This method can run on various terminal devices, including but not limited to computers, smartphones, tablets, smartwatches, and smart bracelets, or can be implemented collaboratively by terminal devices and a network server. See also... Figure 1The method includes the following steps:

[0043] Step S101: Synchronous acquisition and validity verification of multimodal data.

[0044] This step aims to capture comprehensive and high-quality physiological and behavioral data that reflects the user's emotional state. The terminal device, such as a smartphone, synchronously collects the user's multimodal data through its built-in or external sensors and data interfaces.

[0045] The multimodal data not only includes traditional biometric data, but also innovatively incorporates micro-behavioral and non-contact physiological data. Specifically, data types may include:

[0046] 1) Voice data: Collected through the microphone of the terminal device, used to analyze acoustic features such as pitch, intensity, speech rate, and Mel-frequency cepstral coefficients (MFCC), as well as to analyze the semantic sentiment of the voice content through natural language processing technology.

[0047] 2) Facial Data: Facial video data captured by the front-facing camera. This data can be used to extract information such as facial action units, micro-expressions, and head posture. Additionally, it may include facial blood flow data and facial temperature data. Facial blood flow data can be estimated using refractive indexing (rPPG) technology, which utilizes a visible light camera to capture subtle changes in facial skin color caused by blood flow. Facial temperature data can be acquired using a camera with an integrated infrared thermal sensor module.

[0048] 3) Eye-tracking data: By combining the camera with eye-tracking algorithms, data such as the user's gaze point, gaze duration, saccade path, and pupil diameter changes are obtained.

[0049] 4) Cognitive-behavioral data: Records users' reaction time, accuracy, and key pressure when performing specific cognitive-emotional assessment tasks (such as dot-probe tasks, Stroop tasks, and reward-delayed tasks). These tasks induce emotional responses by presenting emotional stimuli (such as emotional words, images, and videos) and assess their impact on cognitive processes.

[0050] 5) Screen Touch Microvibration Data: This is an innovative data dimension. By sampling the capacitive touchscreen of the terminal device at high frequencies (e.g., sampling frequencies greater than 200Hz), the raw coordinate sequences of the user during screen touch and swipe operations are obtained. Since neuromuscular tremors (especially in the 8-12Hz band) are related to the excitability of the sympathetic nervous system, this method performs a Fast Fourier Transform (FFT) on the acquired coordinate sequences and extracts the energy spectral density or power in the 8-12Hz band as an important feature reflecting the user's physiological arousal and tension levels.

[0051] 6) Radio Frequency Respiratory Disturbances Data: This is another innovative non-contact physiological data acquisition method. Utilizing the radio frequency (RF) transceiver unit (such as a Wi-Fi or cellular network antenna) of a terminal device (such as a mobile phone), a radio signal in a specific frequency band is transmitted. This signal is captured by the receiving unit after being reflected from the user's chest cavity surface. Because breathing causes periodic rises and falls in the chest cavity, the echo signal produces a slight Doppler frequency shift. By analyzing the temporal changes in the Doppler frequency shift of the echo signal relative to the transmitted signal, real-time respiratory rhythm and respiratory depth data of the user can be extracted non-contactly. These data are highly correlated with emotional states such as relaxation and tension.

[0052] To ensure the quality of input data, the system automatically performs online validity verification of the data after or during data collection. This step is fundamental to subsequent accurate analysis.

[0053] Voice data verification: Using voice feature analysis technology and deep semantic parsing algorithms, the collected voice samples are compared with the target voice signals (such as the text to be read aloud) pre-stored in the system. The comparison includes key features such as timbre, pitch, speech rate, and semantic coherence. If the consistency of each feature is lower than a preset threshold, it is judged as invalid data (e.g., the user is not speaking or is rambling), and the user is prompted to collect data again.

[0054] Facial Data Verification: Throughout the facial video acquisition process, the integrity of the user's face and the camera angle are monitored in real time. A deep learning-based facial key point detection algorithm is used to ensure that most key feature points (such as eyes, nose, and mouth) are clearly visible and not obstructed. Simultaneously, by analyzing facial 3D pose information, it is determined whether the camera angle is within a preset allowable range (e.g., yaw and pitch angles less than 15 degrees). If any condition is not met, the data is deemed invalid. For facial blood flow and temperature data, a signal quality assessment algorithm is used to analyze the signal-to-noise ratio (SNR) in real time. If the SNR is lower than a preset threshold, it indicates that the signal is excessively affected by motion or lighting interference, and the data is deemed invalid.

[0055] Eye-tracking data verification: During eye-tracking calibration and task execution, the user's eye movement trajectory is tracked. If the average deviation between the actual eye movement trajectory and the target position to be gazed at on the screen exceeds a preset threshold, the user is deemed to be inattentive, and this segment of eye-tracking data is considered invalid.

[0056] Cognitive behavioral data validation: During the execution of cognitive tasks, the user's response is monitored in real time. If the user's average response time to the task exceeds a reasonable range set based on a large amount of experimental data (e.g., too long or too short), or if the task error rate is higher than a preset threshold, it is determined that the user may not have performed the task seriously, and this segment of cognitive behavioral data is invalid.

[0057] Through the above verification, the system can effectively filter out low-quality data caused by improper user operation, environmental interference, or lack of concentration, thereby ensuring the accuracy of subsequent emotion assessments.

[0058] Step S201: Real-time calculation of the situation-response vector.

[0059] This step transforms raw, high-dimensional, heterogeneous multimodal data into structured, low-dimensional, and information-condensed feature representations. This process involves two parallel paths:

[0060] 1) Context Awareness and Tag Generation: The system runs a lightweight context recognition model in parallel on the device. This model receives environmental information and application usage information from the terminal device, such as:

[0061] Ambient audio: Background sounds collected by the microphone are processed by audio feature extraction networks such as VGGish and classified into categories such as "quiet", "noisy", and "conversation".

[0062] Location information: By locating via GPS, Wi-Fi, or cellular base stations and combining it with the user's historical data, it identifies locations such as "home," "office," and "on the way to work."

[0063] Time information: Current time, day of the week.

[0064] Activity status: Classified into "stationary", "walking", and "moving" based on accelerometer and gyroscope data.

[0065] Application usage information: Applications currently running in the foreground, such as "social media", "work software", and "games".

[0066] Based on the above information, the model outputs a discrete context label Ci, such as C1 = "Stationary office work", C2 = "Listening to music during commuting", and C3 = "Social entertainment with friends".

[0067] 2) Multimodal data fusion and reaction pattern vector generation: All multimodal data that passed the validity verification in step S101 are preprocessed and feature extracted within a synchronous time window (e.g., 5 seconds) to form a high-dimensional feature vector. Then, this high-dimensional vector is input into a pre-trained variational autoencoder (VAE) model.

[0068] The encoder portion of VAE compresses high-dimensional, heterogeneous input data into a low-dimensional, dense latent vector, namely the reaction pattern vector Rj. This vector Rj can be viewed as a comprehensive and holistic "snapshot" of the user's physiological and behavioral state within a given time window. It is decoupled from specific emotion labels and purely describes "how the body and behavior react." This structure of VAE enables it to learn the non-linear relationships between different modalities, achieving efficient deep fusion.

[0069] Step S301: Modeling of complex emotion emergence based on dynamic context-coupled field theory.

[0070] This is the core innovation of the emotion assessment in this invention. This step no longer uses the traditional "feature -> label" classification paradigm, but introduces dynamic context-coupled field theory based on physics concepts to model the emergent process of emotions.

[0071] The system internally pre-defines an emotion emergence model. This model receives the context label Ci and response pattern vector Rj generated in step S201 as input. Its core idea is to construct a dynamic, multi-dimensional "emotional field," see [link to relevant documentation]. Figure 2 .

[0072] The basic components of an emotional field: such as Figure 2 As shown, this abstract multidimensional space can be imagined as a topographic map. Some basic emotions (such as happiness 201, sadness 202, and anger 203) are defined as "potential wells" in the field. The location and depth of each potential well are determined by initialization with a large amount of labeled data, representing the typical response pattern region and its stability for a certain basic emotion.

[0073] The dynamic coupling effect of context: The "topography" of this emotional field is not fixed, but rather a function of the current context label Ci. That is, the model dynamically adjusts the field shape based on the input Ci. For example, in context C1 = "Still working", the potential energy well 203 representing "anger" may be deeper and smaller in range, requiring a stronger stimulus for the user to express anger; while in context C2 = "Traffic jam", the potential energy well may become shallower and wider, making it easier for the user to be triggered with anger. This mechanism achieves dynamic coupling between emotion and context.

[0074] Emergence of Complex Emotions: Complex emotions (such as anxiety, burnout, flow, etc.) are not considered independent labels, but are modeled as special terrains formed by the interaction of multiple basic emotional potential wells under the influence of a specific context Ci, such as a "saddle point" 204 between two potential wells or a broad "shallow valley". A response pattern vector Rj falling into these regions indicates that the user is in a mixed and complex complex emotional state.

[0075] Determining Emotional State: Once a new response pattern vector Rj is calculated, it is "placed" in an emotional field determined by the current context Ci. The system simulates Rj's dynamic evolution trajectory by calculating its energy gradient within this field. Vector Rj moves along the direction of fastest energy decrease, eventually settling at a local energy minimum, i.e., stable state 206. The location of this stable state (whether it's the bottom of a potential well for a basic emotion, or a saddle point or gentle valley corresponding to a complex emotion) determines the user's current complex emotional state Et. The output emotional state Et can be a vector containing the intensity scores of each basic emotion and the identified complex emotion label.

[0076] As a specific implementation method, this invention proposes a composite emotion recognition method based on "multimodal temporal interferograms" and "dynamic attractor clustering." This method analogizes the generation process of composite emotions to the phenomenon of "wave interference" in physics, treating each basic emotion as an "emotional wave" with a specific frequency and phase. When two or more emotional waves are simultaneously excited in a specific cognitive context, they interfere, forming a new and stable "interference pattern." The manifestation of this interference pattern in physiological and behavioral data is the representation of the identified composite emotion. The technical principle lies in the fact that each basic emotion (such as joy, anger, sorrow, and fear) corresponds to a specific set of synchronized cross-modal physiological-behavioral rhythmic patterns. For example, an "anger wave" can manifest as a coordinated oscillation of increased heart rate, tension in the brow muscles, faster speech, and increased gesture amplitude. When external or internal events trigger complex cognitive evaluations, multiple emotional waves may be activated simultaneously. The rhythms of these waves superimpose, cancel each other out, or modulate each other in time, generating a new multimodal temporal pattern, i.e., the interference pattern of the composite emotion. In the state space, this interference pattern forms a temporary dynamic attractor, which attracts the user's state and temporarily keeps it in this area, manifesting as a continuous state of complex emotions.

[0077] The specific technical implementation path of this method includes the following steps:

[0078] 1) Multimodal temporal signal decomposition and "emotional fundamental frequency" extraction. First, high-frequency temporal data of physiological signals (such as PPG, EDA), facial electromyography or video (AU), speech stream, behavioral actions (such as accelerometer), and eye movement trajectories are acquired simultaneously. Second, empirical mode decomposition or wavelet transform is performed on the temporal signals of each modality to extract intrinsic mode functions at different frequency scales to extract rhythms. Third, during the user calibration phase, a task to induce basic emotions is used to identify the dominant oscillation frequencies or patterns that occur synchronously in all modalities when the user is in a pure basic emotion. For example, the "anger fundamental frequency" is identified as a coordinated oscillation of heart rate rhythm at 1.2Hz, interbrow electromyography at 8Hz, and speech energy envelope at 2Hz, thereby establishing a personalized "basic emotion fundamental frequency feature library" for each user.

[0079] 2) Real-time "Emotion Interferogram" Calculation. First, the real-time collected multimodal data is projected onto the user's "basic emotion fundamental frequency feature library," and the instantaneous amplitude and phase of each fundamental frequency component at the current position are calculated to obtain the "multi-emotion activation vector" A(t) and "multi-emotion phase vector" Φ(t) that change over time. Second, a short time window is defined, and the "coherence-phase relationship matrix" C(t) between any two emotion fundamental frequency components i and j within the window is calculated. The matrix element C_ij contains not only the coherence intensity but also the dynamic phase difference φ_i(t) - φ_j(t), which determines whether the interference is constructive or destructive. This C(t) matrix is ​​the "multimodal temporal interferogram," which encodes how different emotional components interact and is a direct basis for identifying complex emotions.

[0080] 3) Composite Emotion Recognition Based on "Dynamic Attractor Clustering". First, the continuous C(t) matrix is ​​expanded into points in a high-dimensional space to form the user's state trajectory, thus constructing a high-dimensional state manifold. Second, recursive graph or phase space reconstruction techniques are used to analyze the dynamic characteristics of the state trajectory, and density peak clustering or temporal clustering algorithms are applied to automatically discover regions on the state manifold where state points are frequently visited and have long dwell times, i.e., "dynamic attractors". Third, in the initial stage, semantic labels (such as "anxious expectation") are assigned to the discovered attractors based on the user's active reports, and a "composite emotion-interferogram fingerprint" database is established, enabling the system to learn the "fingerprint" features of each composite emotion attractor on the interferogram C(t). For example, the fingerprint features of the emotion "awe" may be that the "fear" and "surprise" components are strongly coherent, and the phase of "fear" leads "surprise" by about 90 degrees. Finally, real-time identification is performed by calculating the user's current C(t) in real time and the dynamic similarity between it and the known "fingerprint" features of each composite emotion attractor (such as using dynamic time-normalized distance). The current state is classified into the most similar attractor, and the composite emotion category and intensity of the optimal solution are output based on the attribution confidence level.

[0081] In this way, the present invention can model emotions in a more refined, dynamic and psychologically theoretical way, and is particularly good at handling situation-dependent complex emotions.

[0082] Step S401: Construction and updating of personalized emotion cause-and-effect diagram.

[0083] To address emotions at their root, the system needs to understand the unique emotional triggers and evolution patterns of each user. This step aims to build a dynamic, personalized emotional causal graph for each user.

[0084] The system continuously runs a time-series causal discovery algorithm in the background. This algorithm utilizes historical data accumulated by users over a long period of time, including multimodal data feature sequences, historical context label sequences, and emotional state sequences inferred from step S301.

[0085] Optionally, the time-series causal discovery algorithm may employ Granger causality, constraint-based methods (such as PC algorithm, FCI algorithm), or models based on Dynamic Bayesian Network (DBN).

[0086] These algorithms analyze the temporal relationships and conditional independence between different variables to construct a directed acyclic graph (DAG), i.e., a personalized emotion causal graph. Among them:

[0087] The nodes in the graph can be specific events or states, such as "receiving a negative work email", "ambient noise > 70dB", "using social media for more than 30 minutes", "decreased heart rate variability (HRV)", and "compound emotional state = anxiety".

[0088] The edges of a graph represent causal relationships between nodes, and the weights of the edges indicate the strength of the causal influence. For example, an edge pointing from "using social media" to "anxiety" indicates that prolonged use of social media is a cause of anxiety for that user.

[0089] This cause-and-effect graph is dynamically updated as new data accumulates, making it increasingly accurate in reflecting users' personal emotional patterns.

[0090] Step S501: Enhance the generation of the state vector.

[0091] To make subsequent intervention decisions more targeted, the system needs a comprehensive state representation. This step integrates the current emotional state with deeper causal information.

[0092] When the system determines that intervention is needed (e.g., when persistent negative emotions are detected), it extracts key causal path features related to the current situation from the personalized emotion causal graph constructed in step S401.

[0093] Specifically, the extraction steps include: in the causal graph, identifying the path that terminates at the current negative emotional state (e.g., Et = "anxiety") and has the highest cumulative causal influence weight. This path reveals the most likely root cause of the current negative emotion. For example, the path might be: "Environment = Office" -> "Event = Receiving a project deadline reminder" -> "Behavior = Continuous coding for 2 hours" -> "Physiological = Decreased HRV" -> "Emotion = Anxiety".

[0094] The system concatenates or fuses the key node features (such as "project deadline reminder") and path structure features on the path with the current composite emotional state Et determined in step S301 to generate an enhanced state vector St with richer information.

[0095] Step S601: Personalized intervention decision based on meta-reinforcement learning.

[0096] At the heart of this step is a personalized intervention agent deployed on a terminal device. This agent is a model based on meta-reinforcement learning.

[0097] The enhanced state vector St generated in step S501 is provided as input to the agent. The agent has a policy network π, which makes decisions based on the input St and outputs the most suitable intervention action At.

[0098] The intervention action At is selected from a pre-defined intervention library. This intervention library contains various types of interventions, such as:

[0099] Physiological regulation: Push a guided breathing animation, play a soothing audio clip at a specific frequency (such as binaural beats), and adjust the screen color temperature and brightness.

[0100] Cognitive restructuring category: push a mini mindfulness exercise, a short cognitive behavioral therapy (CBT) exercise (such as a mind recording challenge).

[0101] Behavioral activation: Suggestions include short periods of physical activity and positive social interaction with a specific friend.

[0102] Environment change type: It is recommended to temporarily leave the current environment.

[0103] The agent's decision-making process aims to select the intervention most likely to propel the user's current emotional state towards a target positive state (such as "peaceful" or "focused"). Because the input state St contains causal path information, the agent's decision can be more fundamental. For example, if St indicates that anxiety stems from "continuous coding for 2 hours," the agent might prioritize "suggesting a short break" over "playing music."

[0104] Step S701: Intervention implementation and reward calculation.

[0105] The terminal device executes the corresponding operation based on the intervention action At output in step S601. For example, it displays an animation guiding breathing on the screen.

[0106] After a period of time following the intervention (e.g., 5 minutes), the system will execute steps S101 to S301 again to determine the user's new composite emotional state Et+1 after the intervention, and thereby obtain a new enhanced state vector St+1.

[0107] The system then calculates an intervention reward Rt to evaluate the effectiveness of the intervention. The reward function is designed in a multi-dimensional way, including at least:

[0108] 1. Mood Improvement: Based on the change in emotional state before and after the intervention. For example, whether the change from Et to Et+1 moves closer to the preset target positive emotional state.

[0109] 2. Causal Path Disruption: This examines whether the strength of previously identified key causal paths is weakened after intervention. For example, it can be quantified by checking whether the weights of relevant edges in the causal graph significantly decrease after intervention. Using "causal path disruption" as a reward can effectively guide the agent to learn how to conduct root cause interventions.

[0110] 3. User feedback: If users provide explicit feedback on the intervention (such as likes / dislikes), it will be included in the reward.

[0111] 4. Intervention cost: The degree of disruption to the user; for example, interventions that require prolonged user interaction have higher intervention costs.

[0112] Step S801: Closed-loop adaptive update.

[0113] This is a crucial step in enabling the system to continuously evolve. The system uses the experience tuples (St, At, Rt, St+1) generated in a complete "state-action-reward-new state" interaction to update the various models in the system online.

[0114] 1. Updating the intervention agent: Utilizing this experience tuple, the policy network π of the personalized intervention agent is updated through online policy gradient algorithms (such as Proximal Policy Optimization (PPO), a variant of Proximal Policy Optimization). This makes the agent's intervention policy increasingly adapted to the user's individual response patterns, thus becoming more effective.

[0115] 2. Update the emotion emergence model: Use the latest multimodal data and inferred emotional states as new training samples to fine-tune the emotion emergence model (such as the parameters of the VAE and the shape of the emotional field) to make its modeling of user emotions more accurate.

[0116] 3. Update personalized emotion causal graph: Incorporate the latest time-series data (context, behavior, physiology, emotion) into the causal discovery algorithm to update the personalized emotion causal graph so that it can reflect the latest emotional pattern changes of users.

[0117] Through this closed-loop adaptive update mechanism, the method of this invention constitutes an intelligent system that can grow together with the user, and its performance will be continuously optimized over time.

[0118] Example 2

[0119] This embodiment provides an emotion assessment and personalized intervention system, which is the hardware carrier of the method described in Embodiment 1. See also Figure 3 This system can be integrated into a single electronic device (such as a smartphone), or multiple devices (such as mobile phones and smartwatches) can work collaboratively via a communication network. The system includes:

[0120] A data acquisition unit 301 is configured to synchronously acquire multimodal data from the user through a hardware interface of the terminal device (such as a camera, microphone, touchscreen, RF transceiver, gyroscope, etc.) and perform the validity verification described in Embodiment 1. For example, this unit obtains screen touch micro-vibration data by performing FFT on the high-frequency sampling data of the capacitive touchscreen and extracting energy in the 8-12Hz frequency band; and obtains radio frequency breathing disturbance data non-contactly by analyzing the Doppler frequency shift of the RF echo signal.

[0121] A feature processing unit 302 is configured to generate discrete context labels Ci based on environmental information and application usage information, and to fuse multimodal data that has passed validity verification into a response pattern vector Rj using a variational autoencoder model.

[0122] An emotion modeling unit 303 is configured to receive a context label Ci and a response pattern vector Rj, and input them into an emotion emergence model based on dynamic context-coupled field theory. This unit determines the user's current composite emotion state Et by calculating the stable state of Rj in a dynamic emotion field determined by Ci. The dynamic emotion field models basic emotions as potential wells and composite emotions as saddle points or gentle valleys formed by the interaction of multiple potential wells under specific contextual influences.

[0123] A causal analysis unit 304 is configured to construct and dynamically update a personalized emotion causal graph representing the user's emotional patterns based on the user's historical data and using time-series causal discovery algorithms such as dynamic Bayesian networks.

[0124] A state generation unit 305 is configured to fuse the composite emotional state Et output by the emotion modeling unit 303 with key causal path features extracted from a personalized emotion causal graph to generate an enhanced state vector St. When extracting features, this unit searches for and identifies the causal chain with the highest cumulative influence weight that leads to the current negative emotional state.

[0125] A decision-making intervention unit 306 is configured to deploy a personalized intervention agent. This agent is a model based on meta-reinforcement learning. The unit is configured to receive an reinforcement state vector St and decide to output an intervention action At. After the intervention is executed, it calculates a multi-dimensional intervention reward Rt based on the feedback.

[0126] A model update unit 307 is configured to use experience tuples (St, At, Rt, St+1) to update the agent policy in the decision intervention unit 306, the emotion model parameters in the emotion modeling unit 303, and the personalized emotion causal graph in the causal analysis unit 304 online.

[0127] In addition, the system may include a privacy protection unit (not shown). This unit employs an edge computing framework to ensure that all preprocessing, feature extraction, and model inference of raw multimodal data are completed locally on the terminal device. When it is necessary to upload model update parameters or perform joint learning, this unit will use differential privacy, homomorphic encryption, or biometric desensitization techniques (such as converting raw blood flow signals into abstract vascular motion feature vectors, or only recording relative temperature changes) to process the data, ensuring user privacy and security during model optimization and data sharing.

[0128] The various units of the system in this embodiment work together to fully realize the closed-loop, adaptive emotion assessment and intervention process described in Embodiment 1.

[0129] Example 3

[0130] This embodiment focuses on the training and rapid adaptation mechanism of the personalized intervention agent mentioned in Embodiments 1 and 2. This mechanism solves the problems of "cold start" and "one-size-fits-all" in traditional intervention models. See also Figure 4 The mechanism mainly consists of two phases:

[0131] Phase 1: Meta-Training (Phase 401)

[0132] This phase is typically completed on a cloud server. The system is trained using a large-scale, anonymized dataset of multimodal digital phenotypes and intervention outcomes from numerous users.

[0133] 1. Task Definition: In this stage, each user's data is considered an independent "task." The goal of meta-learning is not to learn how to solve the intervention problem for a specific user, but rather to learn the ability to "quickly learn how to effectively intervene for a new user."

[0134] 2. Data preparation: For each task (user), the data is divided into a support set and a query set.

[0135] 3. Model Architecture: Construct a meta-reinforcement learning model, such as a policy network based on the Model-Agnostic Meta-Learning (MAML) framework.

[0136] 4. Training Process:

[0137] Inner Loop 402: For a sampled task (user), use the data from its support set to perform a small gradient update on the current model parameters to obtain a temporary, adaptive model for that user.

[0138] Outer Loop (403): This temporary model is tested on the query set for the task, and its loss (or reward) is calculated. Then, the initial parameters of the meta-learning model are updated based on the performance on this query set.

[0139] By repeating this process on a large number of different tasks, the resulting meta-model (meta-reinforcement learning agent) acquires a very good set of initial parameters. These parameters are not optimal for any particular user, but they are in a "central position" from which they can be adapted very quickly (with only a small amount of data and a few updates) to any new user.

[0140] Phase Two: Rapid Personalization Fine-tuning (Meta-Testing / Fine-tuning) Phase 404

[0141] This stage occurs on the user's terminal device.

[0142] 1. Model Deployment: When a new user starts using this system, the pre-trained meta-reinforcement learning model in the cloud is downloaded to their terminal device as the initial policy model π_θ for the intervention agent.

[0143] 2. Online interaction and fine-tuning: During daily use, the system will continuously generate online interaction data for the user, namely the experience tuple (St, At, Rt, St+1) 405.

[0144] 3. Model Update: Using this fresh, personalized data that is entirely unique to the user, the initial policy model π_θ is rapidly and continuously fine-tuned using an online policy gradient method (such as PPO).

[0145] Because the initial model already possesses strong generalization and rapid learning capabilities, it can be quickly adapted into a highly personalized intervention agent tailored specifically to the user, without requiring a large amount of data. This adapted model can deeply understand the user's unique emotional response patterns and provide precise and effective intervention strategies.

[0146] By employing a two-stage strategy of "cloud-based meta-training" and "device-based fast tuning," this invention effectively solves the problem of personalizing intervention models, avoids ineffective interventions during cold starts, and allows models to continuously evolve as users use them.

[0147] Example 4

[0148] This embodiment extends the application of the present invention in social contexts, proposing a modeling method for a "multi-agent emotion field." See also... Figure 5 This method aims to extend emotion modeling from the realm of individual psychology to social dynamics.

[0149] When a user's terminal device senses through sensors (such as Bluetooth, GPS, Wi-Fi) that the user is in a social environment (for example, having a party in the same room with several friends who also use the system), the system can trigger a social interaction mode.

[0150] In this mode, the system includes the following steps:

[0151] 1. Proximity device discovery and secure connection: The user's terminal device 501 will discover other users' terminal devices 502 and 503 in the vicinity through near field communication technologies such as Bluetooth Low Energy (BLE) or Wi-Fi Direct.

[0152] 2. Anonymous Parameter Exchange: With authorization from all users, a secure communication link is established between the devices. They exchange anonymized key parameters of each other's emotional field models, rather than the original data. For example, they can exchange a de-identified representation of the current emotional state vector Et, or key parameters of the emotional field "topography" (such as the location and depth of the main potential wells). The entire exchange process is encrypted to strictly protect user privacy.

[0153] 3. Constructing a Multi-Agent Coupled Emotional Field: Each device locally constructs a temporary, localized multi-agent coupled emotional field 504 based on collected emotional field parameters of itself and other nearby users. In this coupled field, the emotional state of an agent (user) is not only a function of its own reaction pattern but is also influenced by the emotional states of other agents within the field, simulating the phenomenon of emotional contagion or empathy in the real world. For example, a user's "happiness" potential energy well may exert a "gravitational pull" on the states of neighboring users, and vice versa.

[0154] 4. Dynamic Analysis and Prediction of Group Emotions: By analyzing the dynamic evolution of this coupled field, the system can analyze and predict trends in group emotions. For example, it can predict whether a negative emotion will spread within a small group, or assess whether the overall emotional atmosphere of the current team is positive or negative.

[0155] 5. Group-oriented intervention: In certain scenarios, the system can even intervene at the group level. For example, if a tense atmosphere is detected in a team meeting, a short group relaxation activity can be recommended, or the mood can be guided collectively by adjusting the environment (such as smart home devices).

[0156] By introducing the concept of a multi-agent emotional field, this invention expands the application scenarios of emotion assessment and intervention from individuals to groups, opening up new possibilities for team atmosphere regulation, emotional guidance in public spaces, and social psychology research.

[0157] As a rich set of variant embodiments, the specific models and algorithms in the above embodiments can be replaced. For example, in step S201, in addition to VAE, other deep generative models such as variants of Generative Adversarial Networks (GANs) can be used for multimodal fusion. In the emotion emergence model, in addition to the potential well analogy, other dynamic systems based on state-space models can be used to describe the evolution of emotions. In step S401, the causal discovery algorithm can also be other more advanced algorithms. In step S601, the reinforcement learning agent can also use other algorithms, such as Deep Q-Networks (DQN), Actor-Critic methods, etc. These alternatives do not depart from the core idea of ​​the present invention and fall within the protection scope of the present invention.

[0158] Furthermore, this invention can be further combined with longitudinal data analysis. For each subject to be evaluated, the system stores multimodal data from all previous emotion assessments. When using the emotion prediction model, in addition to the subject's current data, weighted calculations can be performed on one or more previous emotion assessments, with the weight of each data point dynamically determined based on a predetermined algorithm (e.g., more recent data has a higher weight). This approach can distinguish between a user's transient emotional state (such as brief happiness or anger, mainly predicted based on current data features) and persistent emotional state (such as long-term anxiety or depression, calculated based on long-term and multiple data feature accumulations), thereby providing a more comprehensive emotional profile.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for emotion assessment and personalized intervention based on multimodal data, characterized in that, Includes the following steps: Collect multimodal data from users through terminal devices; Generate discrete context labels describing the current context, and fuse the multimodal data to generate a response pattern vector; The context label and the response pattern vector are input into the emotion emergence model to determine the user's current complex emotional state. The emotion emergence model is a preset model based on dynamic context coupling field theory. The complex emotional state is determined by calculating the stable state of the response pattern vector in the dynamic emotional field determined by the context label. The dynamic emotional field defines basic emotions as potential energy wells with a certain energy depth, and defines complex emotions as saddle points or gentle valleys formed by the interaction of potential energy wells corresponding to multiple basic emotions under the influence of a specific context label. The evolution trajectory and final stable state of the response pattern vector in the field correspond to the complex emotional state. Build and dynamically update a personalized emotion causal graph; The composite emotional state is fused with key causal path features extracted from the personalized emotional causal graph to generate an enhanced state vector; The enhanced state vector is input into the personalized intervention agent, and the intervention action is output. Perform the intervention action and calculate the intervention reward; The personalized intervention agent, the emotion emergence model, and the personalized emotion causal graph are updated using the enhanced state vector, the intervention action, and the intervention reward.

2. The method according to claim 1, characterized in that: The collection of user multimodal data includes: synchronous collection through the data collection interface of the terminal device. The multimodal data includes at least voice data, facial data, eye movement data, screen touch micro-vibration data and radio frequency respiratory disturbance data, and validity verification is performed to screen high-quality data. The generation of discrete context labels is based on the environmental information of the terminal device and the current application usage information; The step of fusing the multimodal data to generate a reaction mode vector includes: using a preset variational autoencoder model to fuse and reduce the dimensionality of the multimodal data that has passed validity verification. The personalized emotion causal graph is constructed and dynamically updated using a time-series causal discovery algorithm based on the user's historical multimodal data, historical context labels, and emotion state sequences. It is used to characterize the triggering and evolution patterns of user emotions. The extracted key causal path features are key causal path features related to the current context; The personalized intervention agent is deployed on the terminal device and is based on meta-reinforcement learning; The calculated intervention reward is determined after the complex emotional state is reassessed following the intervention, and is based at least on the amount of change in the emotional state before and after the intervention and whether the key causal path was successfully interrupted. The update steps include: updating the strategy of the personalized intervention agent online, and updating the emotion emergence model and the personalized emotion causal graph using the latest data feedback to form a closed-loop adaptive system.

3. The method according to claim 2, characterized in that, The steps for acquiring the screen touch micro-vibration data include: performing high-frequency sampling on the capacitive touchscreen of the terminal device at a sampling frequency greater than 200Hz to obtain the original coordinate sequence during the touch process; performing a fast Fourier transform on the original coordinate sequence and extracting the energy spectral density in the 8-12Hz frequency band as a feature reflecting the tension level of the user's neuromuscular system.

4. The method according to claim 2, characterized in that, The steps for acquiring radio frequency respiratory disturbance data include: using the radio frequency transceiver unit of the terminal device to transmit radio signals in a specific frequency band; receiving echo signals generated after reflection by the user's chest cavity fluctuations; and non-contactly extracting the user's real-time respiratory rhythm and respiratory depth data by analyzing the Doppler frequency shift of the echo signals relative to the transmitted signals.

5. The method according to claim 1, characterized in that, The personalized intervention agent is a meta-reinforcement learning model pre-trained on a large-scale general dataset on a cloud server. After being deployed to the terminal device, it continuously receives the user's online interaction data, including the augmented state vector, intervention actions, and intervention rewards, and performs rapid personalized fine-tuning to adapt to the unique emotional response patterns of a specific user.

6. The method according to claim 2, characterized in that, The data validity verification step further includes: verifying the timbre, pitch, speech rate, and semantic coherence of the speech data to exclude non-human speech or invalid expressions; verifying the facial data for facial integrity, illumination uniformity, and camera angle to ensure reliable extraction of key facial feature points; and verifying the deviation between the eye movement trajectory and the target position on the screen to assess the user's level of attention.

7. The method according to claim 2, characterized in that, The steps for constructing and updating the personalized emotion causal graph include: using Granger causality tests or dynamic Bayesian network models to analyze the temporal sequence and conditional dependencies between historical multimodal data features, contextual labels, and emotional states, thereby quantitatively constructing a directed acyclic graph where nodes are states or events and edges represent the intensity of causal influence.

8. The method according to claim 1, characterized in that, The extraction step of the key causal path features includes: in the personalized emotion causal graph, identifying the path that ends with the current negative emotional state and has the highest cumulative causal influence weight, and using the key node features and path structure features on the path as the key causal path features to guide root cause intervention.

9. The method according to claim 1, characterized in that, When the terminal device is in a social context, the method further includes: communicating with the terminal devices of other nearby users via Bluetooth Low Energy or Wi-Fi Direct, exchanging anonymous key parameters of the emotional field while protecting privacy; and constructing a temporary multi-agent coupled emotional field locally based on the collected emotional field parameters of multiple users, for analyzing and predicting the dynamic evolution trend of group emotions.

10. The method according to claim 2, characterized in that, When processing the multimodal data, an edge computing framework is used to complete data preprocessing, feature extraction, and model inference locally on the terminal device, and only the de-identified model update parameters or statistical results are uploaded to the cloud; in addition, before processing biometric data including facial data, differential privacy or homomorphic encryption technology is applied to de-identify biometric features to protect user privacy and security.

11. The method according to claim 1, characterized in that, The intervention actions include, but are not limited to, the following categories: pushing mindfulness practice guidance on the terminal device interface, playing personalized recommended soothing music, adjusting the screen color temperature and brightness, suggesting short breaks or physical activities, or triggering positive interactions with specific friends in social applications.

12. The method according to claim 1, characterized in that, The step of determining the user's current complex emotional state specifically includes calculations based on a multimodal temporal interferogram, the calculations including: Empirical mode decomposition or wavelet transform is performed on the multimodal time-series signals of users in basic emotional states to extract the dominant frequencies and patterns of co-oscillations in heart rate, electromyography, speech and behavioral action modes, so as to establish a personalized basic emotional fundamental frequency feature library. The real-time acquired multimodal data is projected onto the basic emotion fundamental frequency feature library, and the instantaneous amplitude and phase of each fundamental frequency component are calculated; and Within a preset time window, the coherence intensity and phase difference dynamics between any two fundamental frequency components of emotions are calculated, and a coherence-phase relationship matrix is ​​constructed as the multimodal temporal interferogram to characterize the interference between different emotional components.

13. The method according to claim 12, characterized in that, The step of determining the user's current complex emotional state further includes identification based on the multimodal temporal interferogram and dynamic attractor clustering, the identification including: The continuously calculated coherence-phase relation matrix is ​​expanded into state points in a high-dimensional space to form the evolution trajectory of the user state. The evolutionary trajectory is processed using recursive graph analysis or phase space reconstruction techniques, and regions where state points frequently visit and reside on the trajectory are identified using density peak clustering or temporal clustering algorithms, defined as dynamic attractors; and Calculate the dynamic similarity between the coherence-phase relation matrix at the current moment and the features of each attractor in a pre-stored database containing composite emotions and interferogram fingerprints, and classify the current state into the composite emotion category corresponding to the attractor with the highest similarity.

14. An emotion assessment and personalized intervention system, characterized in that, include: The data acquisition unit is configured to collect multimodal data from users via terminal devices; The feature processing unit is configured to generate discrete context labels describing the current context and fuse the multimodal data into a response pattern vector; An emotion modeling unit is configured to input the context label and response pattern vector into an emotion emergence model to determine the user's current complex emotional state. The emotion emergence model is a preset model based on dynamic context-coupled field theory, which determines the complex emotional state by calculating the stable state of the response pattern vector in a dynamic emotion field. The dynamic emotion field models basic emotions as potential energy wells with a defined energy depth and complex emotions as saddle points or gentle energy valleys formed by the interaction of multiple potential energy wells under the influence of a specific context. The stable state of the response pattern vector is the local minimum point reached by the vector after the energy gradient in the field decreases. The causal analysis unit is configured to build and dynamically update a personalized emotion causal graph. The state generation unit is configured to fuse the composite emotional state with key causal path features extracted from the personalized emotional causal graph to generate an enhanced state vector. The decision-making intervention unit is equipped with a personalized intervention agent, which is configured to receive the enhanced state vector and output the intervention action, and calculate the intervention reward based on the feedback after the intervention is executed. The model update unit is configured to update the personalized intervention agent, the emotion emergence model, and the personalized emotion causal graph using the enhanced state vector, intervention action, and intervention reward.

15. The system according to claim 14, characterized in that: The data acquisition unit is further configured to: synchronously acquire data through the hardware interface of the terminal device, wherein the multimodal data includes at least voice data, facial data, eye movement data, screen touch micro-vibration data and radio frequency respiratory disturbance data, and perform validity verification to screen high-quality data; The feature processing unit is further configured to: generate the discrete context labels based on environmental information and application usage information, and use a preset variational autoencoder model to fuse and reduce the multimodal data that has passed the validity verification into the reaction pattern vector; The causal analysis unit is further configured to: construct and dynamically update the personalized emotion causal graph based on historical multimodal data, historical context labels, and emotional state sequences using a time-series causal discovery algorithm to characterize the user's emotional patterns; The key causal path features extracted by the state generation unit are key causal path features related to the current context. In the decision-making intervention unit, the personalized intervention agent is based on meta-reinforcement learning; the calculation of the intervention reward is based at least on a quantitative assessment of the magnitude of the positive shift in emotional state before and after the intervention and whether the causal correlation strength of key causal paths has significantly decreased. The model update unit is further configured to: update the strategy of the personalized intervention agent online, and update the emotion emergence model and the personalized emotion causal graph using the latest data feedback.

16. The system according to claim 15, characterized in that, The data acquisition unit is further configured to: obtain the screen touch micro-vibration data by performing a fast Fourier transform on the high-frequency sampling data of the capacitive touch screen and extracting energy in a specific frequency band; and obtain the radio frequency breathing disturbance data non-contactly by analyzing the Doppler frequency shift of the echo signal of the radio frequency transceiver unit.

17. The system according to claim 14, characterized in that, The personalized intervention agent in the decision intervention unit is a meta-reinforcement learning model that is pre-trained in the cloud and deployed to the terminal. It is configured to use locally generated user interaction data for rapid online fine-tuning to achieve deep personalization of the model.

18. The system according to claim 15, characterized in that, When performing validity verification, the data acquisition unit is further configured to: verify the timbre, pitch, speech rate, and semantic coherence of the speech data to exclude non-human speech or invalid expressions; verify the facial data for facial integrity, illumination uniformity, and camera angle to ensure reliable extraction of key facial feature points; and verify the deviation between the eye movement trajectory and the target position on the screen to assess the user's level of attention.

19. The system according to claim 14, characterized in that, The causal analysis unit is further configured to: employ a dynamic Bayesian network model to analyze the temporal dependencies between various data dimensions, thereby constructing a directional, personalized emotion causal graph that reflects the transition from environmental stimuli to physiological responses and then to emotional states.

20. The system according to claim 14, characterized in that, When extracting key causal path features, the state generation unit is further configured to: identify the path with the current negative emotional state as the endpoint and the highest cumulative causal influence weight in the personalized emotion causal graph, and use the key node features and path structure features on the path as the key causal path features to guide root cause intervention.

21. The system according to claim 14, characterized in that, When the decision-making intervention unit calculates the intervention reward, its reward function is designed to include at least two parts: one part is based on the magnitude of the change in the composite emotional state towards a positive direction before and after the intervention, and the other part is based on a quantitative assessment of whether the causal correlation strength between the nodes of the key causal path is significantly reduced after the intervention.

22. The system according to claim 14, characterized in that, The system also includes a privacy protection unit, which is configured to process raw multimodal data locally on the terminal device via an edge computing framework and use differential privacy technology to add noise and desensitize the user biometric templates generated during the processing, ensuring privacy and security during model optimization and data sharing.

23. The system according to claim 14, characterized in that, The system also includes a social interaction unit configured to establish connections with other users' terminal devices in specific social contexts and securely exchange anonymous emotional field model parameters to construct a local, temporary group emotional field model.

24. The system according to claim 14, characterized in that, The emotion modeling unit is further configured to perform calculations based on multimodal temporal interferograms to determine the composite emotional state. Specifically, the unit is configured to: perform empirical mode decomposition or wavelet transform on the user's multimodal temporal signal in the basic emotional state, extract the dominant frequency and mode of the cooperative oscillation, and establish a personalized basic emotion fundamental frequency feature library; project the real-time collected multimodal data onto the basic emotion fundamental frequency feature library, and calculate the instantaneous amplitude and phase of each fundamental frequency component; Within a preset time window, the coherence intensity and phase difference dynamics between any two fundamental frequency components of emotions are calculated to construct a coherence-phase relationship matrix that serves as the multimodal temporal interferogram.

25. The system according to claim 24, characterized in that, The emotion modeling unit is further configured to perform identification based on the multimodal temporal interferogram and dynamic attractor clustering. Specifically, the unit is configured to: expand the continuously calculated coherence-phase relation matrix into state points in a high-dimensional space to form the user's state evolution trajectory; process the evolution trajectory using recursive graph analysis or phase space reconstruction technology; and identify the regions on the trajectory that are frequently visited and stayed in by the state points as dynamic attractors through density peak clustering or temporal clustering algorithms. It also calculates the dynamic similarity between the coherence-phase relation matrix at the current moment and the features of each attractor in a pre-stored database containing composite emotions and interferogram fingerprints, and classifies the current state into the composite emotion category corresponding to the attractor with the highest similarity.

26. The system according to claim 14, characterized in that, The system is integrated into an electronic device, which includes a memory, a processor, and a computer program stored in the memory. When the processor executes the program, it performs the functions of the system according to any one of claims 14 to 23.

27. An electronic device comprising: A memory, a processor, and a computer program stored on the memory, characterized in that, when the processor executes the program, it implements the method of any one of claims 1 to 13.

28. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method of any one of claims 1 to 13.

29. A computer program product comprising computer instructions, characterized in that, When the instructions are executed by the processor, they implement the method of any one of claims 1 to 13.

Images