Multi-source data fusion and self-adaptive threshold value-based user emotional state judgment method and system
By using multi-source data fusion and adaptive thresholding, the problem of single-modal recognition being susceptible to interference in children's mental health monitoring was solved, achieving highly accurate and robust emotional state determination and providing precise intervention decisions.
Patent Information
- Application Number
- CN202511875724.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-12
- Publication Date
- 2026-04-14
AI Technical Summary
In the monitoring of children's mental health, existing technologies for single-modal emotion recognition are easily affected by environmental factors, and multimodal fusion schemes lack adaptability and cannot effectively handle modality loss or low-quality input, resulting in insufficient recognition accuracy.
By employing a multi-source data fusion and adaptive thresholding approach, this method collects four main modal features: visual, linguistic, behavioral, and textual. It calculates single-modal confidence and dynamic stability weights, and combines time-recursive updates and system adaptive threshold correction to determine emotional states.
It improves the accuracy of emotion recognition and the robustness of the system, possesses trend memory, feedback learning and self-regulation characteristics, and can keenly capture the subtle dynamics of children's emotions and attention, providing accurate intervention decisions.
Smart Images
Figure CN121845579A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method and system for determining user emotional state using multi-source data fusion and adaptive thresholds, belonging to the field of artificial intelligence technology. Background Technology
[0002] In recent years, the mental health of children and adolescents has received widespread attention. Traditional mental health monitoring relies on manual interviews, questionnaires, or centralized assessments, which often suffer from drawbacks such as long processing times, poor real-time performance, and significant susceptibility to subjective factors. With the development of computer vision, speech analysis, and multimodal information processing technologies, emotion recognition and intervention systems based on smart terminals are gradually being applied in homes, schools, and public settings.
[0003] Existing systems typically perform emotion recognition using a single modality (such as video facial expression analysis or voice emotion analysis) and trigger intervention when negative emotions are detected. However, in real-world scenarios, single-modal recognition is easily affected by environmental factors such as occlusion, noise, and changes in lighting, thus impacting recognition accuracy. In the field of child mental health, although multimodal fusion solutions exist, most employ fixed weights or simple voting methods for modality synthesis, failing to adaptively adjust the fusion strategy based on changes in modality quality; and they lack robust handling for missing or low-quality inputs. Summary of the Invention
[0004] Purpose of the invention: This invention provides a robust method for determining user emotional state through multi-source data fusion and adaptive thresholding.
[0005] Technical solution: To achieve the above objectives, the technical solution adopted by this invention is as follows: A method for determining user emotional state using multi-source data fusion and adaptive thresholds includes the following steps: Step 1: Collect and extract the user's four main modal features: visual, linguistic, behavioral, and textual.
[0006] Step 2: Calculate the confidence scores of each of the four main modal features, combine them with historical statistical features to obtain dynamic stability weights, and then obtain the fused sentiment features.
[0007] Step 3: Introduce time-recursive update to update the fused sentiment features to time-based fused sentiment features.
[0008] Step 4, within the time window The system calculates the historical mean and variance, and dynamically adjusts the adaptive threshold based on global fluctuations and feedback biases.
[0009] Step 5: Define the weighted risk ratio based on the obtained system adaptive threshold and introduce a state smoothing term.
[0010] Step 6, according to the maximum probability criterion, when continuous The proportion of high-level warnings within seconds exceeded the warning threshold. When this occurs, adaptive feedback adjustment is triggered and the deviation is fed back.
[0011] Step 7: Determine the user's emotional state based on the time-based fused emotional features and the adjusted system adaptive threshold to obtain the determined user emotional state.
[0012] Preferred: Single-mode confidence level is:
[0013] in, The expression represents the confidence score of the m-th modality in unimodal emotion recognition at time t. This represents the activation function. The feature mapping function represents the m-th mode. This represents the raw data collected at time t in scenario r for the m-th modality. This indicates the type of scenario in which the emotion data was collected. Indicates the modal type identifier. Representing visual modality, Represents language / audio modality. Represents behavioral modality, Represents text modality; Dynamic stability weights:
[0014] in, This represents the dynamic stability weight of the m-th mode at time t. This represents the modal confidence sensitivity adjustment coefficient. This represents the modal stability weighting adjustment coefficient. The expression represents the confidence score of the m-th modality in unimodal emotion recognition at time t. This represents the sliding variance of the confidence level of the m-th mode at time t. Indicates the modal traversal index; Preferred: Integration of emotional features:
[0015] in, This represents the multimodal fusion emotion feature value at time t.
[0016] The preferred formula for calculating time-based fusion emotion features is as follows:
[0017] in, This indicates time-based fusion of emotional characteristics. Indicates the integration of emotional characteristics. This represents the weighting coefficient of the current fused emotional features. The weighting coefficients represent the historical time-based fusion of emotional characteristics. The weighting coefficients representing the rate of change in sentiment. ,satisfy .
[0018] Preferred: Time Window The historical mean and variance were calculated as follows:
[0019] in, This represents the historical mean of the time-based fused sentiment features within time window B. This indicates the size of the time window used for historical statistics. This represents the time-based fused emotional features at time ti. This represents the historical standard deviation of time-based fused sentiment features within time window B.
[0020] Preferred: Dynamically adjust the system's adaptive threshold based on global fluctuations and feedback deviations.
[0021] in, This represents the adaptive emotion warning threshold of the system at time t. This represents the historical mean of the time-based fused sentiment features within time window B. This represents the historical standard deviation of the time-based fused sentiment features within time window B. This represents the time-based fused emotional characteristics at time t. This represents the adjustment coefficient of historical fluctuations on the threshold. This represents the adjustment coefficient of the instantaneous rate of change to the threshold. This represents the adjustment mechanism of the feedback error on the threshold. This represents the system feedback error term;
[0022] in, This represents the system feedback error term. This represents the historical feedback error mean at time t. The adjustment coefficient represents the average historical feedback error. The adjustment coefficient representing the rate of change of feedback error. This indicates the size of the time window for calculating the feedback error. This represents the time-based fused emotional features at time ti. The system adaptive threshold is represented at time ti.
[0023] Preferred weighted risk ratio:
[0024] in, This represents the weighted risk ratio of emotional state at time t. This represents the time-based fused emotional characteristics at time t. This represents the adaptive emotion warning threshold of the system at time t. This represents the historical variance of the time-based fused sentiment features within time window B. Represents the minimum constant; State smoothing term:
[0025] in, This represents the probability that the user's emotional state level is k at time t. This represents the user's emotional state level at time t. A value indicating the level of emotional state. This represents the risk ratio moderating coefficient for the emotional state level k at time t. This represents the state continuity coefficient of emotional state level k at time t. This represents the weighted risk ratio of emotional state at time t. This represents the risk ratio moderating coefficient for emotional state level j at time t. This represents the state continuity coefficient of emotional state level j at time t. Indicates the level of emotional state. This is the smoothing coefficient.
[0026] Preferred adaptive feedback adjustment method is as follows:
[0027] in, This represents the modal confidence sensitivity adjustment coefficient. This indicates the parameter assignment / update symbol. This represents the updated modal confidence sensitivity adjustment coefficient. This represents the feedback adjustment step size of α. This represents the weighted risk ratio of emotional state at time t. This represents the modal stability weighting adjustment coefficient. This represents the updated modal stability weight adjustment coefficient. This represents the feedback adjustment step size of λ. This represents the adjustment coefficient of the threshold based on historical fluctuations.
[0028] Another objective of this invention is to provide a user emotional state determination system based on multi-source data fusion and adaptive thresholding, comprising a data acquisition and extraction module, an emotional feature fusion module, a time-based emotional feature fusion module, a system adaptive thresholding module, a determination module, and an output module, wherein: The acquisition and extraction module is used to acquire and extract four main modal features of the user: visual, linguistic, behavioral, and textual.
[0029] The fusion emotion feature module is used to calculate the single-modal confidence based on the four main modal features, and combine them with historical statistical features to obtain dynamic stability weights, thereby obtaining fusion emotion features.
[0030] The time-based fusion emotion feature module is used to introduce time-recursive updates to update the fusion emotion features to time-based fusion emotion features.
[0031] The system adaptive threshold module is used to [address thresholds] within a time window. The system calculates historical mean and variance, and dynamically adjusts the adaptive threshold based on global fluctuations and feedback bias. A weighted risk ratio is defined based on the obtained adaptive threshold, and a state smoothing term is introduced. According to the maximum probability criterion, when continuous... The proportion of high-level warnings within seconds exceeded the warning threshold. When this occurs, adaptive feedback adjustment is triggered and the deviation is fed back.
[0032] The determination module is used to determine the user's emotional state based on time-based fused emotional features and an adjusted system adaptive threshold, thereby obtaining the determined user emotional state.
[0033] The output module is used to output the user's emotional state.
[0034] Another object of the present invention is to provide an electronic device, characterized in that it includes: at least one processor, at least one memory, and a communication interface. The processor, memory, and communication interface communicate with each other. The memory stores program instructions executable by the processor, which invokes the program instructions to execute the user emotion state determination method based on multi-source data fusion and adaptive thresholds.
[0035] Compared with the prior art, the present invention has the following advantages: 1. By constructing a multimodal fusion assessment system, subtle dynamics in children's emotions and attention can be keenly captured. Compared to traditional solutions relying on a single sensor, this system provides more accurate assessments of children's psychological states, thereby allowing subsequent interactive activities to better align with children's actual needs. 。
[0036] 2. By dynamically aggregating and analyzing multi-source data over time, the system enables global judgment and intervention decisions regarding users' emotional states. Furthermore, through adaptive fusion and closed-loop correction of multimodal data at the system level, the emotion judgment system acquires trend memory, feedback learning, and self-regulation characteristics, significantly improving the overall robustness and intelligence of the system. Attached Figure Description
[0037] Figure 1 The main flowchart of the user emotional state determination method based on multi-source data fusion and adaptive threshold is shown.
[0038] Figure 2 This is a flowchart of the data transmission process.
[0039] Figure 3 This is the system logic state diagram. Detailed Implementation
[0040] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these examples are for illustrative purposes only and are not intended to limit the scope of the invention. After reading this invention, any modifications of the invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.
[0041] Example 1 This embodiment provides a method for determining user emotional state using multi-source data fusion and adaptive thresholds, such as... Figure 1-3 As shown, it includes the following steps: Step 1: Collect and extract the user's four main modal features: visual, linguistic, behavioral, and textual.
[0042] The terminal incorporates sensors such as cameras and microphones to collect children's visual data (facial expressions), auditory data (audio events and voice-text), and tactile / motor data (touch operations). It then extracts four main modal features: visual, linguistic, behavioral, and textual. This multimodal input data is entered into the system for subsequent analysis.
[0043] Step 2: Calculate the confidence scores of each of the four main modal features, combine them with historical statistical features to obtain dynamic stability weights, and then obtain the fused sentiment features.
[0044] The confidence level for a single mode is:
[0045] in, The expression represents the confidence score of the m-th modality in unimodal emotion recognition at time t. This represents the activation function. The feature mapping function represents the m-th mode. This represents the raw data collected at time t in scenario r for the m-th modality. This indicates the type of scenario in which the emotion data was collected. Indicates the modal type identifier. Representing visual modality, Represents language / audio modality. Represents behavioral modality, Represents the text modality.
[0046] The dynamic stability weights are obtained by combining historical statistical characteristics (moving mean and variance):
[0047] in, This represents the dynamic stability weight of the m-th mode at time t. This represents the modal confidence sensitivity adjustment coefficient. This represents the modal stability weighting adjustment coefficient. The expression represents the confidence score of the m-th modality in unimodal emotion recognition at time t. This represents the sliding variance of the confidence level of the m-th mode at time t. This represents the modal traversal index.
[0048] Integrating emotional characteristics:
[0049] in, This represents the multimodal fusion emotion feature value at time t.
[0050] Step 3: To characterize the continuity and evolution trend of emotional states, a time-recursive update is introduced, which updates the fused emotional features into time-based fused emotional features.
[0051]
[0052] in, This indicates time-based fusion of emotional characteristics. Indicates the integration of emotional characteristics. This represents the weighting coefficient of the current fused emotional features. The weighting coefficients represent the historical time-based fusion of emotional characteristics. The weighting coefficients representing the rate of change in sentiment. ,satisfy The last item reflects the instantaneous acceleration of the score.
[0053] Step 4, within the time window The system calculates the historical mean and variance, and dynamically adjusts the threshold based on global fluctuations and feedback biases.
[0054]
[0055] in, This represents the historical mean of the time-based fused sentiment features within time window B. This indicates the size of the time window used for historical statistics. This represents the time-based fused emotional features at time ti. This represents the historical standard deviation of time-based fused sentiment features within time window B.
[0056] The system's adaptive threshold is dynamically adjusted based on global fluctuations and feedback biases.
[0057] in, This represents the adaptive emotion warning threshold of the system at time t. This represents the historical mean of the time-based fused sentiment features within time window B. This represents the historical standard deviation of the time-based fused sentiment features within time window B. This represents the time-based fused emotional characteristics at time t. This represents the adjustment coefficient of historical fluctuations on the threshold. This represents the adjustment coefficient of the instantaneous rate of change to the threshold. This represents the adjustment mechanism of the feedback error on the threshold. This indicates the system feedback error term, which can be calculated from the "false alarm rate" or "intervention effectiveness rate" reported by the terminal:
[0058] in, This represents the system feedback error term. This represents the historical feedback error mean at time t. The adjustment coefficient represents the average historical feedback error. The adjustment coefficient representing the rate of change of feedback error. This indicates the size of the time window for calculating the feedback error. This represents the time-based fused emotional features at time ti. The system adaptive threshold is represented at time ti.
[0059] This design enables the system to perform closed-loop correction of feedback errors, giving the threshold the characteristics of memory and self-learning.
[0060] Step 5: Define the weighted risk ratio based on the obtained threshold and introduce a state smoothing term.
[0061] To ensure the time stability and robustness of the judgment, a weighted risk ratio is defined:
[0062] in, This represents the weighted risk ratio of emotional state at time t. This represents the time-based fused emotional characteristics at time t. This represents the adaptive emotion warning threshold of the system at time t. This represents the historical variance of the time-based fused sentiment features within time window B. This represents the minimum constant.
[0063] And introduce a state smoothing term:
[0064] in, This represents the probability that the user's emotional state level is k at time t. This represents the user's emotional state level at time t. A value indicating the level of emotional state. This represents the risk ratio moderating coefficient for the emotional state level k at time t. This represents the state continuity coefficient of emotional state level k at time t. This represents the weighted risk ratio of emotional state at time t. This represents the risk ratio moderating coefficient for emotional state level j at time t. This represents the state continuity coefficient of emotional state level j at time t. Indicates the level of emotional state. This is the smoothing coefficient.
[0065] Step 6, according to the maximum probability criterion, when continuous The proportion of high-level warnings within seconds exceeded the warning threshold. When this occurs, adaptive feedback adjustment is triggered to obtain the adjusted system adaptive threshold.
[0066] Maximum probability criterion:
[0067] Adaptive feedback adjustment:
[0068] in, This represents the modal confidence sensitivity adjustment coefficient. This indicates the parameter assignment / update symbol. This represents the updated modal confidence sensitivity adjustment coefficient. This represents the feedback adjustment step size of α. This represents the weighted risk ratio of emotional state at time t. This represents the modal stability weighting adjustment coefficient. This represents the updated modal stability weight adjustment coefficient. This represents the feedback adjustment step size of λ. This represents the adjustment coefficient of the threshold based on historical fluctuations.
[0069] Step 7: Determine the user's emotional state based on the time-based fused emotional features and the adjusted system adaptive threshold to obtain the determined user emotional state.
[0070]
[0071] in, Indicates the time-varying global threshold. Indicates the sensitivity bandwidth. This represents the proportion above the threshold. Indicates the duration of continuous exceedance. As a time threshold, For frequency threshold, This is the time threshold. If continuous detection satisfies the three-level conditions in the above formula, the system triggers a higher-level intervention process.
[0072] The algorithm enables a self-evolutionary process of system parameters. It achieves adaptive fusion and closed-loop correction of multimodal data at the system level, giving emotion judgment the characteristics of trend memory, feedback learning, and self-regulation, significantly improving the overall robustness and intelligence level of the system.
[0073] Another embodiment of the present invention provides an electronic device, including: at least one processor, at least one memory, and a communication interface. The processor, memory, and communication interface communicate with each other. The memory stores program instructions executable by the processor, which invokes the program instructions to execute the user emotion state determination method based on multi-source data fusion and adaptive threshold.
[0074] Another embodiment of the present invention provides a user emotion state determination system based on multi-source data fusion and adaptive threshold, comprising a data acquisition and extraction module, an emotion feature fusion module, a time-based emotion feature fusion module, a system adaptive threshold module, a determination module, and an output module, wherein: The acquisition and extraction module is used to collect and extract four main modal features of the user: visual, linguistic, behavioral, and textual. The terminal continuously acquires multimodal data (visual local facial expression key points, audio events and transcriptions, tactile / behavioral events, ranging, etc.), performing feature extraction and preliminary judgment locally. To reduce privacy risks and bandwidth, event summaries and feature vectors are uploaded in most cases; sensitive content is de-identified and encrypted before uploading, and recorded in the archive along with the event summary. Each modality is independently encoded to generate representations (visual: local key points / micro-expression features; audio: event probability / fundamental frequency / emotion vector, etc.; text: STT + emotion word / sensitive word score; tactile / ranging: behavioral trigger identifier). Multimodal data consistency rule: when the weighted confidence of at least one modality exceeds a threshold and is persistent over time, it is considered consistent.
[0075] The fusion emotion feature module is used to calculate the single-modal confidence based on the four main modal features, and combine them with historical statistical features to obtain dynamic stability weights, thereby obtaining fusion emotion features.
[0076] The time-based fusion emotion feature module is used to introduce time-recursive updates to update the fusion emotion features to time-based fusion emotion features.
[0077] The system adaptive threshold module is used to [address thresholds] within a time window. The system calculates historical mean and variance, and dynamically adjusts the adaptive threshold based on global fluctuations and feedback bias. A weighted risk ratio is defined based on the obtained adaptive threshold, and a state smoothing term is introduced. According to the maximum probability criterion, when continuous... The proportion of high-level warnings within seconds exceeded the warning threshold. When this occurs, adaptive feedback adjustment is triggered and the deviation is fed back.
[0078] The determination module is used to determine the user's emotional state based on time-based fused emotional features and an adjusted system adaptive threshold, thereby obtaining the determined user emotional state.
[0079] The output module is used to output the user's emotional state.
[0080] The real-time interaction and tiered early warning module relies on a "know-the-person, time-appropriate" data tiering mechanism, enabling real-time detection and assessment of children's emotional states with immediate feedback. When the assessment results fall into a preset danger threshold, corresponding warning and interaction strategies are triggered: For general emotional state changes (Level 0 or 1), the system initiates personalized, non-emergency proactive interactions such as voice to help children achieve emotional self-regulation; for emergency warnings (Level 2), the system automatically generates prompts and pushes alerts and suggestions to guardians via mobile apps and SMS, simultaneously initiating emergency proactive interaction; if a crisis warning (Level 3) is triggered, the system automatically generates alerts based on sensitive words, sends multiple alarms to guardians and public institutions via mobile apps and SMS, and switches to crisis proactive interaction mode. Simultaneously, the module tracks the intervention effects of each warning level in real time. If the issue remains unresolved within the specified time, the warning is automatically escalated, ensuring that information reaches parents, schools, community governments, and other relevant stakeholders accurately and at the appropriate time. This achieves efficient tiered response while effectively preventing ethical and discrimination risks. 。
[0081] ① Hierarchical intervention library: 1) Real-time scripts for terminals: short voice reassurance, breathing light guidance, and real-time interactive attention diversion scripts; 2) Parent-side script: Personalized communication scripts, family operation suggestions, and observation and record checklist (Level 2 intervention); 3) Expert Referral Package: A structured event package (including event summary, modality weights, timeline, parent confirmation information, etc.) for remote expert assessment and recommendations (Level 3 intervention).
[0082] ② Expert intervention channel: When the system triggers level three or when parents request expert advice, the system automatically packages the event and delivers it to the psychological expert team according to the authorization strategy; the expert submits suggestions on their end, and the system sends the suggestions to the parent's end and records them in the file.
[0083] This invention, by constructing a multimodal fusion assessment system, can keenly capture subtle dynamics in children's emotions and attention. Compared to traditional solutions that rely on a single sensor, this system is more accurate in judging children's psychological states, thus allowing subsequent interactions to better align with children's actual needs. For example, the system can not only recognize the voiceprint signals of a child crying but also simultaneously perceive their facial expressions of sadness. Through comprehensive analysis of these two types of information, it accurately determines whether reassurance intervention is necessary.
[0084] A division of labor model of "local collection, cloud analysis, and encrypted transmission" is established. Locally, multimodal data is identified and collected during interaction. After analysis in the cloud, the data is encrypted and transmitted through specific rule encoding and security protocols, which effectively reduces the risk of privacy leakage and meets data security compliance requirements.
[0085] Under the early warning level classification mechanism, different intervention and reporting paths are matched to achieve hierarchical allocation of technical resources; support for manual review and strategy fine-tuning mechanisms is provided to ensure dynamic optimization of intervention plans in changing scenarios.
[0086] Parents can view complete individual data and intervention records on the web / app platform and can proactively issue remote reassurance instructions; public institutions can access permission-based summary information and statistical analysis, support regional risk monitoring and resource allocation, and achieve a collaborative closed loop between families, institutions, and public health.
[0087] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A method for determining user emotional state using multi-source data fusion and adaptive thresholding, characterized in that, Includes the following steps: Step 1: Collect and extract the user's four main modal features: visual, linguistic, behavioral, and textual. Step 2: Calculate the confidence scores of each of the four main modal features, combine them with historical statistical features to obtain dynamic stability weights, and then obtain the fused sentiment features. Step 3: Introduce time-recursive update to update the fused sentiment features to time-based fused sentiment features; Step 4, within the time window The system calculates historical mean and variance, and dynamically adjusts the adaptive threshold based on global fluctuations and feedback biases. Step 5: Define the weighted risk ratio based on the obtained system adaptive threshold and introduce a state smoothing term; Step 6, according to the maximum probability criterion, when continuous The proportion of high-level warnings within seconds exceeded the warning threshold. When this occurs, adaptive feedback adjustment is triggered and the deviation is fed back. Step 7: Determine the user's emotional state based on the time-based fused emotional features and the adjusted system adaptive threshold to obtain the determined user emotional state.
2. The user emotional state determination method based on multi-source data fusion and adaptive threshold as described in claim 1, characterized in that: The confidence level for a single mode is: in, The expression represents the confidence score of the m-th modality in unimodal emotion recognition at time t. This represents the activation function. The feature mapping function represents the m-th mode. This represents the raw data collected at time t in scenario r for the m-th modality. This indicates the type of scenario in which the emotion data was collected. Indicates the modal type identifier. Representing visual modality, Represents language / audio modality. Represents behavioral modality, Represents text modality; Dynamic stability weights: in, This represents the dynamic stability weight of the m-th mode at time t. This represents the modal confidence sensitivity adjustment coefficient. This represents the modal stability weighting adjustment coefficient. The expression represents the confidence score of the m-th modality in unimodal emotion recognition at time t. This represents the sliding variance of the confidence level of the m-th mode at time t. This represents the modal traversal index.
3. The user emotional state determination method based on multi-source data fusion and adaptive threshold as described in claim 2, characterized in that: Integrating emotional characteristics: in, This represents the multimodal fusion emotion feature value at time t.
4. The user emotional state determination method based on multi-source data fusion and adaptive threshold as described in claim 3, characterized in that: The formula for calculating time-based fusion emotion features is as follows: in, This indicates time-based fusion of emotional characteristics. Indicates the integration of emotional characteristics. This represents the weighting coefficient of the current fused emotional features. The weighting coefficients represent the historical time-based fusion of emotional characteristics. The weighting coefficients representing the rate of change in sentiment. ,satisfy .
5. The user emotional state determination method based on multi-source data fusion and adaptive threshold as described in claim 4, characterized in that: Time window The historical mean and variance were calculated as follows: in, This represents the historical mean of the time-based fused sentiment features within time window B. This indicates the size of the time window used for historical statistics. This represents the time-based fused emotional features at time ti. This represents the historical standard deviation of time-based fused sentiment features within time window B.
6. The user emotional state determination method based on multi-source data fusion and adaptive threshold as described in claim 5, characterized in that: The system's adaptive threshold is dynamically adjusted based on global fluctuations and feedback biases. in, This represents the adaptive emotion warning threshold of the system at time t. This represents the historical mean of the time-based fused sentiment features within time window B. This represents the historical standard deviation of the time-based fused sentiment features within time window B. This represents the time-based fused emotional characteristics at time t. This represents the adjustment coefficient of historical fluctuations on the threshold. This represents the adjustment coefficient of the instantaneous rate of change to the threshold. This represents the adjustment mechanism of the feedback error on the threshold. This represents the system feedback error term; in, This represents the system feedback error term. This represents the historical feedback error mean at time t. The adjustment coefficient represents the average historical feedback error. The adjustment coefficient representing the rate of change of feedback error. This indicates the size of the time window for calculating the feedback error. This represents the time-based fused emotional features at time ti. The system adaptive threshold is represented at time ti.
7. The user emotional state determination method based on multi-source data fusion and adaptive threshold as described in claim 6, characterized in that: Weighted risk ratio: in, This represents the weighted risk ratio of emotional state at time t. This represents the time-based fused emotional characteristics at time t. This represents the adaptive emotion warning threshold of the system at time t. This represents the historical variance of the time-based fused sentiment features within time window B. Represents the minimum constant; State smoothing term: in, This represents the probability that the user's emotional state level is k at time t. This represents the user's emotional state level at time t. A value indicating the level of emotional state. This represents the risk ratio moderating coefficient for the emotional state level k at time t. This represents the state continuity coefficient of emotional state level k at time t. This represents the weighted risk ratio of emotional state at time t. This represents the risk ratio moderating coefficient for emotional state level j at time t. This represents the state continuity coefficient of emotional state level j at time t. Indicates the level of emotional state. This is the smoothing coefficient.
8. The user emotional state determination method based on multi-source data fusion and adaptive threshold as described in claim 7, characterized in that: The adaptive feedback adjustment method is as follows: in, This represents the modal confidence sensitivity adjustment coefficient. This indicates the parameter assignment / update symbol. This represents the updated modal confidence sensitivity adjustment coefficient. This represents the feedback adjustment step size of α. This represents the weighted risk ratio of emotional state at time t. This represents the modal stability weighting adjustment coefficient. This represents the updated modal stability weight adjustment coefficient. This represents the feedback adjustment step size of λ. This represents the adjustment coefficient of the threshold based on historical fluctuations.
9. A determination system for user emotional state determination based on the multi-source data fusion and adaptive threshold method described in claim 1, characterized in that: It includes a data acquisition and extraction module, a sentiment feature fusion module, a time-based sentiment feature fusion module, a system adaptive threshold module, a decision module, and an output module, wherein: The acquisition and extraction module is used to acquire and extract four main modal features of the user: visual, linguistic, behavioral, and textual. The fusion emotion feature module is used to calculate the single-modal confidence based on the four main modal features, combine them with historical statistical features to obtain dynamic stability weights, and then obtain the fusion emotion features. The time-based fusion emotion feature module is used to introduce time recursive updates to update the fusion emotion features to time-based fusion emotion features. The system adaptive threshold module is used to [address thresholds] within a time window. The system calculates historical mean and variance, and dynamically adjusts the adaptive threshold based on global fluctuations and feedback biases. A weighted risk ratio is defined based on the obtained adaptive threshold, and a state smoothing term is introduced. According to the maximum probability criterion, when continuous... The proportion of high-level warnings within seconds exceeded the warning threshold. When this occurs, adaptive feedback adjustment is triggered and the deviation is fed back. The determination module is used to determine the user's emotional state based on time-based fused emotional features and an adjusted system adaptive threshold, thereby obtaining the determined user emotional state. The output module is used to output the user's emotional state.
10. An electronic device, characterized in that, include: At least one processor, at least one memory, and a communication interface; The processor, memory, and communication interface communicate with each other; The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the user emotion state determination method based on multi-source data fusion and adaptive threshold as described in any one of claims 1-8.