Multi-modal emotion recognition psychological counseling system

Abstract

The invention relates to the technical field of emotion recognition, and discloses a multi-mode emotion recognition psychological counseling system. The system comprises a data receiving module for receiving real-time emotion data of a multi-mode sensor and comparing the real-time emotion data with historical emotion data to obtain periodic components and non-periodic components; the mode recognition module recognizes an unexpected emotion fluctuation mode in the emotion data according to the components; a key factor extraction module performs causal relationship and co-occurrence mode discovery on the unexpected fluctuation mode to obtain key emotion factors; the benchmark construction module is used for constructing a benchmark emotion expression curve by taking the key emotion factors as input and combining same-type emotion data among individuals; the risk assessment module assesses the individual emotional state based on the deviation degree of the reference curve and the actual curve in combination with a fuzzy assessment system, and determines a potential psychological risk level; and the prediction module predicts emotion crisis individuals in a preset time period according to the unexpected fluctuation mode and the key emotion factors, and generates a psychological counseling suggestion report.

Description

Technical Field

[0001] This invention relates to the field of emotion recognition technology, specifically a multimodal emotion recognition psychological counseling system. Background Technology

[0002] In modern society, the complexity of individual emotional states is becoming increasingly prominent, making the early detection and intervention of psychological problems an important issue. Traditional emotion recognition methods often rely on single-modal data, such as voice, facial expressions, or physiological signals, which makes it difficult to comprehensively capture the dynamic changes in emotions. Single-modal data often has limitations; for example, voice signals are easily interfered with by environmental noise, facial expressions may contain faking elements, and the invasiveness of physiological signal acquisition equipment may affect the naturalness of the data.

[0003] Sentiment analysis often focuses on extracting instantaneous features from real-time data, lacking longitudinal comparative analysis of historical data. This makes it difficult to distinguish between periodic and non-periodic fluctuations in sentiment data. For example, an individual's mood changes within a specific time period may exhibit regular cycles, such as the difference in mood between weekdays and weekends. However, non-periodic fluctuations caused by sudden events, if not effectively identified, can easily be misjudged as regular mood changes, delaying the discovery of potential psychological risks.

[0004] Traditional pattern recognition methods lack sufficient accuracy in identifying unexpected emotional fluctuations. Because emotional fluctuations are influenced by multiple factors, including physiological state, environmental changes, and social interactions, the interplay of these factors makes it difficult to accurately capture unexpected fluctuation patterns. The extraction of key emotional factors often relies on human experience or simple statistical analysis, lacking in-depth exploration of causal relationships and co-occurrence patterns, thus failing to identify the core factors influencing emotional changes.

[0005] In constructing baseline emotional expression curves, existing technologies are mostly based on individual historical data, without considering the reference value of similar emotional data among individuals. Different individuals exhibit common characteristics in their emotional responses in similar situations. A baseline curve lacking support from such common data has limited reference value and is difficult to accurately measure the degree of deviation from an individual's actual emotional expression.

[0006] In the risk assessment process, traditional methods often use fixed thresholds to determine risk levels, ignoring the ambiguity and continuity of emotional states. The change in emotion from normal to abnormal is a gradual process; rigid threshold divisions can easily lead to distorted risk assessment results, either causing over-warning or underestimating potential risks. Furthermore, existing systems lack a systematic approach to generating emotional crisis prediction and guidance suggestions. The prediction models have limited input dimensions, and the suggestions are mostly general guidelines, failing to meet individualized needs. These problems collectively result in the limited effectiveness of traditional emotion recognition and guidance systems in practical applications, failing to provide effective technical support for psychological counseling. Summary of the Invention

[0007] The purpose of this invention is to provide a multimodal emotion recognition and psychological counseling system to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides a multimodal emotion recognition and psychological counseling system, the system comprising: A data receiving module is used to receive real-time emotion data from a multimodal sensor, compare the real-time emotion data with historical emotion data, and obtain the periodic and non-periodic components in the emotion data. The pattern recognition module is used to identify unexpected emotional fluctuation patterns in emotional data based on the periodic components and the non-periodic components. The key factor extraction module performs causal relationship and co-occurrence pattern discovery processing on the unexpected emotional fluctuation patterns to obtain key emotional factors; The benchmark construction module is used to construct a benchmark emotional performance curve by using the key emotional factors as input and combining similar emotional data among individuals. The risk assessment module is used to assess an individual's emotional state and determine the level of potential psychological risk based on the degree of deviation between the benchmark emotional expression curve and the actual emotional expression curve, combined with a fuzzy assessment system. The prediction module is used to predict individuals who will experience an emotional crisis within a preset time period based on the unexpected emotional fluctuation pattern, the key emotional factors, and the potential psychological risk level, and to generate a psychological counseling suggestion report.

[0009] Preferably, the data receiving module includes: The multidimensional data acquisition unit is used to simultaneously collect multidimensional emotional parameter data of individuals in different emotional states and construct an emotional feature dataset. The component extraction unit is used to perform time-frequency analysis on the emotional feature dataset, extract frequency domain feature vectors, and separate periodic and non-periodic components based on the frequency domain feature vectors.

[0010] Preferably, the pattern recognition module includes: The scene determination unit is used to determine an emotional scene including all emotional elements based on the emotional feature dataset and a preset emotional scene knowledge graph. The region division unit is used to divide the emotional data into large-scale emotional information regions and small-scale emotional information regions according to the emotional scene and the corresponding specification requirements. The fluctuation recognition unit is used to identify unexpected emotional fluctuation patterns based on the large-scale emotional information region and the small-scale emotional information region using a pattern classification algorithm.

[0011] Preferably, the key factor extraction module includes: The data optimization unit is used to construct an emotional fluctuation dataset using the unexpected emotional fluctuation pattern, and to apply a dimensionality reduction algorithm to perform dimensionality reduction processing on the emotional fluctuation dataset to obtain an optimized dataset. The relationship mining unit is used to scan the optimized dataset, identify frequent itemsets, detect the causal relationships and co-occurrence patterns of the frequent itemsets, and generate key sentiment factors.

[0012] Preferably, the benchmark building module includes: The data smoothing unit is used to use the key emotional factors as input, combine similar emotional data between individuals, and smooth the multi-source historical emotional data to obtain smoothed data. The curve fitting unit is used to fit the smoothed data using a regression model to generate a baseline sentiment expression curve.

[0013] Preferably, the risk assessment module includes: The deviation calculation unit is used to calculate the degree of deviation between the actual emotional expression curve and the benchmark emotional expression curve by comparing the actual emotional expression curve with the benchmark emotional expression curve. A scoring generation unit is used to quantify the degree of deviation using an error quantification algorithm to obtain a deviation score. A health assessment unit is used to score the degree of deviation and obtain a health status assessment result; The risk determination unit is used to use the health status assessment results, combined with a time series prediction model, to predict the trend of individual emotional changes within a preset time period, and analyze the prediction results to determine the potential psychological risk level.

[0014] Preferably, the prediction module includes: The crisis prediction unit is used to input the unexpected emotional fluctuation pattern, the key emotional factors, and the potential psychological risk level into the network model to predict emotional crisis events. The report generation unit is used to generate psychological counseling suggestion reports based on predicted emotional crisis events.

[0015] Preferably, the fluctuation recognition unit includes: The sub-data partitioning unit is used to split the emotional data into large-scale sub-data and small-scale sub-data according to the large-scale emotional information region and the small-scale emotional information region; The large-scale analysis unit is used to input large-scale sub-data into the first deep learning model for analysis. The small-scale analysis unit is used to input small-scale sub-data into the second deep learning model for analysis. The pattern fusion unit is used to combine the outputs of the first deep learning model and the second deep learning model to identify unexpected emotional fluctuation patterns.

[0016] Preferably, the mode fusion unit includes: The feedback mechanism unit is used to establish a real-time feedback mechanism between the first deep learning model and the second deep learning model to transmit preliminary analysis results and location information. The fine analysis unit is used to perform detailed hidden danger identification processing on small-scale sub-data based on the preliminary analysis results and location information, and to detect specific emotional abnormal information; The results optimization unit is used to adjust the analysis parameters based on the comparison between multiple analysis results and actual sentiment data, and output the final sentiment anomaly pattern.

[0017] Preferably, the fine analysis unit includes: The region stripping unit is used to calculate the texture complexity and grayscale change rate indices of local regions based on large-scale sub-data. When the indices exceed the preset threshold, the region with potential hidden dangers in detail is identified. The sub-data extraction unit is used to calculate stripping parameters and extract small-scale sub-data from large-scale sub-data to ensure complete coverage of the potential hazard area. The feature analysis unit is used to perform subtle feature recognition processing on small-scale sub-data and output detailed information on emotional anomalies.

[0018] Compared with the prior art, the beneficial effects of the present invention are: This multimodal emotion recognition and psychological counseling system achieves comprehensive optimization of emotion recognition and counseling through the coordinated operation of its various modules. The data receiving module compares and analyzes real-time and historical emotion data from multimodal sensors, accurately distinguishing between periodic and non-periodic components. This process can separate regular patterns and abnormal fluctuations in emotion changes. Compared to traditional methods that rely solely on real-time data, this processing mode that combines historical data can more comprehensively grasp the overall context of emotion changes and avoid misjudgments caused by the limitations of data from a single time point.

[0019] The pattern recognition module identifies unexpected emotional fluctuation patterns based on periodic and aperiodic components. By focusing on emotional changes that deviate from normal patterns, it effectively filters out emotional fluctuations within the normal range, concentrating the analysis on truly noteworthy abnormal fluctuations. This precise identification capability makes subsequent key factor extraction more targeted, reduces interference from irrelevant information, and makes it easier to capture factors that substantially affect emotional changes.

[0020] The key factor extraction module performs causal relationship and co-occurrence pattern discovery processing on unexpected fluctuation patterns, enabling it to identify the core factors influencing emotional states from complex emotional data. By deeply exploring the correlation patterns behind emotional fluctuations, it can clearly reveal which factors are directly or indirectly related to abnormal emotional changes. This process overcomes the limitations of traditional reliance on experience-based judgment, making the identification of key factors more objective and scientific. The benchmark construction module utilizes key emotional factors and combines them with similar emotional data among individuals to construct benchmark emotional performance curves, overcoming the limitations of relying solely on individual historical data to build benchmarks. By introducing common reference data among individuals, the benchmark curves not only reflect individual emotional characteristics but also incorporate the general patterns of similar groups, making the benchmark more representative and valuable for reference.

[0021] The risk assessment module, based on the deviation between the baseline curve and the actual curve, combines a fuzzy assessment system to evaluate an individual's emotional state, thus better reflecting the continuous and ambiguous characteristics of emotional changes. This assessment method avoids rigid judgments caused by fixed thresholds, and can flexibly classify potential psychological risk levels based on subtle differences in deviation, making the assessment results more consistent with the individual's true emotional state. The prediction module integrates unexpected fluctuation patterns, key emotional factors, and potential risk levels to predict emotional crises within a preset time period and generate a guidance and intervention report, achieving a closed-loop process from data collection and analysis to risk warning and intervention recommendations. Through the integration of multi-dimensional information, the prediction results are more forward-looking, and the generated guidance and intervention recommendations are more tailored to individual needs based on specific key factors and risk levels, thus improving the pertinence and effectiveness of psychological counseling. Attached Figure Description

[0022] Figure 1 This is a timing diagram of the multimodal emotion recognition and psychological counseling system described in this invention; Figure 2 This is a flowchart of the data receiving module. Figure 3 A flowchart illustrating the workflow of building modules based on a baseline; Figure 4 This is a flowchart of the prediction module's workflow. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Please see Figure 1This invention provides a multimodal emotion recognition and psychological counseling system, the system comprising: The system collects real-time emotional data using multimodal sensors, analyzes emotional fluctuation patterns by combining them with historical data, identifies key emotional factors, constructs a baseline emotional performance curve, and ultimately assesses the psychological risk level and generates guidance suggestions. The system includes a data receiving module, a pattern recognition module, a key factor extraction module, a baseline construction module, a risk assessment module, and a prediction module. The data receiving module collects multidimensional emotional parameter data and separates periodic and non-periodic components; the pattern recognition module divides emotional information regions and identifies unexpected fluctuation patterns; the key factor extraction module extracts key emotional factors through dimensionality reduction and correlation analysis; the baseline construction module fits the baseline emotional performance curve; the risk assessment module calculates the degree of deviation and determines the psychological risk level; and the prediction module generates a psychological guidance suggestion report.

[0025] Example 1: See Figure 2 The data receiving module synchronously collects physiological signals, speech features, and facial expression data of individuals under different emotional states through a multi-dimensional data acquisition unit. Physiological signal acquisition utilizes wearable devices to monitor heart rate variability, skin conductance, and electroencephalogram (EEG) activity in real time. Heart rate variability data is recorded with millisecond-level precision using a chest strap sensor. Skin conductance is measured using fingertip electrodes to measure conductance changes caused by sweat gland secretion. EEG activity is collected using a portable EEG cap to capture the energy distribution of alpha, beta, and theta waves in the prefrontal and temporal lobes. Speech feature acquisition uses a high-sensitivity microphone to record the acoustic characteristics of individuals in natural dialogue or specific speech tasks, including fundamental frequency trajectory, formant distribution, and short-term energy changes. Facial expression data is captured by a high-definition camera to capture the motion trajectories of key facial points, and computer vision algorithms are used to extract the amplitude and duration characteristics of movement in the eyebrow, corner of the eye, and corner of the mouth areas.

[0026] The collected multidimensional emotional parameter data were time-aligned and normalized to construct an emotional feature dataset. Time alignment employed a dynamic time warping algorithm to eliminate acquisition delays between different sensors, while normalization scaled the data across all dimensions to a uniform numerical range. The emotional feature dataset contains four types of structured data: timestamps, physiological indicators, speech feature vectors, and facial expression feature matrices. Each data point is labeled with a corresponding emotional state tag. These emotional state tags were manually annotated by psychology experts based on standard emotional scales for the collected scenarios, covering five basic emotional categories: calm, joy, anxiety, anger, and sadness.

[0027] The component extraction unit performs time-frequency analysis on the emotional feature dataset, using short-time Fourier transform to convert the time-domain signal into a joint time-frequency representation. Appropriate window function lengths and overlap rates are set during the analysis to achieve a balance between time and frequency resolution. Wavelet transform is used to extract multi-scale features of the signal, capturing transient change patterns in different frequency bands by selecting a mother wavelet function that matches the characteristics of emotional physiological responses. The frequency domain feature vector includes indicators such as the power spectral density distribution, harmonic component ratio, and frequency band energy ratio of each physiological signal. The spectral decomposition algorithm, based on singular spectrum analysis, decomposes the composite emotional signal into periodic and aperiodic components. The periodic components exhibit stable oscillation patterns, reflecting regular emotional changes such as circadian rhythms and environmental adaptation; the aperiodic components contain transient responses caused by sudden events, which are separated from the original signal using residual analysis.

[0028] The scene determination unit of the pattern recognition module loads a pre-defined emotional scene knowledge graph, which uses a graph database to store emotion-related concepts and their relationships. The construction process of the knowledge graph includes extracting entities such as emotional triggering events, typical response patterns, and regulatory strategies from psychological literature, and establishing causal chains and co-occurrence networks between concepts through semantic relationship mining. The matching process between the emotional feature dataset and the knowledge graph uses similarity calculation based on embedding vectors to map data features to the concept space of the graph to find the best matching scene. The determined emotional scene includes a descriptive framework encompassing three dimensions: environmental factors, social interaction characteristics, and individual physiological state, with several quantifiable observation indicators under each dimension.

[0029] Based on the spatiotemporal characteristics of emotional scenarios, the region segmentation unit divides the emotional data into analysis regions of different scales. Large-scale emotional information regions cover long-term trend changes measured in days, with an analysis window length set to 24 hours and updated using a sliding window mechanism. Small-scale emotional information regions focus on instantaneous fluctuations measured in minutes, with the analysis window length dynamically adjusted according to the physiological duration of specific emotional responses, ranging from 1 minute to 30 minutes. The segmentation process considers the lag effect and duration characteristics of emotional responses, appropriately expanding the boundaries of the analysis window for sudden emotional events.

[0030] The pattern classification algorithm for the fluctuation recognition unit employs an ensemble learning framework, combining the advantages of multiple base classifiers to improve recognition robustness. Classification of large-scale sentiment information regions uses a random forest algorithm, constructing multiple decision trees to vote on long-term sentiment trends. Feature selection for each decision tree focuses on the statistical characteristics of periodic components, including daily average fluctuation amplitude, period length, and phase stability. Classification of small-scale sentiment information regions uses a support vector machine algorithm, selecting radial basis functions as the kernel function to adapt to nonlinear sentiment fluctuation patterns. Feature space construction focuses on transient features of non-periodic components, including dynamic parameters such as rising slope, peak intensity, and decay time constant.

[0031] Pattern recognition results are used to generate a final determination of unexpected emotional fluctuation patterns through decision-level fusion. The fusion process considers the confidence weights of large-scale and small-scale analyses and introduces a time consistency verification mechanism for conflict determination results. Identified unexpected patterns include anomalous types such as amplitude fluctuations exceeding normal ranges, abnormal periodic characteristics, and reaction intensities that do not conform to the expected scenario. Each anomalous type is associated with a pattern descriptor, including metadata such as start time, duration, severity, and possible influencing factors. These pattern descriptors provide structured input for subsequent key factor extraction while preserving the traceability links of the original emotional data.

[0032] The emotional scene knowledge graph is continuously updated during operation. By analyzing the matching degree between newly identified unexpected patterns and existing knowledge, it automatically expands the branches of abnormal emotional patterns in the graph. The update mechanism adopts an incremental learning strategy, gradually incorporating validated new emotional response patterns while maintaining the stability of the core emotional theory framework. This dynamically evolving knowledge representation method enables the system to adapt to changes in emotional expression caused by different cultural backgrounds and individual differences, improving the generalization ability of pattern recognition.

[0033] Strict quality control measures are implemented during data acquisition, with real-time detection and handling of anomalies such as sensor failure, signal loss, and motion artifacts. The signal quality assessment module continuously monitors the signal-to-noise ratio and sampling integrity of each data stream, triggering an automatic re-acquisition mechanism for data segments that fail to meet quality standards. In cases where re-acquisition is not possible, a context-information-based interpolation method is used for data repair, ensuring that the spatiotemporal continuity of the sentiment feature dataset is not affected by single points of failure.

[0034] Cross-validation mechanisms among multidimensional data enhance the accuracy of sentiment state determination. When sentiment determination results from different modalities diverge, the system initiates a multimodal fusion algorithm for reassessment. The fusion algorithm considers factors such as measurement accuracy, historical consistency, and scene adaptability of each modality, and calculates the most probable sentiment state interpretation using a probabilistic graphical model.

[0035] Example 2: See Figure 3 The data optimization unit of the key factor extraction module receives unexpected emotional fluctuation pattern data from the pattern recognition module and constructs an emotional fluctuation dataset containing multi-dimensional features. This dataset integrates various indicators such as physiological signal variation features, speech spectrum change patterns, and facial expression trajectory parameters, with each data point associated with a timestamp and emotional scene label. Feature standardization is performed during data processing to eliminate the influence of different dimensions on the analysis results. The standardized data is then processed by a dimensionality reduction algorithm. Principal component analysis (PCA) transforms relevant features into linearly uncorrelated principal components through orthogonal transformation, retaining the main feature components according to their variance contribution rate. The t-SNE algorithm maintains the local structural relationships of the high-dimensional data in the low-dimensional space and iteratively optimizes to cluster similar emotional fluctuation patterns in the dimensionality-reduced space. Feature importance assessment is performed during dimensionality reduction, calculating the loading coefficients of each original feature on the principal components and selecting key indicators with strong explanatory power for emotional fluctuations. The optimized dataset retains the essential characteristics of emotional fluctuations while significantly reducing data dimensionality and computational complexity.

[0036] The relation mining unit performs association rule mining on the optimized dataset, using an improved Apriori algorithm to scan the entire dataset for frequently occurring feature combinations. The algorithm sets a minimum support threshold to filter low-frequency noise patterns and evaluates the rule strength between feature combinations through confidence calculation. A hierarchical search strategy is implemented during frequent itemset generation, gradually expanding from single-feature itemsets to multi-feature combinations, effectively controlling computational growth. Causal relationship analysis employs Bayesian network modeling to construct a directed acyclic graph structure between sentiment features. The network learning process combines rating search and conditional independence testing to determine the dependencies and conditional probability distributions between feature nodes. Key sentiment factor extraction comprehensively considers the support of association rules and the confidence of causal relationships, selecting feature combinations that are stable across multiple analytical dimensions. Key extracted factors include specific physiological indicator co-variation patterns, cross-modal association features between voice and facial expressions, and temporal patterns of emotional responses.

[0037] The data smoothing unit of the baseline construction module receives key emotional factors and similar emotional data from the same group as input, and performs multi-source data fusion processing. The group data comes from a set of individuals with similar demographic characteristics and lifestyle backgrounds, providing a reference distribution range for emotional expression. The smoothing process uses a weighted moving average method, assigning appropriate weights to emotional fluctuations at different time scales, preserving trend changes while suppressing random fluctuations. The Kalman filter algorithm is applied to the smoothing of non-stationary emotional data, estimating the true trajectory of emotional changes through a state-space model. The filtering process dynamically adjusts the covariance matrix of prediction and measurement errors to adapt to the data characteristics under different signal-to-noise ratio conditions. After smoothing, the data maintains the continuity and trend of emotional changes, eliminating abnormal fluctuations caused by measurement noise and transient interference.

[0038] The curve fitting unit selects an appropriate regression model to model and analyze the smoothed data. The Gaussian process regression method uses a kernel function to define a similarity measure between data points, constructing a probability distribution model of emotional changes. Kernel function selection considers the temporal correlation and nonlinearity of emotional data, combining periodic kernels and radial basis kernels to capture fluctuation patterns at different scales. The piecewise linear fitting method automatically segments the data at emotional state transition points, establishing an optimal linear model for each segment. Goodness-of-fit evaluation uses cross-validation, dividing the dataset into training and validation sets for repeated validation. The generated baseline emotional performance curve includes a mean curve and confidence interval boundaries, reflecting the fluctuation range and trend of healthy emotional states under normal conditions. The curve update mechanism periodically incorporates new emotional data for refitting, enabling the baseline performance to adapt to gradual changes in individual emotional characteristics.

[0039] The validation process for key sentiment factors employs a retrospective analysis approach, applying the extracted factors to historical sentiment data to test their explanatory power. Validation indicators include quantitative standards such as the matching degree of sentiment fluctuation patterns and the detection rate of anomalous events. The importance ranking of factors is dynamically adjusted based on the validation results, and factors with declining explanatory power are eliminated and updated. Personalized adaptation of the baseline curve considers the differences in individual sentiment response baselines, adjusting model parameters to better match the baseline range with individual characteristics. The adaptation process uses a stepwise approximation strategy to achieve personalized adjustments while maintaining model stability.

[0040] Multiple quality control measures are implemented during data optimization and benchmark construction. The dimensionality reduction effect is evaluated through reconstruction error analysis to check the degree to which the low-dimensional representation restores the original data. Association rule mining uses reasonable interest metrics to avoid generating a large number of meaningless and trivial rules. Residual analysis of curve fitting checks the rationality of model assumptions, and local refitting is performed on data segments that do not conform to the assumptions. A comprehensive logging and error tracing mechanism is established throughout the entire processing flow to ensure the auditability and repeatability of each processing step.

[0041] The system continuously monitors the timeliness of key factors during operation and establishes an evaluation mechanism for the validity period of these factors. Factors whose explanatory power declines over time are automatically labeled and replaced to maintain the adaptability of the analytical model. The baseline curve update strategy considers the influence of external variables such as seasonality and environmental factors, triggering systematic recalibration at specific time points. This dynamic maintenance mechanism enables the system to maintain accurate emotional state assessment capabilities over the long term, adapting to the natural evolution of individual emotional characteristics.

[0042] The integration and processing of multi-source data employs a unified spatiotemporal reference framework to ensure the comparability of sentiment data from different sources. The data alignment process considers differences in acquisition equipment and environmental conditions, implementing necessary corrections and compensations. Strict quality standards are established for screening group data, excluding individual records with incomplete data or significant biases. The generation of the baseline curve involves multi-level cross-validation to check the curve's applicability and stability across different subgroups. These measures ensure that the final generated baseline sentiment performance curve has reliable reference value.

[0043] The visualization of sentiment data employs a multi-layered design, simultaneously showcasing the comparison between raw fluctuations, smoothed trends, and baseline ranges. The visualization tools support interactive exploration, allowing users to focus on specific time periods or sentiment dimensions for detailed analysis. Dynamic parameter adjustment helps users understand sentiment change patterns from different perspectives, enhancing the interpretability of the system output. The visualization interface integrates anomaly pattern annotation, enabling users to quickly locate the occurrence time and characteristic manifestations of key sentiment events.

[0044] Example 3: See Figure 4 The deviation calculation unit of the risk assessment module receives the baseline emotional performance curve and the actual collected emotional data stream from the baseline construction module. The baseline curve contains three core trajectories: a median line reflecting the average level of the group, a warning line representing the upper limit of the normal fluctuation range, and a baseline describing the lower limit. The actual emotional performance curve is continuously updated through a real-time data acquisition system, with the sampling interval consistent with the temporal resolution of the baseline curve. The deviation degree is calculated using a dynamic time warping algorithm, which finds the optimal matching path between two curves by constructing a cumulative distance matrix. Appropriate local path constraints are set in the algorithm implementation to ensure the temporal rationality of the matching results. The quantification of the deviation degree includes two dimensions: amplitude difference and temporal asynchrony. Amplitude difference reflects the anomaly in the intensity of emotional response, while temporal asynchrony reflects the deviation in the speed of emotional response. During the calculation process, sudden emotional events are specially handled, and their starting point and duration are identified to assess the overall impact.

[0045] The scoring generation unit uses composite indicators to quantify the degree of deviation, with the time axis deviation score calculated as follows:

[0046] in: Indicates time deviation from the score. The total number of sampling points within the analysis period. and These represent the actual curve and the reference curve at the th... The time position of each feature point This serves as a time window adjustment coefficient. The magnitude deviation score is calculated using standardized Euclidean distance, assigning differentiated weighting coefficients to different emotional dimensions. The comprehensive deviation score combines the time score and the magnitude score, generating an overall assessment value through weighted summation. The scoring scale design employs a percentage-based grading system, setting multiple threshold intervals corresponding to different levels of deviation severity.

[0047] The health assessment unit establishes a mapping relationship between deviation scores and health status, constructed using the knowledge of clinical psychology experts. The mapping process considers the differences in affective dimensions, setting stricter judgment criteria for core affective indicators. Assessment results are divided into four levels, each associated with color coding and textual description. Fuzzy logic is used to handle level boundaries, implementing smooth transitions in critical regions to avoid abrupt cuts. The assessment process employs multidimensional cross-validation; a review mechanism is activated when judgments across different affective dimensions conflict. The review algorithm analyzes the historical consistency and environmental relevance of conflicting dimensions to generate the most probable comprehensive judgment.

[0048] The long short-term memory network model for risk assessment employs a three-layer hidden layer structure. The input layer receives historical deviation rating sequences and environmental context features. Network training utilizes a unit structure with forgetting gates to effectively capture long-term dependencies. The model output includes predicted emotional states and their probability distributions for the next three time units. The prediction results are fused with real-time health status assessments, and the confidence level of the risk is calculated using evidence-based algorithms. Risk level classification considers the persistence and acceleration of deviation trends, implementing risk escalation for rapidly deteriorating emotional states. The confidence assessment of the prediction results is achieved through Monte Carlo sampling, quantifying the uncertainty range of the model output.

[0049] The crisis prediction unit of the prediction module constructs a graph neural network model, where nodes represent emotional characteristics and external environmental factors, and edges represent known causal relationships and statistical correlations. A network propagation algorithm integrates node attributes and topological structure to calculate the trigger probability of emotional crisis events. Model input includes feature vectors of unexpected emotional fluctuation patterns, weight distributions of key emotional factors, and temporal changes in risk levels. The dynamic update mechanism of the graph structure allows for the real-time incorporation of newly discovered emotional correlation rules. The crisis prediction output includes descriptions in three dimensions: possible event types, occurrence time windows, and expected severity.

[0050] The psychological counseling suggestion library in the report generation unit adopts a hierarchical organizational structure. The top level is categorized by crisis type, and the lower levels are further subdivided by intervention intensity and applicable scenarios. Suggestion content generation combines template filling with natural language generation, maintaining professional consistency in the basic framework while dynamically adjusting detailed descriptions based on individual case characteristics. The report structure includes four standard parts: a current situation overview, risk analysis, professional recommendations, and resource links. The level of detail in each part is adapted to the recipient's professional background. Report output supports multiple format conversions, including structured documents, visual charts, and audio summaries.

[0051] The deviation calculation unit employs a multi-scale analysis strategy, simultaneously performing instantaneous deviation detection and trend deviation assessment. Instantaneous analysis utilizes a sliding window mechanism, with the window length dynamically adjusted based on the physiological characteristics of emotional responses. Trend analysis identifies the long-term trajectory of emotional changes through piecewise linear regression, comparing it to the expected trend of the baseline curve. The description of deviation characteristics includes directional indicators, distinguishing the psychological significance of positive and negative deviations. Real-time quality control is implemented throughout the calculation process, initiating data repair procedures for missing or abnormal signals.

[0052] The weighting scheme for the scoring generation unit was determined using the Delphi method, integrating domain knowledge from multiple psychology experts. The weighting system design considered the clinical importance and measurement reliability of the affective dimension, assigning higher weights to fundamental indicators. Parameter tuning of the scoring algorithm used historical labeled data, employing a grid search to find the optimal parameter combination. A calibration mechanism for the scoring output regularly checked the consistency between the score distribution and actual conditions, implementing gradual adjustments as necessary. The health assessment unit established a traceability mechanism for assessment results, linking each judgment to specific data characteristics and calculation processes supporting that conclusion. Traceability information was stored in a structured format to support the verification and validation of assessment conclusions. The assessment model was updated using an online learning strategy, gradually incorporating new clinical practice evidence while maintaining the stability of the core logic. Environmental adaptation adjustments to the assessment process considered the influence of external factors such as seasonal changes and social events, dynamically relaxing judgment criteria for special periods.

[0053] The time series prediction model of the risk assessment unit integrates the advantages of multiple algorithms, combining an autoregressive integral moving average model and a deep neural network. Model fusion employs a stacked generalization technique, integrating the prediction results of the base models through a meta-learner. Enhanced interpretability of the prediction output is achieved through feature importance analysis, identifying the input variables with the greatest impact on the prediction results. Performance monitoring of the prediction model continuously tracks changes in the distribution of prediction errors, triggering a retraining process for model components with deteriorating performance. The graph neural network of the crisis prediction unit implements an improved attention mechanism, dynamically adjusting the influence of nodes during information dissemination. Attention weight calculation considers the timeliness and relevance of node attributes, highlighting key influencing factors in the current context. The edge weights of the graph are periodically updated through statistical learning, reflecting the dynamic changes in the strength of emotional connections. The visualization of the prediction results adopts a force-oriented layout, intuitively displaying the networked structure of crisis factors.

[0054] The system implements comprehensive privacy protection measures, and the collection and processing of emotional data comply with medical information confidentiality standards. Differential privacy technology is used for data anonymization to protect individual identity information while maintaining analytical utility. Access control implements role-based access management, allowing users at different levels to view content corresponding to their security level. All data operations are logged in a complete audit log, supporting the tracing and investigation of security incidents. The system communication channel uses end-to-end encryption to prevent information from being intercepted or tampered with during transmission. The operation and maintenance system establishes a multi-level monitoring mechanism to monitor the operational status of each system component in real time. Performance monitoring indicators include key technical parameters such as data processing latency, algorithm execution time, and resource utilization. Anomaly detection algorithms identify deviations from normal operating modes and automatically trigger warnings and recovery procedures. System updates implement a canary release strategy, with new versions first verified in the test environment before being gradually rolled out to the production environment. The disaster recovery plan is designed to consider multiple failure scenarios to ensure the continued availability of critical functions under abnormal conditions.

[0055] Example 4: The sub-data segmentation unit of the fluctuation recognition unit receives the emotion data stream from the region segmentation module and splits the continuous emotion monitoring data into analysis units with different time granularities according to preset scale segmentation rules. Large-scale sub-data covers emotion change trends with a 24-hour cycle, while small-scale sub-data focuses on instantaneous fluctuations within a 5-minute time window. An overlap sampling strategy is implemented during the segmentation process, preserving appropriate temporal overlap between adjacent analysis units to avoid fragmenting key emotional events. The parameter settings for data segmentation consider the physiological characteristics of emotional responses; for example, a shorter analysis window is used for skin conductance response data to capture its rapid changes, while a longer analysis window is used for heart rate variability data to assess its periodicity.

[0056] The deep convolutional neural network configured for large-scale analysis units employs a multi-branch architecture, with each branch specializing in processing different types of emotional feature data. The physiological signal branch contains stacked one-dimensional convolutional layers and max-pooling layers to extract long-term variation patterns of indicators such as heart rate and respiration. The speech feature branch uses two-dimensional convolution to process spectrograms, capturing macroscopic trends in intonation changes. The facial expression analysis branch processes time-series facial action units to identify persistent emotional expression features. Network training utilizes deep structures with residual connections to mitigate the vanishing gradient problem. The output of each analysis cycle includes evaluation results across three dimensions: emotional state score, stability index, and anomaly probability.

[0057] The bidirectional long short-term memory network of the small-scale analysis unit is designed with an attention-enhanced architecture. The forward and backward layers capture the causal and retrospective features of emotional fluctuations, respectively. The attention layer dynamically calculates the weight allocation at each time step, highlighting the impact of key transient events. Input feature engineering considers the noise characteristics of small-scale data and implements adaptive filtering preprocessing. The network output includes three core indicators: instantaneous emotional intensity, change slope, and anomaly type identification. The analysis process implements a sliding window update mechanism, refreshing the evaluation results every 10 seconds to maintain the ability to track rapid emotional changes.

[0058] The pattern fusion unit establishes a collaborative working mechanism between large-scale and small-scale analysis models. After each analysis cycle, the large-scale model transmits its output trend feature vector to the small-scale model via shared memory. The transmitted information includes an estimate of the current sentiment baseline, periodic fluctuation parameters, and time stamps of identified outlier areas. The small-scale model uses this macro-level guidance to adjust its analytical focus and perform enhanced analysis on the data during the marked periods. The fusion decision employs a weighted voting strategy, combining the confidence scores of the two models' outputs to generate the final judgment. When there is a significant discrepancy between the judgments of the large-scale and small-scale models, a data re-analysis process is triggered for arbitration.

[0059] The feedback mechanism unit enables real-time information exchange between analysis models. Large-scale models periodically send feature maps of their convolutional kernel activation patterns, reflecting the cyclical characteristics of the currently observed sentiment changes. Small-scale models return detailed analytical results for local regions, including precise event start points and duration measurements. Information exchange uses a lightweight binary encoding format to optimize transmission efficiency. The feedback channel implements a priority scheduling mechanism, granting higher processing authority to communication requests for critical anomalies. The region stripping algorithm in the fine analysis unit locates suspicious time periods based on large-scale analysis results and identifies abrupt changes in sentiment data through multi-scale edge detection. Stripping parameter calculations consider data sampling rate and signal characteristics to ensure complete coverage of anomalies by the extracted small-scale analysis regions. Time-frequency joint analysis is performed on the stripped data segments, simultaneously observing waveform characteristics in the time domain and spectral distribution changes in the frequency domain. Subtle feature identification employs high-order statistical analysis to calculate nonlinear characteristics such as signal skewness, kurtosis, and complexity. The result optimization unit improves judgment accuracy through iterative analysis. After each analysis, the system compares the predicted results with the actual observed sentiment performance and calculates error indices for each dimension. Error analysis guides hyperparameter tuning, including key configurations such as kernel size, number of network layers, and number of attention heads. The optimization process employs a mini-batch update strategy, gradually fine-tuning model parameters without affecting overall stability. The final output sentiment anomaly pattern includes standardized descriptors, recording structured information such as anomaly type, occurrence time, duration, and impact dimensions (see Table 1).

[0060] Table 1: Parameter Table for Scale Division in Sentiment Data Analysis

[0061] Data quality control is implemented throughout the entire analysis process. Integrity checks are performed on raw input data, and data segments with missing values ​​exceeding a threshold are marked as invalid. During feature extraction, the signal-to-noise ratio (SNR) of each dimension is monitored, and features that fail to meet quality standards are downweighted. Reasonableness checks are implemented at the model output layer to filter out abnormal predictions that clearly violate physiological laws. Detailed log information is recorded at all quality control nodes, supporting traceability and review of the analysis process.

[0062] The system operates within a high-performance computing environment to ensure real-time requirements. Large-scale analysis tasks are assigned to GPU clusters for batch processing, while small-scale analyses are executed locally by edge computing devices. The task scheduler dynamically balances the load across nodes, prioritizing analysis tasks within critical time windows. Memory management employs an intelligent caching strategy, predicting and preloading potentially needed feature data based on data access patterns.

[0063] Privacy protection measures are implemented throughout the entire data processing lifecycle. Raw data undergoes de-identification during collection, removing directly identifiable personal information. Data transmission employs industry-standard encryption protocols to prevent man-in-the-middle attacks. Data storage is subject to strict access control, with differentiated permission levels assigned based on roles. Analysis results are aggregated and obfuscated before output to avoid inferring individual identities. All data operations are logged in complete audit logs to meet compliance requirements.

[0064] The hardware infrastructure is designed with fault tolerance in mind. Compute nodes are configured with redundant power supplies and network connections to prevent service interruptions due to single points of failure. The storage system adopts a distributed architecture, with data automatically replicated across multiple physical devices. The network topology design avoids single points of failure on critical paths, and important communication links are configured with backup channels. Environmental monitoring continuously tracks the data center's temperature, humidity, and air quality to provide early warnings of potential hardware risks.

[0065] The user support system provides multi-tiered help resources. The quick start guide introduces basic operating procedures in a visually appealing way. Interactive tutorials demonstrate system functions through practical examples. The online help system supports natural language queries, allowing direct location of relevant document paragraphs. A professional technical support team provides remote diagnostics and on-site service for complex issues, ensuring stable system operation.

[0066] Example 5: The region stripping algorithm of the fine analysis unit, when processing sentiment data from large-scale analysis, first performs multi-resolution decomposition on the time series. This method can simultaneously observe the overall trend and local details of sentiment fluctuations, capturing feature performance at different time scales by setting different observation windows. The algorithm automatically adjusts the analysis granularity during runtime, using a coarser observation scale for stable sentiment state segments, and automatically switching to fine analysis mode when potential abnormal regions are detected. The data scanning process employs an adaptive threshold strategy, dynamically adjusting the sensitivity of anomaly detection based on the statistical distribution characteristics of historical data. When signal features exceed the dynamic threshold range, the system marks that time period as a candidate region for further analysis.

[0067] Texture complexity calculation employs an analysis method based on signal differential characteristics, quantifying the irregularity of emotional parameters by measuring the frequency and amplitude of their changes per unit time. This calculation considers not only the variation patterns of a single emotional dimension but also assesses the collaborative variation relationships between multiple parameters. The grayscale change rate index identifies potential emotional state inflection points by analyzing the intensity gradient changes of emotional features. The comprehensive evaluation of these two indices effectively distinguishes between normal emotional fluctuations and potentially problematic unstable states. When the index value exceeds the reference range set based on group benchmark data, the system determines that there are potential hidden details in that area that require further analysis.

[0068] The sub-data extraction unit calculates optimal extraction parameters based on the spatiotemporal characteristics of the potential hazard area. The determination of the time window length considers the typical duration of emotional responses, ensuring complete coverage of the entire abnormal event cycle. The spatial scope is defined with reference to correlation analysis of multimodal data, incorporating strongly correlated emotional dimensions into the same analysis batch. Boundary smoothing is implemented during the extraction process to avoid loss of key features due to hard segmentation. A data alignment mechanism ensures precise synchronization of sub-data acquired from different sensors along the timeline, creating favorable conditions for multimodal fusion analysis. The extracted sub-datasets are accompanied by complete metadata descriptions, including the source time period, original signal quality, and relevant contextual information.

[0069] The feature analysis unit implements a multi-level sentiment anomaly detection process. Primary analysis targets unimodal data, identifying deviation features across various sentiment dimensions. Intermediate analysis examines the consistency between multimodal parameters, detecting inconsistencies across dimensions. Advanced analysis integrates temporal contextual information to assess the evolution trend of anomalous features. The analysis process employs an incremental learning strategy, automatically optimizing feature extraction rules as data accumulates. Subtle feature recognition pays particular attention to fluctuations with small amplitudes but distinctive patterns, which often indicate potential sentiment modulation problems. The feature extraction algorithm possesses rotation invariance and scale invariance, reliably detecting sentiment anomalies in various forms.

[0070] The output of detailed information on emotional abnormalities employs a structured representation method. Each abnormal event record includes precise time location, a list of affected emotional dimensions, and a severity score. Detailed descriptions distinguish between primary and secondary features, facilitating professionals' rapid understanding of the core issues. Correlation analysis between events identifies recurring patterns that may indicate difficulties in emotional regulation. The output information is organized in a standardized format, supporting seamless integration with subsequent intervention systems. Data visualization utilizes timeline markers combined with characteristic waveform displays, making abstract analytical results intuitive and understandable.

[0071] Quality control is implemented throughout the entire detailed analysis process. During raw data input, the continuity of timestamps and the reasonableness of numerical values ​​are verified to eliminate obvious collection anomalies. The stability of the algorithm is monitored during the feature extraction stage to prevent the accumulation of numerical calculation errors. Consistency checks are performed before results are output to ensure that conclusions from different analytical paths support each other. A quality log records the verification status of each processing step in detail, providing a complete basis for possible review and analysis. An anomaly handling mechanism can automatically identify common data quality issues and trigger appropriate repair or retry processes.

[0072] System resource allocation takes into account the specific needs of fine-grained analysis. Computation nodes are equipped with high-performance signal processing hardware to accelerate time-series feature extraction. Memory management optimizes data locality, reducing I / O wait times during analysis. The task scheduler prioritizes resource allocation for the real-time analysis pipeline while also rationally scheduling background batch processing tasks. Network bandwidth reservations ensure efficient data exchange between large-scale and fine-grained analysis units. The storage system employs a tiered design, storing frequently accessed data on high-speed storage media to support frequent access.

[0073] The user interface allows for in-depth observation of the detailed analysis process. Expert mode provides a complete parameter adjustment interface, allowing experienced users to customize the analysis workflow. The debug view displays intermediate calculation results, helping to understand the system's decision-making basis. The results browser supports multi-dimensional filtering and sorting, quickly locating sentiment events of interest. The annotation tool allows users to add supplementary explanations to important patterns discovered by the system; this human feedback will be used to optimize subsequent analyses. The interface design follows cognitive ergonomics principles, highlighting important information, and the operation flow conforms to professional work habits.

[0074] The system maintenance establishes a regular calibration mechanism. Sensor calibration ensures the accuracy of data acquisition, providing a reliable foundation for detailed analysis. Algorithm calibration adjusts feature extraction parameters to adapt to changes in individual emotional expressions. Model calibration updates classification boundaries to maintain sensitivity in anomaly detection. The calibration process employs a closed-loop design, automatically evaluating calibration effectiveness and guiding further optimization. A version upgrade strategy ensures continuous improvement in analytical capabilities while maintaining consistency in output results.

[0075] Privacy protection measures are strictly implemented at every stage of data processing. Data anonymization removes direct identifying information, and anonymized identifiers are used during analysis. Access control follows the principle of least privilege, allowing only essential personnel access to raw data. Audit logs record all data access and operations to meet compliance requirements. Output results undergo aggregation processing to prevent the inference of personal identities from analysis conclusions. Strong encryption measures are used for data transmission and storage to prevent unauthorized access.

[0076] The technical documentation system provides a comprehensive explanation of the analysis process. The architecture document describes the relationships between components and data flow. The algorithm white paper explains the theoretical basis of the core analysis methods. The interface specification defines the communication protocols between modules. The operation manual guides daily maintenance and troubleshooting. The case library collects typical analysis scenarios for reference. Documentation updates are synchronized with system development to ensure consistency between content and actual functionality.

[0077] The hardware infrastructure is designed to support high-precision sentiment analysis requirements. A dedicated signal processing unit optimizes time-series data analysis performance. A high-speed data bus ensures real-time transmission of large volumes of sensor data. Redundant storage configurations prevent the loss of intermediate analysis results. An environmental control system maintains stable operating temperatures to prevent hardware performance fluctuations from affecting analysis quality. Network equipment is configured with Quality of Service (QoS) policies to guarantee communication bandwidth for critical analysis tasks.

[0078] The user support system provides multi-channel help resources. An online knowledge base contains frequently asked questions and solutions. Video tutorials demonstrate how to operate in typical analysis scenarios. A community forum facilitates the exchange of experiences among users. A professional technical support team provides tiered responses to complex issues. User feedback channels collect user experiences to guide system improvement. The training system includes introductory courses and advanced topics to help users fully utilize the system's capabilities.

[0079] The application of data analysis results establishes a closed-loop feedback mechanism. Data on the effectiveness of intervention measures is collected and fed back into the analysis system to validate and improve the rules for identifying emotional anomalies. This data-driven continuous optimization enables the system to gradually adapt to the emotional characteristics of different individuals, improving the personalization of the analysis. The integration and processing of feedback data considers the time delay effect, reasonably assessing the long-term impact of intervention measures. The system's scalability design supports the incremental enhancement of analytical capabilities. The modular architecture allows for the flexible addition of new analytical dimensions. The plug-in mechanism facilitates the integration of improved feature extraction algorithms. The abstract interface definition supports plug-and-play functionality for various sensors. The configuration management system records the capability characteristics of each node and intelligently allocates analytical tasks. This design enables the system to upgrade its functionality and scale without service interruption. The anomaly handling process covers all possible technical issues. Data anomaly detection identifies problems such as missing data, noise, and distortion, triggering appropriate remedial measures. Computational anomaly monitoring prevents result deviations caused by algorithm failure. System anomaly handling ensures the continuous availability of core services. The fault recovery mechanism automatically diagnoses the root cause of problems and executes remedial steps; problems that cannot be resolved automatically are escalated to manual handling. All abnormal events are logged in detail for post-event analysis and preventative improvement.

[0080] The performance monitoring system tracks key operational metrics in real time. Data analysis shows throughput, reflecting system processing capacity. Task latency monitoring ensures real-time requirements are met. Resource utilization optimizes hardware efficiency. Service quality metrics assess end-user experience. Performance data visualization displays historical trends and current status, helping operations personnel understand system health. An early warning mechanism promptly notifies relevant personnel when metrics exceed normal ranges.

[0081] Deployment solutions consider the varying needs of different application scenarios. Centralized deployment is suitable for data-intensive analysis tasks. Edge computing solutions meet scenarios with high real-time requirements. Hybrid architectures balance processing efficiency and communication overhead. Containerization technology simplifies system migration and deployment across different environments. Configuration management tools ensure consistency of parameters across all nodes. Load balancing mechanisms dynamically adjust task allocation to optimize overall system performance.

[0082] The security protection system implements a defense-in-depth strategy. Network firewalls filter malicious access attempts. Intrusion detection systems identify suspicious behavior patterns. Data encryption protects sensitive information. Identity authentication ensures user legitimacy. Access control restricts the scope of operations. Security audits track all sensitive operations. Vulnerability management regularly assesses and fixes system weaknesses. Incident response plans prepare contingency plans for various security incidents. These measures collectively build comprehensive security safeguards, ensuring the confidentiality of sentiment data and the reliability of system services.

[0083] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0084] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multimodal emotion recognition and psychological counseling system, characterized in that, include: A data receiving module is used to receive real-time emotion data from a multimodal sensor, compare the real-time emotion data with historical emotion data, and obtain the periodic and non-periodic components in the emotion data. The pattern recognition module is used to identify unexpected emotional fluctuation patterns in emotional data based on the periodic components and the non-periodic components. The key factor extraction module performs causal relationship and co-occurrence pattern discovery processing on the unexpected emotional fluctuation patterns to obtain key emotional factors; The benchmark construction module is used to construct a benchmark emotional performance curve by using the key emotional factors as input and combining similar emotional data among individuals. The risk assessment module is used to assess an individual's emotional state and determine the level of potential psychological risk based on the degree of deviation between the benchmark emotional expression curve and the actual emotional expression curve, combined with a fuzzy assessment system. The prediction module is used to predict individuals who will experience an emotional crisis within a preset time period based on the unexpected emotional fluctuation pattern, the key emotional factors, and the potential psychological risk level, and to generate a psychological counseling suggestion report.

2. The multimodal emotion recognition and psychological counseling system according to claim 1, characterized in that, The data receiving module includes: The multidimensional data acquisition unit is used to simultaneously collect multidimensional emotional parameter data of individuals in different emotional states and construct an emotional feature dataset. The component extraction unit is used to perform time-frequency analysis on the emotional feature dataset, extract frequency domain feature vectors, and separate periodic and non-periodic components based on the frequency domain feature vectors.

3. The multimodal emotion recognition and psychological counseling system according to claim 2, characterized in that, The pattern recognition module includes: The scene determination unit is used to determine an emotional scene including all emotional elements based on the emotional feature dataset and a preset emotional scene knowledge graph. The region division unit is used to divide the emotional data into large-scale emotional information regions and small-scale emotional information regions according to the emotional scene and the corresponding specification requirements. The fluctuation recognition unit is used to identify unexpected emotional fluctuation patterns based on the large-scale emotional information region and the small-scale emotional information region using a pattern classification algorithm.

4. The multimodal emotion recognition and psychological counseling system according to claim 3, characterized in that, The key factor extraction module includes: The data optimization unit is used to construct an emotional fluctuation dataset using the unexpected emotional fluctuation pattern, and to apply a dimensionality reduction algorithm to perform dimensionality reduction processing on the emotional fluctuation dataset to obtain an optimized dataset. The relationship mining unit is used to scan the optimized dataset, identify frequent itemsets, detect the causal relationships and co-occurrence patterns of the frequent itemsets, and generate key sentiment factors.

5. The multimodal emotion recognition and psychological counseling system according to claim 4, characterized in that, The benchmark construction module includes: The data smoothing unit is used to use the key emotional factors as input, combine similar emotional data between individuals, and smooth the multi-source historical emotional data to obtain smoothed data. The curve fitting unit is used to fit the smoothed data using a regression model to generate a baseline sentiment expression curve.

6. The multimodal emotion recognition and psychological counseling system according to claim 5, characterized in that, The risk assessment module includes: The deviation calculation unit is used to calculate the degree of deviation between the actual emotional expression curve and the benchmark emotional expression curve by comparing the actual emotional expression curve with the benchmark emotional expression curve. A scoring generation unit is used to quantify the degree of deviation using an error quantification algorithm to obtain a deviation score. A health assessment unit is used to score the degree of deviation and obtain a health status assessment result; The risk determination unit is used to use the health status assessment results, combined with a time series prediction model, to predict the trend of individual emotional changes within a preset time period, and analyze the prediction results to determine the potential psychological risk level.

7. The multimodal emotion recognition and psychological counseling system according to claim 6, characterized in that, The prediction module includes: The crisis prediction unit is used to input the unexpected emotional fluctuation pattern, the key emotional factors, and the potential psychological risk level into the network model to predict emotional crisis events. The report generation unit is used to generate psychological counseling suggestion reports based on predicted emotional crisis events.

8. The multimodal emotion recognition and psychological counseling system according to claim 3, characterized in that, The fluctuation identification unit includes: The sub-data partitioning unit is used to split the emotional data into large-scale sub-data and small-scale sub-data according to the large-scale emotional information region and the small-scale emotional information region; The large-scale analysis unit is used to input large-scale sub-data into the first deep learning model for analysis. The small-scale analysis unit is used to input small-scale sub-data into the second deep learning model for analysis. The pattern fusion unit is used to combine the outputs of the first deep learning model and the second deep learning model to identify unexpected emotional fluctuation patterns.

9. The multimodal emotion recognition and psychological counseling system according to claim 8, characterized in that, The mode fusion unit includes: The feedback mechanism unit is used to establish a real-time feedback mechanism between the first deep learning model and the second deep learning model to transmit preliminary analysis results and location information. The fine analysis unit is used to perform detailed hidden danger identification processing on small-scale sub-data based on the preliminary analysis results and location information, and to detect specific emotional abnormal information; The results optimization unit is used to adjust the analysis parameters based on the comparison between multiple analysis results and actual sentiment data, and output the final sentiment anomaly pattern.

10. The multimodal emotion recognition and psychological counseling system according to claim 9, characterized in that, The fine analysis unit includes: The region stripping unit is used to calculate the texture complexity and grayscale change rate indices of local regions based on large-scale sub-data. When the indices exceed the preset threshold, the region with potential hidden dangers in detail is identified. The sub-data extraction unit is used to calculate stripping parameters and extract small-scale sub-data from large-scale sub-data to ensure complete coverage of the potential hazard area. The feature analysis unit is used to perform subtle feature recognition processing on small-scale sub-data and output detailed information on emotional anomalies.

Images