A Dynamic Early Warning Method for Emotional Risk Based on Multimodal Data Fusion

Abstract

This invention relates to the field of emotion risk identification technology, and discloses a dynamic early warning method for emotion risk based on multimodal data fusion. The method includes: collecting multimodal emotion-related data of users to obtain a raw multimodal data stream; cleaning and standardizing the raw multimodal data stream to obtain standardized multimodal data; constructing an individual baseline; using the individual baseline as a constraint, extracting features from the standardized multimodal data to obtain a multimodal emotion feature set; performing feature-level weighted fusion and decision-level rule calibration on the multimodal emotion feature set to obtain a comprehensive emotion risk vector; and performing dynamic risk assessment on the comprehensive emotion risk vector to obtain an emotion risk score. This invention employs a dynamic assessment technology scheme using a comprehensive emotion risk vector, achieving continuous risk change identification and enabling continuous evolution early warning of emotion risk, thus overcoming the shortcomings of existing early warning systems that fail to reflect dynamic changes.

Description

Technical Field

[0001] This invention relates to the field of emotion risk recognition technology, specifically to a dynamic early warning method, electronic device, and storage medium for emotion risk based on multimodal data fusion. Background Technology

[0002] In scenarios involving emotional risk early warning in campus psychological services, medical health monitoring, community health and wellness management, and care for employees of enterprises and institutions, the management end needs to continuously identify, assess risks, and intervene in advance for users' emotional abnormalities based on physiological state, language expression, behavioral changes, and contextual information. In this scenario, how to uniformly analyze objective physiological signals and subjective expression information, and combine them with long-term individual state changes to achieve dynamic early warning, has become a key technical issue affecting the accuracy, real-time performance, and effectiveness of intervention.

[0003] Currently, existing technologies mainly complete physiological-level emotion judgment by collecting heart rate variability, heart rate, skin conductance, and sleep status, or complete semantic-level emotion recognition through questionnaires, text content, and voice content. On the platform side, abnormal prompts are output based on preset thresholds, rule models, or classification models to achieve basic emotion monitoring and risk warning.

[0004] While existing technologies can provide basic early warnings based on single physiological signal analysis or single semantic analysis, they still lack a multimodal dynamic fusion mechanism for individual baseline changes. This makes it impossible to adaptively correlate and calculate heart rate variability data, text semantics, voice features, behavioral habits, and scene labels within a unified decision-making framework, resulting in early warning results that do not accurately reflect the continuous evolution of emotional risk. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a dynamic early warning method, electronic device, and storage medium for emotional risk based on multimodal data fusion, so as to at least partially solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a dynamic early warning method for emotional risk based on multimodal data fusion, comprising:

[0008] Multimodal emotion association data of users is collected to obtain raw multimodal data stream, wherein the multimodal emotion association data includes data reflecting the user's autonomic nervous system state. Physiological data, text semantic data reflecting the user's subjective expression, voice emotion data reflecting the user's emotional state, behavioral habit data reflecting the user's daily rhythm, and scene tag data reflecting the user's application environment.

[0009] The original multimodal data stream is cleaned and rectified to obtain standardized multimodal data;

[0010] An individual baseline is constructed, and features are extracted from the standardized multimodal data using the individual baseline as a constraint to obtain a multimodal emotion feature set.

[0011] The multimodal emotion feature set is subjected to feature layer weighted fusion and decision layer rule calibration to obtain a comprehensive emotion risk vector;

[0012] A dynamic risk assessment is performed on the comprehensive emotional risk vector to obtain an emotional risk score;

[0013] The emotional risk score is graded to obtain an emotional risk warning result.

[0014] Preferably, the original multimodal data stream is cleaned and rectified to obtain standardized multimodal data, including:

[0015] The original multimodal data stream is subjected to integrity verification, and data segments lacking valid collected content are removed to obtain the data stream to be normalized;

[0016] The data stream to be normalized is subjected to duplicate content identification and invalid content removal to obtain an effective multimodal data stream;

[0017] The effective multimodal data stream is subjected to noise reduction and abnormal fluctuation correction to obtain a cleaned multimodal data stream;

[0018] The cleaned multimodal data stream is time-aligned and format-unified to obtain a regularized multimodal data stream;

[0019] The regularized multimodal data stream is normalized to obtain the standardized multimodal data.

[0020] Preferably, an individual baseline is constructed, and features are extracted from the standardized multimodal data using the individual baseline as a constraint to obtain a multimodal emotion feature set, including:

[0021] The standardized multimodal data is aggregated to obtain continuous user state data;

[0022] The individual baseline is obtained by calculating the stable range of change, daily range of change, and instantaneous change of state in the continuous state data of the user.

[0023] Using the individual baseline as a reference, the deviation degree, direction of change, and continuous change state of the standardized multimodal data are extracted to obtain candidate emotion features;

[0024] The candidate emotion features are subjected to consistency screening and invalid feature removal to obtain the multimodal emotion feature set.

[0025] Preferably, the multimodal emotion feature set is subjected to feature-level weighted fusion and decision-level rule calibration to obtain a comprehensive emotion risk vector, including:

[0026] The multimodal emotion feature set is distinguished by feature source and its numerical values ​​are unified to obtain the emotion features to be fused;

[0027] The characterization intensity analysis of the emotional features to be fused is performed to obtain the feature weight parameters;

[0028] The emotional features to be fused and the feature weight parameters are weighted and fused to obtain the fused emotional features;

[0029] The fused emotional features are matched using rules to obtain feature-consistent states and feature-conflicting states;

[0030] The feature consistency state and feature conflict state are calibrated and determined to obtain the comprehensive emotion risk vector.

[0031] Preferably, a dynamic risk assessment is performed on the comprehensive emotional risk vector to obtain an emotional risk score, including:

[0032] The comprehensive emotional risk vector is continuously arranged and correlated to obtain the data to be evaluated;

[0033] The deviation of the data to be evaluated is analyzed to obtain the deviation results;

[0034] The deviation results are analyzed for magnitude and duration to obtain risk change results;

[0035] A dynamic risk assessment is performed on the risk changes to obtain the emotional risk score.

[0036] Preferably, the step of classifying and determining the emotional risk score to obtain an emotional risk warning result includes:

[0037] The validity of the emotional risk score is verified to obtain the score to be judged;

[0038] The score to be determined is matched with the grade boundary to obtain the risk level result, which includes any one of the following levels: normal level, attention level, early warning level and crisis level.

[0039] The risk level results and the scores to be determined are correlated and labeled to obtain the classification determination data;

[0040] Based on the grading and determination data, an early warning status is generated, and the emotional risk early warning result is obtained.

[0041] Preferably, the effective multimodal data stream is subjected to noise reduction and abnormal fluctuation correction to obtain a cleaned multimodal data stream, including:

[0042] A continuous analysis is performed on the effective multimodal data stream to obtain the relationship between changes in adjacent data.

[0043] Based on the relationship between adjacent data changes, short-term jump data in the effective multimodal data stream are smoothly replaced to obtain a noise-reduced data stream;

[0044] The variation range of the noise-attenuated data stream is compared to obtain abnormal fluctuation data;

[0045] Based on the continuous data before and after the abnormal fluctuation data, the abnormal fluctuation data is numerically corrected to obtain a corrected multimodal data stream;

[0046] The continuity of the corrected multimodal data stream is verified to obtain the cleaned multimodal data stream.

[0047] Preferably, the emotional features to be fused are subjected to representation intensity analysis to obtain feature weight parameters, including:

[0048] Using the individual baseline as a reference, the degree of deviation of the emotional features to be fused is compared to obtain deviation intensity data;

[0049] The deviation intensity data is analyzed to identify the direction of change, thereby obtaining directional characterization data;

[0050] The direction characterization data is continuously verified to obtain continuous characterization data.

[0051] The sustained characterization data is sorted by strength to obtain a feature intensity sequence;

[0052] The feature intensity sequence is weighted to obtain the feature weight parameters.

[0053] Preferably, the emotional features to be fused and the feature weight parameters are weighted and fused to obtain fused emotional features, including:

[0054] The corresponding matching of the emotion features to be fused and the feature weight parameters is performed to obtain the feature weight corresponding data;

[0055] Based on the data corresponding to the feature weights, the emotional features to be fused are weighted to obtain weighted emotional features;

[0056] The weighted emotional features are integrated in the same direction and canceled out in the opposite direction to obtain the initial fused emotional features;

[0057] The initial fused emotional features are numerically unified and their integrity is verified to obtain the fused emotional features.

[0058] Preferably, the weighted emotional features are integrated in the same direction and canceled out in the opposite direction to obtain the initial fused emotional features, including:

[0059] The direction of change of the weighted emotion features is identified to obtain same-direction weighted emotion features and opposite-direction weighted emotion features;

[0060] The same-direction weighted emotional features are accumulated and integrated to obtain the same-direction integration result;

[0061] Intensity comparison of the inversely weighted emotional features yields an inverse cancellation relationship;

[0062] Based on the aforementioned reverse cancellation relationship, the same-direction integration result is cancelled and corrected to obtain a corrected fusion result;

[0063] The integrity of the corrected fusion result is verified to obtain the initial fused emotional features.

[0064] In a second aspect, the present invention provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method in the first aspect.

[0065] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method in the first aspect.

[0066] The present invention has the following beneficial effects:

[0067] 1. This invention adopts a comprehensive emotional risk vector dynamic assessment technology to achieve continuous risk change identification and realize early warning of continuous evolution of emotional risk, thus solving the shortcomings of existing early warning systems that do not easily reflect dynamic changes.

[0068] 2. This invention adopts a feature layer weighted fusion and decision layer rule calibration technology to achieve the effect of unified association of multi-source features, realize collaborative judgment of emotional risk, and solve the problem of insufficient accuracy of single-modal early warning.

[0069] 3. This invention adopts a multimodal feature extraction technology scheme with individual baseline constraints to achieve the effect of user state differentiation recognition technology, realize personalized representation of emotional changes, and solve the shortcomings of fixed threshold misjudgment. Attached Figure Description

[0070] Figure 1 This is a flowchart illustrating the dynamic early warning method for emotional risk based on multimodal data fusion provided in an embodiment of the present invention. Detailed Implementation

[0071] To enable those skilled in the art to understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort should fall within the scope of protection of the present invention.

[0072] The present invention will now be described in detail with reference to the accompanying drawings:

[0073] Please see the appendix Figure 1 This invention provides a dynamic early warning method for emotional risk based on multimodal data fusion, which can be applied to scenarios such as campus psychological services, medical and health monitoring, community health and wellness management, and employee care in enterprises and institutions. This method collects users' multimodal emotional correlation data to form a raw multimodal data stream. Through data cleaning and regularization, individual baseline construction, feature extraction, weighted fusion of features, rule calibration at the decision level, dynamic risk assessment, and hierarchical judgment, it outputs an emotional risk early warning result that reflects the user's current emotional risk status and its changing trends.

[0074] In this embodiment, multimodal emotion association data includes data reflecting the user's autonomic nervous system state. Physiological data, text semantic data reflecting the user's subjective expression, voice emotion data reflecting the user's emotional state, behavioral habit data reflecting the user's daily rhythm, and scene tag data reflecting the user's application environment. Physiological data is obtained from wearable devices, heart rate monitoring terminals, or medical monitoring terminals; textual semantic data is obtained from psychological scale entries, diary entries, consultation dialogue texts, or platform interaction texts; voice emotion data is obtained from authorized voice clips; behavioral habit data is obtained from sleep records, activity records, platform access frequency, and daily routine change records; scene tag data is generated by the management platform based on the user's application scenario, task status, intervention stage, or service type. All of the above data is collected under user authorization and privacy protection conditions, and is anonymized before entering the calculation process to isolate user identity information from the emotional risk calculation process.

[0075] During the data acquisition phase, a data access channel is established for the same user. Data from different sources is uniformly packaged according to acquisition time, user identifier, and data source to form a raw multimodal data stream. To ensure subsequent data correlation calculations, the acquisition time, data source, data type, and scenario marker are recorded synchronously when acquiring each data point. The acquisition time is used for continuous arrangement and time alignment; the data source is used to distinguish feature sources; the data type is used for format unification and numerical normalization; and the scenario marker is used to distinguish different risk assessment contexts in campus, medical, community, or enterprise scenarios.

[0076] for Physiological data includes changes in adjacent heartbeat intervals, heart rate fluctuations, and heart rate variability calculated from heartbeat intervals. For textual semantic data, it involves acquiring user-initiated input or questionnaire entries and converting them into analyzable text fragments. For speech emotion data, it extracts pitch, speech rate, pauses, energy fluctuations, and emotional tendencies from speech fragments. For behavioral habit data, it collects user routines, activities, access frequency, interaction frequency, and behavioral rhythm changes. For scenario tagging data, it records whether the user is currently in a learning stress scenario, a medical follow-up scenario, a health and wellness management scenario, or a workplace care scenario. All of the above data together constitute the original multimodal data stream.

[0077] After obtaining the raw multimodal data stream, it is first cleaned and standardized to obtain standardized multimodal data. This step is used to eliminate missing, duplicate, invalid, noise, abnormal fluctuations, time inconsistencies, and format inconsistencies during the acquisition process, providing a reliable data foundation for subsequent individual baseline construction and feature extraction.

[0078] The integrity of the raw multimodal data stream is verified. Specifically, each data segment is read and its validity is checked to determine whether it contains valid data, data collection time, data source, and resolvable data values. If a data segment lacks valid data, contains records but the data type cannot be identified, or the data collection time is missing and cannot be included in continuous analysis, the data segment is removed. The remaining data after removal constitutes the data stream to be organized.

[0079] Integrity checks do not simply determine if data is empty; they also determine whether the data can be used in subsequent calculations. For example, for For physiological data, if the heartbeat interval recording is missing or the continuous acquisition period is severely interrupted, the segment cannot be used for effective physiological fluctuation analysis. For textual semantic data, if the text content contains unresolvable characters, it cannot be used for semantic tendency judgment. For speech emotion data, if the speech segment has no effective acoustic signal, it cannot be used for speech emotion analysis. For behavioral habit data, if there are empty records without the time of occurrence or type of behavior, it cannot be used for behavior change recognition. All of the above situations are judged as data segments lacking effective acquisition content.

[0080] The process involves identifying duplicate content and removing invalid content from the regularized data stream to obtain a valid multimodal data stream. Duplicate content identification is based on user identifier, collection time, data source, and data value. When multiple data entries share the same user, collection time, data source, and data value, only one valid record can be retained. When multiple data entries have similar collection times and highly consistent content, the data with higher reliability is retained based on the priority of the collection source or the completeness of the record.

[0081] Invalid content removal is used to delete data that is not meaningful for sentiment risk analysis. For example, records containing no semantic symbols in text, records containing environmental noise in speech, and records without user-initiated behavior in behavioral data can all be identified as invalid content. This process avoids duplicate data amplifying the influence of a particular modality and prevents invalid content from entering the subsequent feature extraction process.

[0082] The effective multimodal data stream is subjected to noise reduction and abnormal fluctuation correction to obtain a cleaned multimodal data stream. This process includes continuity analysis, short-term jump smoothing replacement, variation range comparison, abnormal fluctuation correction, and continuity verification.

[0083] First, a continuous analysis is performed on the effective multimodal data stream according to the acquisition sequence to obtain the relationship between adjacent data changes. The relationship between adjacent data changes is used to represent the direction and magnitude of data changes for the same user and the same data source within a continuous acquisition period. When a data point exhibits a short-term surge or drop relative to its preceding and following adjacent data, and this change does not continue to occur in adjacent time periods, the data point can be identified as a short-term jump data. Short-term jump data is usually caused by acquisition jitter, network latency, unstable sensor contact, or momentary non-emotional behavior of the user.

[0084] For short-term abrupt changes in data, a smooth replacement is performed based on the relationship between adjacent data changes to obtain a noise-reduced data stream. The smooth replacement uses a weighted estimation method based on adjacent valid data.

[0085] The following formula is used: ,

[0086] in, Indicates user The Class data in the first Collect the data value at the group number after smooth replacement. This represents the original valid data value before the smooth replacement. This represents the data value of the previously collected group number. This indicates the data value of the next collected group number. Indicates the first The original value retention factor for class data, Indicates user, Indicates the data source category. Indicates the original acquisition sequence number;

[0087] If this type of data collection is highly stable, then It can be set higher; if this type of data is easily affected by noise, then... It can be set to a lower value to enhance the impact of adjacent valid data on the replacement result.

[0088] After noise reduction, the noise-reduced data stream is compared to identify anomalous fluctuation data. The range of variation is determined by the user's historical stable data, reasonable variation intervals for similar data, and consecutive data before and after the anomalous fluctuation. When a data value exceeds the acceptable range of variation and does not have a continuous variation relationship with adjacent data, it is identified as anomalous fluctuation data. Anomalous fluctuation data is not directly deleted; instead, it is numerically corrected based on consecutive data before and after it, resulting in a corrected multimodal data stream. The correction method uses trend estimates from consecutive data before and after the anomalous value to replace outliers, avoiding time series breaks caused by direct deletion.

[0089] The following formula is used: ,

[0090] in, Indicates user The Class data in the first The data value at the group number is corrected for abnormal fluctuations. This represents the data value at the previous data collection group number after noise reduction. This indicates the data value at the next data collection group number after noise reduction. Indicates the first Trend adjustment factor for class data Indicates user, Indicates the data source category. Indicates the original collection sequence number.

[0091] This correction method ensures that abnormal fluctuations do not directly disrupt data continuity and are not simply fixed to an average value, making it suitable for subsequent dynamic risk assessment.

[0092] After correction, the continuity of the corrected multimodal data stream is verified. This verification confirms whether the corrected data still contains breakpoints, jumps, or segments that do not conform to the acquisition order. If the verification result meets the requirements for continuous analysis, the cleaned multimodal data stream is obtained.

[0093] The cleaned multimodal data stream is time-aligned and format-unified to obtain a regularized multimodal data stream. Since different modalities are acquired at different frequencies, Physiological data may be generated continuously, textual semantic data may be generated intermittently, voice emotion data may be generated according to conversation, and behavioral habit data may be generated according to events. Therefore, it is necessary to establish a correspondence between different data according to a unified time and location.

[0094] Time alignment can be achieved using a time window approach. The cleaned multimodal data stream is divided into several consecutive segments according to a preset time window, and multiple types of data generated by the same user within the same time window are grouped into the same time position. If a certain type of data is not generated within a time window, it is marked as a missing position, and supplemented using adjacent valid data or the user's individual baseline without affecting accuracy. Format standardization is used to convert different record formats into a unified record format. For numerical data, the numerical value and unit are retained; for text data, it is converted into semantic vectors or sentiment values; for speech data, it is converted into acoustic feature values ​​or speech sentiment values; for behavioral data, it is converted into frequency, duration, rhythm deviation, or change status. After time alignment and format standardization, a well-organized multimodal data stream is obtained.

[0095] Normalization transformation is performed on regularized multimodal data streams to obtain standardized multimodal data. Normalization transformation is used to eliminate differences in value ranges, units, and representation formats between different data sources, making the data more uniform. Physiological data, textual semantic data, voice emotion data, and behavioral habit data can be incorporated into the same computational framework.

[0096] The following formula is used: ,

[0097] in, Indicates user The Class data in the first Standardized data values ​​within a unified time window This represents the normalized data value before normalization transformation. Indicates the first The preset lower bound of the data type. Indicates the first The preset upper bound of the data type. This represents a stable term to prevent the denominator from being zero. Indicates user, Indicates the data source category. This represents a unified time window sequence number. Through this transformation, data from different sources are mapped to a unified value representation, resulting in standardized multimodal data.

[0098] After obtaining standardized multimodal data, an individual baseline is constructed. Using the individual baseline as a constraint, features are extracted from the standardized multimodal data to obtain a multimodal emotion feature set. This step is a key difference between this invention and fixed threshold-based early warning systems. The core of this invention lies in establishing a reference standard based on the user's own long-term state, rather than simply using a uniform threshold to judge emotional risk.

[0099] Standardized multimodal data is aggregated by user to obtain continuous user status data. During aggregation, standardized multimodal data for the same user within a continuous time window are arranged in chronological order, preserving data source, numerical changes, scene labels, and time position. Continuous user status data includes data from the current time period as well as historical stable time periods, which can be used to construct long-term user status profiles.

[0100] The stable range of change, daily range of change, and immediate change of state in the user's continuous state data are calculated to obtain the individual baseline. The stable range of change is used to represent the normal fluctuation boundary of the user in a non-high-risk state; the daily range of change is used to represent the user's habitual changes in different daily scenarios; and the immediate change of state is used to represent the short-term changes of the current data relative to the historical state.

[0101] Individual baselines are calculated using the following formula: ,

[0102] in, Indicates user In the Individual baselines on class data, Indicates user The set of time windows involved in individual baseline construction. Indicates user The Class data in the first The stable contribution weight to the construction of individual baselines within the time window. Represents standardized data values. This represents a stable term to prevent the denominator from being zero. Indicates user, Indicates the data source category. This indicates the sequence number of the unified time window. The more obvious the stable state, the more continuous the data, and the lower the noise. In this way, the individual baseline can reflect the user's own normal level, rather than using the group average to replace the individual state.

[0103] Using an individual baseline as a reference, the standardized multimodal data is analyzed to extract the degree of deviation, direction of change, and state of continuous change, resulting in candidate emotional features. The degree of deviation represents the difference between the current state and the individual's normal state; the direction of change indicates whether the difference is moving towards increased risk or decreased risk; and the state of continuous change indicates whether the change persists within a continuous time window.

[0104] The following formula is used: ,

[0105] in, Indicates user The Class data in the first The degree of deviation from the individual baseline within the time window. Represents standardized data values. Indicates the individual baseline, Indicates user In the The normal fluctuation scale on similar data This represents a stable term to prevent the denominator from being zero. Indicates user, Indicates the data source category. This indicates the unified time window sequence number.

[0106] If the current data deviates significantly from the individual baseline, then The deviation is relatively high. The direction of change is determined by the difference between the current data and the data from the previous time window, and the state of continuous change is determined by whether the degree of deviation remains or increases within multiple consecutive time windows.

[0107] Candidate emotion features are subjected to consistency screening and invalid feature removal to obtain a multimodal emotion feature set. Consistency screening is used to retain features that have mutually supportive relationships across different modalities. For example, Physiological data showed an increasing trend in stress, while textual semantic data showed an increased negative tendency, and behavioral data showed irregular sleep patterns. Invalid feature removal was used to delete changes unrelated to emotional risk or caused by non-emotional factors. For example, changes in physiological data due to abnormal device use, or changes in platform access caused by automated push notifications, should not be included as valid emotional features in subsequent fusion.

[0108] After obtaining the multimodal emotion feature set, a weighted fusion of features and rule calibration at the decision layer are performed to obtain a comprehensive emotion risk vector. This step addresses the issues of inconsistent contributions from different modalities, potential conflicts in risk directions, and the insufficient professionalism of a single model in judgment.

[0109] The multimodal emotion feature set is differentiated by feature source and its numerical values ​​are unified to obtain the emotion features to be fused. Feature source differentiation records which type of data (physiological, textual, audio, behavioral, or scene) each emotion feature originates from. Numerical unification converts different features into a unified expression, enabling weight allocation and weighted fusion. For directional features, the direction of risk enhancement or reduction is retained; for intensity features, the degree of intensity is retained; and for persistence features, the persistence state is retained.

[0110] The representation strength analysis of the emotional features to be integrated is performed to obtain feature weight parameters. The representation strength analysis includes deviation comparison, change direction identification, persistence status confirmation, strength ranking, and weight transformation.

[0111] Using an individual baseline as a reference, the degree of deviation of the emotional features to be integrated is compared to obtain deviation intensity data; the direction of change of the deviation intensity data is identified to obtain directional representation data; the continuous state of the directional representation data is confirmed to obtain continuous representation data; the continuous representation data is sorted by strength to obtain a feature intensity sequence; and the feature intensity sequence is weighted to obtain feature weight parameters.

[0112] The following formula is used: ,

[0113] in, Indicates user The The emotional features to be fused in the first The intensity of representation within the time window, Indicates the first Reliability factors of class feature sources Indicates the degree of deviation. The factor representing the contribution of the direction of change to emotional risk. This represents the contribution factor of the persistent state to emotional risk. Indicates user, Indicates the data source category. This indicates a uniform time window sequence number. If a feature deviates from the individual baseline, the direction of change points to increased risk, and it persists within a continuous time window, then the characterization strength is high.

[0114] Furthermore, the representation strength is converted into feature weight parameters using the following formula:

[0115] ,

[0116] in, Indicates user The The emotional features to be fused in the first Feature weight parameters within the time window, Indicates the first The representational strength of the emotional features to be fused. This represents the set of data source categories participating in the fusion. Represents any category in the set of data source categories. Indicates user The The emotional features to be fused in the first The intensity of representation within the time window, This represents a stable term to prevent the denominator from being zero. Indicates user, Indicates the data source category. This indicates the unified time window sequence number.

[0117] The process involves weighted fusion of the emotional features to be fused and the feature weight parameters to obtain the fused emotional features. Specifically, the emotional features to be fused and the feature weight parameters are matched to obtain the corresponding data for the feature weights; based on the corresponding data for the feature weights, the emotional features to be fused are weighted to obtain the weighted emotional features; the weighted emotional features are then integrated in the same direction and canceled out in the opposite direction to obtain the initial fused emotional features; finally, the initial fused emotional features are numerically unified and their integrity is verified to obtain the final fused emotional features.

[0118] Correspondence matching refers to binding each emotional feature to be fused with its corresponding weight to avoid weight mismatch. Weight assignment refers to multiplying the weight into the corresponding feature, so that features from different sources participate in the fusion according to their representational strength. Same-direction integration refers to integrating weighted emotional features with the same risk direction, while opposite-direction cancellation refers to canceling and correcting weighted emotional features with opposite risk directions according to their intensity relationship.

[0119] The following formula is used: ,

[0120] in, Indicates user In the Initial fusion of emotional characteristics within the time window, Represents the feature weight parameters. Indicates the first The emotional features to be fused in the first Risk direction indicators within the time window Indicates the first Feature values ​​of the emotion features to be fused This represents the set of data source categories participating in the fusion. Indicates user, Indicates the data source category. This indicates the unified time window sequence number.

[0121] If this characteristic indicates increased risk, then Take the positive sign; if the feature points to risk mitigation, then... Take the opposite. Through this calculation, the features in the same direction are accumulated and integrated, while the features in the opposite direction are canceled and corrected to obtain the initial fused emotional features.

[0122] In practice, the direction of change of weighted sentiment features is identified to obtain both unidirectional and inversely weighted sentiment features. The unidirectional weighted sentiment features are then accumulated and integrated to obtain a unidirectional integration result; the inversely weighted sentiment features are compared in intensity to obtain an inverse cancellation relationship; based on the inverse cancellation relationship, the unidirectional integration result is corrected to obtain a corrected fusion result; the completeness of the corrected fusion result is then verified to obtain the initial fused sentiment features. Completeness verification is used to determine whether the fusion result retains the contributions of all valid sources. If a source is not involved in the fusion due to missing data, the missing status is recorded, and the fusion result remains computable through weight redistribution of the participating sources.

[0123] Rule matching is performed on the fused emotional features to obtain feature-consistent and feature-conflicting states. Rule matching can be established based on common judgment logic used in psychological services, healthcare, and risk warning operations. For example, if increased physiological stress, increased negative textual expression, increased vocal depression, and abnormal behavioral rhythms occur simultaneously, it can be determined as a feature-consistent state; if physiological data shows increased risk, but textual and behavioral changes do not show corresponding changes, it can be determined as a feature-conflicting state; if vocal emotion is depressed but physiological and behavioral aspects remain stable, it can be determined as a conflicting state requiring calibration.

[0124] The system calibrates and determines feature consistency and conflict states to obtain a comprehensive emotion risk vector. When feature consistency is strong, the risk direction and intensity in the fused emotion features are enhanced. When feature conflict is significant, the fused emotion features are corrected based on the reliability of the feature source, persistence, and individual baseline deviation to avoid false alarms caused by a single anomalous feature. The comprehensive emotion risk vector includes risk intensity, risk direction, persistence, source contribution, and credibility, and is used for subsequent dynamic risk assessment.

[0125] A dynamic risk assessment is performed on the comprehensive emotional risk vector to obtain an emotional risk score. The dynamic risk assessment does not make a judgment at a certain point in time, but rather ranks, compares, analyzes deviations, analyzes the magnitude of changes, and analyzes the persistence of risk states within a continuous time window to obtain a score that reflects the evolution trend of emotional risk.

[0126] The comprehensive sentiment risk vectors are arranged sequentially according to their generation order to obtain a continuous risk vector sequence. This sequence preserves the user's risk change trajectory across multiple time windows. Adjacent comprehensive sentiment risk vectors in the continuous risk vector sequence are compared to obtain adjacent change data. This comparison may include differences in risk intensity, consistency of risk direction, changes in the main contributing factors, and extension of the duration of the state.

[0127] The direction of change of adjacent change data is confirmed to obtain direction comparison data; the change magnitude of the direction comparison data is correlated to obtain related change data; the continuity of the related change data is verified to obtain the data to be evaluated. The data to be evaluated includes the current risk status and the impact of previous risk status on the current risk.

[0128] Deviation analysis is performed on the data to be evaluated to obtain deviation results. These deviation results represent the difference between the comprehensive emotional risk vector and the user's normal risk state. Then, the magnitude and persistence of these deviation results are analyzed to obtain risk change results. The magnitude analysis is used to determine the rate of risk escalation; the persistence analysis is used to determine whether the risk is temporary or persistent. If the risk intensity is low but the duration is long, it may still be judged as requiring attention; if the risk intensity is momentarily high but not sustained, rule calibration is used to reduce the probability of false alarms.

[0129] The following formula is used:

[0130]

[0131] in, Indicates user In the Risk changes within the time window Indicates user In the The comprehensive sentiment risk vector within the time window, Indicates user Historical normal risk vector, Indicates user The comprehensive sentiment risk vector within the previous time window, Indicates user In the Risk persistence parameters within the time window This represents the contribution coefficient of deviation from the normal state. This represents the contribution coefficient of adjacent changes. This represents the contribution coefficient of the continuous state. Indicates user, This represents the unified time window sequence number. This formula allows us to simultaneously consider the current degree of anomaly, the rate of change, and its persistence.

[0132] A dynamic risk assessment is performed on the changes in risk to obtain an emotional risk score. This emotional risk score can be converted into a standardized score using a mapping function. The formula is as follows:

[0133] ,

[0134] in, Indicates user In the Emotional risk score within the time window Indicates the result of risk change. Indicates the rating bias. Indicates the scoring criteria item. Indicates user, This indicates the unified time window sequence number.

[0135] This scoring method can uniformly map risk changes of different modalities, intensities, and durations into an emotional risk score that can be used for grading and determination.

[0136] The emotional risk score is graded to obtain an emotional risk warning result. This step is used to convert continuous scores into an understandable, manageable, and traceable warning status.

[0137] First, the emotional risk score is validated to obtain a score to be judged. Validation is used to confirm whether the data source for the score is complete, whether the calculation process is continuous, and whether there are any abnormal jumps in the score. If the score is generated from severely missing data, the reliability of the label is low, and the output of a high-level warning is temporarily suspended.

[0138] The scores to be assessed are then matched against tiered boundaries to obtain risk level results. Risk level results include any one of the following: Normal, Attention, Warning, and Crisis. A Normal level indicates no significant emotional risk is currently identified; an Attention level indicates some emotional fluctuations requiring continued observation; a Warning level indicates a relatively significant risk requiring proactive management attention; and a Crisis level indicates a high level of emotional risk requiring immediate and strong intervention.

[0139] The risk level results and the scores to be judged are correlated and labeled to obtain the classification judgment data. The correlation label is used to record the score and level, as well as the main sources of risk, changing trends, and ongoing status, so that the management can understand the reasons for the warning. The warning status content is generated based on the classification judgment data to obtain the emotional risk warning result. The warning status content may include the user's risk level, risk changing trend, main sources of influence, recommended level of attention, and subsequent tracking status.

[0140] In practical applications, the classification is not determined by a single score. Boundary stability is assessed by considering continuous score changes to avoid frequent jumps in scores near adjacent level boundaries. For example, when the score is near the boundary between the concern level and the warning level, the final level is determined by combining the ongoing status and the direction of risk change. If the risk shows a continuously increasing trend, the warning sensitivity can be increased; if the risk shows a mitigating trend, observation can be maintained while reducing the probability of false alarms.

[0141] In this invention, each processing stage has a clearly defined data continuity relationship. The original multimodal data stream undergoes integrity verification, duplicate content identification, invalid content removal, noise reduction, abnormal fluctuation correction, time alignment, format unification, and normalization transformation to form standardized multimodal data. This standardized multimodal data is used to construct continuous user state data and further establish individual baselines. These individual baselines serve as important references for subsequent feature extraction, characterization intensity analysis, and dynamic risk assessment, enabling personalized judgments based on the long-term state differences of different users.

[0142] The multimodal emotion feature set is extracted from standardized multimodal data under individual baseline constraints, including the degree of deviation, direction of change, and state of continuous change. The emotion features to be fused are obtained from the multimodal emotion feature set after source differentiation and numerical unification. The feature weight parameters are obtained from the representational strength analysis of the emotion features to be fused. The fused emotion features are obtained by weighted fusion of the emotion features to be fused and the feature weight parameters. The comprehensive emotion risk vector is obtained from the fused emotion features through rule matching and calibration. The emotion risk score is obtained from the comprehensive emotion risk vector through dynamic risk assessment. The emotion risk warning result is obtained from the emotion risk score through graded determination.

[0143] Through the aforementioned data links, this invention avoids the shortcomings of existing technologies that rely on single physiological or semantic data for biased judgments, and also avoids the problem that uniform thresholds cannot adapt to individual differences. Because this invention introduces continuous permutation, correlation comparison, deviation analysis, change magnitude analysis, and continuous state analysis in the risk assessment stage, it can reflect the continuous evolution of emotional risk, rather than outputting a static judgment result at a single moment.

[0144] By employing the method of this invention, a unified analytical relationship can be established among multimodal data, thereby enabling... Physiological changes, changes in text expression, changes in voice emotion, changes in behavioral habits, and changes in scene labels are all incorporated into the same risk assessment framework. Data cleaning and standardization reduce the interference of noise, abnormal fluctuations, and invalid data on the early warning results. Individual baseline construction identifies changes in the user's own state, avoiding misjudgments caused by fixed thresholds. Weighted fusion at the feature layer assigns fusion weights based on the representational strength of different emotional features, highlighting key risk signals. Decision-level rule calibration professionally corrects consistent and conflicting features. Through dynamic risk assessment and hierarchical judgment, it outputs emotional risk early warning results that are continuous, interpretable, and targeted for appropriate action.

[0145] Therefore, this invention can achieve individualized identification, dynamic calculation, professional calibration, and hierarchical output of emotional risks, solving the problems of insufficient accuracy of single-modal early warning, fixed thresholds that are not easy to adapt to individual differences, lack of adaptive correlation calculation of multimodal features, and early warning results that do not accurately reflect the continuous evolution of risks in existing technologies.

[0146] Example 1: Personalized Recognition of Emotional Changes under Individual Baseline Constraints

[0147] This embodiment is used to verify that "the present invention adopts a multimodal feature extraction technology scheme with individual baseline constraints to achieve the effect of user state differentiation recognition technology, realize personalized representation of emotional changes, and solve the shortcomings of fixed threshold misjudgment."

[0148] In a campus psychological service setting, continuous emotional risk monitoring was conducted on three student users. Significant differences were observed in the physiological fluctuations, text expression habits, vocal expression, and behavioral rhythms of these three students in their daily lives. Using a uniform, fixed threshold for judgment could easily misclassify users with already significant emotional fluctuations as high-risk users, or overlook users whose normally low emotional state suddenly declines as normal users.

[0149] In this embodiment, multimodal emotion correlation data of student users is first collected to form a raw multimodal data stream. Then, integrity verification, duplicate content identification, invalid content removal, noise reduction, abnormal fluctuation correction, time alignment, format unification, and normalization conversion are performed to obtain standardized multimodal data. Finally, the standardized multimodal data of the same user are aggregated, and the user's stable variation range, daily variation range, and immediate variation state are calculated to construct a corresponding individual baseline.

[0150]

[0151] As shown in Table 1, students current While not the lowest value, it showed a significant decrease relative to the individual baseline, and both negative textual expression and behavioral deviation increased, resulting in the highest overall degree of deviation. (Student) current Value higher than students and students However, judging solely by absolute values ​​cannot reflect changes in state; this invention confirms student status by comparing with an individual baseline. The current changes are relatively small and should not be directly classified as high risk.

[0152]

[0153] This example demonstrates that the fixed threshold method easily overlooks the normal differences between different users. (Students) The absolute value may not meet the unified high-risk judgment criteria, but it has deviated significantly from its own individual baseline; students Some data meet the unified attention criteria, but the changes are not significant relative to their own baseline. This invention constructs an individual baseline and uses it as a constraint for feature extraction, making the identification of emotion changes closer to the user's real state and reducing false positives and false negatives caused by fixed thresholds.

[0154] Example 2: Multi-source collaborative judgment under feature layer weighted fusion and decision layer rule calibration

[0155] This embodiment is used to verify that "the present invention adopts a feature layer weighted fusion and decision layer rule calibration technology scheme to achieve the effect of unified association of multi-source features, realize collaborative judgment of emotional risk, and solve the problem of insufficient accuracy of single-modal early warning".

[0156] In healthcare monitoring scenarios, emotional risk monitoring is conducted for users with chronic diseases. The emotional state of these users is typically influenced by their physiological state, subjective expression, vocalizations, and behavioral rhythms. Judging solely based on physiological data might misinterpret physical discomfort as emotional risk; judging based on textual content might overlook physiological and behavioral abnormalities that the user has not actively expressed. Therefore, this embodiment employs a multimodal feature fusion and rule-based calibration approach for assessment.

[0157] First, multimodal emotion association data of users is collected, and standardized multimodal data is obtained through data cleaning and normalization. Then, an individual baseline is constructed, and a multimodal emotion feature set is extracted under the constraints of the individual baseline. For the multimodal emotion feature set, feature source differentiation and numerical unification are performed to obtain the emotion features to be fused. The representation intensity analysis of the emotion features to be fused is performed to obtain feature weight parameters. Then, weighted fusion is performed to obtain the fused emotion features. Through rule matching and calibration judgment, a comprehensive emotion risk vector is obtained.

[0158]

[0159] Table 1 shows that the user's text semantic features and The high intensity of physiological characteristics indicates that both subjective expression and physiological state point to a significant increase in risk; although there are some changes in vocal emotion characteristics and behavioral habits, their contribution is relatively low.

[0160]

[0161] As shown in Table 2, single Judging by the level of attention and the level of warning based on a single text, there is a possibility that the results may be influenced by the user's expression habits. After weighted fusion at the feature layer, a more stable fusion result is obtained by integrating data from multiple sources; further calibration through decision layer rules, due to... Both physiological and textual semantic features point to increased risk, and the voice emotional features do not form a significant counteracting effect, so the result is calibrated to a warning level.

[0162] As demonstrated in this embodiment, the present invention can assign weights, integrate in the same direction, and cancel out in opposite directions emotional features from different sources, and identify feature consistency and feature conflict states through rule matching. This approach avoids biased judgments caused by single-modal data and avoids the distortion of risk expression caused by simply adding multimodal data, thereby improving the accuracy and professionalism of emotional risk assessment.

[0163] Example 3: Continuous Evolution Early Warning Based on Dynamic Assessment of Comprehensive Emotional Risk Vector

[0164] This embodiment is used to verify that "the present invention adopts a comprehensive emotional risk vector dynamic assessment technology to achieve continuous identification of risk changes, realize continuous evolution early warning of emotional risk, and solve the shortcomings of existing early warning systems that are unable to reflect dynamic changes."

[0165] In the context of employee care within enterprises and institutions, continuous emotional risk monitoring is conducted on employee users. During periods of increased workload, heightened communication pressure, and changes in work and rest schedules, the employee's multimodal emotional correlation data showed continuous changes. Judging based on a single score might fail to detect the process of continuously increasing risk; issuing an alert only when a single score reaches a high value might miss the opportunity for early intervention.

[0166] In this embodiment, the continuously generated comprehensive emotional risk vectors are arranged in the order of their generation to obtain a continuous risk vector sequence. Corresponding terms of adjacent comprehensive emotional risk vectors are compared to obtain adjacent change data. The direction of change, magnitude of change correlation, and continuity verification of adjacent change data are then performed to obtain the data to be evaluated. Deviation analysis, magnitude of change analysis, and persistence analysis are conducted on the data to be evaluated to obtain risk change results, and based on these results, an emotional risk score and graded early warning results are generated.

[0167]

[0168] Table 1 shows that the overall emotional risk vector of this employee user is determined by the cycle. to the cycle Continued to rise, in the cycle While a high-level warning has not yet been reached, the magnitude of adjacent changes and the duration of the status values ​​have shown an increasing trend. This invention captures the gradual accumulation of risk through continuous arrangement and correlation comparison, rather than issuing a warning after the risk score reaches a high value.

[0169]

[0170] As shown in Table 2, the static single-judgment method has a periodicity. The lack of attention prompts can easily lead to missing early state changes; the dynamic assessment method of this invention, before the risk has significantly increased, confirms the periodicity through continuous change trends and persistent states. Identified as an interested state, and within the period Maintain follow-up to provide a basis for subsequent intervention.

[0171] As demonstrated in this embodiment, the present invention does not rely on an isolated judgment based on an emotional risk score at a single moment. Instead, it incorporates the continuous changes, adjacent magnitudes of changes, and persistent states of the comprehensive emotional risk vector into a dynamic risk assessment. This approach can identify the evolutionary process of emotional risk from slight fluctuations to sustained increases, ensuring that the warning results align with the user's actual state changes and addressing the shortcomings of existing warning systems that fail to reflect the continuous evolution of risk.

[0172] Embodiments of the present invention have been presented and described. It will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

[0173] The embodiments of the present invention described above are subject to modification and change of method by those skilled in the art without departing from the embodiments and broader aspects of the present invention. The appended claims are intended to include all such modifications and changes of method that do not depart from the present invention.

[0174] Those skilled in the art should understand that the embodiments of the present invention can be implemented using a pure hardware architecture, a pure software architecture, or an integrated hardware and software architecture. The present invention can be prepared as a computer program product, which can be stored in various non-volatile computer-readable storage media, including but not limited to solid-state drives, flash memory chips, mobile storage devices, optical discs, cloud storage servers, and other standardized storage media, and is not limited to traditional storage media.

[0175] The embodiments of the present invention described above are subject to modification and change of method by those skilled in the art without departing from the embodiments and broader aspects of the present invention. The appended claims are intended to include all such modifications and changes of method that do not depart from the present invention.

[0176] Based on the foregoing description in conjunction with the accompanying drawings, those skilled in the art will understand that the embodiments of this application can also be implemented by software programs. Therefore, this application also provides a computer-readable storage medium. This computer-readable storage medium stores computer-readable instructions thereon, which, when executed by one or more processors, implement the method described above in conjunction with the accompanying drawings.

[0177] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computing device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0178] It should be noted that although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. On the contrary, the steps depicted in the flowchart can be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0179] It should be understood that when the terms "first," "second," "third," and "fourth," etc., are used in the claims, specification, and drawings of this application, they are used only to distinguish different objects and not to describe a specific order. The terms "comprising" and "including" as used in the specification and claims of this application indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0180] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0181] Although the embodiments of this application are described above, the content is merely an example adopted for the purpose of facilitating understanding of this application and is not intended to limit the scope and application scenarios of this application. Any person skilled in the art described in this application may make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in this application, but the scope of patent protection of this application shall still be determined by the scope defined in the appended claims.

Claims

1. A dynamic early warning method for emotional risk based on multimodal data fusion, characterized in that, include: Collect users' multimodal emotion association data to obtain raw multimodal data stream. The multimodal emotion association data includes HRV physiological data reflecting the user's autonomic nervous system state, text semantic data reflecting the user's subjective expression content, voice emotion data reflecting the user's voice emotional state, behavioral habit data reflecting the user's daily life rhythm, and scene label data reflecting the user's application environment. The original multimodal data stream is cleaned and rectified to obtain standardized multimodal data; Using a pre-constructed individual baseline as a constraint, feature extraction is performed on the standardized multimodal data to obtain a multimodal emotion feature set; The multimodal emotion feature set is subjected to feature layer weighted fusion and decision layer rule calibration to obtain a comprehensive emotion risk vector; A dynamic risk assessment is performed on the comprehensive emotional risk vector to obtain an emotional risk score; The emotional risk score is graded to obtain an emotional risk warning result.

2. The dynamic early warning method for emotional risk according to claim 1, characterized in that, The original multimodal data stream is cleaned and rectified to obtain standardized multimodal data, including: The original multimodal data stream is subjected to integrity verification, and data segments lacking valid collected content are removed to obtain the data stream to be normalized; The data stream to be normalized is subjected to duplicate content identification and invalid content removal to obtain an effective multimodal data stream; The effective multimodal data stream is subjected to noise reduction and abnormal fluctuation correction to obtain a cleaned multimodal data stream; The cleaned multimodal data stream is time-aligned and format-unified to obtain a regularized multimodal data stream; The regularized multimodal data stream is normalized to obtain the standardized multimodal data.

3. The dynamic early warning method for emotional risk according to claim 1, characterized in that, Using a pre-constructed individual baseline as a constraint, feature extraction is performed on the standardized multimodal data to obtain a multimodal emotion feature set, including: The standardized multimodal data is aggregated to obtain continuous user state data; The individual baseline is obtained by calculating the stable range of change, daily range of change, and instantaneous change of state in the continuous state data of the user. Using the individual baseline as a reference, the deviation degree, direction of change, and continuous change state of the standardized multimodal data are extracted to obtain candidate emotion features; The candidate emotion features are subjected to consistency screening and invalid feature removal to obtain the multimodal emotion feature set.

4. The dynamic early warning method for emotional risk according to claim 1, characterized in that, The multimodal emotion feature set is subjected to feature-level weighted fusion and decision-level rule calibration to obtain a comprehensive emotion risk vector, including: The multimodal emotion feature set is distinguished by feature source and its numerical values ​​are unified to obtain the emotion features to be fused; The characterization intensity analysis of the emotional features to be fused is performed to obtain the feature weight parameters; The emotional features to be fused and the feature weight parameters are weighted and fused to obtain the fused emotional features; The fused emotional features are matched using rules to obtain feature-consistent states and feature-conflicting states; The feature consistency state and feature conflict state are calibrated and determined to obtain the comprehensive emotion risk vector.

5. The dynamic early warning method for emotional risk according to claim 1, characterized in that, A dynamic risk assessment is performed on the comprehensive emotional risk vector to obtain an emotional risk score, including: The comprehensive emotional risk vector is continuously arranged and correlated to obtain the data to be evaluated; The deviation of the data to be evaluated is analyzed to obtain the deviation results; The deviation results are analyzed for magnitude and duration to obtain risk change results; A dynamic risk assessment is performed on the risk changes to obtain the emotional risk score.

6. The dynamic early warning method for emotional risk according to claim 1, characterized in that, The process of grading and determining the emotional risk score to obtain an emotional risk warning result includes: The validity of the emotional risk score is verified to obtain the score to be judged; The score to be determined is matched with the grade boundary to obtain the risk level result, which includes any one of the following levels: normal level, attention level, early warning level and crisis level. The risk level results and the scores to be determined are correlated and labeled to obtain the classification determination data; Based on the grading and determination data, an early warning status is generated, and the emotional risk early warning result is obtained.

7. The dynamic early warning method for emotional risk according to claim 2, characterized in that, The effective multimodal data stream is subjected to noise reduction and abnormal fluctuation correction to obtain a cleaned multimodal data stream, including: A continuous analysis is performed on the effective multimodal data stream to obtain the relationship between changes in adjacent data. Based on the relationship between adjacent data changes, short-term jump data in the effective multimodal data stream are smoothly replaced to obtain a noise-reduced data stream; The variation range of the noise-attenuated data stream is compared to obtain abnormal fluctuation data; Based on the continuous data before and after the abnormal fluctuation data, the abnormal fluctuation data is numerically corrected to obtain a corrected multimodal data stream; The continuity of the corrected multimodal data stream is verified to obtain the cleaned multimodal data stream.

8. The dynamic early warning method for emotional risk according to claim 4, characterized in that, The characterization intensity analysis of the emotional features to be fused is performed to obtain feature weight parameters, including: Using the individual baseline as a reference, the degree of deviation of the emotional features to be fused is compared to obtain deviation intensity data; The deviation intensity data is analyzed to identify the direction of change, thereby obtaining directional characterization data; The direction characterization data is continuously verified to obtain continuous characterization data. The sustained characterization data is sorted by strength to obtain a feature intensity sequence; The feature intensity sequence is weighted to obtain the feature weight parameters.

9. The dynamic early warning method for emotional risk according to claim 4, characterized in that, The emotional features to be fused and the feature weight parameters are weighted and fused to obtain fused emotional features, including: The corresponding matching of the emotion features to be fused and the feature weight parameters is performed to obtain the feature weight corresponding data; Based on the data corresponding to the feature weights, the emotional features to be fused are weighted to obtain weighted emotional features; The weighted emotional features are integrated in the same direction and canceled out in the opposite direction to obtain the initial fused emotional features; The initial fused emotional features are numerically unified and their integrity is verified to obtain the fused emotional features.

10. The dynamic early warning method for emotional risk according to claim 9, characterized in that, The weighted emotional features are integrated in the same direction and canceled out in the opposite direction to obtain the initial fused emotional features, including: The direction of change of the weighted emotion features is identified to obtain same-direction weighted emotion features and opposite-direction weighted emotion features; The same-direction weighted emotional features are accumulated and integrated to obtain the same-direction integration result; Intensity comparison of the inversely weighted emotional features yields an inverse cancellation relationship; Based on the aforementioned reverse cancellation relationship, the same-direction integration result is cancelled and corrected to obtain a corrected fusion result; The integrity of the corrected fusion result is verified to obtain the initial fused emotional features.

Images