Community susceptible population emotion crisis dynamic monitoring and early warning method based on multi-modal interaction characteristics

Abstract

The invention provides a community susceptible population emotion crisis dynamic monitoring and early warning method based on multi-modal interaction characteristics, and relates to the technical field of intelligent emotion early warning. Comprising the following steps: constructing a multi-modal emotion interaction graph, introducing a self-organizing attention mechanism to realize node-dependent dynamic weighting, and establishing a time-aware migration updating mechanism to capture an emotional energy flow evolution trajectory; aiming at modal deficiency and semantic offset, designing a cross-modal compensation cooperation mechanism to generate deficiency features, constructing an energy propagation model to measure emotional disturbance accumulation and diffusion, and combining reinforcement learning to realize self-learning optimization of an early warning model so as to form a closed-loop dynamic emotional crisis monitoring and early warning system.

Description

Technical Field

[0001] This invention relates to the field of intelligent emotion early warning technology, and in particular to a method for dynamic monitoring and early warning of emotional crises among vulnerable groups in the community based on multimodal interaction features. Background Technology

[0002] With the widespread adoption of social media platforms, wearable devices, and smart interactive terminals, the emotional state of community members can be indirectly perceived through multiple modalities, including language, facial expressions, and physiological signals. Existing research often utilizes deep learning models to fuse multimodal data, enabling individual emotion recognition and group emotion analysis, and subsequently conducting mental health monitoring or emergency intervention. Some systems also incorporate time-series modeling and social network propagation characteristics to describe trends in emotion changes. However, these traditional methods largely rely on complete and synchronized data, which is insufficient for addressing the complexities of modal heterogeneity, information gaps, and dynamic emotional coupling in community settings, resulting in limited stability in recognition and early warning systems.

[0003] The existing technology has the following shortcomings:

[0004] The lack of self-organizing modeling of dynamic relationships between different modalities makes it difficult to capture the true structure of emotion propagation; it cannot maintain representational consistency in the case of missing data modalities or semantic noise, and emotional states are prone to drift; it is difficult to continuously predict emotion evolution by analyzing only a single time period or static features; it lacks a self-updating feedback mechanism, resulting in poor adaptability of the model to long-term evolution trends; and risk assessment is based on thresholds or static indicators, failing to reflect the temporal accumulation and propagation effect of emotional energy.

[0005] Therefore, existing community sentiment monitoring systems are limited in their early warning accuracy and real-time response capabilities when faced with real-world application scenarios involving modal heterogeneity, information gaps, and dynamic evolution. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a method for dynamic monitoring and early warning of emotional crises in vulnerable groups in the community based on multimodal interaction features. This invention solves the problem of inaccurate early warning in existing multimodal emotion monitoring that cannot stably cope with modal heterogeneity and dynamic evolution.

[0007] To achieve the above objectives, the present invention provides the following solution:

[0008] A method for dynamic monitoring and early warning of emotional crises in vulnerable community populations based on multimodal interaction features, comprising:

[0009] For heterogeneous modal data of individuals in the community, an evolvable graph structure of cross-modal relationships is constructed, and the self-organizing attention mechanism is used to dynamically adjust the dependency strength and propagation direction between different modalities to obtain a multimodal emotion interaction graph representation set;

[0010] Based on the multimodal emotion interaction graph representation set, a time-aware migration and update mechanism is established, and the evolution trajectory of emotion energy flow between modalities is captured in continuous time periods to obtain a temporal emotion evolution association representation.

[0011] To address the modality loss and semantic shift in the temporal emotion evolution association representation, a cross-modal compensation and collaboration mechanism is introduced to generate feature fragments of the missing modality by utilizing the semantic mutual guidance relationship between modalities, thereby obtaining a compensated and consistent temporal emotion state space.

[0012] Based on the compensated consistent temporal emotional state space, an energy propagation analysis model is constructed to measure the energy accumulation and diffusion of emotional disturbances in the interaction structure. When the propagation energy exceeds the adaptive threshold, abnormal nodes are identified, and hierarchical emotional crisis risk mapping results and corresponding initial early warning models are obtained.

[0013] The hierarchical emotional crisis risk mapping results and corresponding intervention feedback are input into the initial early warning model. The interaction structure, compensation mechanism and risk judgment parameters in the initial early warning model are updated according to the reinforcement learning mechanism to obtain a dynamic early warning model optimized by self-learning, so as to complete the monitoring and decision-making closed loop.

[0014] The present invention discloses the following technical effects:

[0015] This invention provides a method for dynamic monitoring and early warning of emotional crises in vulnerable community populations based on multimodal interaction features. The method includes: constructing an evolvable graph structure of cross-modal relationships for heterogeneous modal data of community individuals, and dynamically adjusting the dependency strength and propagation direction between different modalities using a self-organizing attention mechanism to obtain a multimodal emotion interaction graph representation set; establishing a time-aware migration and update mechanism based on the multimodal emotion interaction graph representation set, and capturing the evolutionary trajectory of emotional energy flow between modalities in continuous time periods to obtain a temporal emotion evolution association representation; and introducing a cross-modal compensation and collaboration mechanism to address modal missingness and semantic shift in the temporal emotion evolution association representation, thereby utilizing the semantic interaction between modalities. The feature fragments of the missing modality are generated by the guide relationship to obtain a compensated consistent temporal emotional state space. Based on the compensated consistent temporal emotional state space, an energy propagation analysis model is constructed to measure the energy accumulation and diffusion of emotional disturbances in the interaction structure. When the propagation energy exceeds the adaptive threshold, abnormal nodes are identified to obtain hierarchical emotional crisis risk mapping results and corresponding initial warning models. The hierarchical emotional crisis risk mapping results and corresponding intervention feedback are input into the initial warning model, and the interaction structure, compensation mechanism and risk judgment parameters in the initial warning model are updated according to the reinforcement learning mechanism to obtain a self-learning optimized dynamic warning model to complete the monitoring and decision-making closed loop. This invention achieves continuous dynamic modeling of the emotional state of individuals in a community under heterogeneous modal conditions by constructing an evolvable multimodal emotion interaction graph and a time-aware transfer mechanism. This effectively overcomes the problems of incomplete data modalities, weak temporal correlation, and difficulty in capturing the true structure of emotion propagation in existing technologies. By introducing a cross-modal compensation and collaboration mechanism, it improves the consistency and robustness of representation in scenarios with missing modalities and semantic shifts. Based on an energy propagation analysis model, it achieves quantitative modeling of the accumulation and diffusion process of emotional disturbances, transforming risk identification from static threshold judgment to dynamic energy-driven identification. Furthermore, by combining a feedback closed-loop model optimized by reinforcement learning, it possesses self-learning and adaptive evolution capabilities, continuously improving the accuracy of early warning and real-time response performance. This significantly enhances the early identification and intervention effect of community group emotional crises and strengthens the stability and reliability of the system in complex dynamic scenarios. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a method for dynamic monitoring and early warning of emotional crises among vulnerable groups in a community based on multimodal interaction features, provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] like Figure 1 As shown, this invention provides a method for dynamic monitoring and early warning of emotional crises among vulnerable groups in communities based on multimodal interaction features, including:

[0021] Step 100: For heterogeneous modal data of community individuals, construct an evolvable graph structure of cross-modal relationships, and use the self-organizing attention mechanism to dynamically adjust the dependency strength and propagation direction between different modalities to obtain a multimodal emotion interaction graph representation set;

[0022] Step 200: Based on the multimodal emotion interaction graph representation set, establish a time-aware migration update mechanism, and capture the evolution trajectory of emotion energy flow between modalities in continuous time periods to obtain a temporal emotion evolution association representation;

[0023] Step 300: To address the modal missingness and semantic shift in the temporal emotion evolution association representation, a cross-modal compensation and collaboration mechanism is introduced to generate feature fragments of the missing modalities by utilizing the semantic mutual guidance relationship between modalities, thereby obtaining a compensated and consistent temporal emotion state space.

[0024] Step 400: Based on the compensated consistent temporal emotional state space, construct an energy propagation analysis model to measure the energy accumulation and diffusion of emotional disturbances in the interaction structure, and when the propagation energy exceeds the adaptive threshold, mark abnormal nodes to obtain hierarchical emotional crisis risk mapping results and corresponding initial early warning models.

[0025] Step 500: Input the hierarchical emotional crisis risk mapping results and corresponding intervention feedback into the initial early warning model, and update the interaction structure, compensation mechanism and risk judgment parameters in the initial early warning model according to the reinforcement learning mechanism to obtain a self-learning optimized dynamic early warning model, so as to complete the monitoring and decision-making closed loop.

[0026] Furthermore, the implementation process of step 100 is as follows:

[0027] This embodiment first acquires heterogeneous modal data of individuals in the community, including text, speech, visual, and physiological data. These data are then processed through emotion feature extraction and unified dimensional encoding to obtain a preliminary modal feature representation set. In this process, a semantic feature extraction network is used to extract text semantic vectors, speech prosody and emotional frequency domain features, visual facial texture encoding, and emotional index features of physiological signals. Subsequently, a unified mapping layer is used to perform dimensional alignment and semantic normalization on all modal features, thereby obtaining comparable multimodal node feature vectors in a unified feature space, laying the foundation for subsequent cross-modal graph structure construction.

[0028] This embodiment uses the feature representations of each modality as graph nodes. Initial cross-modal connections are established based on semantic similarity and temporal dependencies. The initial association strength between nodes is calculated using a semantic similarity function, forming an initial graph representation with modal correspondence and temporal information. Based on this initial graph structure, this embodiment introduces a self-organizing attention mechanism. By calculating the dependency scores and propagation directionality between nodes, the connection weights and directional distributions of different modal nodes are adaptively adjusted from local to global levels to reflect the dynamic influence between modal pairs under real-time interaction and emotional resonance. After multi-layer attention iterations, an adaptively updated modal interaction graph structure is generated, enabling intelligent reconstruction of inter-modal dependency strength and propagation paths, thus more realistically describing the changes in emotional influence between heterogeneous modalities.

[0029] This embodiment further performs iterative updates, weight decay, and connection pruning operations on the modal interaction graph structure based on the continuous changes in the emotional states of individuals in the community over time. This captures the evolutionary trajectory of intermodal dependencies over time from a dynamic perspective, resulting in an evolvable cross-modal emotion association graph. Subsequently, feature aggregation and embedding training are performed on this evolvable graph. Deep interaction features are extracted through global topological constraints and semantic consistency optimization, thereby obtaining a multimodal emotion interaction graph representation set and achieving high-fidelity dynamic modeling of the multimodal emotional features of individuals in the community.

[0030] Specifically, in constructing the adaptively updated modal interaction graph structure, this embodiment introduces a self-organizing attention mechanism to dynamically adjust the dependency strength and propagation direction between nodes of different modalities. Specifically, this embodiment first performs feature normalization and dimension alignment on the feature vectors of each modal node in the initial graph representation, ensuring that features from text, speech, vision, and physiological modalities are comparable and scale-consistent within the same semantic space. Subsequently, this embodiment calculates the relevance score between any two nodes using the self-organizing attention mechanism. This mechanism is an unsupervised learning structure that automatically adjusts attention weights based on internal statistical relationships within the data. By establishing interaction mappings between multimodal features and calculating similarity relationships, it achieves adaptive evaluation of the dependency strength between nodes. Based on this, this embodiment uses an adaptive nonlinear function to map the relevance score into weighted coefficients, forming an initial attention weight matrix. This matrix is ​​used to measure the relative influence between nodes.

[0031] To address the temporal order and semantic associations between different nodes, this embodiment further constructs an information propagation direction model to determine the directionality and probability of emotion transmission. This model calculates the combined relationship between semantic relevance scores and temporal differences between nodes, combined with the intensity differences of node features, to generate a propagation direction strength value. A standardization function is used to limit the result to between zero and one to represent the probability of the propagation direction. The magnitude of the propagation direction value reflects the strength of the likelihood of emotional information being transmitted from one node to another. The semantic relevance score in the model is derived from the cosine similarity calculation in a unified space after multimodal feature encoding. The temporal difference is calculated based on the timestamps of node behavior or signal occurrences. The multimodal fusion feature vector is obtained through the aforementioned unified mapping layer. By analyzing the positive and negative relationships of the temporal differences, the system can distinguish the propagation order of emotion formation, thereby establishing a directional propagation relationship.

[0032] In this embodiment, the directional strength output by the propagation direction model is used to correct the connectivity in the initial attention weight matrix, thereby obtaining a directionally optimized weighted relation matrix. Based on this matrix, this embodiment performs iterative training through multiple attention layers. Each layer fine-tunes and corrects the residuals of the weight learning results of the previous layer to ensure the stability and continuity of information transmission. Finally, this embodiment applies the obtained iteratively optimized attention mapping to the initial graph structure to reconstruct and update the node connectivity, so that the connections between modalities can reflect both the semantic association strength and dynamically reflect the propagation path of emotional information in the time dimension. Through this process, the modal interaction graph structure can automatically evolve to adapt to the emotional propagation patterns under different time periods and multimodal input conditions, thereby achieving dynamic modeling of cross-modal associations and accurate expression of emotional state evolution.

[0033] Furthermore, the specific implementation process of step 200 is as follows:

[0034] This embodiment first organizes and segments the continuous input data stream based on the multimodal emotion interaction graph representation set, according to the timestamps and event identifiers attached to each modality's data, to construct a time-aware temporal structure. Specifically, based on temporal continuity and event trigger points, this embodiment divides the interaction data into several adjacent time periods, each containing multimodal node relationships and emotional features within the same time window. To maintain the temporal coherence between graph structures, this embodiment establishes a corresponding index mapping for the graph representation of each time period and organizes it into an interaction graph sequence with temporal indexes according to chronological order. This achieves an extension from static multimodal graphs to dynamic temporal representations at the data level, enabling subsequent models to capture the emotional evolution trend across time periods.

[0035] After obtaining the interaction graph sequence, this embodiment performs difference calculations on the node and edge layers for adjacent time periods to measure the changes in modal features and connection strengths over time. Node feature differences are obtained by calculating the rate of change of the feature vector of the same node in consecutive time periods, reflecting the short-term fluctuation trend of an individual's emotional state. Connection strength differences are used to measure the temporal changes in emotional dependencies between different nodes, characterizing the dynamic transfer of influence between modalities. Based on this, this embodiment generates a preliminary temporal migration weight matrix to record the rate of change and transformation direction of nodes and connections between adjacent time periods, thus providing basic parameter support for subsequent time-weighted aggregation. To further enhance sensitivity to the evolutionary process, this embodiment introduces a temporal attention mechanism, adaptively adjusting the weight distribution according to the importance of feature changes in each time period, so that the influence of feature migration in key time periods on the overall sequence is dynamically amplified.

[0036] This embodiment dynamically updates and continuously estimates the emotional energy flow between nodes based on the migration characteristics after time-weighted attention and the historically accumulated emotional energy state. Specifically, this embodiment uses the energy accumulation rate to measure the changing trend of node emotional intensity over a continuous period and maintains both short-term responsiveness and long-term stability through a time-weighted update rule. During the energy flow update process, the model tracks the energy input, output, and dissipation relationships between nodes of each modality in real time, thereby forming a complete temporal energy evolution trajectory. Finally, this embodiment aggregates the migration characteristics and corresponding energy states of each time period, and extracts a temporal emotion evolution correlation representation by learning the collaborative patterns between different modalities. This representation is used to characterize the dynamic dependency of emotion propagation in both time and modality dimensions, achieving continuous modeling and trend prediction of the emotional evolution process of community groups.

[0037] Furthermore, the specific implementation process of step 300 is as follows:

[0038] This embodiment first targets the multimodal features in the temporal emotion evolution association representation, performing modality integrity detection and semantic consistency analysis to determine whether there are modality missing or emotional semantic shifts in continuous time periods. Specifically, this embodiment performs statistical feature analysis on the feature vector distribution of each modality, identifying missing modalities and semantic drift phenomena through feature dimension anomaly detection and distribution offset calculation. The modality missing identifier set is used to mark modality nodes where data is missing due to incomplete collection or signal interruption within a specific time window, while the semantic offset feature set is used to characterize semantic inconsistencies between modalities caused by noise interference, differences in emotional expression, or cross-modal perception bias. This embodiment achieves accurate localization and measurement of modality anomalies through the acquisition of these two sets, providing a basis for subsequent compensation models.

[0039] This embodiment constructs a semantic cross-modal information recovery model based on a set of missing modal identifiers and a set of semantic offset features. The semantic cross-modal model is a structure capable of bidirectional semantic mapping and feature transfer between multiple modalities, used to generate and correct features from known modalities to missing modalities. This embodiment establishes a semantic cross-modal matrix based on the semantic association strength between different modalities to characterize the guiding relationship of each modality's features in the semantic space; each element in this matrix represents the semantic influence weight when a feature from one modality propagates to another. Subsequently, this embodiment uses a collaborative generative network to reconstruct the missing modal features. By inputting the semantic features of the associated modalities and using the weights of the cross-modal matrix to guide the generation process, the temporal feature segments corresponding to the missing modalities are recovered. The generated results are then interpolated and fused with the original features of the retained modalities to obtain a preliminary temporal feature representation containing compensation information.

[0040] After obtaining the initially compensated temporal feature representation, this embodiment designs an integrated consistency constraint and residual correction mechanism to further improve the consistency and semantic alignment of multimodal features. In the consistency constraint, this embodiment minimizes the distribution differences of features corresponding to different modalities in different time periods, ensuring that the compensated features maintain the same direction of change at the semantic level. Simultaneously, a residual correction unit is introduced to dynamically correct the error signal between the generated features and the original features, preventing new deviations from arising during the compensation process. After multiple rounds of alternating optimization, this embodiment obtains a consistent multimodal temporal feature set and maps it uniformly to the emotional state vector space using temporal aggregation and spatial mapping methods, thereby forming a compensated and consistent temporal emotional state space. This achieves complete modeling of temporal emotional information and cross-modal semantic alignment under conditions of modality loss and semantic shift.

[0041] Specifically, in constructing the semantic intermodal model in this embodiment, a semantic embedding mapping network is first designed for each modality feature in the modality missing identifier set and semantic offset feature set. The semantic embedding mapping network is a non-linear feature mapping structure that can transform the original emotional features of different modalities into a unified semantic space. Its function is to enable the semantic information of text, speech, vision, and physiological modalities to be interoperable and comparable through a unified semantic coordinate system. In this embodiment, a loss function-based alignment strategy is used during training to bring the semantic distributions of each modality embedding closer together, forming a standardized set of modality semantic embeddings, thus providing a foundation for subsequent cross-modal similarity measurement and semantic alignment.

[0042] This embodiment calculates the similarity measurement results between different modalities based on the obtained standardized modal semantic embedding set. It achieves cross-modal alignment relationships within a unified semantic space by minimizing the differences in semantic distribution, and forms a cross-modal semantic alignment matrix accordingly. This matrix records the consistency of semantic distribution between modal pairs; a larger value indicates greater semantic similarity. Subsequently, this embodiment establishes a semantic association weight learning model, which is an adaptive parameter update mechanism for learning the semantic dependency strength between modal embeddings. The basic principle of this model is to adjust the semantic association weights according to the alignment degree and deviation distribution between modalities, giving higher weights to highly correlated relationships. The model's calculation involves similarity calculation between related modal embedding vectors. The results are mapped using a normalization function to generate a weight parameter set. The normalization function ensures that the sum of all weight parameters is one, thereby ensuring that the semantic dependency strength between modalities is numerically comparable and constrained.

[0043] Based on the semantic association weight parameter set, this embodiment further designs a cross-modal mutual differentiation function, which describes a semantic guidance transmission relationship from one modality to another. The output of the cross-modal mutual differentiation function is the mutual differentiation mapping result between modal pairs, which can characterize the interaction strength and information transmission direction of the two modalities under semantic alignment conditions. To correct the offset of different modalities in the semantic center, this embodiment introduces an alignment bias in the mutual differentiation calculation. This bias is used to correct the semantic center drift caused by sampling differences or cognitive expression differences, making the mutual differentiation result more consistent with the semantic consistency law. This embodiment uses the hyperbolic tangent function to perform nonlinear mapping on the mutual differentiation output result to enhance the model's sensitivity to extreme values ​​and small differences. Finally, the obtained inter-modal semantic mutual differentiation result is expressed in a matrix to obtain the inter-modal semantic mutual differentiation matrix. Each element of this matrix represents the semantic mutual differentiation strength between two modalities; the larger the value, the stronger the influence of that modality on the other modality during semantic transmission. This embodiment achieves bidirectional transmission and dynamic balance of semantic information among multiple modalities through joint optimization of semantic embedding mapping network, association weight learning model and cross-derivative function, effectively improving the semantic consistency among modalities and the reliability of generating missing modalities.

[0044] Furthermore, the specific implementation process of step 400 is as follows:

[0045] This embodiment constructs an energy propagation analysis model based on a compensated, consistent temporal emotional state space to reveal the accumulation and diffusion patterns of emotional disturbances within the community interaction structure. Specifically, this embodiment first converts the emotional intensity value of each node and its connection weight in the interaction graph into corresponding emotional energy values, thus forming an initial energy distribution representation. Here, emotional intensity reflects the emotional activity level of a node at a specific moment, and interaction weights reflect the correlation strength of emotional influence between nodes. Through this mapping, multimodal emotional features can be uniformly measured in energy form, providing a quantitative basis for energy propagation analysis.

[0046] In the energy modeling process, this embodiment establishes a dynamic energy propagation model to describe the transmission and attenuation of emotional energy between nodes. This model characterizes the flow and redistribution of energy between adjacent nodes by defining a temporal update equation for node energy. The energy transfer rate in the model represents the efficiency of energy transmission from one node to its neighboring nodes. The diffusion path is determined by the node connection relationships in the interaction graph, and the feedback coefficient measures the degree of energy reinforcement caused by external disturbances or self-feedback mechanisms during propagation. Energy is simultaneously affected by the self-holding coefficient, propagation coefficient, and attenuation coefficient during transmission: the self-holding coefficient describes the emotional inertia within a node, i.e., the proportion of energy state maintained by the node without external stimuli; the propagation coefficient reflects the emotional coupling strength between nodes, determining the diffusion speed of energy in the network; and the attenuation coefficient simulates the natural dissipation of energy over time. In the iterative simulation phase, this embodiment performs multiple rounds of calculations on energy propagation based on the time series, gradually forming the energy accumulation value and diffusion range distribution of each node, thereby obtaining the time-series energy accumulation results to reflect the concentrated areas and energy peaks of community emotional propagation.

[0047] After obtaining the time-series energy accumulation distribution results, this embodiment designs an energy determination mechanism based on a dynamic adaptive threshold to achieve abnormal node detection and risk hierarchical mapping. By constructing an adaptive energy threshold function, the instantaneous energy value of a node is adaptively compared with the global energy average, comprehensively considering the energy fluctuation scale within the system to dynamically adjust risk sensitivity. The energy fluctuation scale parameter is automatically updated based on the variance of the overall system energy change within the time window, reflecting the stability level of the community emotional system. When the energy of a node exceeds the adaptive threshold, it is determined that the node has generated an abnormal energy response and is included in the abnormal node identifier set. Subsequently, this embodiment generates a hierarchical risk mapping network based on the node's energy level and adjacency relationship. The energy level is used to define the severity level of the crisis, and the adjacency relationship is used to associate the diffusion pathways between different risk nodes. Finally, this embodiment uses the hierarchical risk mapping results as input, combined with node risk levels and dynamic energy feedback, to establish an initial early warning model to achieve quantitative expression and early warning of emotional risk. Through this mechanism, the model can identify potential crisis nodes in a timely manner before the spread of emotions forms a high-energy cluster, providing a scientific basis and dynamic decision support for group emotional intervention.

[0048] Specifically, this embodiment first extracts the emotional intensity features of each node and their associated interaction weight parameters from the compensated consistent temporal emotional state space, forming a set of node emotional features and a set of interaction weights. The emotional intensity features reflect the emotional activity level of a node within a specific time window. They are calculated from the temporal emotional features after modality fusion, and their values ​​comprehensively consider factors such as semantic incentives, emotional amplitude, and modality consistency. The interaction weight parameters represent the degree of influence between a node and its associated modality during the emotional propagation process. These parameters are calculated using the association frequency and propagation intensity between nodes.

[0049] After obtaining the set of node emotion features and the set of interaction weights, this embodiment performs normalization and scaling to ensure comparability of data from different sources during the calculation process. The normalization operation includes interval scaling and standard deviation constraint adjustment. Interval scaling maps all feature values ​​to the range of zero to one, while standard deviation constraint eliminates the impact of variance differences between modal data on energy calculation. The normalized emotion feature vector and interaction weight vector will serve as the basic variables for energy calculation. Based on this, this embodiment establishes an energy mapping function to linearly combine emotion intensity and interaction weights, thereby obtaining the node energy calculation function. This function reflects the node's emotional energy level at the current moment. Emotion intensity represents the amount of emotion aroused within the node, and interaction weights represent the degree of external emotional influence. The linear combination of these two can accurately characterize the magnitude and distribution trend of emotional energy.

[0050] This embodiment utilizes the constructed node energy calculation function to perform individual energy calculations on all nodes in the compensated, consistent temporal emotional state space, and assigns energy values ​​based on network topology. The energy assignment process is implemented through topological mapping rules, that is, based on the node's position in the graph structure and the distribution of connection weights with neighboring nodes, the energy value of a single node is extended to the local structural range to form a complete node-level energy mapping result. Subsequently, this embodiment aggregates the node-level energy mapping results along the time dimension to form a continuous representation of energy changes over multiple time periods. Through this aggregation process, an overall energy distribution tensor is obtained. This tensor is used to record the spatial distribution pattern of emotional energy in different time windows, ultimately forming an initial energy distribution representation, providing a quantifiable modeling basis for subsequent energy propagation analysis and anomaly identification.

[0051] Furthermore, the specific implementation process of step 500 is as follows:

[0052] After obtaining the hierarchical emotional crisis risk mapping results, this embodiment inputs the mapping results and their corresponding intervention feedback into the initial early warning model. Through self-learning and dynamic parameter adjustment, the model achieves continuous optimization and closed-loop updates. Specifically, this embodiment utilizes a reinforcement learning mechanism to dynamically iteratively update the model's interaction structure, compensation mechanism, and risk judgment parameters, enabling the model to have adaptive learning capabilities. Reinforcement learning is a learning architecture that optimizes strategies based on environmental feedback. Its core idea is to guide the model towards convergence in the optimal risk identification direction by reinforcing or penalizing the target signal. In this embodiment, under this mechanism, intervention feedback is input into the model as an environmental feedback signal. Combined with changes in the response of risk nodes, the internal parameters of the early warning model are adjusted, allowing it to gradually improve the accuracy and response sensitivity of risk prediction through multiple rounds of training.

[0053] During model updates, this embodiment first calculates a feedback signal based on the changing trend of group emotions in the intervention feedback. This signal is used to quantify the reinforcing effect of external intervention measures on node risk changes. The feedback signal reflects the effectiveness of the intervention, i.e., the degree to which the node risk value declines after external behavioral regulation or emotional guidance. Then, this embodiment combines the energy change rate of nodes in adjacent time periods and assesses the dynamic level of risk fluctuations in the system by calculating the temporal difference of the intensity of emotional disturbances. The energy change rate, as a sensitive indicator of short-term risk, can reflect the rate of rise and fall of emotions, thereby allowing for real-time adjustments to the risk assessment.

[0054] This embodiment also introduces node memory representation to retain the model's historical state and semantic residual information during long-term learning. The memory representation mechanism enables the model to simultaneously consider historical emotional evolution trends and current risk characteristics when making prediction updates, achieving a smooth transition under temporal continuity. The feedback response sensitivity parameter controls the fusion strength of external feedback on model updates. When feedback signals fluctuate significantly, the model automatically adjusts the risk update step size based on sensitivity to prevent instability caused by over-adjustment. The energy change influence factor regulates the weight of energy perturbations in the risk update process, determining the importance of energy dynamic characteristics in overall risk identification. The memory update balance coefficient maintains a balance between short-term response and long-term memory, preventing the model from forgetting long-term patterns during continuous learning. This embodiment achieves self-learning optimization of the model through the combined effect of the above parameters, ultimately forming a self-learning optimized dynamic early warning model. This model can autonomously adjust its strategy during continuous monitoring and intervention feedback, completing a closed loop of dynamic identification, prediction, and decision-making for emotional crises.

[0055] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0056] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for dynamic monitoring and early warning of emotional crises among vulnerable groups in a community based on multimodal interaction features, characterized in that, include: For heterogeneous modal data of individuals in the community, an evolvable graph structure of cross-modal relationships is constructed, and the self-organizing attention mechanism is used to dynamically adjust the dependency strength and propagation direction between different modalities to obtain a multimodal emotion interaction graph representation set; Based on the multimodal emotion interaction graph representation set, a time-aware migration and update mechanism is established, and the evolution trajectory of emotion energy flow between modalities is captured in continuous time periods to obtain a temporal emotion evolution association representation. To address the modality loss and semantic shift in the temporal emotion evolution association representation, a cross-modal compensation and collaboration mechanism is introduced to generate feature fragments of the missing modality by utilizing the semantic mutual guidance relationship between modalities, thereby obtaining a compensated and consistent temporal emotion state space. Based on the compensated consistent temporal emotional state space, an energy propagation analysis model is constructed to measure the energy accumulation and diffusion of emotional disturbances in the interaction structure. When the propagation energy exceeds the adaptive threshold, abnormal nodes are identified, and hierarchical emotional crisis risk mapping results and corresponding initial early warning models are obtained. The hierarchical emotional crisis risk mapping results and corresponding intervention feedback are input into the initial early warning model. The interaction structure, compensation mechanism and risk judgment parameters in the initial early warning model are updated according to the reinforcement learning mechanism to obtain a dynamic early warning model optimized by self-learning, so as to complete the monitoring and decision-making closed loop.

2. The method for dynamic monitoring and early warning of emotional crises in vulnerable community populations based on multimodal interaction features according to claim 1, characterized in that, The heterogeneous modal data targeting community individuals is used to construct an evolvable graph structure for cross-modal relationships. A self-organizing attention mechanism is then used to dynamically adjust dependency strength and propagation direction across different modalities, resulting in a multimodal emotion interaction graph representation set, including: Heterogeneous modal data of individuals in the community were obtained, and preliminary modal feature representation sets were obtained by extracting emotional features and encoding unified dimensions respectively. Using each modal feature in the preliminary modal feature representation set as a node, an initial cross-modal connection relationship is established based on semantic similarity and temporal correlation to obtain an initial graph representation with modal association structure; Based on the initial graph representation, a self-organizing attention mechanism is introduced to dynamically adjust the dependency strength and propagation direction between different modal nodes, resulting in an adaptively updated modal interaction graph structure. Based on the changes in emotional state in the time series, the modal interaction graph structure is iteratively updated and the connection is pruned, and the evolution path between modalities is captured to obtain an evolvable cross-modal emotional association graph. The evolving cross-modal emotion association graph is aggregated and embedded for training, and its global semantic and topological features are extracted to obtain a multimodal emotion interaction graph representation set.

3. The method for dynamic monitoring and early warning of emotional crises in vulnerable community populations based on multimodal interaction features according to claim 1, characterized in that, The heterogeneous modal data includes: Text data, voice data, visual data, and physiological data.

4. The method for dynamic monitoring and early warning of emotional crises in vulnerable community populations based on multimodal interaction features according to claim 2, characterized in that, Based on the initial graph representation, a self-organizing attention mechanism is introduced to dynamically adjust the dependency strength and propagation direction between nodes of different modalities, resulting in an adaptively updated modal interaction graph structure, including: For each modal node in the initial graph representation, feature normalization and dimension alignment are performed to obtain a standardized set of node features; Based on the standardized node feature set, the correlation score between nodes is calculated using a self-organizing attention mechanism, and weighting coefficients are formed through an adaptive function mapping to obtain the initial attention weight matrix. Based on the initial attention weight matrix, and combined with the temporal order and semantic association between nodes, an information propagation direction model is established. The directional adjustment and weight redistribution are performed on the bidirectional links to obtain a weighted relation matrix with optimized direction. Based on the weighted relation matrix optimized in the aforementioned direction, iterative training and residual update are performed in multiple attention layers to obtain the iteratively optimized attention mapping structure; The iteratively optimized attention mapping structure is applied to the initial graph representation to adaptively adjust and reconstruct the node connection relationships in the graph, resulting in an adaptively updated modal interaction graph structure. The expression for the information propagation direction model is: ； in, For nodes To the node At any moment The direction and intensity of propagation; For nodes and Semantic relevance score between them; For nodes and The time difference between them; and They are nodes and At any moment Multimodal fusion feature vector; Indicates the magnitude of the vector; This is the Sigmoid function.

5. A method for dynamic monitoring and early warning of emotional crises in vulnerable community populations based on multimodal interaction features, as described in claim 1, is characterized in that... The process involves establishing a time-aware migration and update mechanism based on the multimodal emotion interaction graph representation set, capturing the evolutionary trajectory of intermodal emotion energy flow over continuous time periods, and obtaining a temporal emotion evolution correlation representation, including: For the multimodal emotion interaction graph representation set, the continuous data stream is divided into adjacent time periods based on timestamps and event identifiers, and an index mapping is established for the graph representation of each time period to obtain an interaction graph sequence with time-series index; Based on the interaction graph sequence with temporal index, the differential relationship between node features and connection strength between adjacent time periods is calculated, and a preliminary temporal migration weight matrix is ​​obtained. Based on the preliminary temporal transfer weight matrix, a temporal attention mechanism is introduced to weight and aggregate the feature importance of different time periods to obtain a time-weighted transfer feature representation. By utilizing the time-weighted migration feature representation and combining it with historically accumulated energy states, dynamic updates and continuous estimations are performed on the emotional energy flow between nodes to obtain the temporal energy evolution trajectory. Based on the temporal energy evolution trajectory, the migration characteristics and energy states of each time period are aggregated, and the correlation patterns between different modalities are extracted to obtain the temporal emotion evolution correlation representation.

6. The method for dynamic monitoring and early warning of emotional crises in vulnerable community populations based on multimodal interaction features according to claim 1, characterized in that, To address the modality loss and semantic shift in the temporal emotion evolution association representation, a cross-modal compensation and collaboration mechanism is introduced. This mechanism utilizes the semantic intermodal relationships to generate feature fragments of the missing modalities, resulting in a compensated and consistent temporal emotion state space. This includes: For the temporal emotion evolution association representation, the integrity and consistency of each modality feature are detected to obtain a modality missing identifier set and a semantic offset feature set; Based on the modality missing identifier set and semantic offset feature set, a semantic mutual differentiation model between different modalities is constructed, and the semantic mutual differentiation matrix between modalities is obtained; Based on the intermodal semantic cross-transformation matrix, a collaborative generative network is used to generate feature fragments of the missing modalities, which are then interpolated and fused with the original temporal features to obtain a preliminary compensated temporal feature representation. Based on the temporal feature representation of the preliminary compensation, consistency constraints and residual correction mechanisms are applied between different modes to obtain a set of multimodal temporal features after consistency optimization; The consistency-optimized multimodal temporal feature set is temporally aggregated and spatially mapped to obtain a compensated consistent temporal emotional state space.

7. A method for dynamic monitoring and early warning of emotional crises in vulnerable community populations based on multimodal interaction features, as described in claim 6, is characterized in that... Based on the modality missing identifier set and semantic offset feature set, a semantic intermodality differentiation model is constructed, resulting in an intermodality semantic intermodality differentiation matrix, including: For each modality feature in the modality missing identifier set and semantic offset feature set, a semantic embedding mapping network is constructed to convert the modality features into a unified semantic space representation, thereby obtaining a standardized modality semantic embedding set. Based on the standardized modal semantic embedding set, the similarity metric between different modalities is calculated, and alignment is achieved by minimizing the semantic distribution difference, resulting in a cross-modal semantic alignment matrix; Based on the cross-modal semantic alignment matrix, a semantic association weight learning model is established, and the modal semantic mapping is adaptively updated to obtain a set of semantic association weight parameters. Based on the semantic association weight parameter set, a cross-modal mutual differentiation function was designed, resulting in a cross-modal mutual differentiation mapping model; Based on the cross-modal mutual guidance mapping model, the intensity of mutual influence of features is calculated and expressed in a matrix form to obtain the intermodal semantic mutual guidance matrix; The expression for the semantic association weight learning model is: ； The calculation expression for the cross-modal mutual guidance mapping model is: ； in, For the first The embedding vector of a modality in a unified semantic space; For the first The semantic embedding vector of a modality; For the first Modality and the first The semantic association weights between modalities represent the degree of correlation between the two modalities after semantic alignment; Softmax(•) is the normalization function. for Modal pairs Cross-modal cross-conversion mapping results; This is the intermodal alignment bias, reflecting the amount of correction for semantic center deviation during the mutual guidance process; It is the hyperbolic tangent function.

8. A method for dynamic monitoring and early warning of emotional crises in vulnerable community populations based on multimodal interaction features, as described in claim 1, is characterized in that... Based on the compensated consistent temporal emotional state space, an energy propagation analysis model is constructed to measure the energy accumulation and diffusion of emotional disturbances in the interaction structure. When the propagation energy exceeds an adaptive threshold, abnormal nodes are identified, resulting in a hierarchical emotional crisis risk mapping result and a corresponding initial early warning model, including: For the compensated consistent temporal emotion state space, the node emotion intensity and interaction weight are converted into corresponding energy values ​​to obtain an initial energy distribution representation; Based on the initial energy distribution representation, an energy migration and attenuation equation between nodes is established, and the energy transfer rate, diffusion path and feedback coefficient are defined to obtain a dynamic model of energy propagation. Based on the energy propagation dynamic model, the energy flow within the time series is iteratively simulated to calculate the energy accumulation and diffusion range of the nodes, and the time series energy accumulation distribution results are obtained. Based on the time-series energy accumulation distribution results, an adaptive energy threshold function is constructed, and anomaly detection and labeling are performed on nodes whose energy exceeds the threshold, resulting in an abnormal node identifier set; Based on the abnormal node identifier set, a hierarchical risk mapping network is generated by combining energy level and node adjacency relationship, and a corresponding initial early warning model is established, thus obtaining the hierarchical emotional crisis risk mapping result and the corresponding initial early warning model. The expression for the dynamic model of energy propagation is: ； The expression for the initial early warning model is: ； in, For nodes At any moment The emotional energy value; For energy propagation nodes The renewal energy; For nodes and Connection weights in the interaction structure; For nodes The set of neighboring nodes; The energy self-sustaining coefficient represents the proportion of emotional inertia within a node; The energy propagation coefficient reflects the energy-star coupling strength between nodes; The energy decay coefficient; For nodes At any moment The level of risk; This represents the average energy level of the entire system. It is an adaptive energy fluctuation scale used to dynamically adjust risk sensitivity.

9. A method for dynamic monitoring and early warning of emotional crises in vulnerable community populations based on multimodal interaction features, as described in claim 8, is characterized in that... The process for compensating for the consistent temporal emotional state space involves converting node emotional intensity and interaction weights into corresponding energy values ​​to obtain an initial energy distribution representation, including: The emotional intensity features of each node and the interaction weight parameters of its connected modalities are extracted from the compensated consistent temporal emotional state space to obtain the node emotional feature set and the interaction weight set. Based on the set of node emotion features and the set of interaction weights, the numerical range is normalized and scaled to obtain a normalized emotion and weight vector. Based on the normalized emotion and weight vector, an energy mapping function is constructed, which linearizes the emotion intensity and interaction weight to obtain the node energy calculation function; Using the node energy calculation function, energy calculation is performed on all nodes in the state space, and energy values ​​are assigned according to the network topology to obtain node-level energy mapping results; The node-level energy mapping results are aggregated along the time dimension to form an overall energy distribution tensor, thus obtaining the initial energy distribution representation.

10. A method for dynamic monitoring and early warning of emotional crises in vulnerable community populations based on multimodal interaction features, as described in claim 9, is characterized in that... The expression for the self-learning optimized dynamic early warning model is: ； in, For nodes After self-learning optimization, at any time Dynamic early warning output; For feedback semaphores, representing the reinforcing effect of intervention results or group responses on changes in node risk; For nodes The rate of energy change in adjacent time periods is used to reflect the temporal difference of the intensity of emotional disturbance; For nodes The memory representation expresses the residual information of the model's internal historical state and semantic learning; To improve the sensitivity of the feedback response, the fusion strength of the feedback with risk updates is controlled; This is an energy change influencing factor, used to adjust the weight of energy disturbances in risk updates; The balancing coefficients are used to update the memory, which is used to maintain the stability of the model's memory of long-term evolutionary trends.

Images