A mental state dynamic monitoring and cross-modal early warning method and system

Abstract

The application provides a psychological state dynamic monitoring and cross-modal early warning method and system, relates to the technical field of data fusion analysis and early warning, and comprises the following steps: preprocessing acquired multi-modal behavior data of a target object to obtain time sequence alignment data; extracting time sequence behavior characteristics based on the time sequence alignment data; determining a baseline offset according to the time sequence behavior characteristics and a preset individual normal behavior baseline; weighting and fusing the baseline offset, the behavior evolution slope and the abnormal duration to determine a comprehensive risk degree, wherein the comprehensive risk degree represents the degree of personal psychological risk; determining grading early warning data according to the comprehensive risk degree and the abnormal duration, and performing early warning according to the grading early warning data. The application can realize high-precision, low-false-alarm, long-period and front-end psychological risk dynamic monitoring and early warning based on the fusion analysis of multi-modal behavior data.

Description

Technical Field

[0001] This invention relates to the field of data fusion analysis and early warning technology, and more specifically, to a method and system for dynamic monitoring of psychological states and cross-modal early warning. Background Technology

[0002] The deterioration of the psychological state of target groups such as adolescents is highly time-dependent and insidious. The vast majority of high-risk individuals are not triggered by a single extreme expression, but rather gradually develop through continuous behavioral regression over 72 hours or more, manifesting as dissociative behaviors such as prolonged intervals between interactive responses, decreased frequency of proactive interactions, and long-term inconsistency between body language and expression.

[0003] The relevant psychological monitoring technologies mainly adopt static single-frame emotion classification, general fixed threshold judgment, pure physiological signal monitoring or simple splicing of multimodal features. Most of them can only identify extreme emotions that have already erupted, and cannot track gradual and continuous behavioral abnormalities, resulting in a large number of psychological risks being missed, inaccurate judgments, and missing the best window of early intervention. Summary of the Invention

[0004] The problem that this invention aims to solve is at least one of the following technical issues: inability to track progressive and persistent behavioral anomalies, easy misjudgment or omission, and untimely intervention.

[0005] To address the aforementioned problems, in a first aspect, the present invention provides a method for dynamic monitoring and cross-modal early warning of psychological states, comprising: The acquired multimodal behavior data of the target object is preprocessed to obtain time-aligned data; Temporal behavioral features are extracted based on the temporal alignment data. These features include at least temporal degradation features reflecting the degree of behavioral activity degradation and evolutionary trend features reflecting the long-term trend of behavioral changes. The evolutionary trend features include behavioral evolution slope and abnormal duration. The behavioral evolution slope represents the changes in an individual's behavior over time. The abnormal duration represents the duration of abnormal behavior within the individual's behavior. Based on the temporal behavioral characteristics and the preset personalized normal behavior baseline, a baseline offset is determined; the baseline offset represents the degree of deviation between the individual behavior and the personalized normal behavior. The baseline offset, the slope of behavioral evolution, and the duration of the anomaly are weighted and fused to determine the overall risk level; the overall risk level represents the degree of individual psychological risk. The graded early warning data is determined based on the comprehensive risk level and the duration of the anomaly, and an early warning is issued based on the graded early warning data.

[0006] In practical application, this embodiment obtains time-aligned and format-uniform time-series aligned data through compliant collection and preprocessing of multimodal behavioral data. This provides standardized input for subsequent cross-modal analysis, solving the problem that raw multi-source data cannot be directly analyzed together, and laying a data foundation for high-precision monitoring. By extracting time-series degradation features and evolutionary trend features from the time-series aligned data, the raw behavioral data is transformed into quantifiable behavioral activity degradation indicators and long-term change trend indicators (where the evolutionary trend features include behavioral evolution slope and abnormal duration). This solves the problem that related technologies can only identify instantaneous emotions and cannot capture continuous behavioral degradation, achieving long-term, proactive risk identification. By comparing time-series behavioral features with personalized normal behavioral baselines, the baseline offset is determined, using the target object itself as the base. This invention achieves accurate anomaly detection, resolving the issue that general fixed thresholds cannot adapt to individual differences, and further improving the accuracy of personalized monitoring. By combining the established behavioral evolution slope and anomaly duration, a weighted fusion of three indicators yields a comprehensive risk level. Psychological risk is comprehensively assessed from three dimensions: deviation magnitude, trend of change, and duration, addressing the problem of misjudgment caused by single-dimensional assessment. Furthermore, by implementing tiered early warning based on the comprehensive risk level and anomaly duration, and using dual thresholds to filter temporary fluctuations and differentiate risk levels, the high false alarm rate of related technologies is resolved, achieving low-false-alarm routine monitoring. This facilitates early warning and timely intervention. In summary, this invention, based on the fusion analysis of multimodal behavioral data, enables high-precision, low-false-alarm, long-term, and proactive dynamic monitoring and early warning of psychological risks in scenarios such as school mental health education.

[0007] Secondly, the present invention also provides a dynamic monitoring and cross-modal early warning system for psychological states, which applies the dynamic monitoring and cross-modal early warning method for psychological states as described in any of the above claims, including: The preprocessing module is used to preprocess the acquired multimodal behavior data of the target object to obtain time-aligned data. The feature extraction module is used to: extract temporal behavioral features based on the temporal alignment data, wherein the temporal behavioral features include at least temporal degradation features reflecting the degree of degradation of behavioral activity and evolutionary trend features reflecting the long-term trend of behavioral change; the evolutionary trend features include behavioral evolution slope and abnormal duration; the behavioral evolution slope represents the change of individual behavior on the time axis; the abnormal duration represents the duration of abnormal behavior in the individual behavior; The baseline matching module is used to: determine the baseline offset based on the time-series behavioral characteristics and a preset personalized normal behavior baseline; the baseline offset represents the degree of deviation between the individual behavior and the personalized normal behavior. The fusion calculation module is used to: perform weighted fusion of the baseline offset, the behavioral evolution slope, and the duration of the anomaly to determine the comprehensive risk level; the comprehensive risk level represents the degree of individual psychological risk. The graded early warning module is used to: determine graded early warning data based on the comprehensive risk level and the duration of the anomaly, and issue early warnings based on the graded early warning data.

[0008] Thirdly, the present invention provides an electronic device, including a memory and a processor; The memory is used to store computer programs; The processor is configured to implement the method for dynamic monitoring of mental states and cross-modal early warning as described in the first aspect when executing the computer program.

[0009] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for dynamic monitoring of psychological states and cross-modal early warning as described in the first aspect.

[0010] The psychological state dynamic monitoring and cross-modal early warning system, electronic device and computer-readable storage medium provided by this invention have the same beneficial effects as the psychological state dynamic monitoring and cross-modal early warning method compared with the prior art, and will not be repeated here. Attached Figure Description

[0011] Figure 1 A flowchart illustrating the dynamic monitoring and cross-modal early warning method for psychological states in an embodiment of the present invention is shown; Figure 2 A timing diagram of the dynamic monitoring and cross-modal early warning method for psychological states in an embodiment of the present invention is shown; Figure 3 A schematic diagram of the structure of the dynamic monitoring and cross-modal early warning system for psychological states in an embodiment of the present invention is shown; Figure 4 A schematic diagram of the structure of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0012] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0013] It should be noted that relational terms such as "first" and "second" in this invention are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0014] In the description of this specification, references to terms such as "embodiment," "one embodiment," and "one implementation" indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or implementation is included in at least one embodiment or illustrative implementation of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or implementation. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or implementations.

[0015] Reference Figure 1 As shown in the figure, this invention proposes a method for dynamic monitoring of psychological states and cross-modal early warning; The method for dynamic monitoring and cross-modal early warning of psychological states includes: S100: Preprocess the acquired multimodal behavior data of the target object to obtain time-aligned data.

[0016] Specifically, during the data acquisition phase, informed consent from the target subject or their legal guardian is required. The scope, purpose, and privacy protection measures for data collection must be clearly stated. The data collection process adheres to the principle of minimum necessity, collecting only behavioral data directly related to psychological state monitoring. The collected content undergoes anonymization (e.g., anonymizing text content, extracting only acoustic features from speech data, and processing visual data on-device without retaining the original images). Multimodal behavioral data refers to behavioral data collected from two or more sensory channels, such as text interaction data, speech signal data, and visual body posture data. Preprocessing includes data cleaning and data alignment: data cleaning removes noise, null values, and outliers from the original data; data alignment synchronizes and calibrates the timestamps of different modalities to eliminate time discrepancies. After preprocessing, time-series aligned data that is strictly corresponding in time and uniform in numerical values ​​is obtained.

[0017] S200: Extract temporal behavioral features based on the temporal alignment data. The temporal behavioral features include at least temporal degradation features reflecting the degree of degradation of behavioral activity and evolutionary trend features reflecting the long-term trend of behavioral change. The evolutionary trend features include behavioral evolution slope and abnormal duration. The behavioral evolution slope represents the change of individual behavior on the time axis. The abnormal duration represents the duration of abnormal behavior in the individual behavior.

[0018] Specifically, temporal behavioral features are a set of quantitative indicators extracted from temporally aligned data that characterize the behavioral state of a target object. Temporal degradation features quantify the degree of degradation in behavioral activity, such as slower interaction response, decreased frequency of proactive interaction, and prolonged periods of inactivity. Evolutionary trend features quantify the long-term trajectory of behavioral changes. The behavioral evolution slope reflects the trend of behavior over time; for example, the rate of change is obtained by linearly fitting feature values ​​within a continuous time window, with negative values ​​indicating continuous deterioration and positive values ​​indicating gradual recovery. Abnormal duration reflects the duration of the abnormal state, starting from the first detection of the abnormality until its end, used to determine whether the current abnormality is a short-term fluctuation or a persistent stability risk. Both features jointly depict the evolutionary process of psychological state from normal to abnormal. Temporal degradation features and evolutionary trend features describe the behavioral state of the target object from the dimensions of current activity and long-term trend, respectively, providing quantitative basis for subsequent baseline comparison and risk assessment.

[0019] S300: Determine the baseline offset based on the time-series behavioral characteristics and the preset personalized normal behavior baseline; the baseline offset represents the degree of deviation between the personal behavior and the personalized normal behavior.

[0020] Specifically, the personalized normal behavior baseline is a pre-established personalized reference benchmark for the target individual, representing their behavioral pattern under normal conditions. The baseline offset is the difference value obtained by comparing the real-time extracted temporal behavioral features with this baseline, used to determine whether the current behavior deviates from the individual's normal state; the greater the deviation, the more significant the abnormality. Combining the behavioral evolution slope and abnormal duration determined in S200, the three indicators assess psychological risk from three dimensions: deviation magnitude, trend of change, and duration of duration, respectively, and serve as inputs for subsequent weighted fusion.

[0021] S400: The baseline offset, the behavior evolution slope, and the duration of the anomaly are weighted and fused to determine the overall risk level.

[0022] Specifically, weighted fusion calculation is the process of combining three dimensions of indicators into a single risk value. Since the three indicators have different dimensions and numerical ranges, they need to be normalized first to ensure they fall within the same comparable range. Then, based on the importance of each indicator in the risk assessment, a corresponding preset weight coefficient is assigned to each indicator (the sum of the three is 1). The three normalized indicators are then multiplied by their respective weight coefficients and summed to obtain the comprehensive risk score. The comprehensive risk score can be a continuous value between 0 and 100, directly reflecting the target's current psychological risk level; a higher score indicates a greater risk.

[0023] S500: Determine graded early warning data based on the comprehensive risk level and the duration of the anomaly, and issue an early warning based on the graded early warning data.

[0024] Specifically, the tiered early warning system employs a dual-condition judgment mechanism, considering both the degree of risk and its duration. Multiple risk levels can be preset, each corresponding to a different range of overall risk level and duration of abnormality. When both the overall risk level and the duration of abnormality meet the judgment criteria for a particular level, the tiered early warning data for that level is determined. Different levels correspond to different early warning strategies; for example, low-level warnings may only require recording or a light reminder, while high-level warnings trigger immediate notifications and intervention processes. Early warning data may include information such as target object identification, overall risk level, duration of abnormality, and risk level to support subsequent intervention decisions.

[0025] In practical application, this embodiment obtains time-aligned and format-uniform time-series aligned data through compliant collection and preprocessing of multimodal behavioral data. This provides standardized input for subsequent cross-modal analysis, solving the problem that raw multi-source data cannot be directly analyzed together, and laying a data foundation for high-precision monitoring. By extracting time-series degradation features and evolutionary trend features from the time-series aligned data, the raw behavioral data is transformed into quantifiable behavioral activity degradation indicators and long-term change trend indicators (where evolutionary trend features include behavioral evolution slope and abnormal duration). This solves the problem that related technologies can only identify instantaneous emotions and cannot capture continuous behavioral degradation, achieving long-term, proactive risk identification. By combining time-series behavioral features with... By comparing personalized routine behavior baselines to determine baseline deviations, and using the target individual as a benchmark for anomaly detection, the problem of general fixed thresholds failing to adapt to individual differences is solved, further improving the accuracy of personalized monitoring. Combining the established behavioral evolution slope and anomaly duration, three indicators are weighted and fused to obtain a comprehensive risk level. Psychological risk is comprehensively assessed from three dimensions: deviation magnitude, trend of change, and duration, solving the problem of misjudgment caused by single-dimensional assessment. By implementing tiered early warnings based on comprehensive risk level and anomaly duration, and using dual thresholds to filter temporary fluctuations and differentiate risk levels, the problem of high false alarm rates in related technologies is solved, achieving low-false-alarm routine monitoring, facilitating early warning and timely intervention.

[0026] In summary, this invention, based on the fusion analysis of multimodal behavioral data, enables high-precision, low-false-report, long-term, and proactive dynamic monitoring and early warning of psychological risks in scenarios such as campus mental health education, providing stable, reliable, and efficient technical support for the mental health protection system of the target group.

[0027] As an optional embodiment of the present invention, the personalized normal behavior baseline is extracted from a pre-established temporal behavior baseline library, and the method further includes: Acquire incremental data, which includes one or more of the following: time-series aligned data, time-series behavioral features, baseline offset, behavioral evolution slope, anomaly duration, comprehensive risk level, and graded early warning data.

[0028] Specifically, after each monitoring cycle, the system automatically collects all core data generated during the monitoring process as incremental data. This includes: time-series alignment data (raw preprocessed data); time-series behavioral characteristics (time-series degradation characteristics, synchronization anomaly characteristics, and evolutionary trend characteristics); synchronization characteristic data (overall modal synchronization coefficient, modal offset value, and number of consistency collapses); baseline offset, behavioral evolution slope, and anomaly duration as the three core inference indicators; comprehensive risk level (comprehensive risk score); classification results as normal, watchful, or high-risk; and early warning records including generated early warning data and push notifications. This incremental data forms the basis for subsequent baseline updates and the storage of time-series behavioral trajectories.

[0029] The personalized normal behavior baseline corresponding to the time-series behavior baseline library is adaptively updated based on the incremental data.

[0030] Specifically, incremental data is stored in the temporal behavior archive of the corresponding target object in the temporal behavior baseline library, forming a complete long-term behavior trajectory. Based on the normal behavior data in the incremental data (such as the temporal behavior characteristics when the classification result is normal), the personalized normal behavior baseline is adaptively and iteratively updated through an unsupervised learning algorithm, so that the baseline can be dynamically adjusted as the normal behavior of the target object changes, including updating the core parameters of the baseline, the baseline generation time, and update records.

[0031] In practical application, this embodiment utilizes closed-loop feedback of incremental data to store all time-series aligned data, time-series behavioral characteristics, synchronization characteristic data, three major inference indicators, comprehensive risk level, graded judgment results, and early warning records generated from each monitoring session into a time-series behavioral baseline library, forming a complete long-cycle behavioral trajectory. Based on normal behavioral data, a personalized normal behavioral baseline is adaptively updated using an unsupervised algorithm, enabling the baseline to dynamically adjust as the target object's behavior changes. This solves the problem of rigidity in traditional fixed threshold judgment standards and achieves precise anomaly identification tailored to each individual.

[0032] As an optional embodiment of the present invention, the temporal behavior baseline library also stores temporal behavior trajectories; The step of adaptively updating the personalized normal behavior baseline in the time-series behavior baseline library based on the incremental data includes: The time-series behavior trajectory is updated based on the incremental data.

[0033] Specifically, incremental data generated in each monitoring cycle is appended and stored in the time-series behavior trajectory of the corresponding target object, forming a complete long-term behavior record. The time-series behavior trajectory is organized in chronological order and includes raw preprocessed data (time-series aligned data), extracted feature vectors (time-series behavior features and synchronization feature data), risk inference results (baseline offset, behavior evolution slope, and duration of anomalies), comprehensive risk score, classification judgment results, early warning records, and manual intervention records, providing historical data support for subsequent baseline updates and trend analysis.

[0034] Based on the temporal behavior features used to characterize normal behavior in the temporal behavior trajectory, the personalized normal behavior baseline is adaptively and iteratively updated using an unsupervised algorithm, wherein the updated personalized normal behavior baseline is used to extract the baseline feature vector corresponding to the personalized normal behavior.

[0035] Specifically, data representing normal behavior is selected from the time-series behavioral trajectories; that is, the time-series behavioral characteristics and synchronous characteristic data when the classification result is normal. This normal behavior data is used as training samples, and the normal behavior distribution of the target object is recalculated using unsupervised learning algorithms (such as clustering algorithms or incremental learning algorithms) to generate an updated personalized normal behavior baseline. The updated baseline includes updated baseline core parameters, baseline generation time, and update records, which are used to extract baseline feature vectors for use in subsequent monitoring cycles.

[0036] The baseline offset is the deviation of the temporal feature vector corresponding to the temporal behavior feature relative to the baseline feature vector.

[0037] Specifically, the baseline offset is obtained by calculating the distance between the time-series feature vector and the baseline feature vector (e.g., weighted Euclidean distance). The time-series feature vector is obtained by integrating the time-series behavioral features of the current monitoring period, while the baseline feature vector is extracted from the updated personalized normal behavior baseline. The baseline offset quantifies the degree of deviation between the current behavior and the individual's normal behavior and is one of the core inputs for calculating the comprehensive risk level.

[0038] In practical application, this embodiment records long-term behavioral data of the target object through time-series behavioral trajectories. This includes raw preprocessed data, feature vectors, risk inference results, comprehensive risk scores, grading judgment results, early warning records, and manual intervention records, forming a complete closed loop of "monitoring—early warning—intervention—feedback." Based on normal behavioral data, an unsupervised algorithm adaptively updates the personalized normal behavioral baseline, enabling the baseline to match the target object's natural behavioral changes in real time. The baseline offset, as the deviation between the time-series feature vector and the baseline feature vector, accurately quantifies the degree of deviation between the current behavior and the individual's normal state, providing a personalized judgment benchmark for identifying psychological anomalies.

[0039] As an optional embodiment of the present invention, the multimodal behavioral data includes at least text interaction trajectories, speech rhythm features, and visual body postures. The preprocessing of the acquired multimodal behavioral data of the target object to obtain time-series aligned data includes: The multimodal behavior data is subjected to at least time alignment processing to obtain the time-aligned data.

[0040] Specifically, taking students as the target group as an example, multimodal time-series data acquisition units deployed on campus student interaction terminals (including online mental health education platforms and offline intelligent sensing devices) acquire multimodal behavioral data of the target group at a millisecond-level acquisition frequency, covering the following three types of data: Text interaction trajectory: Collect anonymized response content (only retain non-semantic features such as word frequency statistics, and do not store the complete original dialogue text), sending interval, and interaction frequency.

[0041] Speech rhythm features: This method does not involve speech privacy content, but only extracts acoustic feature parameters such as speech rate, intonation, pause duration, and volume changes. It does not collect speech content or perform speech recognition, ensuring that the original speech cannot be reproduced.

[0042] Visual posture representation: Facial micro-expressions, body movements, and sitting postures are collected after authorization in compliant scenarios. Among them, facial micro-expression collection adopts an edge processing method, and feature extraction is completed locally on the collection terminal. Only the intensity value of micro-expression action units (such as the intensity of frowning and the intensity of mouth corners) is output. No original facial images or video streams are stored or transmitted. The original data is destroyed locally immediately after feature extraction. If the scenario does not meet the conditions for compliant collection (such as the inability to obtain authorization or the existence of privacy risks), facial micro-expressions are not collected, and only body movements and sitting postures are collected.

[0043] The data collection process follows the principle of minimum necessity, collecting only behavioral data directly related to psychological state monitoring and excluding sensitive personal identification information (such as real name, ID number, home address, etc.). All data is encrypted during transmission and stored using hierarchical encryption, with student identification data stored separately from behavioral data. The data collection process preserves the original data timestamps and behavioral continuity.

[0044] Secondly, a sliding time window is used to segment the original continuous time series data. The window length and sliding step can be configured according to actual needs (e.g., window length of 30 minutes and sliding step of 10 minutes). Preprocessing operations such as denoising, null filling and outlier removal are performed on each time series segment.

[0045] The above ensures that data acquisition complies with relevant laws and regulations.

[0046] Then, a normalization algorithm is used to map time series data of different dimensions to the 0-1 interval, eliminating the impact of data dimension differences on subsequent calculations.

[0047] Finally, using the system's global unified clock as the anchor point, the three types of heterogeneous time-series data—text, speech, and body posture—are rigidly aligned frame by frame to eliminate time delay deviations caused by multi-device and multi-modal acquisition, ensuring one-to-one correspondence of the three types of behavioral data at the same time node, and obtaining time-aligned data (including synchronized time-series data of text interaction trajectory, speech rhythm features, and visual body posture performance).

[0048] In practical applications, this embodiment involves a series of preprocessing steps, including data acquisition, slicing, denoising, normalization, and time axis alignment, to transform the original heterogeneous multimodal data into time-aligned data with a unified time reference, unified dimensions, and standardized format. This provides accurate and standardized data input for subsequent behavioral synchronization calculation, temporal behavioral feature extraction, and baseline comparison. This preprocessing process ensures strict consistency of multimodal data in the time dimension, which is the foundation for subsequent cross-modal synchronization anomaly identification and evolution trend analysis.

[0049] As an optional embodiment of the present invention, the extraction of temporal behavior features based on the temporal alignment data includes: Using a sliding time window as a unit, determine the behavioral synchronization degree between the temporal alignment data corresponding to each of the two items in the text interaction trajectory, the speech rhythm features, and the visual body posture.

[0050] Specifically, behavioral synchronization is calculated using a cosine similarity algorithm. For any two modalities (such as text-speech), the two types of time-series data within the same time window are represented as vectors, and the cosine angle between the two vectors is calculated as the behavioral synchronization of that modal pair.

[0051] Behavioral synchronicity is used to quantify the degree of consistency of different modal behaviors over the same time dimension. Using a sliding time window (e.g., a 30-minute window or a 10-minute step) as the unit, behavioral synchronicity is calculated for three types of data pairs: text-speech, text-body language, and speech-body language. Each synchronicity value is a value between 0 and 1, with values ​​closer to 1 indicating stronger consistency and values ​​closer to 0 indicating a higher degree of dissociation.

[0052] By combining the synchronization degree of each behavior and the normal duration threshold, the abnormality judgment result of the individual behavior is determined and the abnormal duration is extracted.

[0053] Specifically, the system comprehensively judges whether there is a cross-modal synchronization anomaly based on the synchronization degree of three types of behaviors, and continuously times the abnormal state. When the comprehensive judgment result meets the abnormal conditions, the timer starts from the moment the anomaly is first triggered to obtain the duration of the anomaly; when the abnormal state ends, the timer is reset to zero.

[0054] The behavioral evolution slope is extracted from the time-series aligned data, and the evolution trend characteristics are determined by combining the duration of the anomaly.

[0055] Specifically, the behavioral evolution slope reflects the trend of individual behavior over time, and the duration of abnormality reflects the duration of the abnormal state. Together, they constitute the evolutionary trend feature, which is used to characterize the long-term trajectory of psychological state changes.

[0056] The response interval extension rate, interaction frequency decrease rate, and silent cumulative duration are extracted from the temporal alignment data corresponding to the text interaction trajectory to determine the temporal degradation characteristics.

[0057] Specifically, the response interval extension rate reflects the degree of sluggishness in interactive response, the interaction frequency decline rate reflects the degree of reduction in the willingness to actively interact, and the cumulative silence duration reflects the degree of persistence of behavioral silence. Together, these three constitute the temporal degradation feature, which is used to characterize the degree of degradation of behavioral activity.

[0058] In practical application, this embodiment extracts quantitative indicators such as behavioral synchronization degree, anomaly duration, behavioral evolution slope, and temporal degradation characteristics from the temporal alignment data. These indicators are used for: generating the overall modal synchronization coefficient (to identify anomalies and time them), calculating baseline offset by comparing with the baseline, participating in the comprehensive risk fusion calculation, and serving as the basis for graded early warning judgment. Simultaneously, all extracted temporal behavioral features are stored as incremental data in the temporal behavioral baseline library to update students' temporal behavioral trajectories, providing a data foundation for subsequent adaptive iterations of personalized baselines.

[0059] As an optional embodiment of the present invention, determining the abnormality judgment result of the individual behavior and extracting the abnormal duration by comprehensively considering the synchronization degree of each behavior and the normal duration threshold includes: Based on the synchronization degree of each behavior, the overall modal synchronization coefficient is determined by the first weighted fusion algorithm.

[0060] Specifically, the synchronization degrees of three types of behaviors—text-speech, text-body language, and speech-body language—are each assigned a preset weight, and the overall modal synchronization coefficient is obtained by weighted summation. The overall modal synchronization coefficient comprehensively reflects the overall consistency of the three types of modal behaviors, and its value ranges from 0 to 1.

[0061] Within the sliding time window, if the overall modal synchronization coefficient is lower than the synchronization coefficient threshold and the duration exceeds the normal duration threshold, it is determined that the individual behavior has cross-modal behavior abnormality, and the duration of the abnormality is obtained from the time when the abnormality is first triggered; otherwise, the individual behavior is determined to be normal.

[0062] Specifically, a synchronization coefficient threshold (e.g., 0.6) and a normal duration threshold (e.g., 6 hours) are set. When the overall modal synchronization coefficient is below 0.6 and this state continues for more than 6 hours, it is determined to be a cross-modal synchronization anomaly. The start time of the anomaly is recorded, and the duration of the anomaly is the time difference between the current time and the start time of the anomaly. If the synchronization coefficient recovers to above 0.6, the anomaly state ends, and the timer is reset to zero.

[0063] In practical applications, this embodiment can accurately identify cross-modal synchronization anomalies by using both the overall modal synchronization coefficient and duration for determination, effectively distinguishing between temporary acquisition deviations and persistent behavioral dissociation, and providing a reliable basis for extracting the duration of abnormalities.

[0064] As an optional embodiment of the present invention, the temporal behavioral characteristics further include synchronous abnormal characteristics, which represent the degree of dissociation between the internal and external manifestations of psychological abnormalities; The extraction of temporal behavior features based on the temporal alignment data also includes: Based on the overall modal synchronization coefficient, determine the modal offset value between the temporal alignment data corresponding to each of the two items in the text interaction trajectory, the speech rhythm features, and the visual body posture.

[0065] Specifically, the modal offset value is used to quantify the maximum time deviation of the three types of time-series data after timeline alignment, reflecting the degree of time synchronization deviation between multimodal behaviors. The modal offset value is calculated as follows: after timeline alignment, the timestamp differences of the three data pairs—text and speech, text and body language, and speech and body language—are calculated separately, and the maximum value is taken as the modal offset value. Excessive modal offset values ​​may be caused by two types of reasons: first, device deviation due to hardware acquisition delay, which can be corrected through timeline alignment; second, behavioral dissociation caused by inconsistencies in the target object's internal and external performance, such as a significant time misalignment between facial expression changes and speech output, which is the implicit psychological risk signal that this invention focuses on.

[0066] Within a preset unit time period, the number of times the overall modal synchronization coefficient falls below the synchronization coefficient threshold is counted, and this number is taken as the consistency collapse count.

[0067] Specifically, the consistency collapse count is used to quantify the frequency of multimodal behavioral dissociation. Using a sliding time window as the unit, the overall modal synchronization coefficient is checked window by window to see if it falls below a synchronization coefficient threshold (e.g., 0.6). The number of windows with a synchronization coefficient below the threshold within a unit of time (e.g., 1 hour) is counted as the consistency collapse count. This indicator complements the anomaly duration: anomaly duration measures the duration of a single anomaly, while the consistency collapse count measures the frequency of anomalies. Even if each collapse is short in duration, frequent occurrences still indicate that the target individual may have persistent psychological problems.

[0068] The synchronization anomaly characteristics are determined based on the overall modal synchronization coefficient, the modal offset value, and the number of consistency collapses.

[0069] Specifically, the synchronization anomaly characteristics are composed of three quantitative indicators: the overall modal synchronization coefficient, the modal offset value, and the number of consistency collapses. These indicators comprehensively characterize the degree of dissociation between the target object's internal and external behaviors. The overall modal synchronization coefficient reflects the consistency of current multimodal behavior and is the core indicator of the synchronization anomaly characteristics. The modal offset value reflects the degree of temporal synchronization deviation in multimodal behavior, and the number of consistency collapses reflects the frequency of dissociative behavior. These three indicators jointly characterize dissociative latent risks from three dimensions: current consistency, temporal deviation, and frequency of occurrence. An abnormality in any of these indicators may indicate psychological abnormalities in the target object.

[0070] In practical applications, this embodiment effectively identifies dissociative latent psychological abnormalities such as "normal text expression but disordered speech rhythm and body avoidance" by introducing synchronization anomaly features. This solves the problem of core feature loss caused by existing technologies that only perform multimodal feature splicing without temporal linkage verification. The three indicators of overall modal synchronization coefficient, modal offset value, and consistency collapse number quantify the consistency of internal and external performance from different dimensions, which can capture the hidden risk of the target object "pretending to be normal" and significantly reduce the false negative rate of psychological screening.

[0071] It should be noted that the baseline offset is obtained by comparing temporal behavioral features with a personalized normal behavior baseline. The specific calculation process is as follows: the temporal behavioral features are integrated into a temporal feature vector, which is then compared with the baseline feature vector corresponding to the personalized normal behavior baseline. A weighted Euclidean distance algorithm is used to calculate the distance between the two, and this distance value is used as the baseline offset. The larger the distance, the more significant the deviation between the current behavior and the individual's normal behavior. According to the aforementioned embodiment, the baseline offset is obtained by comparing temporal behavioral features with a personalized normal behavior baseline. The behavioral evolution slope and abnormal duration have been determined in S200, calculated based on the rate of change from temporal alignment data and timed based on modal synchronization feature data, respectively. These three indicators quantify the psychological state from three dimensions: deviation magnitude, change trend, and duration, providing accurate input for weighted fusion. To address the issue that existing technologies cannot adapt to individual differences due to the inability of general thresholds, the baseline offset is determined based on the target object itself, achieving accurate identification. To address the issue of prioritizing instantaneous events over temporal sequences, the behavioral evolution slope extends risk identification to long-term trend analysis, enabling proactive identification. To address the issue of triggering early warnings with a single fluctuation, the duration of anomalies is used to effectively distinguish between temporary fluctuations and persistent anomalies through dual threshold conditions, significantly reducing the false alarm rate.

[0072] As an optional embodiment of the present invention, the weighted fusion of the baseline offset, the behavioral evolution slope, and the duration of the anomaly to determine the comprehensive risk level includes: Obtain the preset weight coefficients corresponding to the baseline offset, the behavior evolution slope, and the duration of the anomaly.

[0073] Specifically, preset weighting coefficients are used to reflect the importance of the three indicators in the comprehensive risk calculation. Baseline deviation reflects the extent of deviation between current behavior and an individual's normal state, serving as a fundamental indicator for determining the existence of anomalies. Behavioral evolution slope reflects the trend of behavior over time; negative values ​​indicate deterioration, and positive values ​​indicate recovery, used to judge the direction of risk development. Abnormal duration reflects the stability of the abnormal state; the longer the duration, the more persistent the risk. These three indicators jointly assess psychological risk from three dimensions: deviation magnitude, trend of change, and duration, and none can be omitted. The weighting coefficients can be adjusted according to the needs of the campus setting; for example, the weight of baseline deviation can be set to 0.4, the weight of behavioral evolution slope to 0.3, and the weight of abnormal duration to 0.3, with a sum of 1.

[0074] The baseline offset, the behavior evolution slope, and the duration of the anomaly are normalized respectively. The three normalized indicators are then weighted and summed with the corresponding preset weight coefficients to obtain the initial risk score.

[0075] Specifically, the weighted summation involves multiplying each of the three indicators by its corresponding weight coefficient and then adding them together. Since the three indicators have different dimensions, normalization is required before weighted summation to map each indicator to a uniform numerical range (e.g., 0-1) to eliminate the impact of dimensional differences on the fusion result. The initial risk score obtained from the weighted summation is a value between 0 and 1, comprehensively reflecting the risk contribution of the three dimensions.

[0076] The initial risk score is range-mapped to generate the comprehensive risk level.

[0077] Specifically, interval mapping transforms the initial risk score between 0 and 1 into a range of 0 to 100, resulting in a comprehensive risk level that is easy to understand and classify. The mapping method can use linear scaling, which involves multiplying the initial risk score by 100. A higher comprehensive risk score indicates a higher psychological risk level; for example, 0-39 is the normal range, 40-69 is the attention range, and 70-100 is the high-risk range, providing a quantitative basis for subsequent graded early warning systems.

[0078] In practical application, this embodiment normalizes the baseline offset, behavioral evolution slope, and abnormal duration separately, and then weights and sums them with their respective preset weight coefficients. This integrates the three dimensions of deviation magnitude, trend of change, and duration into a single comprehensive risk level, achieving a multi-dimensional quantitative assessment of psychological risk. Compared with single-indicator judgment, this embodiment can more comprehensively reflect the psychological state of the target: the baseline offset is used to determine whether the current behavior deviates from the individual's normal state, the behavioral evolution slope is used to determine whether the behavior is continuously deteriorating or gradually recovering, and the abnormal duration is used to determine whether the abnormality is a temporary fluctuation or a persistent risk. The integration of the three effectively avoids risk underreporting or false reporting caused by misjudgment of a single dimension, significantly improving the accuracy and reliability of psychological risk identification.

[0079] As an optional embodiment of the present invention, the step of determining graded early warning data based on the comprehensive risk level and the duration of the anomaly, and issuing an early warning based on the graded early warning data, includes: The overall risk level is compared with a preset first risk threshold and a second risk threshold, and the duration of the anomaly is compared with a preset first duration threshold and a second duration threshold; the second risk threshold is higher than the first risk threshold; the second duration threshold is higher than the first duration threshold.

[0080] Specifically, the first and second risk thresholds are used to classify psychological risk levels, while the first and second duration thresholds are used to distinguish between short-term fluctuations and long-term persistent abnormalities. When the overall risk level is lower than the first risk threshold, it indicates that the target's current behavioral characteristics deviate little from the individual's normal baseline; when the overall risk level is higher than the second risk threshold, it indicates that the deviation is already quite serious. The duration of the abnormality reflects the continuous duration of the abnormal state from its initial trigger to the current moment; the longer the duration, the more stable the abnormal state and the more likely it is to develop into a persistent risk.

[0081] For example, the first risk threshold can be set to 40 points, and the second risk threshold can be set to 70 points; the first duration threshold can be set to 6 hours, and the second duration threshold can be set to 24 hours. The comprehensive risk score ranges from 0 to 100 points, with higher scores indicating higher psychological risk levels; the duration of the anomaly refers to the continuous duration from the first triggering of the cross-modal synchronicity anomaly to the current moment. For example, if a student is first diagnosed with cross-modal synchronicity anomaly at 8:00 AM and has not recovered by 2:00 PM, the duration of the anomaly is 6 hours.

[0082] If the overall risk level is lower than the first risk threshold or the duration of the abnormality is lower than the first duration threshold, it is determined to be a normal state without warning, and no warning is triggered.

[0083] Specifically, this criterion is used to filter out temporary and non-persistent behavioral fluctuations. When the overall risk level is below the first risk threshold, it indicates that the target's temporal behavioral characteristics deviate little from their normal baseline, the slope of behavioral evolution does not show a significant deterioration trend, and the overall psychological state is within the normal range. When the duration of the abnormality is below the first duration threshold, it indicates that the current abnormal state is short-lived and may be caused by sporadic factors such as a single emotional fluctuation, temporary environmental interference (such as exam pressure, conflicts with classmates), or equipment malfunctions, and has not yet formed a persistent or stable psychological risk. Triggering an alert in this situation would result in a high false alarm rate, interfering with the normal work of mental health teachers. Therefore, the system determines it to be a normal, no-alert state, does not trigger an alert, and only uses the current monitoring data for adaptive updates to the personalized baseline.

[0084] For example, if a student's interaction frequency decreases and response intervals lengthen due to a poor exam performance, resulting in a comprehensive risk score of 45 (above 40), but the anomaly lasts only 2 hours (below 6 hours), the system classifies it as a normal, no-warning state and does not trigger a warning. Similarly, if a student's comprehensive risk score is 35 (below 40), regardless of the duration of the anomaly, the system classifies it as a normal, no-warning state.

[0085] If the overall risk level is between the first risk threshold and the second risk threshold, and the duration of the anomaly is between the first duration threshold and the second duration threshold, it is determined to be in a state of concern, and first warning data is generated.

[0086] Specifically, this criterion is used to identify psychological risks that have become persistent but have not yet reached an emergency level. When the overall risk level is between the first and second risk thresholds, it indicates that the target individual's temporal behavioral characteristics have significantly deviated from their normal baseline, including but not limited to abnormal changes in quantitative indicators such as an increased rate of prolonged response intervals, a greater rate of decreased interaction frequency, and a reduced modal synchronization coefficient. Simultaneously, if the duration of the abnormality is between the first and second duration thresholds, it indicates that the abnormal state has persisted for a certain period (exceeding the short-term fluctuation threshold), but has not yet reached the duration standard required for emergency intervention. In this state, the target individual may be in the nascent or developing stage of psychological risk, without exhibiting extreme behavioral manifestations, but without intervention, the risk may escalate further. Therefore, the system determines this as a state of concern, generating first warning data to remind relevant personnel to proactively monitor and observe the target individual, achieving early identification and early intervention of psychological risks.

[0087] For example, if a student exhibits the following abnormalities for 10 consecutive hours: the response interval extension rate increases by 45% compared to the baseline, the interaction frequency decreases by 30%, and the modal synchronization coefficient drops to 0.55 (the preset threshold for modal synchronization coefficient is 0.6), with a comprehensive risk score of 55 (between 40 and 70), and the abnormality lasts for 10 hours (between 6 and 24 hours), the system will determine that the student is in a state of concern, generate the first warning data, and push it to the homeroom teacher and the psychological counselor, reminding them to have a heart-to-heart talk with the student and conduct daily observation.

[0088] If the overall risk level is higher than the second risk threshold and the duration of the anomaly is higher than the second duration threshold, it is determined to be in a state requiring early warning, and second early warning data is generated.

[0089] Specifically, this criterion is used to identify serious and persistent emergency psychological risks. When the overall risk level exceeds the second risk threshold, it indicates that the target individual's temporal behavioral characteristics deviate significantly from their normal baseline, with a significantly increased baseline offset, a negative slope in behavioral evolution with a large absolute value (indicating continuous behavioral deterioration), and a modal synchronization coefficient that remains below the threshold for an extended period, potentially indicating significant dissociative latent risks. Simultaneously, if the duration of the abnormality exceeds the second duration threshold, it indicates that the serious abnormal state has persisted for a considerable period, is not a temporary fluctuation, and the target individual's psychological state may have entered a high-risk phase. In this state, the target individual has a high probability of experiencing a psychological crisis and requires immediate human intervention. Therefore, the system determines this as a pending warning state, generates second warning data, and triggers the emergency warning process.

[0090] For example, if a student exhibits the following serious abnormalities within 30 consecutive hours: the response interval extension rate increases by 80% compared to the baseline, the interaction frequency decreases by 60%, the modal synchronization coefficient remains below 0.4, the behavioral evolution slope is -0.15 (negative values ​​indicate continuous deterioration), the overall risk score is 85 (higher than 70), and the duration of the abnormality is 30 hours (higher than 24 hours), the system determines it to be in a state awaiting warning, generates second warning data and pushes it synchronously to the school counselor, homeroom teacher and student's parents, triggering the campus psychological crisis intervention process.

[0091] An early warning is issued based on either the first or the second early warning data.

[0092] Specifically, based on the risk level determined by the assessment, a differentiated early warning information push strategy is implemented. The first early warning data corresponds to the "attention" status, used to remind relevant personnel to proactively monitor and observe the target individual. The early warning information includes the target individual's identifier, overall risk level, duration of abnormality, description of core abnormal characteristics, and evolutionary trend information. This information is pushed to homeroom teachers and student counselors. The second early warning data corresponds to the "pending early warning" status, used to trigger emergency intervention procedures. This early warning information, based on the first early warning data, includes risk management suggestions and intervention procedure guidelines. This information is pushed to student counselors, homeroom teachers, and parents, achieving simultaneous push notifications across multiple platforms to ensure timely response from relevant personnel.

[0093] In practical applications, this embodiment effectively separates temporary fluctuations from persistent risks through a dual threshold judgment mechanism, achieving a graded early warning strategy of "no disturbance in the short term, early warning in the long term, and emergency response in the high-risk area." This significantly reduces the false alarm rate while ensuring monitoring sensitivity, meeting the stable operation requirements of routine psychological screening.

[0094] like Figure 2 As shown, taking students as the target audience, Figure 2 The demonstration showcases the complete time-series process from multimodal data collection to tiered early warning: Student interactive terminals upload multimodal behavioral data to the system, which is then preprocessed and time-series aligned to obtain time-series aligned data. Next, time-series degradation features and evolutionary trend features (including behavioral evolution slope and abnormal duration) are extracted from the time-series aligned data, and the student's personalized normal behavioral baseline is retrieved from the behavioral baseline library. The baseline offset is then determined, and a comprehensive risk level is calculated through weighted fusion, followed by tiered early warning. Tiered early warning data is generated based on the comprehensive risk level and abnormal duration and pushed to the teacher management terminal. Simultaneously, incremental data is stored in the behavioral baseline library for adaptive updates of personalized baselines. Finally, the teacher management terminal provides feedback to the student interactive terminals regarding human intervention and care responses, forming a complete closed loop of "monitoring—early warning—intervention—feedback."

[0095] like Figure 3 As shown, the present invention also provides a dynamic monitoring and cross-modal early warning system 200 for psychological states, which applies the dynamic monitoring and cross-modal early warning method for psychological states as described in the above embodiments, including: Preprocessing module 210 is used to: preprocess the acquired multimodal behavior data of the target object to obtain time-aligned data; The feature extraction module 220 is used to: extract temporal behavioral features based on the temporal alignment data, wherein the temporal behavioral features include at least temporal degradation features reflecting the degree of degradation of behavioral activity and evolutionary trend features reflecting the long-term trend of behavioral changes; the evolutionary trend features include behavioral evolution slope and abnormal duration; the behavioral evolution slope represents the change of individual behavior on the time axis; the abnormal duration represents the duration of abnormal behavior in the individual behavior; The baseline matching module 230 is used to: determine the baseline offset based on the time-series behavioral characteristics and a preset personalized normal behavior baseline; the baseline offset represents the degree of deviation between the individual behavior and the personalized normal behavior. The fusion calculation module 240 is used to: perform weighted fusion of the baseline offset, the behavioral evolution slope, and the duration of the anomaly to determine the comprehensive risk level; the comprehensive risk level represents the degree of individual psychological risk. The graded early warning module 250 is used to: determine graded early warning data based on the comprehensive risk level and the duration of the anomaly, and issue an early warning based on the graded early warning data.

[0096] The specific implementation method of this embodiment can be referred to the corresponding implementation method described above, and will not be described again here.

[0097] like Figure 4 As shown, an electronic device 300 provided in this embodiment of the invention includes a memory 310 and a processor 320; the memory 310 is used to store a computer program; the processor 320 is used to implement the above-described method for dynamic monitoring of psychological states and cross-modal early warning when the computer program is executed.

[0098] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0099] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

[0100] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.

Claims

1. A method for dynamic monitoring and cross-modal early warning of psychological states, characterized in that, include: The acquired multimodal behavior data of the target object is preprocessed to obtain time-aligned data; Temporal behavioral features are extracted based on the temporal alignment data. These temporal behavioral features include at least temporal degradation features reflecting the degree of degradation of behavioral activity and evolutionary trend features reflecting the long-term trend of behavioral change. The evolutionary trend features include the slope of behavioral evolution and the duration of abnormality; the slope of behavioral evolution represents the changes in individual behavior over time; and the duration of abnormality represents the duration of abnormal behavior within the individual's behavior. Based on the temporal behavioral characteristics and the preset personalized normal behavior baseline, a baseline offset is determined; the baseline offset represents the degree of deviation between the individual behavior and the personalized normal behavior. The baseline offset, the slope of the behavior evolution, and the duration of the anomaly are weighted and fused to determine the overall risk level. The overall risk level represents the degree of an individual's psychological risk. The graded early warning data is determined based on the comprehensive risk level and the duration of the anomaly, and an early warning is issued based on the graded early warning data.

2. The method for dynamic monitoring and cross-modal early warning of psychological states according to claim 1, characterized in that, The personalized routine behavior baseline is extracted from a pre-established time-series behavior baseline library, and the method further includes: Acquire incremental data, which includes one or more of the following: time-series aligned data, time-series behavioral features, baseline offset, behavioral evolution slope, anomaly duration, comprehensive risk level, and graded early warning data. The personalized normal behavior baseline corresponding to the time-series behavior baseline library is adaptively updated based on the incremental data.

3. The method for dynamic monitoring and cross-modal early warning of psychological states according to claim 2, characterized in that, The temporal behavior baseline library also stores temporal behavior trajectories; The step of adaptively updating the personalized normal behavior baseline in the time-series behavior baseline library based on the incremental data includes: Update the time-series behavior trajectory based on the incremental data; Based on the temporal behavior features used to characterize normal behavior in the temporal behavior trajectory, the personalized normal behavior baseline is adaptively iteratively updated using an unsupervised algorithm, wherein the updated personalized normal behavior baseline is used to extract the baseline feature vector corresponding to the personalized normal behavior. The baseline offset is the deviation of the temporal feature vector corresponding to the temporal behavior feature relative to the baseline feature vector.

4. The method for dynamic monitoring and cross-modal early warning of psychological states according to any one of claims 1-3, characterized in that, The multimodal behavioral data includes at least text interaction trajectories, speech rhythm features, and visual body postures. The preprocessing of the acquired multimodal behavioral data of the target object to obtain time-aligned data includes: The multimodal behavior data is subjected to at least time alignment processing to obtain the time-aligned data.

5. The method for dynamic monitoring and cross-modal early warning of psychological states according to claim 4, characterized in that, The extraction of temporal behavior features based on the temporal alignment data includes: Using a sliding time window as a unit, determine the behavioral synchronization degree between the temporal alignment data corresponding to each two items in the text interaction trajectory, the speech rhythm features, and the visual body posture performance; By combining the synchronization degree of each behavior and the normal duration threshold, the abnormality judgment result of the individual behavior is determined and the abnormal duration is extracted; The behavioral evolution slope is extracted from the time-series aligned data, and the evolution trend characteristics are determined by combining the duration of the anomaly. The response interval extension rate, interaction frequency decrease rate, and silent cumulative duration are extracted from the temporal alignment data corresponding to the text interaction trajectory to determine the temporal degradation characteristics.

6. The method for dynamic monitoring and cross-modal early warning of psychological states according to claim 5, characterized in that, The process of determining the abnormality detection result of the individual behavior and extracting the abnormal duration by comprehensively considering the synchronization degree of each behavior and the normal duration threshold includes: Based on the synchronization degree of each behavior, the overall modal synchronization coefficient is determined by the first weighted fusion algorithm; Within the sliding time window, if the overall modal synchronization coefficient is lower than the synchronization coefficient threshold and the duration exceeds the normal duration threshold, it is determined that the individual behavior has cross-modal behavior abnormality, and the duration of the abnormality is obtained from the first trigger time of the abnormality; otherwise, the individual behavior is determined to be normal.

7. The method for dynamic monitoring and cross-modal early warning of psychological states according to claim 6, characterized in that, The temporal behavioral characteristics also include synchronous abnormal characteristics, which represent the degree of dissociation between the internal and external manifestations of psychological abnormalities; The extraction of temporal behavior features based on the temporal alignment data also includes: Based on the overall modal synchronization coefficient, determine the modal offset value between the temporal alignment data corresponding to each of the two items in the text interaction trajectory, the speech rhythm features, and the visual body posture; Within a preset unit time period, the number of times the overall modal synchronization coefficient falls below the synchronization coefficient threshold is counted, which is taken as the consistency collapse count. The synchronization anomaly characteristics are determined based on the overall modal synchronization coefficient, the modal offset value, and the number of consistency collapses.

8. The method for dynamic monitoring and cross-modal early warning of psychological states according to any one of claims 1-3, characterized in that, The weighted fusion of the baseline offset, the behavioral evolution slope, and the duration of the anomaly to determine the comprehensive risk level includes: Obtain the preset weight coefficients corresponding to the baseline offset, the behavior evolution slope, and the duration of the anomaly, respectively; The baseline offset, the behavior evolution slope, and the duration of the anomaly are normalized respectively. The three indicators obtained after normalization are weighted and summed with the corresponding preset weight coefficients to obtain the initial risk score. The initial risk score is range-mapped to generate the comprehensive risk level.

9. The method for dynamic monitoring and cross-modal early warning of psychological states according to any one of claims 1-3, characterized in that, The step of determining graded early warning data based on the comprehensive risk level and the duration of the anomaly, and issuing early warnings based on the graded early warning data, includes: The overall risk level is compared with a preset first risk threshold and a second risk threshold, and the duration of the anomaly is compared with a preset first duration threshold and a second duration threshold; the second risk threshold is higher than the first risk threshold; the second duration threshold is higher than the first duration threshold; If the overall risk level is lower than the first risk threshold or the duration of the abnormality is lower than the first duration threshold, it is determined to be a normal state without warning, and no warning is triggered. If the overall risk level is between the first risk threshold and the second risk threshold, and the duration of the anomaly is between the first duration threshold and the second duration threshold, it is determined to be in a state of concern, and first warning data is generated. If the overall risk level is higher than the second risk threshold and the duration of the anomaly is higher than the second duration threshold, it is determined to be a state requiring early warning, and second early warning data is generated. An early warning is issued based on either the first or the second early warning data.

10. A dynamic monitoring and cross-modal early warning system for psychological states, characterized in that, The method for dynamic monitoring and cross-modal early warning of psychological states as described in any one of claims 1-9 includes: The preprocessing module is used to preprocess the acquired multimodal behavior data of the target object to obtain time-aligned data. The feature extraction module is used to: extract temporal behavioral features based on the temporal alignment data, wherein the temporal behavioral features include at least temporal degradation features reflecting the degree of degradation of behavioral activity and evolutionary trend features reflecting the long-term trend of behavioral change; the evolutionary trend features include behavioral evolution slope and abnormal duration; the behavioral evolution slope represents the change of individual behavior on the time axis; the abnormal duration represents the duration of abnormal behavior in the individual behavior; The baseline matching module is used to: determine the baseline offset based on the time-series behavioral characteristics and a preset personalized normal behavior baseline; the baseline offset represents the degree of deviation between the individual behavior and the personalized normal behavior. The fusion calculation module is used to: perform weighted fusion of the baseline offset, the behavioral evolution slope, and the duration of the anomaly to determine the comprehensive risk level; the comprehensive risk level represents the degree of individual psychological risk. The graded early warning module is used to: determine graded early warning data based on the comprehensive risk level and the duration of the anomaly, and issue early warnings based on the graded early warning data.

Images