Early warning system and method for predicting depression state of the elderly based on multi-modal data

Abstract

The present application relates to the technical field of mental health monitoring, in particular to an early warning system and method for predicting the depression state of the elderly based on multi-modal data, the system comprising: a trajectory deviation collection module, a language retardation extraction module, a sleep deviation analysis module, a rhythm aggregation trigger module, and a synchronization imbalance quantification module.In the present application, the positioning trajectory, voice response, sleep stage behavior anchor points, and multi-source physiological signals are sequentially processed, the behavior continuity deviation is described by using the trajectory connection change amount, the language response retardation trend is exhibited by using the start delay difference, the sleep structure deviation situation is extracted by using the stage conversion trend, the behavior drift and rhythm change are associated and judged, the abnormal fragments of the heart rate, skin electricity, and respiration body temperature signals are subjected to overlapping screening, the behavior change, language retardation, sleep fluctuation, and physiological synchronization imbalance form a continuous mapping link, and through the mutual pointing deviation structure, the coherent identification and early warning of the psychological risk are realized.

Description

Technical Field

[0001] This invention relates to the field of mental health monitoring technology, and in particular to an early warning system and method for predicting depressive states in the elderly based on multimodal data. Background Technology

[0002] The field of mental health monitoring technology involves technical aspects of continuous observation, quantitative analysis, and risk identification of individual mental states. Its core content includes acquiring data related to changes in mental state through various methods such as physiological parameter monitoring, behavioral characteristic recording, language and facial expression analysis, sleep pattern observation, and environmental and lifestyle information collection. This data is then combined with psychological assessment criteria to organize, compare, and analyze the monitoring information, forming a dynamic monitoring system for mental health. This system encompasses the construction of data collection methods, methods for identifying mental state characteristics, long-term tracking and recording methods, and mental health risk monitoring procedures for specific populations, thus forming a comprehensive system for acquiring and organizing information on changes in mental state. The systematic technical system for diagnosis includes the traditional elderly depression prediction and early warning system, which refers to a system that assesses the psychological state of the elderly by collecting data from a single or limited range of categories to identify potential depressive moods. It typically completes a basic assessment of the elderly's emotional state through methods such as collecting language content through manual interviews, relying on caregivers to observe the elderly's daily behavior, recording their emotional self-reports based on questionnaires, reading physiological data such as heart rate or blood pressure using fixed measuring instruments, and recording the elderly's sleep and activity status through care institutions. The data from these sources is then manually sorted, compared, and judged to obtain a preliminary inference on whether the elderly person is at risk of depression.

[0003] Current technologies for monitoring the psychological state of the elderly primarily rely on methods such as manual interviews, observations, questionnaires, and records to collect information related to language, behavior, sleep, and physiology. This process depends on manual judgment to organize and compare various types of information. When dealing with continuous behavioral changes and potential psychological fluctuations, it is difficult to form a stable quantitative logic. As a result, subtle features such as trajectory changes, slowed language responses, and fluctuations in sleep structure are difficult to continuously capture. Synchronous anomalies in physiological signals are also difficult to identify in a timely manner due to the lack of sequence-level screening. Consequently, the correlation between behavioral changes and physiological imbalances cannot form a coherent link, leading to a lag in the identification of psychological states and a break in the correlation between various types of signals. Summary of the Invention

[0004] To address the technical problems existing in the prior art, embodiments of the present invention provide an early warning system and method for predicting depressive states in the elderly based on multimodal data. The technical solution is as follows:

[0005] On the one hand, an early warning system based on multimodal data to predict the depressive state of the elderly is provided. This system includes: The trajectory offset acquisition module extracts the trajectory point sequence of the positioning device, analyzes the time stamps of adjacent trajectory points, divides continuous trajectory segments, compares the start and end coordinates of trajectory segments to calculate the displacement difference, analyzes the direction and speed of movement to form direction and speed differences, forms a behavioral offset trend structure, and forms behavioral connection offset. Based on the behavior connection offset, the language delay extraction module analyzes the prompting time and the first utterance time, calculates the start delay, analyzes the start delay sequence to establish a reference benchmark, and performs differential processing on the start delay and the reference interval to obtain the start offset, forming the language start offset. The sleep shift analysis module extracts the stage label sequence recorded by the sleep monitoring device, calculates the proportion of duration of adjacent stages to obtain the stage transition ratio, analyzes the sorting change of the transition ratio in time sequence to form the transition trend direction, and combines it with the stage weight to form the sleep transition shift degree. The rhythm aggregation triggering module analyzes the daily behavior anchor time based on the behavior connection offset, language initiation offset, and sleep transition offset, compares the time difference between adjacent dates to form the anchor drift, and outputs behavior status warning information by combining the behavior offset trend structure and the daily language sluggishness. The synchronization anomaly quantification module monitors and analyzes the continuous sequences of heart rate, skin conductance, respiration and body temperature signals based on the behavioral state warning information, marks abnormal segments, and determines the degree of abnormal superposition and classifies synchronization anomaly windows by analyzing the overlap of abnormal segments of physiological signals within each time window. Based on the proportion of synchronization anomaly windows in all windows, physiological synchronization anomaly information is formed.

[0006] As a further aspect of the present invention, the behavioral connection offset includes the trajectory connection offset intensity distribution, the number of occurrences of trajectory connection offset, and the trajectory connection offset duration range; the language initiation offset includes the initiation delay offset magnitude level, the initiation delay offset frequency, and the initiation delay fluctuation dispersion; the sleep transition offset includes the sleep stage transition stability coefficient, the sleep stage structure offset direction marker, and the sleep stage dwell time offset ratio; the behavioral state warning information includes the geriatric depression risk grading results, abnormal behavior pattern feature entries, and warning prompt generation time records; and the physiological synchronization abnormal information includes abnormal segment summary records, abnormal segment overlap analysis results, and abnormal proportion summary information.

[0007] As a further aspect of the present invention, the trajectory offset acquisition module includes: The trajectory segment generation submodule extracts the trajectory point sequence of the positioning device, analyzes the time stamps of adjacent trajectory points, divides continuous trajectory segments, analyzes the coordinate information of the start and end points of the trajectory segments, calculates the displacement difference, and generates the trajectory segment information. The connection change analysis submodule analyzes the movement direction of the trajectory segment based on the information of the trajectory segment, forms direction parameters, compares the direction parameters of adjacent continuous trajectory segments to obtain the direction difference, compares the movement speed of adjacent trajectory segments to obtain the speed difference, and combines the direction difference and speed difference to obtain the connection change index. The trend structure determination submodule analyzes the changes in all trajectory connections within each cycle based on the connection change index, analyzes the changes in connection characteristics between trajectory segments, determines the stable characteristics of behavior, constructs the behavior offset trend structure, and establishes the behavior connection offset.

[0008] As a further aspect of the present invention, the language delay extraction module includes: The startup delay calculation submodule obtains the prompt time and the first speech time in the voice dialogue scenario based on the behavior connection offset, analyzes each set of prompt and speech times, calculates the startup delay parameter between the two time points, and establishes a startup delay parameter set. The offset differential processing submodule calls the set of startup delay parameters, analyzes the startup delay parameter sequence of continuous dialogue and arranges them according to the startup delay parameters, determines the center position parameter of the arranged sequence and uses it as the startup delay reference benchmark, performs differential processing on each startup delay parameter and the reference benchmark to obtain the set of startup offsets; The sluggishness trend assessment submodule analyzes all sluggishness changes within the period based on the set of sluggishness offsets, constructs a set of sluggishness changes, assesses the daily language sluggishness, and establishes a language sluggishness offset.

[0009] As a further aspect of the present invention, the sleep offset parsing module includes: The stage information extraction submodule extracts the stage tag sequence recorded by the sleep monitoring device, identifies the start and end times of the daily light sleep stage, deep sleep stage and REM stage, calculates the duration ratio of adjacent stages, and obtains the stage transition ratio set. The conversion trend analysis submodule calls the set of stage conversion ratios to analyze the sorting changes of the stage conversion ratios in time sequence, determines the direction of change of each stage conversion, and obtains the stage conversion trend parameters. The structural weight combination submodule combines the structural importance weights of each sleep stage with the stage transition trend parameters to establish a sleep transition offset.

[0010] As a further aspect of the present invention, the rhythm aggregation triggering module includes: The anchor point drift calculation submodule extracts and analyzes the time points of the elderly’s daily behavioral anchor points based on the behavioral connection offset, language initiation offset, and sleep transition offset, compares the time difference of behavioral anchor points on adjacent dates, and generates an anchor point drift set. The direction aggregation filtering submodule calls the anchor point drift set, analyzes the directional relationship of multiple anchor point drifts, filters drifts with consistent directions, calculates the proportion of drifts with consistent directions, and obtains the aggregation direction ratio parameter. The risk assessment output submodule, based on the aggregation direction ratio parameter, combined with the behavioral deviation trend structure and daily language slowness, determines the degree of abnormal deviation in the elderly's daily rhythm, assesses the degree of risk of geriatric depression, and establishes behavioral status early warning information.

[0011] As a further aspect of the present invention, the synchronization loss quantification module includes: The abnormal segment marking submodule monitors the continuous sequence of heart rate signal, skin conductance signal, respiratory signal and body temperature signal according to the behavioral state warning information. By calculating the rate of change of the difference between continuous sampling points, it judges the fluctuation rate of each sampling point, filters and marks abnormal segments of each signal, and obtains a set of physiological abnormal segments. The synchronization window classification submodule calls the set of physiological abnormal segments, analyzes the overlap of abnormal segments of each physiological signal within each time window, determines the degree of abnormal overlap, classifies them into synchronization abnormal windows, and generates a set of synchronization abnormal windows. The physiological indicator output submodule analyzes the proportion of the synchronization abnormal windows in all windows based on the set of synchronization abnormal windows, and establishes physiological synchronization abnormal information.

[0012] As a further aspect of the present invention, the process of filtering and marking abnormal segments of each signal specifically includes: The rate of change of the difference between adjacent sampling points is calculated for continuous sequences of heart rate, skin conductance, respiratory and body temperature signals to form a rate of change of difference sequence, and recorded according to time index to correspond with the time axis of the original signal; The difference change rate sequence is arranged from smallest to largest, and the continuous parameter segment located in the center of the arrangement is set as the reference fluctuation band; The upper limit threshold is set by the gap between the upper limit of the reference fluctuation band and the corresponding upper adjacent parameter segment, and the lower limit threshold is set by the gap between the lower limit of the reference fluctuation band and the corresponding lower adjacent parameter segment. When the rate of change of the difference continuously exceeds the upper threshold to form a time interval, the corresponding time interval is marked as a sudden change abnormal segment; when the rate of change of the difference continuously falls below the lower threshold to form a time interval, the corresponding time interval is marked as a decay abnormal segment. The abrupt change anomaly segments and the decay anomaly segments are recorded according to their start and end times and summarized into a set of physiological anomaly segments.

[0013] On the other hand, an early warning method for predicting depressive states in the elderly based on multimodal data, wherein the early warning method for predicting depressive states in the elderly based on multimodal data is executed based on the aforementioned early warning system for predicting depressive states in the elderly based on multimodal data, includes the following steps: S1: Extract the trajectory point sequence of the positioning device, analyze the time stamp of adjacent trajectory points, divide the continuous trajectory segments, compare the start and end coordinates of the trajectory segments to calculate the displacement difference, analyze the direction and speed of movement to form direction difference and speed difference, form behavior offset trend structure, and form behavior connection offset. S2: Based on the behavior connection offset, analyze the prompt time and the first utterance time, calculate the start delay, analyze the start delay sequence to establish a reference benchmark, perform differential processing on the start delay and the reference interval to obtain the start offset, and form the language start offset. S3: Extract the stage label sequence recorded by the sleep monitoring device, calculate the proportion of duration of adjacent stages to obtain the stage transition ratio, analyze the sorting change of the transition ratio in time sequence to form the transition trend direction, and combine it with the stage weight to form the sleep transition offset. S4: Based on the behavior connection offset, language initiation offset, and sleep transition offset, analyze the daily behavior anchor time, compare the time difference between adjacent dates to form the anchor drift, and combine the behavior offset trend structure and the daily language sluggishness to output behavior status warning information. S5: Based on the behavioral state warning information, monitor and analyze the continuous sequence of heart rate, skin conductance, respiration and body temperature signals, mark abnormal segments, and determine the degree of abnormal superposition and classify synchronous abnormal windows by analyzing the overlap of abnormal segments of physiological signals in each time window. Based on the proportion of synchronous abnormal windows in all windows, physiological synchronous abnormal information is formed.

[0014] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: By serializing the anchor points of speech response during sleep stages and multi-source physiological signals based on the positioning trajectory, the continuity of behavior is characterized by changes in trajectory connection, the slowing trend of language response is shown by the start-up delay difference, and the sleep structure shift is extracted by the stage transition trend. Behavioral drift is correlated with rhythm changes, and abnormal segments of heart rate, skin conductance, respiration, and body temperature signals are screened for overlap. This creates a continuous mapping link between behavioral changes, language slowing, sleep fluctuations, and physiological synchronization imbalance. Through this mutually pointing shift structure, coherent identification and early warning of psychological risks can be achieved. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of an early warning system for predicting depressive states in the elderly based on multimodal data, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the system framework of the present invention; Figure 3 This is a flowchart of the trajectory offset acquisition module in this invention; Figure 4 This is a flowchart of the language delay extraction module in this invention; Figure 5 This is a flowchart of the sleep offset analysis module in this invention; Figure 6 This is a flowchart of the rhythm aggregation triggering module in this invention; Figure 7 This is a flowchart of the synchronization loss quantification module in this invention; Figure 8 This is a flowchart of an early warning method for predicting depressive states in the elderly based on multimodal data, provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0018] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0019] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0020] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0021] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0022] This invention provides an early warning system for predicting depressive states in the elderly based on multimodal data, such as... Figure 1-2 The diagram shown illustrates an early warning system for predicting depressive states in older adults based on multimodal data. The system includes: The trajectory offset acquisition module extracts the trajectory point sequence of the positioning device, analyzes the time stamps of adjacent trajectory points, divides continuous trajectory segments, compares the start and end coordinates of trajectory segments to calculate the displacement difference, analyzes the direction and speed of movement to form direction and speed differences, forms a behavioral offset trend structure, and forms behavioral connection offset. The language delay extraction module analyzes the prompting time and the first utterance time based on the behavior connection offset, calculates the start delay, analyzes the start delay sequence to establish a reference benchmark, and performs differential processing on the start delay and the reference interval to obtain the start offset, forming the language start offset. The sleep shift analysis module extracts the stage label sequence recorded by the sleep monitoring device, calculates the proportion of duration of adjacent stages to obtain the stage transition ratio, analyzes the sorting change of the transition ratio in time sequence to form the transition trend direction, and combines it with the stage weight to form the sleep transition shift degree. The rhythm aggregation triggering module analyzes the daily behavior anchor time based on behavior connection offset, language initiation offset, and sleep transition offset, compares the time difference between adjacent dates to form the anchor drift, and outputs behavior status warning information by combining the behavior offset trend structure and the daily language sluggishness. The synchronization anomaly quantification module monitors and analyzes continuous sequences of heart rate, skin conductance, respiration, and body temperature signals based on behavioral status warning information, marks abnormal segments, and determines the degree of abnormal overlap by analyzing the overlap of abnormal segments of physiological signals within each time window and classifies the synchronization anomaly windows. Based on the proportion of the synchronization anomaly windows in all windows, physiological synchronization anomaly information is formed.

[0023] Behavioral cohesion offset includes trajectory cohesion offset intensity distribution, trajectory cohesion offset occurrence frequency, and trajectory cohesion offset duration range; language priming offset includes priming delay offset amplitude level, priming delay offset occurrence frequency, and priming delay fluctuation dispersion; sleep transition offset includes sleep stage transition stability coefficient, sleep stage structure offset direction marker, and sleep stage dwell time offset ratio; behavioral state early warning information includes geriatric depression risk grading results, abnormal behavior pattern feature items, and early warning prompt generation time record; physiological synchronization abnormal information includes abnormal segment summary record, abnormal segment overlap analysis results, and abnormal percentage summary information.

[0024] Specifically, such as Figure 2 , 3 As shown, the trajectory offset acquisition module includes: The trajectory segment generation submodule extracts the trajectory point sequence of the positioning device, analyzes the time stamps of adjacent trajectory points, divides continuous trajectory segments, analyzes the coordinate information of the start and end points of the trajectory segments, calculates the displacement difference, and generates the trajectory segment information. First, the system uses the built-in positioning unit (supporting both GPS and BeiDou dual-mode) of the smart bracelet worn by Subject_A to continuously collect geographic location data during the activity period from 06:00 to 18:00 on the same day at a sampling frequency of 1Hz. The system extracts the trajectory point sequence of the positioning device, which contains... Each data point contains longitude, latitude, and a timestamp accurate to milliseconds. The system reads adjacent trajectory points. and Time markers, calculating time differences The system sets a time threshold of 300 seconds (5 minutes), based on the average pause tolerance of elderly people during daily walking, to distinguish between continuous movement and stillness. When the time difference between adjacent points is less than 300 seconds, it is determined to be a continuous trajectory; when the time difference is greater than or equal to 300 seconds, the system will cut off the continuous trajectory at that point, dividing it into segments. For the data of the day, the system divided a total of 12 valid trajectory segments (denoted as...). to Subsequently, the system targets each trajectory segment, such as trajectory segment... (Corresponding to the morning exercise period from 07:15:00 to 07:45:00), extract its starting coordinates. coordinates of the endpoint The system uses the Haversine Formula or the projected Euclidean distance formula to calculate the straight-line distance between the starting and ending points, thus obtaining the trajectory segment. The displacement difference is 780 meters. The system traverses all 12 trajectory segments, calculates the displacement difference for each segment, and packages the data containing the trajectory segment ID, start and end time, start and end coordinates, and displacement difference to generate trajectory segment information.

[0025] The connection change analysis submodule analyzes the movement direction of the trajectory segment based on the information content of the trajectory segment, forms direction parameters, compares the direction parameters of adjacent continuous trajectory segments to obtain the direction difference, compares the movement speed of adjacent trajectory segments to obtain the speed difference, and combines the direction difference and speed difference to obtain the connection change index. Based on the information content of the trajectory segments, the system performs vectorized analysis on the movement characteristics of the trajectory segments. (This refers to the analysis of the trajectory segments.) The system, combining the displacement difference (780 meters) and the duration (1800 seconds), calculates the average moving speed of this segment to be 0.43 meters per second. Simultaneously, the system establishes a local coordinate system and calculates the azimuth angle from the starting point to the ending point, for example... The azimuth angle is 42 degrees (northeast), which is quantified into a direction parameter. Next, the system extracts information from adjacent continuous trajectory segments, such as trajectory segments... And the trajectory segment that followed (Starting at 07:50:00, moving due north with an azimuth of 0 degrees and an average speed of 0.35 m / s). The system compares the direction parameters of adjacent continuous trajectory segments and calculates the direction difference. Degree; compare the moving speeds of adjacent trajectory segments and calculate the speed difference. meters per second. The system introduces a normalized weighted algorithm to combine the direction difference and velocity difference, and sets a direction weight coefficient. Speed ​​weighting coefficient (This coefficient is set based on statistical data of elderly kinesiology; sudden changes in direction better reflect behavioral confusion.) The system normalizes the direction difference (divided by 180 degrees) and speed difference (divided by the reference speed of 1.5 m / s) before substituting them into the weighted formula for calculation: The value 0.161 represents the trajectory segment. to The system tracks the transition indicators for all adjacent trajectory segments throughout the day. , ,…, Repeat this process to obtain a series of indicators of transition changes.

[0026] The trend structure determination submodule analyzes the changes in all trajectory connections within each cycle based on the connection change index, analyzes the changes in connection characteristics between trajectory segments, judges the stable characteristics of behavior, constructs the behavior offset trend structure, and establishes the behavior connection offset. Based on the calculated sequence of 11 sets of connection change indicators for the entire day (e.g., 0.12, 0.16, 0.05, 0.88, ...), the system analyzes the total trajectory connection change within each period. The system sets a connection stability threshold of 0.5, which is determined by analyzing the historical average connection change indicator (0.25) and its standard deviation (0.12) of Subject_A under no-abnormal conditions over the past 30 days. The system statistically analyzes the number of abnormal connection points exceeding 0.5 in the daily sequence and their distribution density on the time axis. For example, between 2:00 PM and 4:00 PM on a given day, three consecutive high connection change indices exceeding 0.8 appeared, accompanied by drastic fluctuations in direction difference (mean direction difference greater than 120 degrees), indicating a tendency to linger in place. By analyzing the changes in connection characteristics between trajectory segments, the system identifies this high-frequency, high-amplitude oscillation pattern and judges the behavioral stability characteristics as "non-steady-state." The system constructs a behavioral offset trend structure and uses a time series trend term extraction algorithm (such as the moving average method) to calculate the overall Euclidean distance or dynamic time warping (DTW) distance of the daily connection change index sequence relative to the historical baseline sequence. Assuming the calculated DTW distance is 15.4, the system standardizes this distance value (mapping range 0-100), ultimately establishing a behavioral connection offset of 65. This value quantifies the coherence and degree of lack of purpose in the elderly person's outdoor activity trajectory on that day.

[0027] Specifically, such as Figure 2 , 4 As shown, the language delay extraction module includes: The startup delay calculation submodule obtains the prompt time and the first speech time in the voice dialogue scenario based on the behavior connection offset, analyzes the prompt and speech time of each group of prompts and speech times, calculates the startup delay parameters between the two time points, and establishes a startup delay parameter set. Based on the behavioral transition offset (65, indicating a moderate behavioral anomaly), the system retrieves the interaction logs of the home smart voice terminal from 08:00 to 20:00 on the same day. The system obtains the prompt time (the moment the system finishes playing the wake word or question) and the first speech time (the start time when the microphone detects valid voice input from the elderly person) in the voice dialogue scenario. For example, if the system asks "Do you want to play the news now?" at 10:30, the prompt end timestamp is... The starting timestamp for the elderly person's reply "okay" is... The system analyzes the timing of each prompt and sound, and calculates the start-up delay parameter between the two time points. Seconds. A total of 20 valid interactions occurred that day. The system calculated the startup delay for each interaction, resulting in a sequence of 20 values ​​(e.g., 1.2s, 1.6s, 2.5s, 0.9s, ...), and established a startup delay parameter set. For invalid interactions or records triggered by background noise, the system discarded them based on a signal-to-noise ratio threshold (SNR < 10dB), retaining high-quality startup delay data.

[0028] The offset differential processing submodule calls the startup delay parameter set, analyzes the startup delay parameter sequence of continuous dialogue and arranges them according to the startup delay parameters, determines the center position parameter of the arranged sequence and uses it as the startup delay reference benchmark, performs differential processing on each startup delay parameter and the reference benchmark to obtain the startup offset set; The system retrieves the current day's activation delay parameter set and analyzes the sequence of activation delay parameters in continuous dialogues. It selects approximately 1500 voice interaction records from Subject_A's emotionally stable periods over the past three months, arranges them according to the magnitude of the activation delay parameter values, and calculates the arithmetic mean using the interquartile range (interquartile range), obtaining a historical baseline mean of 0.8 seconds. The system determines the center position parameter of the sequence (i.e., 0.8 seconds) and uses it as the activation delay reference benchmark. Next, the system performs difference processing on the activation delay parameter of each of the 20 interactions of the day and compares it to the reference benchmark of 0.8 seconds. For example, for the 1.6-second delay at 10:30, the activation offset is calculated as follows: Seconds; for a given delay of 0.9 seconds, the offset is... Seconds. If the difference is negative, the offset is recorded as 0 or a negative value is retained to reflect agile response (in this embodiment, the sign is retained). After calculation, the system obtains a set of 20 signed values ​​for the start-up offset (e.g., +0.4, +0.8, +1.7, +0.1…).

[0029] The sluggishness trend assessment submodule analyzes all sluggishness changes within a period based on the sluggishness offset set, constructs a sluggishness change set, assesses the daily language sluggishness, and establishes the language sluggishness offset. Based on the set of startup offsets, the system analyzes all startup changes within a given period. The system introduces a "positive cumulative offset index" algorithm, weighting and accumulating only positive offsets greater than 0 (representing slower response times) in the set. Assuming 15 out of 20 interactions on a given day are positive offsets, and 5 of these offsets exceed 1.5 seconds (severe lag), the system constructs a set of startup changes and calculates the mean (e.g., +0.9 seconds) and variance of the positive offsets. The system assesses the daily language lag by dividing the daily mean of positive offsets by the historical baseline's allowable fluctuation range (e.g., 0.3 seconds) to calculate the ratio. This ratio indicates that the average daily reaction time is three times the upper limit of normal fluctuation. The system combines this ratio with interaction frequency density to map it to a score of 0-10, establishing a language priming offset. In this example, due to multiple significant delays, the system determined the language priming offset to be 7.5, indicating a clear sign of cognitive lag.

[0030] Specifically, such as Figure 2 , 5 As shown, the sleep offset parsing module includes: The stage information extraction submodule extracts the stage tag sequence recorded by the sleep monitoring device, identifies the start and end times of the daily light sleep stage, deep sleep stage and REM stage, calculates the duration ratio of adjacent stages, and obtains the stage transition ratio set. The sleep monitoring device (piezoelectric smart mattress) recorded the sleep stage tag sequence from 22:00 yesterday to 06:00 today. The device categorizes sleep into wakefulness, light sleep, deep sleep, and REM sleep stages based on body movement and heart rate variability. As shown in Table 1, the system identifies the start and end times of each stage daily.

[0031] Table 1. Record of Nighttime Sleep Stages in the Elderly (Partial Excerpt) As shown in Table 1, the system calculates the duration ratio of adjacent stages. For example, the transition from Stage 1 (Light, 30 min) to Stage 2 (Deep, 30 min) has a ratio of... Phase 2 (Deep, 30 min) transitions to Phase 3 (Light, 20 min), with the following ratio: The system iterates through all adjacent stages throughout the night to obtain a set of stage transition ratios.

[0032] The conversion trend analysis submodule calls the stage conversion ratio set, analyzes the sorting changes of the stage conversion ratio in time sequence, determines the direction of change of each stage conversion, and obtains the stage conversion trend parameters. The system calls upon a set of stage transition ratios to analyze the temporal changes in these ratios. The system focuses on the "fragmentation" trend, specifically the ability to maintain deep sleep stages. In a normal sleep structure, deep sleep accounts for a higher proportion and transitions are more stable in the first half of the night. The system determines the direction of change in each stage transition. For example, if it finds a continuous cycle of "Light->Wake->Light" starting at 02:00, with a decreasing duration (e.g., 20min->5min->15min), indicating significant fluctuations and shortening in the ratio sequence, the system calculates the first-order difference of the transition ratio sequence and counts the number of sign flips (i.e., the frequency of fluctuations). Assuming 30 transitions occur throughout the night, with 22 sign flips, and the Wake stage intervention frequency in the second half of the night is 30% higher than the historical average, the system parameterizes these characteristics to obtain a stage transition trend parameter, which quantifies the degree of disruption to the continuity of sleep structure.

[0033] The structural weight combination submodule combines the structural importance weights of each sleep stage with the stage transition trend parameters to establish a sleep transition offset. Based on the stage transition trend parameter (quantified as 0.65, indicating moderate to high fragmentation), the system combines the structural importance weights of each sleep stage. The system sets weights according to the consensus in psychiatry regarding sleep characteristics of depression: weight for deep sleep stage (N3). (Because reduced deep sleep is a significant feature), REM phase weighting (Focus on REM latency), weighting of light sleep stage The system calculates the deviation rate between the actual total duration of each stage and the standard norm for that age group (e.g., deep sleep should account for 15%-20%). Assuming Subject_A's total deep sleep duration that day was only 45 minutes (approximately 10%, deviation rate -33%), and the REM latency shortened to 50 minutes (deviation rate +40%), the system weighted and integrated the stage deviation rate with the transition trend parameter. Calculated The system normalizes this result and establishes a sleep transition offset of 8.35 (out of 10), indicating a severe shift in sleep structure.

[0034] Specifically, such as Figure 2 , 6 As shown, the rhythm aggregation triggering module includes: The anchor point drift calculation submodule extracts and analyzes the time points of the elderly’s daily behavioral anchor points based on behavioral connection offset, language initiation offset, and sleep transition offset, compares the time difference of behavioral anchor points on adjacent dates, and generates an anchor point drift set. Based on the behavioral transition offset (65), language initiation offset (7.5), and sleep transition offset (8.35), the system proceeds to the rhythm analysis stage. The system extracts and analyzes the time points of the elderly person's daily behavioral anchors. The system defines "behavioral anchors" as key events with high repetition and relatively fixed times each day, including: wake-up time (first time getting out of bed), breakfast time (first time the refrigerator door is opened), nap start time, and bedtime. The system uses sensor data to lock the daily anchors: wake-up time 06:15, breakfast time 08:30, and bedtime 21:45. The system retrieves the average anchors from the past 30 days: wake-up time 06:30, breakfast time 07:30, and bedtime 22:30. The system compares the time differences of behavioral anchors on adjacent dates (daily and average): wake-up 15 minutes earlier (-15), breakfast 60 minutes later (+60), and bedtime 45 minutes earlier (-45). The system converts these time differences to absolute values ​​or retains their signs, generating a set of anchor drift values. .

[0035] The direction aggregation filtering submodule calls the anchor point drift set, analyzes the directional relationship of multiple anchor point drifts, filters drifts with consistent directions, calculates the proportion of drifts with consistent directions, and obtains the aggregation direction ratio parameter. The system calls upon a set of anchor point drift values ​​to analyze the directional relationships between multiple anchor point drift values. The system focuses on the contradictory phenomena of "early awakening" and "delayed morning activity," as well as the characteristics of "overall rhythm shifting forward or backward." In the example, earlier waking (-15) and earlier falling asleep (-45) exhibit a consistent "rhythm shift," which aligns with the common symptom of phase advance in the diurnal rhythm in geriatric depression; while a significantly delayed breakfast (+60) reflects a lack of motivation. The system filters for drift values ​​with consistent direction (here, the number of time points with negative shifts, a total of 2), and calculates the proportion of drift values ​​with consistent direction. Simultaneously, the system calculates the weighted average of the drift amount, setting breakfast as the highest weight (reflecting appetite and motivation). The system comprehensively calculates the aggregation direction ratio parameter, which reflects not only the amplitude of the drift but also the synergy of the drift direction; a higher value indicates that the circadian rhythm disorder is more systematic rather than accidental.

[0036] The risk assessment output submodule determines the degree of abnormal deviation in the elderly's daily rhythm based on the aggregation direction ratio parameter, combined with the behavioral deviation trend structure and the daily language slowness, assesses the degree of risk of elderly depression, and establishes behavioral status early warning information. Based on the aggregation direction ratio parameter (0.75), combined with the behavioral shift trend structure (65 / 100) and daily language delay level (7.5 / 10), the system employs a multi-dimensional feature fusion algorithm to assess the degree of geriatric depression risk. The system constructs a risk assessment vector. (These correspond to normalized values ​​for behavior, language, rhythm, and sleep, respectively). The system calculates the magnitude of this vector or calculates the probability value using a pre-trained logistic regression model. In this example, due to significant language delay, fragmented sleep structure, and pre-rhythm shift, the calculated comprehensive risk score is 82 (out of 100). The system compares this to a risk grading standard (0-40 low risk, 41-70 medium risk, 71-100 high risk), classifying the elderly person's abnormal circadian rhythm deviation as "severe" and assessing the risk of geriatric depression as "high risk." The system establishes a behavioral status warning, including: "High-risk warning: Significant sleep fragmentation, insufficient morning motivation, and delayed language response detected; attention is recommended."

[0037] Specifically, such as Figure 2 , 7 As shown, the synchronization loss quantization module includes: The abnormal segment marking submodule monitors the continuous sequences of heart rate, skin conductance, respiratory and body temperature signals based on behavioral status warning information. By calculating the rate of change of the difference between continuous sampling points, it determines the fluctuation rate of each sampling point, filters and marks abnormal segments of each signal, and obtains a set of physiological abnormal segments. Based on the behavioral status warning information (high risk), the system activates the synchronous loss measurement module for physiological verification. The system monitors continuous sequences of heart rate (HR), electrical skin conductance (EDA), respiratory rate (RESP), and body temperature (TEMP) signals, with a sampling frequency set to 50Hz. The system calculates the rate of change of the difference between adjacent sampling points in the continuous sequences of heart rate, EDA, respiratory rate, and body temperature signals to form a rate of change difference sequence, and records it according to the time index, corresponding to the time axis of the original signal. The system selects a sedentary period from 10:00 to 11:00 on the same day and arranges the rate of change difference sequence from smallest to largest. As shown in Table 2, the system sets a reference fluctuation band.

[0038] Table 2. Threshold setting table for the rate of change of physiological signal difference (example) An upper threshold (e.g., heart rate of 5.55) is set based on the gap between the upper limit of the reference fluctuation band and the corresponding adjacent parameter segment above it, and a lower threshold (e.g., heart rate of -5.25) is set based on the gap between the lower limit of the reference fluctuation band and the corresponding adjacent parameter segment below it. When the rate of change of the difference continuously exceeds the upper threshold for a time interval (e.g., lasting 3 seconds), the system marks the corresponding time interval as a drastic abnormal segment; when the rate of change of the difference continuously falls below the lower threshold for a time interval, the corresponding time interval is marked as a decaying abnormal segment. The system iterates through the four types of signals, records the drastic abnormal segments and decaying abnormal segments according to their start and end times, and summarizes them into a set of physiological abnormal segments.

[0039] The synchronization window classification submodule calls the physiological abnormal segment set, analyzes the overlap of abnormal segments of each physiological signal within each time window, determines the degree of abnormal overlap, classifies them into synchronization abnormal windows, and generates a synchronization abnormal window set. The system retrieves a set of abnormal physiological segments and divides the monitoring time of the day into sliding time windows with a width of 60 seconds. The system analyzes the overlap of abnormal segments of various physiological signals within each time window. For example, within the time window of 10:15:00 to 10:16:00, the system detects a "drastic change abnormal segment" in the heart rate signal from 10:15:20 to 10:15:25, and simultaneously, a "drastic change abnormal segment" in the electrodermal signal from 10:15:22 to 10:15:28, with an overlap duration of 3 seconds. The system determines the degree of abnormal overlap; if the overlap involves two or more types of signals and the overlap duration exceeds 2 seconds, it determines that there is non-specific synchronous activation of autonomic nervous system function within that window (usually associated with anxiety or mood disorders). The system categorizes this window as a synchronous abnormal window and records the combination of signal types involved in the synchronization (e.g., HR+EDA). The system iterates through all windows throughout the day to generate a set of synchronous abnormal windows.

[0040] The physiological indicator output submodule analyzes the proportion of synchronization abnormal windows in all windows based on the set of synchronization abnormal windows, and establishes physiological synchronization abnormality information. Based on the set of synchronization anomaly windows, the system counted a total of 120 windows marked as "synchronization anomaly" on that day. The total number of valid monitoring windows for the entire day is known to be 960 (calculated based on a 16-hour awake time). The system analyzes the proportion of synchronization anomaly windows among all windows and calculates the synchronization anomaly rate. The system incorporates a baseline control, indicating that the rate of physiological synchronization abnormalities in normal elderly individuals under calm conditions is typically below 5%. The system compares the calculated final value of 12.5% ​​with the preset normal range (0%-5%). This result suggests that Subject_A's physiological system is in a highly unstable stress state, exhibiting significant autonomic nervous system dysfunction. This result, along with the aforementioned high-risk warnings from the behavior, sleep, and language modules, provides strong physiological evidence, confirming the reliability of the depression risk assessment. The system establishes this percentage (12.5%) and specific signal combination characteristics (dominated by HR and EDA) as physiological synchronization abnormality information, serving as the physiological support data for the final warning report, and is transmitted to the caregiver's terminal or medical assistance platform.

[0041] Please see Figure 8 The early warning method for predicting depressive state in the elderly based on multimodal data is executed based on the aforementioned early warning system for predicting depressive state in the elderly based on multimodal data, and includes the following steps: S1: Extract the trajectory point sequence of the positioning device, analyze the time stamp of adjacent trajectory points, divide the continuous trajectory segments, compare the start and end coordinates of the trajectory segments to calculate the displacement difference, analyze the direction and speed of movement to form direction difference and speed difference, form behavior offset trend structure, and form behavior connection offset. S2: Based on the behavioral connection offset, analyze the prompting time and the first utterance time, calculate the start delay, analyze the start delay sequence to establish a reference benchmark, and perform differential processing on the start delay and the reference interval to obtain the start offset, forming the language start offset. S3: Extract the stage label sequence recorded by the sleep monitoring device, calculate the proportion of duration of adjacent stages to obtain the stage transition ratio, analyze the sorting change of the transition ratio in time sequence to form the transition trend direction, and combine it with the stage weight to form the sleep transition offset. S4: Based on the behavioral connection offset, language initiation offset, and sleep transition offset, analyze the daily behavioral anchor time, compare the time difference between adjacent dates to form the anchor drift, and combine the behavioral offset trend structure and the daily language sluggishness to output behavioral status warning information. S5: Based on behavioral status warning information, monitor and analyze continuous sequences of heart rate, skin conductance, respiration and body temperature signals, mark abnormal segments, and determine the degree of abnormal superposition and classify synchronous abnormal windows by analyzing the overlap of abnormal segments of physiological signals within each time window. Based on the proportion of synchronous abnormal windows in all windows, physiological synchronous abnormal information is formed.

[0042] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An early warning system for predicting depressive states in the elderly based on multimodal data, characterized in that, The system includes: The trajectory offset acquisition module extracts the trajectory point sequence of the positioning device, analyzes the time stamps of adjacent trajectory points, divides continuous trajectory segments, compares the start and end coordinates of trajectory segments to calculate the displacement difference, analyzes the direction and speed of movement to form direction and speed differences, forms a behavioral offset trend structure, and forms behavioral connection offset. Based on the behavior connection offset, the language delay extraction module analyzes the prompting time and the first utterance time, calculates the start delay, analyzes the start delay sequence to establish a reference benchmark, and performs differential processing on the start delay and the reference interval to obtain the start offset, forming the language start offset. The sleep shift analysis module extracts the stage label sequence recorded by the sleep monitoring device, calculates the proportion of duration of adjacent stages to obtain the stage transition ratio, analyzes the sorting change of the transition ratio in time sequence to form the transition trend direction, and combines it with the stage weight to form the sleep transition shift degree. The rhythm aggregation triggering module analyzes the daily behavior anchor time based on the behavior connection offset, language initiation offset, and sleep transition offset, compares the time difference between adjacent dates to form the anchor drift, and outputs behavior status warning information by combining the behavior offset trend structure and the daily language sluggishness.

2. The early warning system for predicting depressive states in the elderly based on multimodal data according to claim 1, characterized in that, The behavioral connection offset includes the trajectory connection offset intensity distribution, the number of times the trajectory connection offset occurs, and the trajectory connection offset duration range. The language initiation offset includes the initiation delay offset magnitude level, the initiation delay offset frequency, and the initiation delay fluctuation dispersion. The sleep transition offset includes the sleep stage transition stability coefficient, the sleep stage structure offset direction marker, and the sleep stage dwell time offset ratio. The behavioral state warning information includes the geriatric depression risk grading results, abnormal behavior pattern feature items, and warning prompt generation time records.

3. The early warning system for predicting depressive states in the elderly based on multimodal data according to claim 1, characterized in that, The trajectory offset acquisition module includes: The trajectory segment generation submodule extracts the trajectory point sequence of the positioning device, analyzes the time stamps of adjacent trajectory points, divides continuous trajectory segments, analyzes the coordinate information of the start and end points of the trajectory segments, calculates the displacement difference, and generates the trajectory segment information. The connection change analysis submodule analyzes the movement direction of the trajectory segment based on the information of the trajectory segment, forms direction parameters, compares the direction parameters of adjacent continuous trajectory segments to obtain the direction difference, compares the movement speed of adjacent trajectory segments to obtain the speed difference, and combines the direction difference and speed difference to obtain the connection change index. The trend structure determination submodule analyzes the changes in all trajectory connections within each cycle based on the connection change index, analyzes the changes in connection characteristics between trajectory segments, determines the stable characteristics of behavior, constructs the behavior offset trend structure, and establishes the behavior connection offset.

4. The early warning system for predicting depressive states in the elderly based on multimodal data according to claim 3, characterized in that, The language delay extraction module includes: The startup delay calculation submodule obtains the prompt time and the first speech time in the voice dialogue scenario based on the behavior connection offset, analyzes each set of prompt and speech times, calculates the startup delay parameter between the two time points, and establishes a startup delay parameter set. The offset differential processing submodule calls the set of startup delay parameters, analyzes the startup delay parameter sequence of continuous dialogue and arranges them according to the startup delay parameters, determines the center position parameter of the arranged sequence and uses it as the startup delay reference benchmark, performs differential processing on each startup delay parameter and the reference benchmark to obtain the set of startup offsets; The sluggishness trend assessment submodule analyzes all sluggishness changes within the period based on the set of sluggishness offsets, constructs a set of sluggishness changes, assesses the daily language sluggishness, and establishes a language sluggishness offset.

5. The early warning system for predicting depressive states in the elderly based on multimodal data according to claim 4, characterized in that, The sleep offset analysis module includes: The stage information extraction submodule extracts the stage tag sequence recorded by the sleep monitoring device, identifies the start and end times of the daily light sleep stage, deep sleep stage and REM stage, calculates the duration ratio of adjacent stages, and obtains the stage transition ratio set. The conversion trend analysis submodule calls the set of stage conversion ratios to analyze the sorting changes of the stage conversion ratios in time sequence, determines the direction of change of each stage conversion, and obtains the stage conversion trend parameters. The structural weight combination submodule combines the structural importance weights of each sleep stage with the stage transition trend parameters to establish a sleep transition offset.

6. The early warning system for predicting depressive states in the elderly based on multimodal data according to claim 5, characterized in that, The rhythm aggregation triggering module includes: The anchor point drift calculation submodule extracts and analyzes the time points of the elderly’s daily behavioral anchor points based on the behavioral connection offset, language initiation offset, and sleep transition offset, compares the time difference of behavioral anchor points on adjacent dates, and generates an anchor point drift set. The direction aggregation filtering submodule calls the anchor point drift set, analyzes the directional relationship of multiple anchor point drifts, filters drifts with consistent directions, calculates the proportion of drifts with consistent directions, and obtains the aggregation direction ratio parameter. The risk assessment output submodule, based on the aggregation direction ratio parameter, combined with the behavioral deviation trend structure and daily language slowness, determines the degree of abnormal deviation in the elderly's daily rhythm, assesses the degree of risk of geriatric depression, and establishes behavioral status early warning information.

7. The early warning system for predicting depressive states in the elderly based on multimodal data according to claim 1, characterized in that, The system also includes: The synchronization loss quantification module monitors and analyzes the continuous sequences of heart rate, skin conductance, respiration and body temperature signals based on the behavioral state warning information, marks abnormal segments, and judges the degree of abnormal superposition and classifies the synchronization abnormal windows by analyzing the overlap of abnormal segments of physiological signals within each time window. Based on the proportion of the synchronization abnormal window in all windows, physiological synchronization abnormal information is formed. The abnormal physiological synchronization information includes a summary record of abnormal segments, the results of abnormal segment overlap analysis, and a summary of abnormal percentage information.

8. The early warning system for predicting depressive states in the elderly based on multimodal data according to claim 7, characterized in that, The synchronous offset scaling module includes: The abnormal segment marking submodule monitors the continuous sequence of heart rate signal, skin conductance signal, respiratory signal and body temperature signal according to the behavioral state warning information. By calculating the rate of change of the difference between continuous sampling points, it judges the fluctuation rate of each sampling point, filters and marks abnormal segments of each signal, and obtains a set of physiological abnormal segments. The synchronization window classification submodule calls the set of physiological abnormal segments, analyzes the overlap of abnormal segments of each physiological signal within each time window, determines the degree of abnormal overlap, classifies them into synchronization abnormal windows, and generates a set of synchronization abnormal windows. The physiological indicator output submodule analyzes the proportion of the synchronization abnormal windows in all windows based on the set of synchronization abnormal windows, and establishes physiological synchronization abnormal information.

9. The early warning system for predicting depressive states in the elderly based on multimodal data according to claim 8, characterized in that, The process of filtering and marking abnormal segments of each signal is as follows: The rate of change of the difference between adjacent sampling points is calculated for continuous sequences of heart rate, skin conductance, respiratory and body temperature signals to form a rate of change of difference sequence, and recorded according to time index to correspond with the time axis of the original signal; The difference change rate sequence is arranged from smallest to largest, and the continuous parameter segment located in the center of the arrangement is set as the reference fluctuation band; The upper limit threshold is set by the gap between the upper limit of the reference fluctuation band and the corresponding upper adjacent parameter segment, and the lower limit threshold is set by the gap between the lower limit of the reference fluctuation band and the corresponding lower adjacent parameter segment. When the rate of change of the difference continuously exceeds the upper threshold to form a time interval, the corresponding time interval is marked as a sudden change abnormal segment; when the rate of change of the difference continuously falls below the lower threshold to form a time interval, the corresponding time interval is marked as a decay abnormal segment. The abrupt change anomaly segments and the decay anomaly segments are recorded according to their start and end times and summarized into a set of physiological anomaly segments.

10. An early warning method for predicting depressive states in the elderly based on multimodal data, characterized in that, The early warning system for predicting depressive states in the elderly based on multimodal data, as described in any one of claims 1-9, comprises the following steps: S1: Extract the trajectory point sequence of the positioning device, analyze the time stamp of adjacent trajectory points, divide the continuous trajectory segments, compare the start and end coordinates of the trajectory segments to calculate the displacement difference, analyze the direction and speed of movement to form direction difference and speed difference, form behavior offset trend structure, and form behavior connection offset. S2: Based on the behavior connection offset, analyze the prompt time and the first utterance time, calculate the start delay, analyze the start delay sequence to establish a reference benchmark, perform differential processing on the start delay and the reference interval to obtain the start offset, and form the language start offset. S3: Extract the stage label sequence recorded by the sleep monitoring device, calculate the proportion of duration of adjacent stages to obtain the stage transition ratio, analyze the sorting change of the transition ratio in time sequence to form the transition trend direction, and combine it with the stage weight to form the sleep transition offset. S4: Based on the behavior connection offset, language initiation offset, and sleep transition offset, analyze the daily behavior anchor time, compare the time difference between adjacent dates to form the anchor drift, and combine the behavior offset trend structure and the daily language sluggishness to output behavior status warning information. S5: Based on the behavioral state warning information, monitor and analyze the continuous sequence of heart rate, skin conductance, respiration and body temperature signals, mark abnormal segments, and determine the degree of abnormal superposition and classify synchronous abnormal windows by analyzing the overlap of abnormal segments of physiological signals in each time window. Based on the proportion of synchronous abnormal windows in all windows, physiological synchronous abnormal information is formed.

Images