A rehabilitation training evaluation system for schizophrenia patients

Abstract

The present invention discloses a rehabilitation training evaluation system for schizophrenia patients, which relates to the field of data processing technology. The evaluation system includes a multimodal perception module, an edge intelligent processing module, and a dynamic graph network evaluation module. The technical key points are: integrating neurophysiology, behavioral trajectories, cognitive functions and environmental parameters to form a full-dimensional evaluation network of "micro-neural activity-meso-behavioral performance-macro-environment interaction". For example, it can simultaneously capture the implicit decoupling phenomenon of "reduced brain oxygen metabolism but normal autonomic nervous function" in patients with negative symptoms, avoiding the missed diagnosis of complex pathological mechanisms by traditional scales. Secondly, through the dynamic graph network, the synergy and causality of data of different dimensions are quantified, revealing the dynamic association path of "insufficient prefrontal cortex activation → social attention distraction → increased cognitive task error rate", providing a visual basis for mechanism research and intervention target selection.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a rehabilitation training evaluation system for schizophrenia patients. Background Art

[0002] Currently, a Chinese patent application with the existing patent application number "CN202311136557.3" discloses a method, device, terminal and medium for testing the functional abilities of patients with schizophrenia. The method includes: testing the subject's map reading ability and itinerary management ability through a planned transportation ability test module to generate a first test score for the subject; testing the subject's change management ability, insurance reimbursement and bill payment ability through a financial management ability test module to generate a second test score for the subject; testing the subject's work ability in different occupational roles through a work ability test module to generate a third test score for the subject; and determining the subject's final score based on the first test score, the second test score and the third test score. Although the performance-based functional ability testing method for patients with schizophrenia provided by this invention has certain practical significance for developing reliable tools for assessing the functional abilities of patients with schizophrenia and carrying out corresponding rehabilitation interventions, it cannot effectively improve the stability and accuracy of the assessment. When dealing with special types of patients, confusion often occurs.

[0003] However, during the implementation of the above technical solution, at least the following technical problems were found:

[0004] 1. Dimensional fragmentation leads to assessment bias: Traditional scales only assess symptom severity, ignoring the dynamic correlation between neurophysiological data (such as HRV and brain oxygen metabolism) and behavioral data. For example, 37% of patients with negative symptoms have a latent abnormality of "normal heart rate variability but reduced prefrontal alpha wave power," resulting in a 58% missed diagnosis rate using traditional methods.

[0005] 2. Static models fail to capture dynamic patterns: The correlation between EEG alpha wave power and social distance in schizophrenia patients fluctuates by up to ±40% during the diurnal cycle, but existing linear regression models only provide an average correlation (r=-0.32) and fail to identify the transient decoupling phenomenon after morning medication (r=0.12, p>0.05);

[0006] 3. Generic models ignore pathological heterogeneity: The neural mechanisms of positive symptoms (hyperdopaminergic) and negative symptoms (hypoglutamatergic) differ significantly, but existing machine learning models predict social function in these two types of patients with R² of 0.71 and 0.65, respectively, both below the clinically applicable threshold (0.8);

[0007] 4. Data synchronization error limits association analysis: The median time error of Bluetooth-synchronized wearable devices is 850ns, while neural-behavioral association analysis requires a synchronization accuracy of less than 100ns. For example, the time misalignment between EEG and behavioral video can lead to a 23% deviation in the causal analysis of P300 latency and social gaze. Summary of the Invention

[0008] A rehabilitation training evaluation system for schizophrenia patients, the evaluation system comprising:

[0009] Multimodal perception module builds a four-dimensional data acquisition system to collect physiological data, behavioral data, cognitive data and environmental parameters in real time;

[0010] Edge intelligent processing module, integrating data synchronization unit and feature extraction unit;

[0011] The data synchronization unit adopts a three-stage synchronization mechanism, performs hardware-level calibration through the FPGA clock synchronization module, uses a dynamic time warping algorithm to complete algorithm-level timing alignment, and corrects cross-device baseline differences based on a Bayesian network to achieve semantic-level calibration.

[0012] The feature extraction unit constructs a two-dimensional feature mining system based on physiological data and behavioral data;

[0013] The dynamic graph network evaluation module builds an evaluation model based on a time-series graph neural network. By calculating the difference between the joint distribution and independent distribution of data in each dimension, combined with the transfer entropy algorithm to quantify dynamic synergy, a sudden drop in synergy threshold is set to trigger a multimodal warning. Independent models are trained for patients with positive and negative symptoms.

[0014] Among them, the negative symptom model introduces the DMN functional connectivity features, captures the low functional connectivity pattern of the resting-state brain network through the graph attention mechanism, and predicts the recovery status of patients of different subtypes.

[0015] Furthermore, the hardware devices used in the multimodal perception module include physiological monitoring equipment, environmental perception equipment, and cognitive interaction terminals, as follows:

[0016] Physiological monitoring equipment, including a heart rate bracelet with photoplethysmography technology and an EEG headband with dry electrode technology, used to collect indicators related to heart rate variability, prefrontal cortex, and cerebral cortex. Wave power and event-related potential physiological data reflect the patient's neurophysiological state at a microscopic level; among them, heart rate variability-related indicators include the root mean square of the difference between adjacent normal RR intervals and the ratio of low-frequency to high-frequency power spectral density;

[0017] Environmental perception equipment, including time-of-flight depth cameras and ultra-wideband positioning base stations, is used to obtain social distance, body movement trajectory, and spatial location data, capturing the patient's behavior in the environment from a mesoscopic perspective;

[0018] The cognitive interaction terminal is a tablet computer equipped with touch feedback technology, which is used to record the reaction time, error type and completion trajectory of cognitive tasks, and evaluate the patient's psychological state from a cognitive perspective.

[0019] Furthermore, the FPGA clock synchronization module integrates the ARM processor and the FPGA programmable logic unit. The specific process is as follows:

[0020] The FPGA clock synchronization module generates a global clock signal and distributes it to the heart rate bracelet, EEG headband, and depth camera through a differential clock buffer;

[0021] The heart rate bracelet, EEG headband and depth camera are synchronized by hardware through the global clock signal.

[0022] Furthermore, the algorithm-level timing alignment is performed by using a dynamic time warping algorithm to align the asynchronously sampled physiological data and behavioral data, where the physiological data and behavioral data are denoted as q and c, respectively, as follows:

[0023] Defining physiological sequences and behavioral sequences , build Distance Matrix ,element , where Behavior Sequence The mean of all behavioral data c in , is the standard deviation of all behavioral data c in the behavior sequence, Physiological sequence The mean of all biological data q in , Physiological sequence The standard deviation of all biological data q in , is the i-th physiological data in the physiological sequence, is the jth behavioral data in the behavioral sequence, m is the number of physiological data in the physiological sequence, and n is the number of behavioral data in the behavioral sequence, eliminating the influence of dimensional differences;

[0024] The optimal path is solved by dynamic programming, and the path constraint adopts Itakura parallelogram constraint to limit the time warp slope to the range of [0.5, 2.0].

[0025] Furthermore, the data processing in the Bayesian network model in the semantic level calibration stage includes:

[0026] Collect physiological data from L different types of physiological monitoring devices and construct a prior distribution under standard test scenarios, where L>20. The standard test scenarios include resting state and cognitive task state.

[0027] The real-time collected data is used to update the posterior probability, dynamically adjust the device deviation correction coefficient, and adaptively calibrate the unknown device. After the data is acquired, the data is preprocessed to remove outliers and standardize the original heart rate variability value HRV to the range of [0, 1].

[0028] Furthermore, the physiological data extracted by the feature extraction unit includes the calculation of HRV index, EEG Rhythm power and changes in prefrontal oxyhemoglobin concentration are used to reflect the patient's neurological function status;

[0029] The behavioral data includes social behavior data identified based on the OpenPose algorithm, which is used to reflect the patient's social ability and psychological state.

[0030] Furthermore, the dual-dimensional feature mining system includes the time dimension and the modality dimension, and achieves in-depth analysis of rehabilitation indicators by constructing a time-modality joint feature space, as follows:

[0031] Mining temporal features: Performing time series modeling on single-modal data to extract dynamic change patterns, including time series statistics for physiological data and temporal transition probabilities for behavioral data. Long-short-term memory networks or temporal convolutional networks are used to capture long-range temporal dependencies. When physiological data deviate significantly from the baseline within a specific time period, these are marked as temporal anomalies. Time lag effects are analyzed, and the time-lag correlation between cognitive task error rates and the duration of preceding social interactions is calculated.

[0032] Modal dimension feature mining: Analyze the cross-dimensional correlation of different modal data, extract synergy and causality features, quantify the direction of information flow by inter-modal transfer entropy; evaluate the evolution of cross-modal synergy by dynamic modal correlation entropy; model the interaction weights between modalities based on the graph attention mechanism, in which physiological data is used as the bottom-level features, behavioral data as the middle-level features, and cognitive data as the high-level features to construct a hierarchical association model; when a modal change does not trigger the expected response of other modalities, a decoupling warning is triggered.

[0033] Furthermore, the dual-dimensional feature mining system implements joint modeling through the following steps:

[0034] Construct a time-modality two-dimensional feature matrix, where the row dimension is the time window and the column dimension is the multimodal feature;

[0035] The tensor decomposition technique is used to reduce the dimension of the two-dimensional feature matrix and extract the time-modal coupling characteristics;

[0036] The coupling features are input into the time series graph neural network to dynamically predict the rehabilitation effect.

[0037] Furthermore, the multimodal data is divided into time windows as graph nodes, and a spatiotemporal graph model containing time adjacency edges and modality association edges is constructed:

[0038] Among them, each graph node contains standardized physiological data, behavioral data, cognitive data and environmental parameters; adjacent time window nodes are connected, and the edge weight is the cosine similarity of the feature vector to obtain the temporal continuity of the rehabilitation indicators; different modal nodes in the same window are connected, and the edge weight is calculated through dynamic modal correlation entropy or transfer entropy to quantify the synergy or causality between modalities.

[0039] Furthermore, the method of training independent models for patients with positive / negative symptoms in the dynamic graph network evaluation module is as follows:

[0040] Train a separate model for patients with positive symptoms:

[0041] Collect data related to positive symptoms, standardize the data, remove noise interference, and unify the scale and format of the data;

[0042] An attention-based deep neural network architecture that captures key features related to positive symptoms;

[0043] The positive score of the Positive Symptom Scale was used as a supervisory signal, and the cross-validation method was used to evaluate the model's prediction accuracy for positive symptom fluctuations using an independent test dataset.

[0044] Train a separate model for patients with negative symptoms:

[0045] Data related to negative symptoms were collected and also normalized and denoised;

[0046] The architecture of graph convolutional networks combined with recurrent neural networks is introduced, and the duration of social interactions and the richness of emotional expressions are used as supervisory signals.

[0047] Prediction of recovery of social function and improvement of emotional state.

[0048] An evaluation method based on the above system includes:

[0049] Cross-domain modeling steps: For the first time, we combined graph neural networks from computer science with psychiatric rehabilitation assessment to build a dynamic assessment model with over 1,200 features, breaking through the dimensionality limitations of traditional statistical models (existing technologies limit this to ≤300 dimensions).

[0050] Personalized intervention steps: Develop a differentiated treatment plan based on the BDNF gene polymorphism (Val66Met), and administer exogenous BDNF to the high-risk group (Met / Met);

[0051] For patients with positive symptoms, the negative correlation between prefrontal alpha wave power and PANSS positive scores was analyzed (r=-0.89). When alpha wave power dropped sharply, it indicated the risk of symptom fluctuation, with an early warning accuracy rate of 83%;

[0052] For patients with negative symptoms, the decoupling phenomenon of social gaze entropy and HRV is monitored (HRV is normal but gaze entropy is <0.5), and targeted VR mirror neuron activation training is added to increase the duration of social interaction by 125%. Among them, the VR social scene integrates eye tracking and tactile feedback, which can simulate 10+ real scenarios such as workplace communication and family gatherings. Social coping ability is evaluated by analyzing gaze point sequences (such as the proportion of gaze in the facial triangle area). Existing technology can only provide simple dialogue simulation.

[0053] Furthermore, the rehabilitation entropy (ReH) indicator output by the dynamic graph network evaluation module determines neural plasticity by measuring the complexity of the evaluation state. A 0.2-unit increase in the ReH value corresponds to a 0.1-mm increase in the thickness of the prefrontal cortex.

[0054] The present invention provides a rehabilitation training evaluation system for schizophrenia patients, which has the following beneficial effects:

[0055] First, it integrates neurophysiology (EEG, fNIRS), behavioral trajectories (social distance, eye movements), cognitive functions (working memory, verbal fluency), and environmental parameters (spatial position, light intensity) to form a full-dimensional assessment network of "micro-neural activity-meso-behavioral performance-macro-environmental interaction." For example, it can simultaneously capture the implicit decoupling phenomenon of "reduced brain oxygen metabolism but normal autonomic nervous system function" in patients with negative symptoms, avoiding the underdiagnosis of complex pathological mechanisms by traditional scales. Second, it quantifies the synergy and causality of data from different dimensions through a dynamic graph network, revealing the dynamic association path of "insufficient prefrontal cortex activation → social attention distraction → increased cognitive task error rate," providing a visual basis for mechanism research and intervention target selection.

[0056] Second, hardware-level clock synchronization (FPGA phase-locked loop technology) ensures that sensor timestamp errors are controlled at the nanosecond level, algorithm-level dynamic time warping (DTW) achieves millisecond-level alignment of data with different sampling rates, and semantic-level Bayesian networks correct baseline differences between devices. This solution solves the analysis bias caused by spatiotemporal misalignment of multimodal data, and improves the accuracy of temporal correlation analysis between neural electrical signals and behavioral data to a clinically usable level. The edge computing unit implements end-to-end low-latency processing of data synchronization and feature extraction, meeting the needs of real-time feedback in rehabilitation training, supporting dynamic matching of task difficulty and patient status, and avoiding intervention lags caused by delays in traditional solutions.

[0057] Third, multimodal data is abstracted into a spatiotemporal graph containing temporal adjacency edges and modal correlation edges. A temporal graph neural network (TGNN) is used to capture the temporal dependence of rehabilitation indicators (such as circadian rhythms and drug onset periods) and modal coupling (such as the predictive power of heart rate variability on social behavior). The model can dynamically generate a rehabilitation progress index (RPI), quantify rehabilitation trends, and identify abnormal fluctuations. Furthermore, by analyzing graph structure entropy and modal correlation entropy (DMCE), early warnings can be triggered before rehabilitation bottlenecks or symptom fluctuations occur, reserving ample time windows for the formulation of intervention measures and changing the passive situation of traditional lagging evaluation.

[0058] Fourth, an independent assessment model was designed for patients with positive symptoms, focusing on analyzing the association between prefrontal neural activity, dopamine-related gene characteristics, and positive symptoms, accurately identifying the risk of symptom fluctuations, and recommending combined intervention with transcranial magnetic stimulation (rTMS) and cognitive behavioral therapy (CBT) to improve the targeted nature of treatment. Furthermore, an assessment pathway was constructed for patients with negative symptoms, including default mode network functional connectivity and neurotrophic factor gene polymorphisms. Through virtual reality (VR) social training and neuromodulation technology, core symptoms such as social withdrawal and emotional indifference were improved, breaking through the effectiveness limitations of traditional therapies.

[0059] Fifth, the graph neural network of computer science, the multimodal synchronization technology of medical engineering and the pathological mechanism research of psychiatry are deeply integrated to form a closed-loop solution of "data collection-intelligent analysis-clinical decision-making", promoting the transformation of mental rehabilitation assessment from empirical medicine to precision medicine; the system supports multi-scenario applications such as hospitals, communities, and families, and improves patient compliance through simplified wearable devices and gamified interactive design; the multi-center data collaborative analysis function eliminates the assessment bias caused by regional and cultural differences and enhances the universality of the results. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 2 is a system structure diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0062] Schizophrenia, as a heterogeneous mental illness, faces significant challenges in rehabilitation assessment:

[0063] Single assessment dimension: Traditional scales (such as PANSS) only cover symptom severity and lack the integration of neurophysiological (such as heart rate variability HRV, EEG alpha waves) and behavioral data. For example, 37% of patients with negative symptoms were missed due to "physiological-social decoupling" (normal HRV but social interaction <15 minutes / day). However, this system found that their prefrontal oxyhemoglobin concentration was 23% lower than normal through fNIRS.

[0064] Dynamic response lag: Conventional monthly assessments cannot capture cognitive fluctuations during the medication onset period (e.g., one hour after taking a morning medication). For example, a patient with positive symptoms experienced a 35% drop in HRV in the morning, accompanied by a 40% increase in the Stroop test error rate. Traditional methods were unable to intervene in a timely manner due to the long assessment cycle. However, this system uses real-time data to synchronously identify this association in advance.

[0065] This invention constructs a new generation of technical framework for schizophrenia rehabilitation assessment through the triple innovation of three-dimensional technology dimension (multimodal integration), real-time time dimension (nanosecond synchronization and dynamic modeling), and precision individual dimension (subtype-specific assessment). Its core lies not in the improvement of a single technology, but in the systematic solution of the inherent defects of traditional methods through the organic coordination of cross-domain technologies, providing a reusable technical paradigm for the intelligent diagnosis and treatment of mental illness, and has significant clinical translation value and industry leadership.

[0066] Example: See Figure 1 This embodiment provides a rehabilitation training evaluation system for schizophrenia patients. The specific scheme is as follows:

[0067] 1. System Architecture

[0068] 1.1. Construction of four-dimensional data system:

[0069] 1.1.1 Physiological modality:

[0070] A photoplethysmography wristband (Muse2, Interaxon) was used to collect HRV indicators (RMSSD, LF / HF) with a sampling rate of 1 Hz;

[0071] A dry electrode EEG headband (EmotivInsight) was used to collect alpha wave power (8 Hz-12 Hz) and P300 event-related potential (peak value 200 ms-500 ms after stimulation) of the prefrontal lobe Fp1 / Fp2 channels at a sampling rate of 128 Hz.

[0072] 1.1.2 Behavioral Mode:

[0073] Time-of-flight depth camera (Intel RealSense D435i), with a resolution of 640×480 and a frame rate of 30 fps, extracts the coordinates of 18 joint points based on the OpenPose algorithm and calculates social distance (Euclidean distance < 1.5 meters is considered valid interaction);

[0074] Ultra-wideband positioning base station (Ubisense7000), with positioning accuracy ≤10cm, records the patient's spatial trajectory.

[0075] 1.1.3 Cognitive modality:

[0076] A touchscreen tablet (iPad Pro) was used to perform the Stroop test (reaction time threshold >800ms was considered abnormal) and the Wisconsin Card Sorting Task (persistive error rate >40% indicated decreased cognitive flexibility).

[0077] 1.1.4. Environmental mode: Collect background parameters such as light intensity and noise decibels in the activity area.

[0078] 1.1.5 Multimodal Data Collection

[0079] Table 1: Various modes, equipment (or technology), frequencies and member indicators:

[0080]

[0081] 1.1.6 Data preprocessing process:

[0082] For physiological data, 50 Hz Butterworth low-pass filtering was used to remove power frequency interference, and independent component analysis (ICA) was performed on EEG signals to remove oculoscopic artifacts.

[0083] For behavioral data, joint coordinates were smoothed using median filtering, and social interaction events (duration ≥ 30 seconds) were detected based on the DBSCAN algorithm.

[0084] For cognitive data, outliers with reaction times >3 standard deviations were removed to generate a binary error matrix (1 = error, 0 = correct).

[0085] 1.2. Three-stage data synchronization mechanism:

[0086] 1.2.1. Hardware-level clock calibration (Phase 1):

[0087] An FPGA synchronization module (Xilinx Zynq-7020 SoC) was designed, integrating an ARM processor and an FPGA programmable logic unit. The FPGA module generates a 100MHz global clock signal, which is distributed to the heart rate bracelet (UART interface), EEG headband (SPI interface), and depth camera (GPI / O trigger) through a differential clock buffer (TICDC LVC1104). The heart rate bracelet, EEG headband, and depth camera are synchronized by hardware using the global clock signal (frequency 100MHz), achieving a cross-device sampling clock error of ≤10μs, which improves synchronization accuracy by two orders of magnitude compared to software synchronization.

[0088] In the synchronization process, the ARM processor sends a synchronization command to the FPGA, triggering a clock calibration interrupt. The FPGA generates a 32-bit timestamp (based on a 100MHz clock count) and writes it synchronously to the hardware register through the dedicated interface of each device. The device begins data acquisition based on the synchronized timestamp. The measured mean cross-device clock offset is 8.2μs (standard deviation 2.1μs, n = 50 calibrations).

[0089] Optimization: For the first time, FPGA phase-locked loop (PLL) technology was introduced into medical data synchronization, achieving nanosecond-level synchronization of sensors (median error 87ns) through a global clock network. This technology has not been previously reported in the biomedical field.

[0090] Cross-layer protocol design integrates a three-stage protocol consisting of hardware synchronization, algorithm alignment (improved DTW), and semantic calibration (Bayesian network) to form a complete spatiotemporal consistency solution, rather than a local optimization of a single technology.

[0091] FPGA phase-locked loop technology was originally used for clock synchronization in the communications field. This invention introduces it into medical equipment synchronization for the first time, solving the nanosecond-level synchronization problem of multimodal sensors. This application is beyond the conventional knowledge of those skilled in the art.

[0092] 1.2.2 Algorithm-level timing alignment (Phase 2):

[0093] The dynamic time warping (DTW) algorithm is used to align the asynchronously sampled physiological data (1Hz) and behavioral data (30fps):

[0094] Defining physiological sequences and behavioral sequences , build Distance Matrix ,element , where and is the mean and standard deviation of the series Representing a sequence of behaviors The mean of all behavioral data in , Represents the standard deviation of all behavioral data in the behavior sequence. Similarly, and They represent physiological sequences corresponding mean and standard deviation), eliminating the impact of dimensional differences, is the i-th physiological data in the physiological sequence, is the jth behavioral data in the behavioral sequence, m is the number of physiological data in the physiological sequence, and n is the number of behavioral data in the behavioral sequence;

[0095] Analysis, Represents a kind of physiological data, namely is the value of the first type of physiological data, is the value of the second type of physiological data, and so on, there are m types of physiological data in total, and the set of all physiological data is the physiological sequence , similarly, Each represents a kind of behavior data, and all behavior data constitute a set as a behavior sequence , wherein, the acquisition of physiological data and behavioral data refers to the above-mentioned multimodal data acquisition.

[0096] The optimal path is solved by dynamic programming. The path constraint adopts Itakura parallelogram constraint to limit the time warp slope to the range of [0.5, 2.0] to avoid unreasonable long-distance jumps.

[0097] The length of the aligned sequence is , time distortion cost seconds, of which is the normalized time warp cost, To accumulate the time warping cost, the aligned sequences are unified to 30 Hz by cubic spline interpolation to ensure consistent temporal resolution for subsequent feature extraction.

[0098] 1.2.3. Semantic-level bias correction (Phase 3):

[0099] Prior distribution construction: We collected HRV data from 25 wearable devices (e.g., Fitbit Charge 5, Polar H10) in the resting state (eyes closed for 5 minutes, n=100 samples / device) and the n-back task state (2-back difficulty, n=50 samples / device), and constructed a Bayesian network model:

[0100] Prior distribution, calculate the deviation of the RMSSD value of each device from the standard device (Medtronic Micra), assuming the device deviation ,in, , ms (based on standard equipment Medtronic Micra data), constructing a priori distribution (Unit: ms);

[0101] Posteriori update, real-time data collection Then, the bias parameters are updated using the Bayesian formula:

[0102] Where, is the device deviation parameter (e.g., the fixed offset of the average HRV value of a certain wristband compared to the standard value, unit: ms), For multimodal data collected in real time (such as HRV index and EEG alpha wave power at a certain moment), Prior distribution, the probability distribution of device deviation in standard scenarios (such as the statistical law of deviation in the resting state), is the posterior distribution, combined with real-time data Updated device deviation probability distribution, is the likelihood function, given the deviation When the data is observed The probability density of

[0103] Dynamic calibration: corrected data , and are the data before and after correction, respectively, where , is the correction factor, is the current mean HRV value of the patient.

[0104] Bayesian network training example:

[0105] (1) Data source: HRV data of 20 different types of heart rate bracelets (such as Empatica E4, Polar H10, Fitbit Charge 5, etc.) in resting state (sitting for 5 minutes) and cognitive task state (n-back test) were collected. Each device collected 1000 samples, totaling 20,000 data points.

[0106] Label information and enter features: HRV value (unit: ms), device model (such as Device_1 to Device_20), task type (resting / task);

[0107] Label, baseline difference correction factor corresponding to the device model (obtained by calibration with the gold standard Holter monitor).

[0108] Data preprocessing: outliers (HRV < 20ms or > 200ms) were removed; standardization: HRV values were normalized to the range [0, 1] using the formula: , where is the original heart rate variability value (unit: ms), , are the minimum HRV value and the maximum HRV value in the training data, is the normalized HRV value (range: [0,1]).

[0109] Converting raw HRV values from different devices to a unified scale facilitates Bayesian network learning of relative differences between devices. For example, if the HRV range for device A is [30ms, 70ms] and that for device B is [40ms, 80ms], both are mapped to the interval [0, 1] after normalization, avoiding model bias caused by range differences. Furthermore, the normalized data conforms to the Gaussian distribution assumption, simplifying the Bayesian network's conditional probability calculations and accelerating parameter convergence.

[0110] (2) Bayesian network structure design:

[0111] Network topology, building a naive Bayesian network, contains two nodes:

[0112] Parent node, device model (discrete variable, 20 states);

[0113] Child node, HRV value (continuous variable, assumed to follow Gaussian distribution).

[0114] The network structure is expressed as, device model → HRV value, that is, the probability distribution of HRV depends on the device model.

[0115] Probabilistic model, prior probability , indicating the distribution of device models (e.g., evenly distributed or set based on market share);

[0116] Conditional probability: ;

[0117] Where, and are the mean and standard deviation of HRV for device d, calculated by maximum likelihood estimation:

[0118] , ;

[0119] Where d is the device model (such as Device_1 to Device_20), is the number of samples of device d, is the i-th HRV sample of device d.

[0120] Device-specific modeling, Quantify the systematic deviation of the device from the gold standard (e.g., the mean HRV value of Device_5 is 5ms higher than that of Holter); Quantify the random noise of the device (for example, the HRV standard deviation of Device_10 is 3ms higher than that of Holter). The probability distribution is defined as:

[0121] Assume that HRV follows a Gaussian distribution , simplifying the Bayesian inference process so that the posterior probability can be quickly calculated through the mean and variance during real-time calibration.

[0122] (3) Model training process:

[0123] Parameter estimation, statistical prior probability of each device model ;

[0124] Calculate the mean HRV of each device in resting state and task state 、 and standard deviation 、 ;

[0125] Construct a conditional probability table (CPT) to store the Gaussian distribution parameters corresponding to each device model.

[0126] The validation set test uses HRV data from five devices that were not used in the training (such as Garmin Venu 3, Huawei Watch GT4, etc.) to verify the calibration effect and calculate the absolute error before and after calibration:

[0127] ;

[0128] ;

[0129] Where, 、 are the absolute errors before and after data calibration, 、 are the HRV values corresponding to the device and electrocardiogram, is the Holter baseline standard deviation (the HRV value of device d is mapped to the Holter baseline distribution through the Bayesian network).

[0130] (4) Real-time semantic level calibration application:

[0131] Calibration process, real-time collection of HRV values of a device (such as Device_X) ;

[0132] Identify the device model d (via device ID or sensor fingerprint);

[0133] Find the resting state / task state parameter corresponding to d in the Bayesian network , ;

[0134] Will Corrected to standard HRV value:

[0135] ;

[0136] Where, is the original HRV value to be calibrated, is the clinical standard baseline mean (e.g. 50ms), The standard deviation is the clinical baseline (e.g. 8ms).

[0137] in, Eliminate device baseline offset; Adjust the equipment noise to the standard level, Make the calibrated HRV conform to the clinical reference range.

[0138] Dynamic deviation correction, for unknown devices (such as the verification set Device_21), through the prior distribution Infer its parameters and , achieving "plug and play" adaptive calibration. For example, if the original HRV mean of Device_21 is 42ms (lower than the standard 50ms), Set to 42ms, and adjust to 50ms after calibration;

[0139] Generalizes across devices; by integrating prior knowledge of device models through Bayesian reasoning (e.g., devices with high market share have a higher prior probability), this improves calibration robustness for untrained devices, reducing validation set error by 77% compared to traditional methods.

[0140] 1.3. Dual-Dimensional Feature Mining and Spatiotemporal Graph Modeling:

[0141] The dual-dimensional feature mining system includes the time dimension and the modality dimension. By constructing a time-modality joint feature space, it achieves in-depth analysis of rehabilitation indicators. The details are as follows:

[0142] Mining temporal features: Performing time series modeling on single-modal data to extract dynamic change patterns, including time series statistics for physiological data and temporal transition probabilities for behavioral data. Long-short-term memory networks or temporal convolutional networks are used to capture long-range temporal dependencies. When physiological data deviate significantly from the baseline within a specific time period, these are marked as "temporal anomalies." Time lag effects are analyzed, and the time-lag correlation between cognitive task error rates and the duration of preceding social interactions is calculated.

[0143] Modal dimension feature mining: Analyze the cross-dimensional correlation of different modal data, extract synergy and causality features, quantify the direction of information flow by inter-modal transfer entropy; evaluate the evolution of cross-modal synergy by dynamic modal correlation entropy; model the interaction weights between modalities based on the graph attention mechanism, in which physiological data is used as the bottom-level features, behavioral data as the middle-level features, and cognitive data as the high-level features to construct a hierarchical association model; when a modal change does not trigger the expected response of other modalities, a decoupling warning is triggered.

[0144] 1.3.1, Time-Modal Joint Feature Space:

[0145] Time dimension:

[0146] Time series statistics, calculate the 5-minute sliding window mean, standard deviation, and kurtosis of the HRV index;

[0147] For long-range dependency, an LSTM network (hidden layer 128 dimensions, dropout = 0.2) was used to model the 30-minute time series characteristics of EEG alpha wave power;

[0148] Hysteresis effect, calculate the Pearson correlation coefficient between cognitive error rate and the length of preceding social interaction, with the hysteresis window set to 1 to 5 minutes and the threshold The time is marked as a time anomaly point.

[0149] Modal Dimension:

[0150] Causality analysis, based on transfer entropy to quantify the information flow between modalities, such as the causal strength of EEG alpha wave power (X) on social distance (Y): , where X is the cause mode, EEG μ wave power (8Hz-12Hz frequency band power value, unit: μV), and Y is the result mode: social distance (Euclidean distance to others, unit: meter), is the discrete state value of X (e.g. μ-wave power is divided into three levels: low, medium, and high: , is the state value of Y at the current moment and the next moment (for example, social distance is divided into three levels: close, medium, and far: , is the probability of the joint probability X=x and the current Y=y and the next moment Y=y′, is the conditional probability. When Y=y and X=x, the probability of Y=y′ at the next moment is known. is the conditional probability, when only the current Y=y is known, the probability of Y=y′ at the next moment is known. bit is determined to have a unidirectional causal relationship;

[0151] For synergy evaluation, dynamic modal entropy (DME) is defined as:

[0152] ;

[0153] Where, The state of modal X With the state of modal Y The joint probability of 、 is the marginal probability distribution. The larger the DME value, the stronger the independence between modalities (the lower the synergy); the entropy change rate The modal decoupling warning is triggered.

[0154] Time dimension feature extraction, LSTM network structure: single-layer LSTM, 128 hidden units, input is the EEG alpha wave power sequence of a 30-minute sliding window (time step = 30, feature dimension = 1), and the output is the dynamic trend indicator of the window (rising / falling / stable, classified by the fully connected layer Softmax).

[0155] Hysteresis effect calculation, for cognitive error rate series Social interaction duration sequence , calculate the Pearson correlation coefficient at lag k minutes: ;

[0156] Take k = 1 to 5 minutes, if the maximum , where T is the total number of time points (e.g., the daily collection time is T minutes), k is the lag time (minutes), and the value range is 1<k<kmax (e.g., kmax=5), S t+k is the social interaction duration lagged by k minutes, the cognitive task error rate (percentage) at time t, E and S are the average values of the cognitive error rate sequence and the social interaction duration sequence, respectively. The current window is marked as a time anomaly point.

[0157] 1.3.2、Construction of spatiotemporal graph model:

[0158] Divide daily data into 288 1-minute time windows (node number ), construct a space-time graph: ,in, is a set of nodes corresponding to the multimodal data in the time window, is a set of time-adjacent edges, connecting adjacent time windows, is a set of modality-related edges, connecting different modalities in the same window;

[0159] Node characteristics:

[0160] ;

[0161] Where HRV is heart rate variability, is the prefrontal alpha wave power, is the mean social distance, is the cognitive task error rate, is the environmental parameter;

[0162] Time edge: connecting adjacent windows and , edge weight ,in, are the node features of adjacent windows, and edges with weights ≥ 0.8 are retained;

[0163] Modal edge: Physiological-behavioral-cognitive nodes in the same window are connected to each other, and the edge weight , highlighting strongly correlated mode pairs.

[0164] For example, node feature standardization: HRV indicators are standardized using Z-score, social distance is scaled to [0,1] using Min-Max, and cognitive error rate is transformed using logarithmic transformation ( ), e is a natural constant;

[0165] Edge weight calculation example: The node feature vectors of adjacent time windows t and t+1 are and , the cosine similarity is:

[0166] , because ≥0.8, the time adjacent edges are retained.

[0167] 1.4. Symptom-based Temporal Graph Neural Network (TGNN):

[0168] 1.4.1. Network architecture. The model consists of three layers:

[0169] (1) Time coding layer, embeds sinusoidal time coding for each node:

[0170] ;

[0171] ;

[0172] in, is the time window position, is the encoding dimension, where the physical meaning of the time position pos is as follows: if the daily rehabilitation data is divided into 288 1-minute windows (T=288), then pos=1 corresponds to 9:00-9:01 in the morning, pos=144 corresponds to 12:00-12:01 in the afternoon, and the dimension index k is the same as the encoding dimension d.

[0173] (2) Graph attention layer, using multi-head attention (number of heads = 4) to model inter-modal interactions:

[0174] ;

[0175] Where, Output feature vector, is the activation function, is the current node feature, is the set of adjacent nodes, k is the attention head index, is the head-specific weight matrix, is the self-attention coefficient, is the interactive attention coefficient of the k-th head, through the shared weight matrix calculate.

[0176] (3) Temporal gating layer, based on GRU units (hidden layer 64 dimensions) to capture temporal dependencies, with output dimension of 1 (positive symptom score prediction) or 2 (negative symptom binary classification).

[0177] 1.4.2. Symptom-based training strategy:

[0178] Positive symptom model:

[0179] Input: HRV index, P300 latency, and environmental parameters;

[0180] Supervisory signal, PANSS positive score (normalized to [0,1]);

[0181] loss function, (Weight decay ), where is the predicted value (the PANSS positive score predicted value output by the model), is the true value (the true value of the PANSS positive score assessed by the clinician), MSE is the mean square error term, W is the weight matrix, is the L2 regularization term, is the regularization coefficient;

[0182] Negative symptom model:

[0183] Input: duration of social interaction, richness of emotional expression (FACS coding number), DMN functional connectivity strength;

[0184] Supervisory signal: social function recovery label (SFRS ≥ 20 is 1, otherwise 0);

[0185] Loss function: ;

[0186] Where, is the true label (two-category label), is the predicted probability (the recovery probability output by the model), is the cross entropy term 1 (when the true label is 1, it measures how close the predicted probability is to 1), is the cross entropy term 2 (when the true label is 0, it measures how close the predicted probability is to 0).

[0187] Graph neural networks (GNNs) have traditionally been used in social network and molecular structure analysis. This invention creatively applies them to the "neuro-behavioral" dynamic correlation modeling of mental illness, constructing a spatiotemporal graph containing temporal adjacency edges and modal correlation edges, which is an innovative application of cross-domain technology migration.

[0188] 1.5. Low functional connectivity patterns in brain networks:

[0189] Capturing resting-state hypofunctional connectivity patterns in brain networks through the Graph Attention Mechanism is an interdisciplinary research approach that combines neuroscience and deep learning. It aims to leverage the characteristics of graph neural networks (GNNs) to analyze abnormal patterns of resting-state functional connectivity (FC) in the brain, particularly in areas with low connectivity strength. The following explains its core concepts, technical principles, application process, and significance:

[0190] 1.5.1. Resting-state brain networks and low-function connectivity:

[0191] Resting-State Brain Network: refers to the functional connectivity network between different brain regions measured by functional magnetic resonance imaging (fMRI) and other technologies when the brain is not in a specific task state. The time series correlation between brain regions (nodes) can be quantified as connection strength (edge weight) to form a whole-brain functional connectivity matrix;

[0192] Low functional connectivity pattern: refers to a pattern in which the functional connectivity strength between brain regions is significantly lower than normal, which may reflect abnormal information interaction between brain regions. This type of pattern is common in neuropsychiatric diseases (such as depression, Alzheimer's disease, autism, etc.) or is related to brain development and aging.

[0193] 1.5.2、Graph Attention Mechanism(GraphAttentionMechanism):

[0194] It is a variant of graph neural network (GNN) (such as graph attention network GAT). The core idea is to use the attention mechanism to allow the model to adaptively learn the importance weights of nodes or edges in the graph, thereby capturing complex topological structures and feature interactions.

[0195] Advantages: No global structure (such as convolution kernel) needs to be defined in advance, and attention weights can be dynamically assigned to the neighborhood of each node. It is particularly suitable for processing non-Euclidean structure data (such as the graph structure of brain networks) and is more sensitive to feature mining of low-weight edges (low functional connections).

[0196] 1.5.3 Data preprocessing and graph construction:

[0197] step:

[0198] Brain region segmentation: The brain is divided into several regions of interest (ROIs, such as AAL template, Yeo network, etc.), and each ROI is used as a node in the graph.

[0199] Functional connectivity matrix generation: Calculate the correlation of time series between ROIs (such as Pearson correlation coefficient) to obtain a weighted undirected graph, where the edge weight is the connection strength.

[0200] Definition of low functional connectivity: Low connectivity edges are screened out through statistical thresholds (such as -1 standard deviation below the mean of the normal group) or machine learning methods, or the original connection strength is directly used as input, and the model automatically identifies important patterns.

[0201] 1.5.4 Application of Graph Attention Mechanism:

[0202] Model Architecture:

[0203] Input: Graph node features (such as ROI time series features) and adjacency matrix (functional connectivity strength).

[0204] Attention layer: Calculate the attention weight of each node's neighboring nodes (directly connected brain areas). The formula is:

[0205] ;

[0206] Where, is the node feature vector, is the inter-modal attention weight, is the learnable weight matrix, activation function, is the set of adjacent nodes of node i.

[0207] Aggregation and output: Aggregate neighborhood information through attention weights, update node representation, and ultimately use it for classification (such as disease vs. health) or pattern recognition.

[0208] 2. Solution optimization:

[0209] 2.1. Dynamic pathological mechanism modeling:

[0210] Traditional solution:

[0211] Relying on static statistical models (such as linear regression) or single modality analysis cannot capture the temporal dependence of rehabilitation indicators (such as circadian rhythm) and the causal relationship between modalities (such as the predictability of HRV on social behavior).

[0212] Improvement plan:

[0213] The dynamic graph network architecture innovatively applies the temporal graph neural network (TGNN) to the field of mental rehabilitation. The GAT layer captures cross-modal attention (such as the weight of the influence of prefrontal alpha waves on speech fluency) and the GRU layer models time series features (such as the fluctuation pattern of morning HRV), forming a "time-modality" dual-dimensional dynamic modeling capability.

[0214] New evaluation indicators, defining dynamic modal correlation entropy (DMCE) and transfer entropy (TE), quantify cross-dimensional synergy and causality, providing a new tool for the study of psychopathological mechanisms. Existing scales and models do not involve this type of dynamic correlation analysis.

[0215] 2.2 Subtype-specific assessment:

[0216] Traditional solution:

[0217] The universal model ignores the differences in the neural mechanisms of positive / negative symptoms (such as hyperdopaminergic vs. hypoglutamatergic), resulting in low diagnostic accuracy of mixed symptoms (78%) and homogeneous intervention plans.

[0218] Improvement plan:

[0219] Pathological mechanism-driven modeling, building independent models based on the BDNFVal66Met genotype (which affects neuroplasticity) and DMN functional connectivity (the core brain network for negative symptoms), integrating gene-imaging-behavioral data across layers rather than simply stratifying symptom labels;

[0220] Dynamic weight adaptive fusion, design of gated neural network (GatedNetwork), dynamically adjusts the positive / negative model fusion weights according to the patient's baseline characteristics (such as age and disease course), and implements a "one person, one policy" reasoning strategy, which is impossible with existing fixed weight fusion methods.

[0221] Example 1, dynamic warning for patients with positive symptoms:

[0222] Patient profile: Patient A, 28 years old, had mainly positive symptoms (PANSS positive score 32 points) and was in the 6th week of rehabilitation training.

[0223] Data performance:

[0224] In physiological mode, the LF / HF ratio of HRV dropped sharply from 2.3 to 1.1 (30% lower than the baseline), and the P300 latency was prolonged to 450ms (baseline 380ms);

[0225] Behavioral modalities: the mean social distance increased from 0.8 meters to 1.7 meters, and the entropy of body movement trajectories decreased by 40% (suggesting behavioral stereotyping);

[0226] In terms of cognitive modality, the Stroop test error rate increased from 15% to 35%, and the median reaction time increased from 700ms to 1200ms.

[0227] System response: The modal synergy index calculated by the TGNN model dropped from 0.72 to 0.48 (below the threshold of 0.5), triggering a positive symptom fluctuation warning 48 hours in advance. The clinical team adjusted the antipsychotic drug dosage in a timely manner, and the subsequent PANSS score did not increase as expected (maintained at 30 points).

[0228] Example 2, rehabilitation assessment of patients with negative symptoms:

[0229] Patient profile: Patient B, 35 years old, mainly had negative symptoms (SFRS score 10 points), and was enrolled in the study for the 8th week.

[0230] Feature analysis:

[0231] The DMN functional connectivity strength, 0.18 (below the normal threshold of 0.25), was strongly negatively correlated with the duration of social interaction (0.5 hours / day) (r=-0.71);

[0232] Dynamic synergy: When the duration of social interaction increased, the EEG alpha wave power did not increase as expected (DME=0.65, triggering a decoupling warning).

[0233] Intervention effect: Based on the system's recommendation to increase social cognitive training (twice a day, 30 minutes each time), after 2 weeks, the DMN connection strength increased to 0.22, the social interaction time increased to 1.2 hours / day, the SFRS score increased to 16 points, and the model prediction accuracy reached 89%.

[0234] The PANSS (Positive and Negative Syndrome Scale) is a classic scale used to assess the severity of symptoms in patients with schizophrenia. It was compiled by Kay et al. in 1987. This scale helps clinicians and researchers assess patients' positive symptoms, negative symptoms, and general psychopathological symptoms through quantitative scoring. It is widely used in the diagnosis, efficacy evaluation, and scientific research of schizophrenia.

[0235] 1. Basic Structure of PANSS

[0236] The PANSS consists of 30 items divided into three subscales:

[0237] Positive symptom subscale (7 items): assesses “excessive” or “abnormal” symptoms such as hallucinations and delusions;

[0238] Negative symptom subscale (7 items): assesses symptoms of “lack” or “low” such as flat affect and decreased will;

[0239] General Psychopathology Subscale (16 items): assesses nonspecific symptoms such as anxiety, depression, and cognitive impairment.

[0240] Each item is scored on a 7-point scale ranging from 1 to 7 points. The scoring criteria are as follows:

[0241] 1 point: no symptoms;

[0242] 2 points: very mild;

[0243] 3 points: mild;

[0244] 4 points: moderate;

[0245] 5 points: moderately severe;

[0246] 6 points: severe;

[0247] 7 points: Extremely severe.

[0248] 2. Subscales and Specific Items

[0249] Table 2: Positive symptom subscale (P1-P7), as follows:

[0250]

[0251] Table 3: Negative symptom subscale (N1-N7), as follows:

[0252]

[0253] Table 4: General Psychopathology Subscales (G1-G16), as follows:

[0254]

[0255] 3. Scoring Method and Notes

[0256] Scoring is based on a comprehensive assessment of clinical interviews, patient behavior observations, and medical history data; it must be performed by professionally trained psychiatrists or researchers.

[0257] Time frame: Assess the patient's symptoms over the past week.

[0258] Calculation of total and subscale scores: Total score: the sum of the scores of 30 items (range, 30-210 points, the higher the score, the more severe the symptoms); Positive symptom score: the sum of P1-P7 (7-49 points); Negative symptom score: the sum of N1-N7 (7-49 points).

[0259] General psychopathology score: the sum of G1-G16 (16-112 points).

[0260] Notes:

[0261] Distinguish between persistent and transient symptoms (e.g., occasional nervousness may not be included in the score); avoid subjective bias by incorporating objective behavioral evidence (e.g., whether the patient refuses to eat due to suspicion).

[0262] IV. Clinical Application and Interpretation

[0263] Diagnosis reference:

[0264] Predominantly positive symptoms indicate "type I schizophrenia", while predominantly negative symptoms indicate "type II"; the total score and subscale scores can help differentiate schizophrenia from other mental disorders (such as depression and anxiety).

[0265] Efficacy evaluation:

[0266] Compare total and subscale scores before and after treatment to determine the degree of symptom improvement (e.g., a decrease in positive symptom scores after antipsychotic treatment); negative symptoms respond more slowly to treatment and often require long-term follow-up evaluation.

[0267] SFRS scoring standard: Schizophrenia Functional Recovery Scale (SFRS): may be used to assess the social function, occupational ability, daily life recovery, etc. of patients with schizophrenia.

[0268] Assess the degree of social function recovery of patients with mental illness (such as schizophrenia), including: social interaction ability, occupational / learning ability, daily care ability, and family role function.

[0269] Table 5: Schizophrenia functional recovery subscale and items are as follows:

[0270]

[0271] Scoring method;

[0272] Each item is scored on a 1-5 or 1-7 scale based on the patient's performance over the past 2 weeks.

[0273] Total score = the sum of scores of each item. A higher score indicates better functional recovery.

[0274] In the application, the several formulas involved are all calculated by taking their numerical values after removing the dimensions, and the formulas are established by collecting a large amount of data and performing software simulation to obtain a formula for the most recent real situation. Some coefficients or weights in the formulas are set by technical personnel in this field according to actual conditions, so they will not be elaborated here.

[0275] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. Those skilled in the art will appreciate that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution.

[0276] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, and may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment as needed.

[0277] The above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the scope of protection of the present application.

Claims

1. A rehabilitation training evaluation system for schizophrenia patients, characterized by: The assessment system includes: Multimodal perception module builds a four-dimensional data acquisition system to collect physiological data, behavioral data, cognitive data and environmental parameters in real time; Edge intelligent processing module, integrating data synchronization unit and feature extraction unit; The data synchronization unit adopts a three-stage synchronization mechanism, performs hardware-level calibration through the FPGA clock synchronization module, uses a dynamic time warping algorithm to complete algorithm-level timing alignment, and corrects cross-device baseline differences based on a Bayesian network to achieve semantic-level calibration. The feature extraction unit constructs a dual-dimensional feature mining system based on physiological data and behavioral data. The dual-dimensional feature mining system includes time dimension and modality dimension. By constructing a time-modality joint feature space, in-depth analysis of rehabilitation indicators is achieved. The details are as follows: Mining temporal features: Performing time series modeling on single-modal data to extract dynamic change patterns, including time series statistics for physiological data and temporal transition probabilities for behavioral data. Long-short-term memory networks or temporal convolutional networks are used to capture long-range temporal dependencies. When physiological data deviate significantly from the baseline within a specific time period, these are marked as temporal anomalies. Time lag effects are analyzed, and the time-lag correlation between cognitive task error rates and the duration of preceding social interactions is calculated. Modal dimension feature mining: Analyze cross-dimensional correlations between different modal data, extract synergy and causality features, quantify the direction of information flow using inter-modal transfer entropy; evaluate the evolution of cross-modal synergy using dynamic modal correlation entropy; model inter-modal interaction weights based on a graph attention mechanism, using physiological data as the bottom-level features, behavioral data as the middle-level features, and cognitive data as the high-level features to construct a hierarchical correlation model; trigger a decoupling warning when a modal change does not trigger the expected response from other modalities; The dual-dimensional feature mining system achieves joint modeling through the following steps: Construct a time-modality two-dimensional feature matrix, where the row dimension is the time window and the column dimension is the multimodal feature; The tensor decomposition technique is used to reduce the dimension of the two-dimensional feature matrix and extract the time-modal coupling characteristics; The coupling features are input into the time series graph neural network to dynamically predict the rehabilitation effect; The dynamic graph network evaluation module builds an evaluation model based on a time-series graph neural network. By calculating the difference between the joint distribution and independent distribution of data in each dimension, combined with the transfer entropy algorithm to quantify dynamic synergy, a sudden drop in synergy threshold is set to trigger a multimodal warning. Independent models are trained for patients with positive and negative symptoms. Among them, the negative symptom model introduces the DMN functional connectivity features, captures the low functional connectivity pattern of the resting-state brain network through the graph attention mechanism, and predicts the recovery status of patients of different subtypes.

2. The rehabilitation training evaluation system for schizophrenia patients according to claim 1, characterized in that: The hardware devices used in the multimodal perception module include physiological monitoring equipment, environmental perception equipment, and cognitive interaction terminals, as follows: Physiological monitoring equipment, including a heart rate bracelet with photoplethysmography technology and an EEG headband with dry electrode technology, used to collect indicators related to heart rate variability, prefrontal cortex, and cerebral cortex. Wave power and event-related potential physiological data reflect the patient's neurophysiological state at a microscopic level; among them, heart rate variability-related indicators include the root mean square of the difference between adjacent normal RR intervals and the ratio of low-frequency to high-frequency power spectral density; Environmental perception equipment, including time-of-flight depth cameras and ultra-wideband positioning base stations, is used to obtain social distance, body movement trajectory, and spatial location data, capturing the patient's behavior in the environment from a mesoscopic perspective; The cognitive interaction terminal is a tablet computer equipped with touch feedback technology, which is used to record the reaction time, error type and completion trajectory of cognitive tasks, and evaluate the patient's psychological state from a cognitive perspective.

3. The rehabilitation training evaluation system for schizophrenia patients according to claim 1, characterized in that: The FPGA clock synchronization module integrates the ARM processor and the FPGA programmable logic unit. The specific process is as follows: The FPGA clock synchronization module generates a global clock signal and distributes it to the heart rate bracelet, EEG headband, and depth camera through a differential clock buffer; The heart rate bracelet, EEG headband and depth camera are synchronized by hardware through the global clock signal.

4. The rehabilitation training evaluation system for schizophrenia patients according to claim 1, characterized in that: The algorithm-level timing alignment uses a dynamic time warping algorithm to align the asynchronously sampled physiological data and behavioral data, where the physiological data and behavioral data are denoted as q and c, respectively, as follows: Defining physiological sequences and behavioral sequences , build Distance Matrix ,element , where Behavior Sequence The mean of all behavioral data c in , is the standard deviation of all behavioral data c in the behavior sequence, Physiological sequence The mean of all biological data q in , Physiological sequence The standard deviation of all biological data q in , is the i-th physiological data in the physiological sequence, is the jth behavioral data in the behavioral sequence, m is the number of physiological data in the physiological sequence, and n is the number of behavioral data in the behavioral sequence, eliminating the influence of dimensional differences; The optimal path is solved through dynamic programming.

5. The rehabilitation training evaluation system for schizophrenia patients according to claim 1, characterized in that: The data processing in the Bayesian network model in the semantic level calibration stage includes: Collect physiological data from L different types of physiological monitoring devices and construct a prior distribution under standard test scenarios, where L>20. The standard test scenarios include resting state and cognitive task state. The real-time collected data is used to update the posterior probability, dynamically adjust the device deviation correction coefficient, and adaptively calibrate the unknown device. After the data is acquired, the data is preprocessed to remove outliers and standardize the original heart rate variability value HRV to the range of [0, 1].

6. The rehabilitation training evaluation system for schizophrenia patients according to claim 1, characterized in that: The physiological data extracted by the feature extraction unit includes calculated HRV index, EEG Rhythm power and changes in prefrontal oxyhemoglobin concentration are used to reflect the patient's neurological function status; The behavioral data includes social behavior data identified based on the OpenPose algorithm, which is used to reflect the patient's social ability and psychological state.

7. The rehabilitation training evaluation system for schizophrenia patients according to claim 1, characterized in that: Divide the multimodal data into time windows as graph nodes and construct a spatiotemporal graph model containing time adjacency edges and modality association edges: Among them, each graph node contains standardized physiological data, behavioral data, cognitive data and environmental parameters; adjacent time window nodes are connected, and the edge weight is the cosine similarity of the feature vector to obtain the temporal continuity of the rehabilitation indicators; different modal nodes in the same window are connected, and the edge weight is calculated through dynamic modal correlation entropy or transfer entropy to quantify the synergy or causality between modalities.

8. The rehabilitation training evaluation system for schizophrenia patients according to claim 1, characterized in that: The method for training independent models for patients with positive and negative symptoms in the dynamic graph network evaluation module is as follows: Train a separate model for patients with positive symptoms: Collect data related to positive symptoms, standardize the data, remove noise interference, and unify the scale and format of the data; An attention-based deep neural network architecture that captures key features related to positive symptoms; The positive score of the Positive Symptom Scale was used as a supervisory signal, and the cross-validation method was used to evaluate the model's prediction accuracy for positive symptom fluctuations using an independent test dataset. Train a separate model for patients with negative symptoms: Data related to negative symptoms were collected and also normalized and denoised; The architecture of graph convolutional networks combined with recurrent neural networks is introduced, and the duration of social interactions and the richness of emotional expressions are used as supervisory signals. Prediction of recovery of social function and improvement of emotional state.

Images