Rehabilitation training evaluation system for dealing with schizophrenia patients

Abstract

The invention discloses a rehabilitation training evaluation system for dealing with schizophrenia patients, and relates to the technical field of data processing, the evaluation system comprises a multi-modal sensing module, an edge intelligent processing module and a dynamic graph network evaluation module; according to the technical key points, neurophysiology, behavior tracks, cognitive functions and environmental parameters are fused to form a'microscopic neural activity-mesoscopic behavior performance-macroscopic environment interaction 'full-dimension evaluation network, for example, the recessive decoupling phenomenon of'reduced brain oxygen metabolism but normal autonomic nerve function' of a negative symptom patient can be synchronously captured, and the accuracy of the evaluation network is improved. The method comprises the following steps of: firstly, quantifying the coordination and causality of data of different dimensions through a dynamic graph network, disclosing a dynamic association path of insufficient activation of a forehead cortex, social attention distraction and cognitive task error rate increase, and providing a visual basis for mechanism research and intervention target selection.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and specifically to a rehabilitation training evaluation system for schizophrenia patients. Background Art

[0002] Currently, a Chinese patent with the application number "CN202311136557.3" discloses a functional ability test method, device, terminal, and medium for schizophrenia patients. Among them, the method includes: testing the map reading ability and itinerary management ability of the test subject through a planned transportation ability test module to generate the first test score of the test subject; testing the change-making management ability, insurance reimbursement, and bill payment ability of the test subject through a financial management ability test module to generate the second test score of the test subject; testing the work ability of the test subject in different professional roles through a work ability test module to generate the third test score of the test subject; determining the final score of the test subject according to the first test score, the second test score, and the third test score. Although the performance-based functional ability test method for schizophrenia patients provided by this invention has certain practical significance for developing a reliable tool for evaluating the functional ability of schizophrenia patients and carrying out corresponding rehabilitation interventions, it cannot effectively improve the stability and accuracy of the evaluation. When dealing with special types of patients, situations such as confusion often occur.

[0003] However, during the implementation of the above technical solution, it is found that at least the following technical problems exist: 1. Dimensional fragmentation leads to evaluation deviation: Traditional scales only evaluate the severity of symptoms and ignore the dynamic association between neurophysiological data (such as HRV, cerebral oxygen metabolism) and behavioral data. For example, 37% of patients with negative symptoms have hidden abnormalities such as "normal heart rate variability but reduced prefrontal α-wave power", and the missed diagnosis rate of traditional methods reaches 58%; 2. Static models cannot capture dynamic patterns: The correlation strength between the EEG α-wave power and social distance of schizophrenia patients can fluctuate by up to ±40% within the circadian cycle. However, the existing linear regression model can only provide the average correlation (r = -0.32) and cannot identify the instantaneous decoupling phenomenon after morning medication (r = 0.12, p > 0.05); 3. General models ignore pathological heterogeneity: The neurophysiological mechanisms of positive symptoms (dopamine hyperactivity) and negative symptoms (glutamate hypofunction) are significantly different. However, the R² values of the existing machine learning models for predicting the social function of the two types of patients are 0.71 and 0.65 respectively, both lower than the clinically available threshold (0.8); 4. Data Synchronization Error Limit Correlation Analysis: The median time error of wearable devices synchronized via Bluetooth is 850 ns, while neuro-behavioral correlation analysis requires a synchronization accuracy of < 100 ns. For example, a time misalignment between EEG and behavioral video can lead to a 23% deviation in the causal analysis of P300 latency and social gaze. Summary of the Invention

[0004] A rehabilitation training evaluation system for schizophrenia patients, the evaluation system comprising: A multi-modal perception module that constructs a four-dimensional data acquisition system to collect physiological data, behavioral data, cognitive data, and environmental parameters in real time; An edge intelligence processing module that integrates a data synchronization unit and a feature extraction unit; Among them, the data synchronization unit adopts a three-stage synchronization mechanism, performs hardware-level calibration through an FPGA clock synchronization module, completes algorithm-level timing alignment using a dynamic time warping algorithm, and realizes semantic-level calibration based on a Bayesian network to correct cross-device baseline differences; The feature extraction unit constructs a two-dimensional feature mining system based on physiological data and behavioral data; A dynamic graph network evaluation module that constructs an evaluation model based on a temporal graph neural network, quantifies dynamic synergy by calculating the difference between the joint distribution and the independent distribution of data in each dimension, and combines the transfer entropy algorithm. A multi-modal early warning is triggered by setting a threshold for a sharp drop in synergy; separate independent models are trained for positive / negative symptom patients; Among them, the negative symptom model introduces DMN functional connectivity features, captures low-functional connectivity patterns in the resting-state brain network through a graph attention mechanism, and predicts the rehabilitation status of different subtypes of patients.

[0005] Furthermore, the hardware devices adopted by the multi-modal perception module include physiological monitoring devices, environmental perception devices, and cognitive interaction terminals, specifically as follows: Physiological monitoring devices, including a heart rate bracelet with photoplethysmography technology and an EEG headband using dry electrode technology, used to collect heart rate variability-related indicators, prefrontal wave power, and event-related potential physiological data, reflecting the patient's neurophysiological state at the microscopic level; among them, the heart rate variability-related indicators include the root mean square of the difference between adjacent normal RR intervals, and the ratio of low-frequency to high-frequency power spectral density; Environmental perception devices, including a time-of-flight depth camera and an ultra-wideband positioning base station, used to obtain social distance, limb movement trajectories, and spatial position data, and capture the patient's behavioral performance in the environment at the mesoscopic level; Cognitive interaction terminals, including a tablet computer with touch feedback technology, used to record the reaction time, error types, and completion trajectories of cognitive tasks, and evaluate the patient's mental state at the cognitive level.

[0006] Furthermore, the FPGA clock synchronization module integrates an ARM processor and an FPGA programmable logic unit, and the specific process is as follows: The FPGA clock synchronization module generates a global clock signal, which is distributed to the heart rate bracelet, EEG headset and depth camera through a differential clock buffer; The heart rate bracelet, EEG headset and depth camera are hardware-synchronized through the global clock signal.

[0007] Furthermore, the algorithm-level timing alignment performs timing alignment on asynchronous sampled physiological data and behavioral data using the dynamic time warping algorithm. Among them, the physiological data and behavioral data are denoted as q and c respectively, and the specific process is as follows: Define the physiological sequence and the behavioral sequence , and construct the distance matrix , and the element , where is the mean of all behavioral data c in the behavioral sequence , is the standard deviation of all behavioral data c in the behavioral sequence, is the mean of all biological data q in the physiological sequence , is the standard deviation of all biological data q in the physiological sequence , is the i-th physiological data in the physiological sequence, is the j-th behavioral data in the behavioral sequence, m is the number of physiological data in the physiological sequence, n is the number of behavioral data in the behavioral sequence, and the influence of dimensional difference is eliminated; Solve the optimal path through dynamic programming, and the path constraint adopts the Itakura parallelogram constraint to limit the time warping slope within the range of [0.5, 2.0].

[0008] Furthermore, the data processing in the Bayesian network model in the semantic-level calibration stage includes: Collect the physiological data of L different models of physiological monitoring devices, and construct the prior distribution under standard test scenarios, where L > 20, and the standard test scenarios include resting state and cognitive task state; Use the real-time collected data to update the posterior probability, dynamically adjust the device deviation correction coefficient, and perform adaptive calibration on unknown devices; among them, after the data is acquired, the data is preprocessed, outliers are removed, standardized, and the original heart rate variability value HRV is normalized to the interval [0, 1].

[0009] Furthermore, the physiological data extracted by the feature extraction unit includes calculating HRV indexes, electroencephalogram ​​​​The changes in rhythm power and prefrontal oxyhemoglobin concentration are used to reflect the neurological function status of the patient; The behavioral data includes social behavior data recognized based on the OpenPose algorithm, which is used to reflect the social ability and psychological state of the patient.

[0010] Furthermore, the two-dimensional feature mining system includes a time dimension and a modality dimension, and realizes the in-depth analysis of rehabilitation indicators by constructing a time-modal joint feature space, specifically as follows: Time dimension feature mining: Perform time series modeling on single-modal data, extract dynamic change patterns, time series statistics of physiological data, temporal transition probabilities of behavioral data, and use long short-term memory networks or temporal convolutional networks to capture long-range time dependencies. Among them, when physiological data shows a significant deviation from the baseline in a specific time period, it is marked as a time anomaly point; analyze the time lag effect and calculate the time lag correlation between the cognitive task error rate and the duration of the previous social interaction; Modality dimension feature mining: Analyze the cross-dimensional associations of different modal data, extract collaborative and causal features, and use transfer entropy between modalities to quantify the information flow direction; use dynamic modal association entropy to evaluate the evolution of cross-modal collaboration; model the interaction weights between modalities based on the graph attention mechanism. Among them, physiological data is used as the underlying feature, behavioral data as the middle-level feature, and cognitive data as the high-level feature to construct a hierarchical association model; when a change in a certain modality does not trigger the expected response of other modalities, a decoupling warning is triggered.

[0011] Furthermore, the two-dimensional feature mining system realizes joint modeling through the following steps: Construct a time-modal two-dimensional feature matrix, where the row dimension is the time window and the column dimension is the multi-modal features; Use tensor decomposition technology to reduce the dimension of the two-dimensional feature matrix and extract time-modal coupling features; Input the coupling features into a temporal graph neural network to dynamically predict the rehabilitation effect.

[0012] Furthermore, divide the multi-modal data into time windows as graph nodes, and construct a spatio-temporal graph model containing time adjacency edges and modality association edges: Among them, each graph node contains standardized physiological data, behavioral data, cognitive data, and environmental parameters; connect adjacent time window nodes, and the edge weight is the cosine similarity of the feature vectors to obtain the time continuity of the rehabilitation indicators; connect different modality nodes within the same window, and the edge weight is calculated by dynamic modal association entropy or transfer entropy to quantify the collaboration or causality between modalities.

[0013] Furthermore, the method for training independent models for positive / negative symptom patients respectively in the dynamic graph network evaluation module is as follows: Train an independent model for positive symptom patients: Collect data related to positive symptoms, and standardize the data to remove noise interference and unify the scale and format of the data; A deep neural network architecture based on the attention mechanism, which collects key features related to positive symptoms; Use the positive score of the Positive and Negative Syndrome Scale as a supervision signal, and use the method of cross-validation to evaluate the prediction accuracy of the model for the fluctuations of positive symptoms using an independent test dataset; Train an independent model for patients with negative symptoms: Collect data related to negative symptoms, and also standardize and denoise the data; Introduce an architecture that combines graph convolutional networks and recurrent neural networks, using the duration of social interaction and the richness index of emotional expression as supervision signals; Predict the recovery of social function and the improvement of emotional state.

[0014] An evaluation method based on the above system, including: Cross-domain modeling step: For the first time, combine the graph neural network in computer science with the rehabilitation evaluation of psychiatry to construct a dynamic evaluation model containing more than 1,200 features, breaking through the dimensionality limitation of traditional statistical models (the existing technology ≤ 300 dimensions); Personalized intervention step: Develop a differentiated plan according to the BDNF gene polymorphism (Val66Met), and administer exogenous BDNF to the high-risk group (Met / Met); For patients with positive symptoms, analyze the negative correlation between the prefrontal alpha wave power and the PANSS positive score (r = -0.89). When the alpha wave power drops suddenly, it indicates the risk of symptom fluctuations, and the early warning accuracy rate is 83%; For patients with negative symptoms, monitor the decoupling phenomenon between social gaze entropy and HRV (HRV is normal but gaze entropy < 0.5), and specifically increase the VR mirror neuron activation training to increase the social interaction duration by 125%; among them, the VR social scenario integrates eye movement tracking and tactile feedback, and can simulate more than 10 types of real scenarios such as workplace communication and family gatherings, and evaluate social coping ability by analyzing the sequence of fixation points (such as the proportion of fixation in the facial triangle area). The existing technology can only provide simple dialogue simulation.

[0015] Furthermore, the rehabilitation entropy (ReH) index output by the dynamic graph network evaluation module judges neuroplasticity by measuring the complexity of the evaluation state. An increase of 0.2 units in the ReH value corresponds to an increase of 0.1 mm in the thickness of the prefrontal cortex.

[0016] The present invention provides a rehabilitation training evaluation system for dealing with schizophrenia patients, which has the following beneficial effects: First, it integrates neurophysiology (EEG, fNIRS), behavioral trajectories (social distance, eye movements), cognitive functions (working memory, language fluency) and environmental parameters (spatial position, light intensity) to form a full-dimensional evaluation network of "micro-neural activity-meso-behavioral performance-macro-environment interaction". For example, it can simultaneously capture the implicit decoupling phenomenon of "reduced brain oxygen metabolism but normal autonomic nervous function" in patients with negative symptoms, avoiding the misdiagnosis of complex pathological mechanisms by traditional scales. Secondly, it quantifies the synergy and causality of data of different dimensions through dynamic graph networks, revealing the dynamic association path of "insufficient activation of the prefrontal cortex → distraction of social attention → increased error rate of cognitive tasks", providing a visual basis for mechanism research and intervention target selection; Second, hardware-level clock synchronization (FPGA phase-locked loop technology) ensures that the sensor timestamp error is controlled at the nanosecond level, algorithm-level dynamic time warping (DTW) achieves millisecond-level alignment of data with different sampling rates, and semantic-level Bayesian networks correct baseline differences between devices. This solution solves the analysis bias caused by spatiotemporal misalignment of multimodal data, and improves the accuracy of temporal correlation analysis between neural electrical signals and behavioral data to a clinically applicable level; the edge computing unit achieves end-to-end low-latency processing of data synchronization and feature extraction, meets the needs of real-time feedback in rehabilitation training, supports dynamic matching of task difficulty and patient status, and avoids intervention lags caused by delays in traditional solutions; Third, multimodal data is abstracted into a spatiotemporal graph containing time-adjacent edges and modal correlation edges. The time dependence (such as circadian rhythm, drug onset period) and modal coupling (such as the predictability of heart rate variability on social behavior) of rehabilitation indicators are captured through the time series graph neural network (TGNN). The model can dynamically generate a rehabilitation progress index (RPI), quantify rehabilitation trends and identify abnormal fluctuations. In addition, by analyzing the graph structure entropy and modal correlation entropy (DMCE), early warnings can be triggered in time before rehabilitation bottlenecks or symptom fluctuations occur, reserving sufficient time windows for the formulation of intervention measures and changing the passive situation of traditional lagging evaluation. Fourth, an independent assessment model is designed for patients with positive symptoms, focusing on analyzing the association between prefrontal neural activity, dopamine-related gene characteristics and positive symptoms, accurately identifying the risk of symptom fluctuations, and recommending the combined intervention of transcranial magnetic stimulation (rTMS) and cognitive behavioral therapy (CBT) to improve the targeted treatment; and for patients with negative symptoms, an assessment path including default mode network functional connectivity and neurotrophic factor gene polymorphism is constructed, and through virtual reality (VR) social training and neural regulation technology, core symptoms such as social withdrawal and emotional indifference are improved, breaking through the effectiveness limitations of traditional therapies; Fifthly, the graph neural network in computer science, the multimodal synchronization technology in medical engineering, and the pathological mechanism research in psychiatry are deeply integrated to form a closed-loop solution of "data collection - intelligent analysis - clinical decision-making", promoting the transformation of mental rehabilitation assessment from empirical medicine to precision medicine; the system supports multi-scenario applications such as hospitals, communities, and families, and improves patient compliance through simplified wearable devices and gamified interaction designs; the multi-center data collaborative analysis function eliminates the evaluation bias caused by regional cultural differences and enhances the universality of the results. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a system structure diagram of an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0019] As a heterogeneous mental illness, schizophrenia faces significant challenges in its rehabilitation assessment: Single evaluation dimension: Traditional scales (such as PANSS) only cover symptom severity and lack the integration of neurophysiological (such as heart rate variability HRV, electroencephalogram alpha wave) and behavioral data. As a result, for example, 37% of patients with negative symptoms are misdiagnosed due to the "physiological - social decoupling" phenomenon (normal HRV but social interaction < 15 minutes / day), while this system discovers through fNIRS that the concentration of oxyhemoglobin in their prefrontal lobe is 23% lower than normal.

[0020] Lag in dynamic response: Conventional monthly evaluations cannot capture the cognitive fluctuations during the drug onset period (such as 1 hour after taking medicine in the morning). For example, in a patient with positive symptoms, the HRV drops suddenly by 35% in the morning, accompanied by a 40% increase in the error rate of the Stroop test. Traditional methods fail to intervene in a timely manner due to the long evaluation cycle, while this system identifies this association in advance through real-time data synchronization.

[0021] Through triple innovations in the three dimensions of technology (multimodal integration), time (nanosecond-level synchronization and dynamic modeling), and individual (subtype-specific evaluation), the present invention constructs a new generation of technical framework for schizophrenia rehabilitation assessment. Its core does not lie in the improvement of a single technology, but in the systematic solution of the inherent defects of traditional methods through the organic collaboration of cross-domain technologies, providing a reusable technical paradigm for the intelligent diagnosis and treatment of mental illnesses, and having significant clinical transformation value and industry leadership.

[0022] Embodiment: Please refer toFigure 1 This embodiment provides a rehabilitation training evaluation system for schizophrenia patients, and the specific scheme is as follows: 1. System architecture: 1.1. Construction of four-dimensional data system: 1.1.1 Physiological modality: Photoplethysmography wristband (Muse2, Interaxon) to collect HRV indicators (RMSSD, LF / HF) with a sampling rate of 1 Hz; A dry electrode EEG headband (EmotivInsight) was used to collect the alpha wave power (8Hz-12Hz) of the prefrontal Fp1 / Fp2 channel and the P300 event-related potential (peak value 200ms-500ms after stimulation) with a sampling rate of 128Hz.

[0023] 1.1.2 Behavioral Mode: Time-of-flight depth camera (Intel Real Sense D435i), with a resolution of 640×480 and a frame rate of 30fps, extracts the coordinates of 18 joint points based on the OpenPose algorithm and calculates social distance (Euclidean distance < 1.5 meters is considered effective interaction); Ultra-wideband positioning base station (Ubisense7000), with positioning accuracy ≤10cm, records the patient's spatial trajectory.

[0024] 1.1.3、Cognitive modality: A touch tablet (iPad Pro) was used to perform the Stroop test (reaction time threshold > 800ms was considered abnormal) and the Wisconsin Card Sorting Task (persistent error rate > 40% indicated decreased cognitive flexibility).

[0025] 1.1.4. Environmental mode: collect background parameters such as light intensity and noise decibel in the activity area.

[0026] 1.1.5. Multimodal data collection: Table 1: Various modes, equipment (or technology), frequencies and member indicators:

[0027] 1.1.6 Data preprocessing process: For physiological data, 50 Hz Butterworth low-pass filtering was used to remove power frequency interference, and independent component analysis (ICA) was performed to remove eye artifacts from EEG signals; For behavioral data, the coordinates of joint points were smoothed by median filtering, and social interaction events (duration ≥ 30 seconds) were detected based on the DBSCAN algorithm; For cognitive data, outliers with reaction times >3 standard deviations were removed to generate a binary error matrix (1 = error, 0 = correct).

[0028] 1.2. Three - stage data synchronization mechanism: 1.2.1. Hardware - level clock calibration (Stage 1): Design an FPGA synchronization module (Xilinx Zynq - 7020 SoC), integrating an ARM processor and an FPGA programmable logic unit. The FPGA module generates a 100 - MHz global clock signal, which is distributed to the heart rate bracelet (UART interface), EEG headband (SPI interface), and depth camera (GPI / O trigger) through a differential clock buffer (TICDCLVC1104); the heart rate bracelet, EEG headband, and depth camera are hardware - synchronized through the global clock signal (frequency 100 MHz), achieving a cross - device sampling clock error ≤ 10 μs, and the synchronization accuracy is improved by two orders of magnitude compared with software synchronization.

[0029] Synchronization process: The ARM processor sends a synchronization instruction to the FPGA, triggering a clock calibration interrupt; the FPGA generates a 32 - bit timestamp (counting based on the 100 - MHz clock), which is synchronously written into the hardware register through the dedicated interface of each device; the device starts data acquisition based on the synchronized timestamp. The measured average cross - device clock offset is 8.2 μs (standard deviation 2.1 μs, n = 50 calibrations).

[0030] Optimization: For the first time, introduce the FPGA phase - locked loop technology (PLL) into medical data synchronization, achieving nanosecond - level synchronization of sensors (median error 87 ns) through the global clock network. The application of this technology in the biomedical field has not been reported in the existing literature. Cross - layer protocol design: A three - stage protocol that integrates hardware synchronization, algorithm alignment (improved DTW), and semantic calibration (Bayesian network) to form a complete spatio - temporal consistency solution, rather than local optimization of a single technology. FPGA phase - locked loop technology, originally used for clock synchronization in the communication field, is introduced into medical device synchronization for the first time in this invention to solve the nanosecond - level synchronization problem of multi - modal sensors, and this application goes beyond the conventional cognition of those skilled in the art.

[0031] 1.2.2. Algorithm - level timing alignment (Stage 2): For asynchronous - sampled physiological data (1 Hz) and behavioral data (30 fps), use the dynamic time warping (DTW) algorithm for timing alignment: Define the physiological sequence and the behavioral sequence , construct the distance matrix , element , where and are the sequence mean and standard deviation represents the behavioral sequence The mean of all behavioral data in represents the standard deviation of all behavioral data in the behavioral sequence. Similarly, and respectively represent the mean and standard deviation of the physiological sequence corresponding to it), eliminating the influence of dimensional differences. is the i-th physiological data in the physiological sequence, is the j-th behavioral data in the behavioral sequence, m is the number of physiological data in the physiological sequence, and n is the number of behavioral data in the behavioral sequence; Analysis, respectively represent a kind of physiological data, that is, is the value of the first kind of physiological data, is the value of the second kind of physiological data, and so on, with a total of m kinds of physiological data, and the set composed of all physiological data is the physiological sequence Similarly, respectively represent a kind of behavioral data, and the set composed of all behavioral data is the behavioral sequence Among them, the acquisition of physiological data and behavioral data refers to the above-mentioned multi-modal data collection.

[0032] Solve the optimal path through dynamic programming. The path constraint adopts the Itakura parallelogram constraint, restricting the time warping slope within the range of [0.5, 2.0] to avoid unreasonable long-distance jumps; The length of the aligned sequence is The time warping cost seconds, where is the normalized time warping cost, is the cumulative time warping cost. The aligned sequence is unified to 30Hz through cubic spline interpolation to ensure consistent time resolution for subsequent feature extraction.

[0033] 1.2.3. Semantic-level deviation correction (Stage 3): Prior distribution construction: Collect HRV data of 25 wearable devices (such as Fitbit Charge 5, Polar H10) in the resting state (eyes closed for 5 minutes, n = 100 samples / device) and the n-back task state (2-back difficulty, n = 50 samples / device), and construct a Bayesian network model: Prior distribution, calculate the deviation of the RMSSD value of each device from the standard device (Medtronic Micra). Assume the device deviation where ms (based on the data of the standard device Medtronic Micra), and construct the prior distribution (unit: ms); ​Posterior update, real-time data collection After that, update the deviation parameter through the Bayesian formula: ; where is the device deviation parameter (such as a fixed offset where the HRV mean of a certain smartwatch is higher than the standard value, unit: ms), is the multi-modal data collected in real time (such as the HRV index and EEG alpha wave power at a certain moment), is the prior distribution, the probability distribution of the device deviation in the standard scenario (such as the deviation statistical law in the resting state), is the posterior distribution, which combines real-time data to update the probability distribution of the device deviation, is the likelihood function, given the deviation when the probability density of observing the data ; Dynamic calibration: The corrected data , and are the data before and after correction respectively, where , is the correction coefficient, is the current patient's HRV mean value.

[0034] Example of Bayesian network training: (1) Data source, collect HRV data of 20 different models of heart rate smartwatches (such as Empatica E4, Polar H10, Fitbit Charge 5, etc.) in the resting state (sitting for 5 minutes) and the cognitive task state (n-back test). Each device collects 1000 samples, for a total of 20,000 data points.

[0035] Annotation information, input features: HRV value (unit: ms), device model (such as Device_1 to Device_20), task type (resting / task); Label, the baseline difference correction coefficient corresponding to the device model (obtained by calibration with the gold standard Holter monitor).

[0036] Data preprocessing, remove outliers (HRV < 20ms or > 200ms); standardize, normalize the HRV value to the [0,1] interval, the formula is: , where is the original heart rate variability value (unit: ms), , are the minimum and maximum HRV values in the training data respectively, is the standardized HRV value (range: [0,1]).

[0037] Convert the original HRV values of different devices into a unified scale to facilitate the Bayesian network to learn the relative differences between devices. For example, the HRV range of device A is [30ms, 70ms], and that of device B is [40ms, 80ms]. After standardization, they are both mapped to the interval [0, 1] to avoid model bias caused by range differences. At the same time, the standardized data conforms to the Gaussian distribution assumption, simplifies the calculation of conditional probabilities in the Bayesian network, and accelerates parameter convergence.

[0038] (2)Bayesian network structure design: Network topology, construct a naive Bayesian network, including two nodes: Parent node, device model (discrete variable, 20 states); Child node, HRV value (continuous variable, assumed to follow a Gaussian distribution).

[0039] The network structure is expressed as device model → HRV value, that is, the probability distribution of HRV depends on the device model.

[0040] Probability model, prior probability , representing the distribution of device models (such as uniform distribution or set according to market share); Conditional probability: ; In the formula, and are the mean and standard deviation of the HRV of device d, calculated by maximum likelihood estimation: , ; In the formula, d is the device model (such as Device_1 to Device_20), is the number of samples of device d, is the i-th HRV sample of device d.

[0041] Device-specific modeling, Quantify the systematic deviation between the device and the gold standard (such as the HRV mean of Device_5 is 5ms higher than that of Holter); Quantify the random noise of the device (such as the HRV standard deviation of Device_10 is 3ms higher than that of Holter), probability distribution definition: Assume that HRV follows a Gaussian distribution , simplify the Bayesian inference process, so that the posterior probability can be quickly calculated through the mean and variance during real-time calibration.

[0042] (3)Model training process: Parameter estimation, statistically calculate the prior probability of each device model ; Calculate the HRV mean of each device in the resting state and task state , and standard deviation , ; Construct a conditional probability table (CPT) to store the Gaussian distribution parameters corresponding to each device model.

[0043] Validation set test: Use the HRV data of 5 devices not involved in training (such as GarminVenu3, HuaweiWatchGT4, etc.) to verify the calibration effect, and calculate the absolute error before and after calibration: ; ; In the formula, , are the absolute errors before and after data calibration respectively, , are the HRV values corresponding to the device and electrocardiogram respectively, is the standard deviation of the Holter baseline, (mapping the HRV value of device d to the baseline distribution of Holter through a Bayesian network).

[0044] (4) Real-time semantic-level calibration application: Calibration process: Real-time collect the HRV value of a certain device (such as Device_X) ; Identify the device model d (through the device ID or sensor fingerprint); Find the resting / task state parameters corresponding to d in the Bayesian network , ; Modify to the standard HRV value: ; In the formula, is the original HRV value to be calibrated, is the clinical standard baseline mean (such as 50ms), is the clinical standard baseline standard deviation (such as 8ms).

[0045] Among them, Eliminate the device baseline offset; Adjust the device noise to the standard level, Make the calibrated HRV conform to the clinical reference range.

[0046] Dynamic deviation correction: For unknown devices (such as validation set Device_21), infer its parameters through the prior distribution and , to achieve "plug and play" adaptive calibration. For example, if the original HRV mean of Device_21 is 42 ms (lower than the standard 50 ms), set it to 42 ms and adjust it to 50 ms after calibration; Cross-device generalization; by fusing prior knowledge of device models through Bayesian inference (such as devices with higher market share having higher prior probabilities), the calibration robustness for un-trained devices is improved, and the validation set error is reduced by 77% compared to traditional methods.

[0047] 1.3. Dual-dimensional feature mining and spatio-temporal graph modeling: The dual-dimensional feature mining system includes the time dimension and the modality dimension, and realizes the in-depth analysis of rehabilitation indicators by constructing a time-modal joint feature space, specifically as follows: Time dimension feature mining: perform time series modeling on single-modal data to extract dynamic change patterns, the time series statistics of physiological data, the temporal transition probability of behavioral data, and use long short-term memory networks or temporal convolutional networks to capture long-range time dependencies. Among them, when physiological data shows significant deviation from the baseline in a specific time period, it is marked as a "time anomaly point"; analyze the time lag effect and calculate the time lag correlation between the cognitive task error rate and the previous social interaction duration. Modality dimension feature mining: analyze the cross-dimensional associations of different modal data, extract collaborative and causal features, use transfer entropy between modalities to quantify the information flow direction; use dynamic modal association entropy to evaluate the evolution of cross-modal collaboration; model the interaction weights between modalities based on graph attention mechanism. Among them, use physiological data as the underlying feature, behavioral data as the middle-level feature, and cognitive data as the high-level feature to construct a hierarchical association model; when a change in a certain modality does not trigger the expected response of other modalities, a decoupling warning is triggered.

[0048] 1.3.1. Time-modal joint feature space: Time dimension: Time series statistics, calculate the 5-minute sliding window mean, standard deviation, and kurtosis of the HRV index; Long-range dependence, use an LSTM network (128-dimensional hidden layer, dropout = 0.2) to model the 30-minute time series features of EEG alpha wave power; Lag effect, calculate the Pearson correlation coefficient between the cognitive error rate and the previous social interaction duration, and the lag window is taken from 1 to 5 minutes, and the threshold is marked as a time anomaly point when.

[0049] Modality dimension: Causality analysis, quantify the information flow between modalities based on transfer entropy, such as the causal intensity of EEG alpha wave power (X) on social distance (Y): , where X is the cause modality, the EEG mu-wave power (the power value in the 8 Hz - 12 Hz frequency band, unit: μV), Y is the result modality: the social distance (the Euclidean distance from others, unit: meter), is the discrete state value of X (for example, the mu-wave power is divided into 3 levels: low, medium, and high: , is the state value of Y at the current moment and the next moment (for example, the social distance is divided into 3 levels: near, medium, and far: , is the probability of the joint probability X = x and the current Y = y and the next moment Y = y′, is the conditional probability, the probability that Y = y′ at the next moment given the current Y = y and X = x, is the conditional probability, the probability that Y = y′ at the next moment given only the current Y = y. When bits, it is determined that there is a one-way causal association; Cooperativity evaluation, the dynamic modal correlation entropy (DME) is defined as: ; In the formula, is the state of modality X and the state of modality Y of the joint probability, , are the marginal probability distributions. The larger the DME value, the stronger the independence between modalities (the lower the cooperativity); the entropy change rate triggers the modal decoupling warning.

[0050] Time dimension feature extraction, LSTM network structure: single-layer LSTM, 128 units in the hidden layer, the input is the EEG alpha-wave power sequence of a 30-minute sliding window (time step = 30, feature dimension = 1), and the output is the dynamic trend index of this window (rising / falling / steady, classified by the fully connected layer Softmax).

[0051] Lag effect calculation, for the cognitive error rate sequence and the social interaction duration sequence , calculate the Pearson correlation coefficient with a lag of k minutes: ; Take k = 1 to 5 minutes. If the maximum , in the formula, T is the total number of time points (for example, the daily acquisition duration is T minutes), k is the lag time (minutes), the value range is 1 < k < kmax (for example, kmax = 5), S t+k is the social interaction duration with a lag of k minutes, the cognitive task error rate at time t (percentage), E, S are the average values of the cognitive error rate sequence and the social interaction duration sequence respectively, then mark the current window as a time anomaly point.

[0052] 1.3.2. Construction of Spatiotemporal Graph Model: Divide the daily data into 288 one-minute time windows (the number of nodes ), and construct a spatiotemporal graph: , where is the set of nodes, corresponding to the multimodal data within the time window, is the set of time adjacent edges, connecting adjacent time windows, is the set of modal association edges, connecting different modalities within the same window; Node features: ; In the formula, HRV is heart rate variability, is the prefrontal alpha wave power, is the average social distance, is the cognitive task error rate, is the environmental parameter; Time edge: Connect adjacent windows and , and the edge weight , where are the node features of adjacent windows respectively, and retain the edges with weights ≥ 0.8; Modal edge: Physiological-behavioral-cognitive nodes within the same window are connected pairwise, and the edge weight , highlighting strong association modal pairs.

[0053] Example, node feature standardization: Adopt Z-score standardization for the HRV index, adopt Min-Max scaling for the social distance to [0, 1], and adopt logarithmic transformation for the cognitive error rate ( ), where e is the natural constant; Example of edge weight calculation: The node feature vectors of adjacent time windows t and t + 1 are respectively and , and the cosine similarity is: , because ≥ 0.8, retain the time adjacent edge.

[0054] 1.4. Temporal Graph Neural Network for Sub-symptoms (TGNN): 1.4.1. Network architecture, the model contains three layers: (1) Temporal encoding layer, embed sinusoidal temporal encoding for each node: ; ; where is the time window position, is the encoding dimension. Among them, examples of the physical meaning values of the time position pos are as follows: If the daily rehabilitation data is divided into 288 1-minute windows (T = 288), then pos = 1 corresponds to 9:00 - 9:01 in the morning, pos = 144 corresponds to 12:00 - 12:01 at noon, the dimension index k and the encoding dimension d.

[0055] (2) Graph attention layer, using multi-head attention (number of heads = 4) to model the interaction between modalities: ; In the formula, output feature vector, is the activation function, is the current node feature, is the set of adjacent nodes, k is the attention head index, is the head-specific weight matrix, is the self-attention coefficient, is the interactive attention coefficient of the k-th head, calculated through the shared weight matrix calculate.

[0056] (3) Temporal gating layer, based on the GRU unit (64-dimensional hidden layer) to capture time dependence, with the output dimension being 1 (positive symptom score prediction) or 2 (negative symptom binary classification).

[0057] 1.4.2. Sub-symptom training strategy: Positive symptom model: Input, HRV index, P300 latency, environmental parameters; Supervision signal, PANSS positive score (normalized to [0, 1]); Loss function, (weight decay ), in the formula, is the predicted value (the predicted value of the PANSS positive score output by the model), is the true value (the true value of the PANSS positive score evaluated by the clinician), MSE is the mean square error term, W is the weight matrix, is the L2 regularization term, is the regularization coefficient; Negative symptom model: Input: Social interaction duration, emotional expression richness (FACS encoding number), DMN functional connectivity strength; Supervision signal: Social function recovery label (SFRS ≥ 20 is 1, otherwise 0); Loss function: ; In the formula, is the true label (binary classification label), is the predicted probability (the recovery probability output by the model), is the cross-entropy term 1 (when the true label is 1, measuring the closeness of the predicted probability to 1), is the cross-entropy term 2 (when the true label is 0, measuring the closeness of the predicted probability to 0).

[0058] Graph neural network (GNN), traditionally applied to social networks and molecular structure analysis, is creatively used in this invention for the "neural-behavioral" dynamic association modeling of mental diseases, constructing a spatio-temporal graph containing temporal adjacency edges and modal association edges, which belongs to the innovative application of cross-domain technology transfer.

[0059] 1.5. Low-functional connection pattern of brain network: Capturing the low-functional connection pattern of the resting-state brain network through the Graph Attention Mechanism is an interdisciplinary research method combining neuroscience and deep learning, aiming to analyze the abnormal patterns of functional connectivity (FC) in the resting-state brain using the characteristics of graph neural network (GNN), especially the regions with lower connection strength. The following will elaborate from aspects such as core concepts, technical principles, application processes, and significance: 1.5.1. Resting-state brain network and low-functional connection: Resting-State Brain Network: It refers to the functional connection network measured by techniques such as functional magnetic resonance imaging (fMRI) between different brain regions when the brain is in a state without specific tasks. The temporal sequence correlation between brain regions (nodes) can be quantified as the connection strength (edge weight), forming a whole-brain functional connection matrix; Low-functional connection pattern: It refers to the pattern where the functional connection strength between brain regions is significantly lower than the normal level, which may reflect abnormal information interaction between brain regions. Such patterns are common in neuropsychiatric diseases (such as depression, Alzheimer's disease, autism, etc.), or are related to brain development and aging.

[0060] 1.5.2. Graph Attention Mechanism: It belongs to a variant of graph neural network (GNN) (such as Graph Attention Network GAT). The core idea is to let the model adaptively learn the importance weights of nodes or edges in the graph through the attention mechanism, so as to capture complex topological structures and feature interactions.

[0061] Advantages: It does not require pre-defining a global structure (such as a convolutional kernel), can dynamically allocate attention weights for the neighborhood of each node, is especially suitable for processing non-Euclidean structure data (such as the graph structure of the brain network), and is more sensitive to feature mining of low-weight edges (low-functional connections).

[0062] 1.5.3. Data Preprocessing and Graph Construction: Steps: Brain Region Partitioning: The brain is divided into several regions of interest (ROIs, such as AAL template, Yeo network, etc.), and each ROI serves as a node in the graph.

[0063] Functional Connectivity Matrix Generation: Calculate the correlation of time series between ROIs (such as Pearson correlation coefficient) to obtain a weighted undirected graph, where the edge weight is the connection strength.

[0064] Definition of Low Functional Connectivity: Low-connectivity edges are screened out through statistical thresholds (such as below the mean of the normal group minus 1 standard deviation) or machine learning methods, or the original connection strength is directly used as input, and the model automatically identifies important patterns.

[0065] 1.5.4. Application of Graph Attention Mechanism: Model Architecture: Input: Node features of the graph (such as time series features of ROIs) and adjacency matrix (functional connection strength).

[0066] Attention Layer: Calculate the attention weights for the neighborhood nodes (directly connected brain regions) of each node, and the formula is: ; In the formula, is the node feature vector, is the attention weight between modalities, is the learnable weight matrix, is the activation function, is the set of adjacent nodes of node i.

[0067] Aggregation and Output: Aggregate neighborhood information through attention weights, update node representations, and finally use them for classification (such as disease vs. health) or pattern recognition.

[0068] 2. Scheme Optimization: 2.1. Dynamic Pathological Mechanism Modeling: Traditional Scheme: Relying on static statistical models (such as linear regression) or single-modal analysis, it cannot capture the time dependence of rehabilitation indicators (such as circadian rhythm) and the causal relationship between modalities (such as the predictability of HRV for social behavior).

[0069] Improved Scheme: Dynamic graph network architecture, innovatively applying the temporal graph neural network (TGNN) to the field of mental rehabilitation. Through the GAT layer, cross-modal attention is captured (such as the influence weight of prefrontal alpha waves on speech fluency), and the GRU layer models time series features (such as the morning HRV fluctuation pattern), forming a "time-modal" two-dimensional dynamic modeling ability; A novel evaluation index defines the Dynamic Modal Correlation Entropy (DMCE) and Transfer Entropy (TE) to quantify cross-dimensional synergy and causality, providing a new tool for the study of psychopathological mechanisms. Such dynamic correlation analysis is not covered by existing scales and models.

[0070] 2.2 Subtype-specific evaluation: Traditional approach: The general model ignores the neurobiological mechanism differences between positive / negative symptoms (such as dopaminergic hyperactivity vs. glutamatergic hypofunction), resulting in a low diagnostic accuracy for mixed symptoms (78%) and homogeneous intervention programs.

[0071] Improved approach: Pathological mechanism-driven modeling: Construct independent models based on the BDNF Val66Met genotype (which affects neuroplasticity) and DMN functional connectivity (the core brain network for negative symptoms), integrating gene-image-behavior data across layers instead of simply stratifying by symptom labels. Dynamic weight adaptive fusion: Design a Gated Network to dynamically adjust the fusion weights of positive / negative models according to the patient's baseline characteristics (such as age, disease duration), achieving a "one-person-one-strategy" inference strategy that cannot be realized by existing fixed-weight fusion methods.

[0072] Example 1, dynamic early warning for patients with positive symptoms: Patient profile: Patient A, 28 years old, mainly presenting positive symptoms (PANSS positive score of 32), during the 6th week of rehabilitation training after enrollment.

[0073] Data performance: Physiological modality: The LF / HF ratio of HRV dropped suddenly from 2.3 to 1.1 (30% lower than the baseline), and the P300 latency extended to 450 ms (from 380 ms at baseline). Behavioral modality: The average social distance increased from 0.8 m to 1.7 m, and the entropy value of limb movement trajectories decreased by 40% (indicating behavioral stereotypy). Cognitive modality: The error rate of the Stroop test increased from 15% to 35%, and the median reaction time increased from 700 ms to 1200 ms.

[0074] System response: The modal synergy index calculated by the TGNN model decreased from 0.72 to 0.48 (below the threshold of 0.5), triggering an early warning of positive symptom fluctuations 48 hours in advance. The clinical team timely adjusted the dosage of antipsychotic drugs, and the subsequent PANSS score did not show the expected increase (remaining at 30 points).

[0075] Example 2, rehabilitation assessment for patients with negative symptoms: Patient profile: Patient B, 35 years old, mainly presenting negative symptoms (SFRS score of 10), at the 8th week after enrollment.

[0076] Feature analysis: The strength of DMN functional connectivity, 0.18 (below the normal threshold of 0.25), was strongly negatively correlated with the duration of social interaction (0.5 h / day) (r=-0.71); Dynamic synergy: When the duration of social interaction increased, the EEG alpha wave power did not increase as expected (DME=0.65, triggering a decoupling warning).

[0077] Intervention effect: Based on the system's recommendation to increase social cognitive training (twice a day, 30 minutes each time), after 2 weeks, the DMN connection strength increased to 0.22, the social interaction time increased to 1.2 hours / day, the SFRS score increased to 16 points, and the model prediction accuracy reached 89%.

[0078] The PANSS scoring standard (Positive and Negative Syndrome Scale) is a classic scale used to assess the severity of symptoms in patients with schizophrenia. It was compiled by Kay and other scholars in 1987. This scale helps clinicians and researchers assess patients' positive symptoms, negative symptoms and general psychopathological symptoms through quantitative scoring. It is widely used in the diagnosis, efficacy evaluation and scientific research of schizophrenia.

[0079] 1. Basic structure of PANSS The PANSS contains 30 items divided into three subscales: Positive symptom subscale (7 items): assesses “excessive” or “abnormal” symptoms such as hallucinations and delusions; Negative symptom subscale (7 items): assesses symptoms of “lack” or “low” such as flat affect and decreased will; General psychopathology subscale (16 items): assesses nonspecific symptoms such as anxiety, depression, and cognitive impairment.

[0080] Each item uses a 7-point rating system ranging from 1 to 7 points. The rating criteria are as follows: 1 point: no symptoms; 2 points: very light; 3 points: mild; 4 points: moderate; 5 points: moderately severe; 6 points: severe; 7 points: Extremely severe.

[0081] 2. Subscales and Specific Items Table 2: Positive symptom subscale (P1-P7), as follows:

[0082] Table 3: Negative Symptom Subscale (N1 - N7) is as follows:

[0083] Table 4: General Psychopathology Subscale (G1 - G16) is as follows:

[0084] III. Scoring Method and Precautions Scoring basis: Comprehensive judgment through clinical interviews, patient behavior observation, and medical history data; it needs to be performed by a professionally trained psychiatrist or researcher.

[0085] Time range: Evaluate the symptom manifestations of the patient in the recent week.

[0086] Calculation of total score and subscale scores: Total score: The sum of the scores of 30 items (range 30 - 210 points, the higher the score, the more severe the symptoms); Positive symptom score: The sum of P1 - P7 (7 - 49 points); Negative symptom score: The sum of N1 - N7 (7 - 49 points).

[0087] General psychopathology score: The sum of G1 - G16 (16 - 112 points).

[0088] Precautions: Distinguish between persistent and transient symptoms (e.g., occasional nervousness may not be included in the scoring); Avoid being affected by subjective biases and need to combine objective behavioral evidence (such as whether the patient refuses to eat due to suspicion).

[0089] IV. Clinical Application and Interpretation Diagnostic reference: Positive symptoms as the main manifestation suggest "Type I schizophrenia", and negative symptoms as the main manifestation suggest "Type II"; The total score and subscale scores can assist in differentiating schizophrenia from other mental disorders (such as depression, anxiety disorder).

[0090] Efficacy evaluation: Compare the total score and subscale scores before and after treatment to judge the degree of symptom improvement (such as the decrease in positive symptom score after antipsychotic drug treatment); Negative symptoms respond slowly to treatment and often require long-term follow-up evaluation.

[0091] SFRS scoring criteria: Schizophrenia Functional Recovery Scale (SFRS): It may be used to evaluate the social function, vocational ability, and daily life recovery of schizophrenia patients, etc.

[0092] Evaluate the degree of social function recovery of patients with mental illnesses (such as schizophrenia), including: social interaction ability, vocational / learning ability, daily life self-care ability, family role function.

[0093] Table 5: Schizophrenia Functional Recovery Subscale and Items are as follows:

[0094] Scoring method; Each item uses a scale of 1 - 5 or 1 - 7 points, and is scored according to the patient's performance in the recent 2 weeks.

[0095] Total score = sum of scores of each item. The higher the score, the better the functional recovery is indicated.

[0096] In the application, several formulas involved are calculated by taking their numerical values after dimensionless. The establishment of the formula is obtained by collecting a large amount of data for software simulation to get a formula closest to the real situation. Some coefficients or weights in the formula are set by those skilled in the art according to the actual situation, so no more details will be elaborated here.

[0097] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any arbitrary combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution.

[0098] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units. They can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0099] As described above, only the specific embodiments of this application are provided, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or substitutions, which should all be covered within the protection scope of this application.

Claims

1. A rehabilitation training evaluation system for schizophrenic patients, characterized in that, The evaluation system includes: A multimodal perception module that constructs a four-dimensional data acquisition system to collect physiological data, behavioral data, cognitive data, and environmental parameters in real time; An edge intelligence processing module that integrates a data synchronization unit and a feature extraction unit; Among them, the data synchronization unit adopts a three-stage synchronization mechanism, performs hardware-level calibration through an FPGA clock synchronization module, uses a dynamic time warping algorithm to complete algorithm-level timing alignment, and corrects cross-device baseline differences based on a Bayesian network to achieve semantic-level calibration; The feature extraction unit constructs a two-dimensional feature mining system based on physiological data and behavioral data; A dynamic graph network evaluation module that constructs an evaluation model based on a temporal graph neural network, quantifies dynamic synergy by calculating the difference between the joint distribution and the independent distribution of each dimension of data, and combines the transfer entropy algorithm. A synergy sudden drop threshold is set to trigger multimodal early warning; independent models are trained for positive / negative symptom patients respectively; Among them, the negative symptom model introduces DMN functional connection features, captures the low functional connection pattern of the resting-state brain network through a graph attention mechanism, and predicts the recovery status of different subtypes of patients.

2. The rehabilitation training evaluation system for schizophrenia patients according to claim 1, characterized in that: The hardware devices used by the multimodal perception module include physiological monitoring devices, environmental perception devices, and cognitive interaction terminals, specifically as follows: A physiological monitoring device, including a heart rate bracelet with photoplethysmography technology and an electroencephalogram headband using dry electrode technology, is used to collect physiological data related to heart rate variability, prefrontal wave power, and event-related potentials, reflecting the patient's neurophysiological state at the microscopic level; among them, the indicators related to heart rate variability include the root mean square of the difference between adjacent normal RR intervals and the ratio of low-frequency to high-frequency power spectral density; Environmental perception devices, including a time-of-flight depth camera and an ultra-wideband positioning base station, are used to obtain social distance, limb movement trajectories, and spatial position data, and capture the patient's behavioral performance in the environment from the mesoscopic level; Cognitive interaction terminals, which are tablet computers with touch feedback technology, are used to record the reaction time, error types, and completion trajectories of cognitive tasks, and evaluate the patient's mental state from the cognitive level.

3. The rehabilitation training evaluation system for schizophrenia patients according to claim 1, characterized in that: The FPGA clock synchronization module integrates an ARM processor and an FPGA programmable logic unit, and the specific process is as follows: The FPGA clock synchronization module generates a global clock signal and distributes it to the heart rate bracelet, EEG headset, and depth camera through a differential clock buffer; Hardware synchronization of the heart rate bracelet, EEG headset, and depth camera is performed through the global clock signal.

4. The rehabilitation training evaluation system for schizophrenia patients according to claim 1, characterized in that: The algorithm-level timing alignment performs timing alignment on asynchronously sampled physiological data and behavioral data using a dynamic time warping algorithm. Among them, the physiological data and behavioral data are denoted as q and c respectively, specifically as follows: Define the physiological sequence and the behavior sequence , construct a distance matrix , with elements , where is the mean of all behavior data c in the behavior sequence , and is the standard deviation of all behavior data c in the behavior sequence. is the mean of all biological data q in the physiological sequence , and is the standard deviation of all biological data q in the physiological sequence . is the i-th physiological data in the physiological sequence, is the j-th behavior data in the behavior sequence, m is the number of physiological data in the physiological sequence, n is the number of behavior data in the behavior sequence, to eliminate the influence of dimensional difference. Solve the optimal path through dynamic programming.

5. The rehabilitation training evaluation system for schizophrenia patients according to claim 1, characterized in that: The data processing in the Bayesian network model in the semantic-level calibration stage includes: Collect physiological data of L different models of physiological monitoring devices and construct a prior distribution in a standard test scenario, where L > 20, and the standard test scenario includes the resting state and the cognitive task state; Update the posterior probability using real-time collected data, dynamically adjust the device deviation correction coefficient, and perform adaptive calibration of unknown devices; among them, the data is preprocessed after acquisition, outliers are removed, standardized, and the original heart rate variability value HRV is normalized to the interval [0, 1].

6. The rehabilitation training evaluation system for schizophrenia patients according to claim 1, characterized in that: The physiological data extracted by the feature extraction unit includes calculating HRV indexes, electroencephalogram rhythm power, and changes in the concentration of oxyhemoglobin in the prefrontal lobe, which are used to reflect the neurological function status of the patient; The behavioral data includes social behavior data identified based on the OpenPose algorithm, which is used to reflect the patient's social ability and mental state.

7. A rehabilitation training evaluation system for schizophrenia patients according to claim 1, characterized in that: The two-dimensional feature mining system includes a time dimension and a modality dimension, and realizes the in-depth analysis of rehabilitation indicators by constructing a time-modality joint feature space, specifically as follows: Time dimension feature mining: Perform time series modeling on single-modal data to extract dynamic change patterns, such as time series statistics of physiological data, time series transition probabilities of behavioral data, and use long short-term memory networks or temporal convolutional networks to capture long-range time dependencies. Among them, when physiological data shows a significant deviation from the baseline in a specific time period, it is marked as a time anomaly point; analyze the time lag effect and calculate the time lag correlation between the cognitive task error rate and the duration of the previous social interaction; Modality dimension feature mining: Analyze the cross-dimensional associations of different modal data, extract collaborative and causal features, and use transfer entropy between modalities to quantify the information flow direction; use dynamic modal association entropy to evaluate the evolution of cross-modal collaboration; model the interaction weights between modalities based on the graph attention mechanism. Among them, physiological data is used as the underlying feature, behavioral data as the middle-level feature, and cognitive data as the high-level feature to construct a hierarchical association model; when a change in one modality does not trigger the expected response of other modalities, a decoupling warning is triggered.

8. A rehabilitation training evaluation system for schizophrenia patients according to claim 7, characterized in that: The two-dimensional feature mining system realizes joint modeling through the following steps: Construct a time-modality two-dimensional feature matrix, where the row dimension is the time window and the column dimension is multi-modal features; Use tensor decomposition technology to reduce the dimension of the two-dimensional feature matrix and extract time-modality coupling features; Input the coupling features into a temporal graph neural network to dynamically predict the rehabilitation effect.

9. The rehabilitation training evaluation system for schizophrenia patients according to claim 8, characterized in that: Divide the multi-modal data into time windows as graph nodes, and construct a spatio-temporal graph model containing time adjacency edges and modality association edges: Among them, each graph node contains normalized physiological data, behavioral data, cognitive data, and environmental parameters; connect adjacent time window nodes, and the edge weight is the cosine similarity of the feature vectors to obtain the time continuity of the rehabilitation indicators; connect different modality nodes within the same window, and the edge weight is calculated by dynamic modal association entropy or transfer entropy to quantify the collaboration or causality between modalities.

10. A rehabilitation training evaluation system for schizophrenia patients according to claim 1, characterized in that: The method of training independent models for positive / negative symptom patients in the dynamic graph network evaluation module is as follows: Train an independent model for positive symptom patients: Collect data related to positive symptoms, and perform standardization processing on the data to remove noise interference and unify the scale and format of the data; Based on the deep neural network architecture with attention mechanism, this architecture collects key features related to positive symptoms; Use the positive score of the Positive Symptom Scale as the supervision signal, and use the cross-validation method to evaluate the prediction accuracy of the model for positive symptom fluctuations using an independent test data set; Train an independent model for negative symptom patients: Collect data related to negative symptoms, and also perform standardization and denoising processing on the data; Introduce an architecture that combines a graph convolutional network and a recurrent neural network, and use the duration of social interaction and the richness index of emotional expression as supervision signals; Predict the recovery of social function and the improvement of emotional state.

Images