A virtual reality and augmented reality based intensive psychological and limb rehabilitation system

Abstract

The application provides a severe psychological and limb rehabilitation system based on virtual reality and augmented reality, which comprises a multi-modal data acquisition unit, which acquires limb movement data, physiological index data and psychological state data of a patient in real time; an edge-cloud collaborative processing unit, which comprises an edge computing node and a cloud server; and an immersive interaction unit, which guides the patient to complete rehabilitation training tasks, including upper limb grasping training, lower limb gait training and psychological intervention scenes, through a virtual scene and tactile / auditory / visual feedback; wherein the multi-modal data acquisition unit is in communication connection with the edge computing node, the edge computing node and the cloud server interact data through an encryption protocol, and rehabilitation strategies generated by the cloud server are fed back to the patient in real time through the immersive interaction unit. The VR / AR technology can provide an immersive training environment, and the movement, psychological and physiological states of the patient can be monitored in real time, so that a personalized rehabilitation plan is automatically generated, and precise treatment and personalized intervention are realized.

Description

Technical Field

[0001] This invention belongs to the field of interdisciplinary technology of smart medical care and rehabilitation robots, specifically relating to a critical care psychological and physical rehabilitation system based on virtual reality and augmented reality. Background Technology

[0002] In recent years, with the increasing aging of the population and the rise in patients with serious illnesses, rehabilitation treatment for critically ill patients has become an important topic in the medical field. Traditional rehabilitation training methods, such as physical therapy and speech therapy, while able to help patients recover function to some extent, often suffer from low patient participation and limited effectiveness due to their monotonous, tedious, and inefficient nature. This is especially true for critically ill patients with both physical and psychological impairments, where the effectiveness of traditional methods is even more limited.

[0003] Shortcomings of traditional rehabilitation methods: Low participation: Traditional rehabilitation training methods often rely on the patient's subjective efforts, and the training content and form are relatively monotonous, lacking sufficient interest and interactivity, making it difficult to stimulate the patient's motivation to participate. Many patients easily give up treatment or training due to a lack of sufficient enthusiasm and patience.

[0004] Insufficient Psychological Intervention: Psychological intervention is often overlooked in traditional rehabilitation treatments. Severely ill patients often suffer from psychological disorders such as depression and anxiety. These problems not only affect the patient's emotions and mental state but also directly hinder the physical recovery process. Existing psychological intervention methods typically lack personalization and cannot monitor and regulate the patient's mental state in real time, resulting in limited effectiveness.

[0005] Lack of real-time monitoring and feedback: Traditional rehabilitation methods typically involve face-to-face treatment, making it difficult to assess treatment effectiveness in real time. Patient progress is often judged by medical staff based on experience, lacking precise quantitative evaluation. This not only affects the development of rehabilitation plans but may also lead to inefficient rehabilitation processes.

[0006] The Current Status of Virtual Reality (VR) and Augmented Reality (AR) Technologies in Rehabilitation: With the advancement of technology, virtual reality (VR) and augmented reality (AR) technologies have been gradually introduced into the field of rehabilitation. Through immersive virtual environments, patients can train without the limitations of the real world, increasing the fun and interactivity of training. In particular, VR technology, by simulating daily life environments through virtual scenes, helps patients recover limb and cognitive functions, demonstrating great potential in motor rehabilitation. However, existing VR / AR rehabilitation systems still face the following shortcomings:

[0007] Insufficient data integration and multimodal analysis: Most existing VR / AR rehabilitation systems rely solely on video and image analysis, lacking sufficient integration and processing of motion and physiological data, thus failing to comprehensively assess the patient's rehabilitation status. Traditional systems typically only allow interaction through single visual feedback, lacking in-depth analysis of multimodal data and real-time feedback mechanisms, making it impossible to adjust training content promptly based on the patient's actual condition.

[0008] Lack of personalized rehabilitation plans: Existing VR / AR systems often use fixed training scenarios, and the training plans cannot be personalized according to the patient's rehabilitation progress. For example, the patient's movement or physiological data is not fully collected and analyzed, resulting in an inability to achieve precise and personalized rehabilitation plans. Existing systems are mostly based on a fixed treatment plan for standardized treatment, ignoring the individual differences and dynamic changes of different patients.

[0009] Security and privacy protection issues: With the collection, transmission, and storage of patient data, ensuring that patient privacy is not leaked has become a critical issue that urgently needs to be addressed. Traditional rehabilitation systems lack effective encryption technologies and privacy protection mechanisms, making sensitive patient information vulnerable to leakage, which affects the widespread application of the system and patient trust.

[0010] The inability to effectively integrate edge computing and cloud computing technologies: Although VR / AR technology has been initially applied in rehabilitation training, existing systems rely heavily on cloud computing resources for real-time processing and analysis of multimodal data. This results in high response latency, high bandwidth consumption, and poor real-time performance, especially when processing real-time patient motion and physiological data, where latency issues are particularly severe. Furthermore, existing systems lack effective edge computing support, hindering efficient real-time data processing at edge nodes, leading to low data transmission and processing efficiency.

[0011] The lack of rehabilitation equipment and robotic assistance remains a significant challenge. While many modern rehabilitation devices, such as rehabilitation robots and exoskeletons, have achieved some success in improving limb function recovery, their clinical application still faces limitations. Most existing devices cannot intelligently adjust to the patient's physiological state and lack real-time feedback and interaction, making it impossible to provide customized training programs. Furthermore, the high cost and complexity of rehabilitation robots limit their widespread adoption in practice.

[0012] With the increasing sophistication of medical informatization, the collection, storage, and analysis of patient personal data have become routine operations. However, existing rehabilitation systems often lack data privacy protection measures, making patients' physiological, motor, and psychological data vulnerable to unauthorized access or tampering during transmission and storage. Traditional encryption storage and security protection methods are insufficient to cope with the development of modern technology, posing a significant risk of privacy breaches.

[0013] Addressing the shortcomings of existing technologies, this invention proposes a critical care psychological and physical rehabilitation system based on virtual reality and augmented reality. Through the comprehensive application of technologies such as multimodal data acquisition, edge-cloud collaborative computing, personalized rehabilitation strategies, real-time feedback mechanisms, and data privacy protection, it significantly improves patient engagement, rehabilitation outcomes, and data security. This system not only provides an immersive training environment through VR / AR technology but also automatically generates personalized rehabilitation plans by monitoring patients' movement, psychological, and physiological states in real time, achieving precise treatment and personalized intervention. Real-time data processing via edge computing nodes significantly reduces data transmission latency and improves system response speed, while leveraging the powerful computing capabilities of the cloud for multimodal data fusion and long-term strategy optimization. Furthermore, the system fully utilizes blockchain technology and a federated learning framework to ensure the security and privacy of patient data. Summary of the Invention

[0014] A critical care psychological and physical rehabilitation system based on virtual reality and augmented reality, comprising:

[0015] The multimodal data acquisition unit acquires real-time data on the patient's limb movement, physiological indicators, and psychological state.

[0016] An edge-cloud collaborative processing unit includes an edge computing node and a cloud server, wherein the edge computing node performs real-time analysis of the multimodal data and generates instant feedback instructions, and the cloud server performs multimodal data fusion and long-term rehabilitation strategy optimization.

[0017] The immersive interactive unit guides patients to complete rehabilitation training tasks through virtual scenes and tactile / auditory / visual feedback, including upper limb grasping training, lower limb gait training, and psychological intervention scenarios.

[0018] The data security unit uses blockchain technology and a federated learning framework to encrypt and store patient data and protect its privacy.

[0019] The communication architecture unit includes a multimodal data acquisition unit that is communicatively connected to the edge computing node. The edge computing node interacts with the cloud server via an encrypted protocol, and the rehabilitation strategy generated by the cloud server is fed back to the patient in real time through the immersive interaction unit. This system can provide an immersive training environment using VR / AR technology and automatically generate personalized rehabilitation plans by monitoring the patient's movement, psychological, and physiological states in real time, achieving precision treatment and personalized intervention.

[0020] Furthermore, the multimodal data acquisition unit includes:

[0021] Multiple inertial measurement units (IMUs) are fixed to the patient's shoulder, elbow, wrist, hip, knee, and ankle joints, respectively, to acquire joint motion data at a sampling frequency of not less than 200Hz, with a measurement error of less than 0.05mm;

[0022] A millimeter-wave radar array, installed at the four corners of the training space, operates at a frequency of 60GHz and has a coverage area of ​​no less than 5m×5m. It captures the patient's whole-body movement trajectory through point cloud data.

[0023] A depth camera, positioned 1.5 meters in front of the patient at a height of no less than 1280×720, extracts the patient's joint coordinates using a 3D skeleton tracking algorithm;

[0024] The physiological monitoring components include an electroencephalogram (EEG) sensor, a skin conductance response (GSR) sensor, and a surface electromyography (sEMG) sensor, which monitor the patient's anxiety index, skin resistance, and muscle activation, respectively.

[0025] Furthermore, the edge computing node is further configured as follows:

[0026] A lightweight neural network model is run to calculate joint angles from the patient's real-time motion data. When a motion deviation is detected to exceed a preset threshold, a virtual correction trajectory is generated and superimposed onto the VR scene of the immersive interactive unit.

[0027] The physiological data were preprocessed, including the calculation of heart rate variability (HRV) and muscle activation (MA), using the following formulas:

[0028] , where Ri is the interval between adjacent heartbeats, and T = 1 second.

[0029] Furthermore, the cloud server is further configured as follows:

[0030] Graph Neural Networks (GNNs) are used to fuse and analyze multimodal data. The node feature update formula is as follows:

[0031]

[0032] Wherein, N(v) is the set of neighboring nodes, and the output is the patient's joint coordination score; the patient's rehabilitation progress is predicted based on the Long Short-Term Memory Network (LSTM), with the input being gait deviation, joint angle and heart rate load data from the past 30 days, and the output being a personalized training plan for the next 3 months.

[0033] Furthermore, the immersive interaction unit includes:

[0034] The upper limb training module generates graspable objects in a virtual scene and simulates resistance feedback using haptic gloves. The resistance value is dynamically adjusted based on the weight of the virtual object, calculated using the following formula:

[0035]

[0036] The lower limb training module monitors the patient's center of gravity shift in real time through an AR balance board and maps it to the position of a balance ball in a virtual scene. If the stride difference exceeds 10cm or the stride speed deviation exceeds 20%, the tactile prompt of the vibration strap is triggered.

[0037] The psychological intervention module dynamically switches the meditation environment based on the proportion of beta waves in EEG data. When the proportion of beta waves exceeds 30%, it plays natural sound effects with a frequency below 1kHz and reduces the difficulty of the task.

[0038] Furthermore, the tactile glove includes:

[0039] Twelve piezoelectric ceramic contacts, distributed in the palm and fingertips, provide an actuation force of 0.1N to 5N with a resolution of 0.01N;

[0040] The temperature simulation module, based on a Peltier element, adjusts the temperature within the range of 20°C to 40°C with an accuracy of ±1°C and a response time of less than 3 seconds.

[0041] Furthermore, the data security unit is further configured as follows:

[0042] Patient data is locally encrypted using the AES-256 algorithm and distributed storage is achieved through blockchain technology, with access control controlled by smart contracts.

[0043] In the federated learning framework, edge nodes only upload model gradient parameters, while the cloud aggregation server adds Gaussian noise (σ=0.1) to protect data privacy.

[0044] Furthermore, it also includes:

[0045] The rehabilitation robot interface communicates with the lower limb exoskeleton device via the ROS protocol, and the command format includes joint target angle, torque limit and emergency stop signal;

[0046] The interface for medical systems conforms to the HL7FHIR standard, enabling synchronization of medical record data with the hospital information system.

[0047] Furthermore, the control logic of the lower limb exoskeleton device includes:

[0048] If the joint load exceeds 25 Nm or the patient's heart rate load is greater than 0.85, it will automatically switch to zero resistance mode.

[0049] The assist intensity is dynamically adjusted based on real-time gait symmetry scoring. The calculation formula is as follows:

[0050] .

[0051] Furthermore, the clinical validation methods for the system include:

[0052] A double-blind, randomized, controlled trial was conducted, with the Fugl-Meyer score and Berg balance scale as the primary assessment indicators.

[0053] The rehabilitation efficiency ratio is defined as the ratio of functional recovery to cumulative training time, and the calculation formula is:

[0054] ...

[0055] The critical care psychological and physical rehabilitation system based on virtual reality and augmented reality provided by this invention has significant technical advantages and practical application effects in several aspects, specifically manifested as follows:

[0056] Enhancing patient engagement and treatment outcomes: Through immersive virtual environments and multimodal feedback mechanisms (including visual, auditory, and tactile feedback), patients can participate in rehabilitation through engaging training, significantly increasing their sense of involvement and motivation. Especially in tasks such as upper limb grasping training, lower limb gait training, and psychological intervention for critically ill patients, patients are able to actively interact, reducing the low engagement issues common in traditional rehabilitation methods and thus effectively improving treatment outcomes.

[0057] Personalized Rehabilitation Plans and Dynamic Adjustments: This system monitors patients' motor, psychological, and physiological data in real time and uses edge computing nodes for real-time data analysis. It can automatically generate personalized rehabilitation plans based on the patient's rehabilitation progress, psychological state, and physiological responses. Especially during long-term rehabilitation, the system can dynamically adjust training content and intensity, avoiding the fixed treatment plans of traditional systems. This significantly improves the accuracy and personalization of rehabilitation, meeting the individualized needs of different patients.

[0058] Multimodal data fusion and precise feedback: This system comprehensively monitors patients through multimodal data acquisition units (such as inertial measurement units, millimeter-wave radar, depth cameras, and physiological monitoring components). This not only improves the accuracy of patient status identification but also provides precise real-time feedback. Particularly in the fusion analysis of patient motion and physiological data, it can comprehensively assess the patient's rehabilitation progress and guide the patient's training through correction trajectories in a virtual environment and physiological data feedback, ensuring maximum rehabilitation effectiveness.

[0059] Data security and privacy protection: This invention fully utilizes blockchain technology and a federated learning framework to encrypt and protect patient data, ensuring that sensitive patient data is not leaked during collection, transmission, and storage. Local encryption using the AES-256 algorithm, combined with smart contracts and federated learning methods, effectively protects data privacy while avoiding the privacy risks associated with data sharing in traditional systems.

[0060] Low-latency, high-efficiency, real-time processing: This system utilizes edge computing nodes to perform real-time analysis and processing of multimodal data, significantly reducing data transmission and processing latency. Edge computing nodes can process the patient's motion and physiological data locally in real time, generating instant feedback commands to ensure the system can respond to the patient's performance without delay. This design greatly improves the system's response speed, ensuring patients receive real-time feedback during training and avoiding the latency issues common in traditional cloud computing systems.

[0061] Integration of Rehabilitation Robots and Medical Systems: This invention provides an integration interface with lower limb exoskeleton devices and rehabilitation robots, enabling real-time adjustment of robot-assisted forces to support and guide patients' exercise training, further improving rehabilitation efficiency. The interface with hospital information systems also allows for seamless integration of patient medical records, ensuring consistency between treatment data and medical records during rehabilitation, facilitating comprehensive evaluation and decision-making by physicians.

[0062] In summary, this invention significantly improves the psychological and physical rehabilitation outcomes of critically ill patients by integrating virtual reality, augmented reality, edge computing, cloud computing, artificial intelligence, and data security technologies. It overcomes many limitations of traditional rehabilitation methods and has significant clinical application value and promising prospects for wider application. Attached Figure Description

[0063] Figure 1 This is a schematic diagram of the overall structure of the present invention. Detailed Implementation

[0064] This invention presents a critical care psychological and physical rehabilitation system based on virtual reality and augmented reality, combining multimodal data acquisition, edge computing and cloud computing collaborative processing, personalized rehabilitation plan generation, real-time feedback mechanisms, and data privacy protection. Through precise hardware deployment, sophisticated data analysis, and efficient feedback mechanisms, the system helps patients achieve better training results during rehabilitation. The following details the implementation of this system, including technical details such as hardware deployment, algorithm implementation, data processing flow, and security measures.

[0065] I. Hardware Components and Working Principle

[0066] The hardware deployment of this system is divided into four main modules: motion capture, physiological monitoring, edge computing, and immersive feedback. These modules work together to achieve precise rehabilitation training.

[0067] The motion capture module consists of an inertial measurement unit (IMU), a millimeter-wave radar array, and a depth camera. First, the IMU sensors are the key motion capture devices. Each IMU device uses accelerometers and gyroscopes to collect real-time data on the patient's joint accelerations (ax, ay, az) and angular velocities (ωx, ωy, ωz). This data reflects the patient's movements and joint angles. Each IMU sensor samples at a frequency of 200Hz with an accuracy of 0.05mm, ensuring the capture of every minute change in motion. These sensors are fixed to the patient's shoulder, elbow, wrist, hip, knee, and ankle, respectively, and transmit the data in real-time to the edge computing node via Bluetooth 5.2.

[0068] Secondly, millimeter-wave radar arrays are deployed at the four corners of the training space to capture the patient's full-body movement trajectory. Each radar device operates at a frequency of 60 GHz, with a bandwidth of 4 GHz, a point cloud data update frequency of 30 Hz, and a spatial resolution of 1 mm. The point cloud data obtained through the radar... Where (xi,yi,zi) are the three-dimensional coordinates of each point, and NNN is the total number of point cloud data. This data is transmitted to edge computing nodes via the Wi-Fi 6 protocol, helping the system accurately capture the dynamic changes of the patient.

[0069] The depth camera extracts the patient's joint coordinates using the OpenPose framework. For example, the right elbow coordinates (Pelbow) provide 3D spatial location data. The depth camera has a resolution of 1280×720 and a depth accuracy of ±1mm, enabling it to accurately capture the patient's movement trajectory.

[0070] Physiological monitoring module

[0071] The physiological monitoring module uses multiple sensors to monitor the patient's physiological data in real time to assist in rehabilitation training and psychological intervention. The electroencephalogram (EEG) sensor monitors brainwave signals using the OpenBCI Ganglion headband at a sampling rate of 256Hz. The system calculates the energy percentage (Eβ) of beta waves (12-30Hz) to assess the patient's anxiety level. The calculation formula is as follows:

[0072] Where PSD(f) is the power spectral density at frequency f, T is the time window, EEG(t) is the raw EEG signal, and f is the frequency (12-30Hz).

[0073] A skin conductance response (GSR) sensor helps monitor a patient's anxiety state by measuring skin resistance (RGSR). If RGSR < 100 kΩ, an anxiety state is identified.

[0074] Surface electromyography (sEMG) sensors are used to monitor muscle activation (MA) of target muscle groups, such as the electromyographic signals of the biceps brachii. The formula for calculating MA is:

[0075] Where sEMG(t) is the surface electromyography signal, and T is the calculation time window (unit: seconds).

[0076] Edge computing module

[0077] Edge computing nodes are responsible for processing data transmitted from the motion capture and physiological monitoring modules in real time. Data from each IMU sensor is fed into a lightweight neural network model (such as MobileNet V3) for real-time joint angle calculation. For example, the formula for calculating the angle of the right elbow joint is:

[0078] in: upper is the acceleration vector of the upper arm (from the shoulder IMU); lower is the acceleration vector of the forearm (from the elbow IMU).

[0079] If the angle is deviated If the deviation exceeds 10°, the system will generate a virtual correction trajectory and overlay it onto the VR scene in real time, prompting the patient to adjust their posture, with a feedback delay of less than 50ms.

[0080] Cloud computing module

[0081] The cloud computing module uses a graph neural network (GNN) to fuse and analyze multimodal data from edge computing nodes, generating a patient's motor coordination score. Each human joint (Pjoint) is considered a graph node, and the adjacency matrix A is constructed based on anatomical connections (e.g., shoulder-elbow-wrist is a chain connection). The node feature update formula is:

[0082] in:

[0083] hv(l) is the feature vector of the node in the l-th layer.

[0084] N(v) is the set of neighbors of node v.

[0085] W(l) is the weight matrix after training.

[0086] b(l) is the bias term.

[0087] The cloud-based system also uses a Long Short-Term Memory (LSTM) network to predict rehabilitation progress. Inputting gait deviation Δx, joint angle θknee, and heart rate load (HR Load) from the past 30 days, it outputs a personalized training plan for the next 3 months. The loss function of LSTM is mean squared error (MSE).

[0088] Where: yi represents the actual recovery progress, and y^i represents the recovery progress predicted by the model.

[0089] II. Immersive Interaction and Feedback Mechanism

[0090] Upper limb grasping training

[0091] A virtual kitchen scene was built using the Unity engine, where patients grasped virtual cups for upper limb training. The virtual objects weighed 1.5 kg, and tactile gloves provided feedback by simulating resistance. The resistance calculation formula is as follows:

[0092] Where m = 1.5 kg is the mass of the cup, g = 9.8 m / s² is the acceleration due to gravity, and k is the dynamically adjusted difficulty coefficient. When the grasping accuracy exceeds 90%, the difficulty coefficient k will automatically increase; when the failure rate exceeds 50%, the system will issue a voice prompt "It is recommended to adjust your posture".

[0093] Lower limb gait training

[0094] The AR balance board monitors the patient's center of gravity coordinates (x, y) in real time and maps them to the position of the balance ball in the virtual scene. If the stride difference ΔL > 10 cm, or the stride speed deviation exceeds 20%, the system will trigger the vibration straps to provide tactile feedback.

[0095] Exoskeleton Control: The lower limb exoskeleton device is controlled via the ROS protocol, sending joint angle commands. (If joint load...) The system will automatically switch to zero-resistance mode to avoid overload.

[0096] III. Data Security and Privacy Protection

[0097] Data encryption and storage

[0098] All patient motion, physiological, and psychological data are encrypted using the AES-256 algorithm and distributed stored using the Hyperledger Fabric blockchain. Each data block generates a Merkle Root hash value (e.g., 0x3a7b...e9c1) to ensure data integrity and immutability.

[0099] Federal Learning and Privacy Protection

[0100] Under the federated learning framework, edge nodes only upload the model gradient parameters ΔW, not the patient's raw data. The cloud protects data privacy through Gaussian noise ϵ∼N(0,0.12) to ensure that each gradient update process satisfies differential privacy protection (δ=10−5).

[0101] IV. Clinical Validation and Experimental Data

[0102] Through a double-blind randomized controlled trial, patients in the experimental group showed significant improvements in Fugl-Meyer scores, Berg balance scales, and anxiety index (β-wave proportion), indicating that this system has a significant effect on improving patients' rehabilitation efficiency and psychological state.

[0103]

[0104] This system achieves precise and personalized rehabilitation treatment for critically ill patients through deep integration of hardware and algorithms. By employing technologies such as detailed hardware deployment, real-time data analysis, accurate feedback generation, and data privacy protection, this system has achieved significant advantages in accuracy, real-time performance, and security, demonstrating broad clinical application prospects and industrialization potential.

[0105] This system achieves precise, personalized, and safe rehabilitation treatment for critically ill patients through the synergy of the entire chain of hardware, algorithms, and data, and has significant clinical value and industrialization potential.

[0106] The circuits and controls involved in this invention are all existing technologies and will not be described in detail here.

[0107] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A critical care psychological and physical rehabilitation system based on virtual reality and augmented reality, characterized in that, include: The multimodal data acquisition unit acquires real-time data on the patient's limb movement, physiological indicators, and psychological state. An edge-cloud collaborative processing unit includes an edge computing node and a cloud server, wherein the edge computing node performs real-time analysis of the multimodal data and generates instant feedback instructions, and the cloud server performs multimodal data fusion and long-term rehabilitation strategy optimization. The immersive interactive unit guides patients to complete rehabilitation training tasks through virtual scenes and tactile / auditory / visual feedback, including upper limb grasping training, lower limb gait training, and psychological intervention scenarios. The data security unit uses blockchain technology and a federated learning framework to encrypt and store patient data and protect its privacy. The communication architecture unit includes a multimodal data acquisition unit that is communicatively connected to the edge computing node, an edge computing node that interacts with the cloud server via an encryption protocol, and a rehabilitation strategy generated by the cloud server that is fed back to the patient in real time through the immersive interaction unit.

2. The critical care psychological and physical rehabilitation system based on virtual reality and augmented reality according to claim 1, characterized in that, The multimodal data acquisition unit includes: Multiple inertial measurement units (IMUs) are fixed to the patient's shoulder, elbow, wrist, hip, knee, and ankle joints, respectively, to acquire joint motion data at a sampling frequency of not less than 200Hz, with a measurement error of less than 0.05mm; A millimeter-wave radar array, installed at the four corners of the training space, operates at a frequency of 60GHz and has a coverage area of ​​no less than 5m×5m. It captures the patient's whole-body movement trajectory through point cloud data. A depth camera, positioned 1.5 meters in front of the patient at a height of no less than 1280×720, extracts the patient's joint coordinates using a 3D skeleton tracking algorithm; The physiological monitoring components include an electroencephalogram (EEG) sensor, a skin conductance response (GSR) sensor, and a surface electromyography (sEMG) sensor, which monitor the patient's anxiety index, skin resistance, and muscle activation, respectively.

3. The critical care psychological and physical rehabilitation system based on virtual reality and augmented reality according to claim 1, characterized in that, The edge computing node is further configured as follows: A lightweight neural network model is run to calculate joint angles from the patient's real-time motion data. When a motion deviation is detected to exceed a preset threshold, a virtual correction trajectory is generated and superimposed onto the VR scene of the immersive interactive unit. The physiological data were preprocessed, including the calculation of heart rate variability (HRV) and muscle activation (MA), using the following formulas: , where Ri is the interval between adjacent heartbeats, and T = 1 second.

4. The critical care psychological and physical rehabilitation system based on virtual reality and augmented reality according to claim 1, characterized in that, The cloud server is further configured as follows: Graph Neural Networks (GNNs) are used to fuse and analyze multimodal data. The node feature update formula is as follows: Wherein, N(v) is the set of neighboring nodes, and the output is the patient's joint coordination score; the patient's rehabilitation progress is predicted based on the Long Short-Term Memory Network (LSTM), with the input being gait deviation, joint angle and heart rate load data from the past 30 days, and the output being a personalized training plan for the next 3 months.

5. A critical care psychological and physical rehabilitation system based on virtual reality and augmented reality according to claim 1, characterized in that, The immersive interactive unit includes: The upper limb training module generates graspable objects in a virtual scene and simulates resistance feedback using haptic gloves. The resistance value is dynamically adjusted based on the weight of the virtual object, calculated using the following formula: The lower limb training module monitors the patient's center of gravity shift in real time through an AR balance board and maps it to the position of a balance ball in a virtual scene. If the stride difference exceeds 10cm or the stride speed deviation exceeds 20%, the tactile prompt of the vibration strap is triggered. The psychological intervention module dynamically switches the meditation environment based on the proportion of beta waves in EEG data. When the proportion of beta waves exceeds 30%, it plays natural sound effects with a frequency below 1kHz and reduces the difficulty of the task.

6. A critical care psychological and physical rehabilitation system based on virtual reality and augmented reality according to claim 5, characterized in that, The tactile gloves include: Twelve piezoelectric ceramic contacts, distributed in the palm and fingertips, provide an actuation force of 0.1N to 5N with a resolution of 0.01N; The temperature simulation module, based on a Peltier element, adjusts the temperature within the range of 20°C to 40°C with an accuracy of ±1°C and a response time of less than 3 seconds.

7. A critical care psychological and physical rehabilitation system based on virtual reality and augmented reality according to claim 1, characterized in that, The data security unit is further configured as follows: Patient data is locally encrypted using the AES-256 algorithm and distributed storage is achieved through blockchain technology, with access control controlled by smart contracts. In the federated learning framework, edge nodes only upload model gradient parameters, while the cloud aggregation server adds Gaussian noise (σ=0.1) to protect data privacy.

8. The critical care psychological and physical rehabilitation system based on virtual reality and augmented reality according to claim 1, characterized in that, Also includes: The rehabilitation robot interface communicates with the lower limb exoskeleton device via the ROS protocol, and the command format includes joint target angle, torque limit and emergency stop signal; The interface for medical systems conforms to the HL7FHIR standard, enabling synchronization of medical record data with the hospital information system.

9. A critical care psychological and physical rehabilitation system based on virtual reality and augmented reality according to claim 8, characterized in that, The control logic of the lower limb exoskeleton device includes: If the joint load exceeds 25 Nm or the patient's heart rate load is greater than 0.85, it will automatically switch to zero resistance mode. The assist intensity is dynamically adjusted based on real-time gait symmetry scoring. The calculation formula is as follows: 。 10. A critical care psychological and physical rehabilitation system based on virtual reality and augmented reality according to claim 1, characterized in that, The clinical validation methods for the system include: A double-blind, randomized, controlled trial was conducted, with the Fugl-Meyer score and Berg balance scale as the primary assessment indicators. The rehabilitation efficiency ratio is defined as the ratio of functional recovery to cumulative training time, and the calculation formula is: 。

Images