A hand training device based on touch-visual double synchronization

Abstract

Traditional rehabilitation methods lack interest, and the training enthusiasm of the elderly is low; the existing training equipment is large in size and high in cost, and is not suitable for family use; the existing VR training system lacks tactile feedback and cannot realize multi-sensory coordinated stimulation; there is a lack of professional solutions and quantitative evaluation means for finger fine motor actions. The application discloses a hand training device based on touch-visual double synchronization, and relates to the technical field of medical instruments, which comprises a collecting terminal and a remote processing terminal, the remote processing terminal is arranged on a VR glasses, and comprises a data collection layer, which is used for acquiring the rotation angle, pressing force and hand inclination angle uploaded by the collecting terminal; a data processing and synchronization layer, which comprises an action mapping module, which is used for mapping the collected rotation angle into a grabbing / release action in a VR scene, mapping the pressing force into an object weight perception, and mapping the hand inclination angle into a view angle offset; and a VR scene interaction layer, which is used for realizing the action mapped by the action mapping module in the VR scene.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, specifically to a hand training device based on tactile-visual dual synchronization. Background Technology

[0002] With the accelerating aging of the population, motor function impairment among the elderly is becoming increasingly prominent. Statistics show that approximately 30% of people aged 60 and above in my country experience varying degrees of decreased finger dexterity, primarily manifested as stiff finger joints, weakened grip strength, and reduced fine motor skills. These problems severely impact the quality of life for the elderly, limiting basic activities such as dressing, eating, and writing.

[0003] Currently, the following methods are mainly used for finger training in the elderly: (1) Traditional physical therapy: manual massage and passive exercise by a rehabilitation therapist, but there are problems such as high labor costs and limited training frequency; (2) Mechanical rehabilitation equipment: such as finger rehabilitation robots, but the equipment is bulky and expensive, and is not suitable for home use; (3) Simple rehabilitation devices: such as grip balls, rubber bands, etc., lack feedback mechanisms, patients have low training enthusiasm, and the rehabilitation effect is difficult to evaluate.

[0004] In recent years, virtual reality (VR) technology has been applied in the field of rehabilitation medicine. VR technology can create immersive training environments and stimulate patients' training motivation through visual feedback. However, existing VR rehabilitation systems mainly focus on the large joint movements of the upper limbs, with few rehabilitation programs targeting fine motor skills of the fingers; at the same time, existing systems lack tactile feedback mechanisms, failing to achieve bidirectional synchronization of "tactile and visual" sensations, resulting in limited rehabilitation effects.

[0005] In summary, the existing technology has the following problems: 1. Mismatch between sensor accuracy and rehabilitation needs: The positioning accuracy of the controllers in existing consumer-grade VR devices is at the millimeter level, which cannot meet the fine detection requirements of finger joint angles (requiring degree level) and pressure (requiring 0.1N level); 2. The mapping algorithm lacks physiological adaptability: Simple linear mapping does not take into account the mechanical properties of human finger tendons and the nonlinear characteristics of tactile perception, resulting in a disconnect between training actions and visual feedback; 3. Lack of personalized adaptive mechanism: The training is not graded according to the degree of patients' functional impairment. Severe patients have difficulty completing active training, while mild patients lack the challenge, resulting in low training compliance. 4. Excessive closed-loop feedback latency: The latency from motion acquisition to visual / tactile feedback exceeds 200ms, which is beyond the human perception-motor integration time window (approximately 100ms), making effective neuroplasticity training impossible.

[0006] Therefore, there is an urgent need for a lightweight, low-cost rehabilitation device suitable for the elderly, capable of accurately detecting finger movements, providing tactile feedback and real-time synchronization with visual scenes, to address the rehabilitation pain points of the elderly who are "unwilling to move, unable to move, and afraid to move". Summary of the Invention

[0007] To address the aforementioned problems, this invention aims to provide a hand training device based on tactile-visual dual synchronization, which can achieve closed-loop rehabilitation training of passive / active finger movements through bidirectional synchronization of tactile feedback and virtual reality visual feedback.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A hand training device based on haptic-visual dual synchronization includes a data acquisition terminal and a remote processing terminal. The remote processing terminal is mounted on VR glasses and includes: The data acquisition layer is used to acquire the rotation angle, pressure intensity, and hand tilt angle uploaded by the acquisition terminal. The data processing and synchronization layer includes a motion mapping module and a data synchronization module. The motion mapping module is used to map the collected rotation angle to the grab / release action in the VR scene, the pressure intensity to the object weight perception, and the hand tilt angle to the viewpoint offset. The VR scene interaction layer is used to build different scenes and implement the actions mapped by the action mapping module in the VR scene.

[0009] Based on the above technical solutions, the data acquisition terminal further includes: Bracelet; The interactive module, located on the wristband, includes at least three rotatable and pressable bionic joint knobs for simulating finger flexion and extension movements; The sensor module, located inside the wristband, includes a gyroscope, a pressure sensor, and an accelerometer, used to collect hand movement data; The communication module, located inside the wristband, is used to pair with a remote processing terminal; The feedback module, located inside the wristband, includes a miniature vibration motor for providing tactile guidance feedback.

[0010] Furthermore, based on the above technical solutions, the rotation angle range of the bionic joint knob is 0°-90° with an accuracy of ±1°; the pressure detection range is 0.1-5N with an accuracy of ±0.1N; and the hand tilt angle detection range is ±30° with an accuracy of ±0.5°.

[0011] Furthermore, the VR scene interaction layer also includes an assessment feedback module to assess the patient's finger movement ability. Patients with a finger range of motion <30° or a pressure intensity <1N enter passive guidance mode; patients with a finger range of motion ≤30° and a pressure intensity <60° or a pressure intensity ≤1N enter active assistance mode; patients with a finger range of motion ≥60° and a pressure intensity ≥3N enter self-training mode.

[0012] Furthermore, through the above technical solutions, in the active assistance mode, motion assistance guide lines are superimposed on the VR scene.

[0013] Furthermore, through the above technical solutions, in the passive guidance mode, the vibration motor built into the wristband provides pulsed vibrations to guide the fingers to passively follow the movement according to the rhythm of the VR scene.

[0014] Furthermore, through the above technical solution: the motion mapping module maps the collected rotation angle to a grab / release action in the VR scene, and the mapping formula is: GrabValue = ; θ is the rotation angle of the knob, GrabValue is the intensity of the grab / release action, k1=0.02, k2=0.013.

[0015] Through the above technical solutions, further: the pressure intensity is mapped to the object weight perception, and the mapping formula is: Weight = 200g×ln(F+1) / ln(6); Weight is the perceived weight of the object, and F is the pressure applied.

[0016] Furthermore, using the above technical solutions, the hand tilt angle is mapped to the view offset, and the mapping formula is: ViewOffset = α × 1.5°; ViewOffset is the view offset angle, and α is the hand tilt angle.

[0017] A hand training method based on haptic-visual dual synchronization, implemented using a hand training device based on haptic-visual dual synchronization as described above, includes the following steps: The patient's finger dexterity is assessed through an interactive module; the assessment includes measuring the maximum angle of finger flexion and extension, measuring the maximum pressure applied, and assessing cognitive function through question-and-answer sessions. The training mode is automatically matched based on the evaluation results; Patients with a maximum finger flexion / extension angle <30° or a pressing force <1N enter the passive guidance mode; patients with a maximum finger flexion / extension angle of 30° ≤ 60° or a pressing force of 1N ≤ 3N enter the active assistance mode; and patients with a maximum finger flexion / extension angle ≥60° and a pressing force ≥3N enter the self-training mode.

[0018] The beneficial effects of this invention are: (1) Physiological adaptive mapping: A nonlinear mapping algorithm with piecewise linear and logarithmic compression is used to match the mechanical properties of human finger tendons and tactile perception properties, solve the problem of "no feedback for micro-movement and early saturation for large-movement" caused by existing linear mapping technology, and improve the precision of rehabilitation training.

[0019] (2) Millisecond-level dual-synchronization closed loop: Through hardware timestamps, dynamic buffer queues and delay prediction algorithms, the delay of VR presentation collected by the sensor is less than 100ms, and the tactile feedback is strictly synchronized with the visual scene, forming an effective neuroplasticity training stimulus.

[0020] (3) Adaptive graded training: Based on the quantitative assessment of ROM and pressure intensity, three training modes are automatically matched and the difficulty parameters are dynamically adjusted to solve the compliance problem of "severe patients cannot complete the training and mild patients lack challenge".

[0021] (4) Lightweight medical design: The wristband weighs no more than 80g and is made of medical-grade silicone material. The cost per unit is controlled within 300 yuan. Through the combination of FSR402 thin film pressure sensor and rotary encoder, medical-grade detection accuracy is achieved in the range of 0.1N-5N and 1° resolution, which is suitable for bulk purchase by hospitals and long-term use by families.

[0022] (5) Quantitative rehabilitation assessment: The system automatically records multi-dimensional data such as ROM change curve, pressure distribution, action completion time, and tactile feedback response delay. It uses sliding window and exponential smoothing algorithm to generate rehabilitation trend prediction to assist doctors in developing personalized rehabilitation plans.

[0023] (6) Age-friendly design: The VR screen uses high-contrast colors, the text size is not less than 24pt, and key operations are equipped with voice guidance; overload protection and anti-dizziness mode are set to ensure the safety of the elderly. Attached Figure Description

[0024] Figure 1 This is a block diagram of the overall structure of a hand training device based on tactile-visual dual synchronization according to the present invention. Figure 2 This is a schematic diagram of the data acquisition terminal structure of the present invention; Figure 3 This is a flowchart of the action mapping and data synchronization process of the present invention; Figure 4 This is a schematic diagram of the VR training scene of the present invention; Figure 5 This is a flowchart of the hierarchical training mode of the present invention; Figure 6 This is a schematic diagram illustrating the working principle of the present invention; Figure 7Images depicting actual VR rehabilitation training scenarios for elderly patients; Figure 8 This is the interface for the recovery trend report. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0026] Figure 1 This is a block diagram of the overall structure of a hand training device based on tactile-visual dual synchronization according to the present invention. As shown in the figure, the system includes two main parts: a data acquisition terminal and a remote processing terminal. Real-time data transmission is achieved through Bluetooth 5.0, forming a closed-loop training mechanism of "finger micro-movement → screen response → tactile feedback".

[0027] Figure 2 This is a schematic diagram of the hardware structure of the wristband of the present invention, which is the physical part of the data collection terminal. The main body of the wristband is made of medical-grade silicone material in one piece, with an arc shape to fit the wrist curve of the elderly. The total weight of the wristband is controlled within 80g, so there is no pressure when wearing it, making it suitable for long-term use.

[0028] The bracelet's front interactive area integrates three bionic joint knobs arranged in a straight line, 25mm apart. Each knob is 20mm in diameter and 8mm in height, with a diamond-patterned anti-slip texture designed to accommodate the weakened finger strength of the elderly. Internally, each knob is connected to a rotary encoder (model: EC11, 1° resolution, 1 million cycles lifespan), capable of detecting rotation angles from 0° to 90° with a resolution of 1°.

[0029] Each knob is embedded with an FSR402 thin-film pressure sensor (sensing area diameter 12.7mm, thickness 0.3mm). After being acquired by a voltage divider circuit and a 12-bit ADC, the force detection range of 0.1N-5N is achieved through a lookup table method with an accuracy of ±0.1N. The resistance-pressure characteristic of the FSR402 exhibits non-linear exponential decay. In this embodiment, a piecewise linear lookup table method is used for calibration: 20 calibration points are set in the 0.1N-1N range, and 40 calibration points are set in the 1N-5N range. Accurate measurement across the entire range is achieved through linear interpolation.

[0030] The wristband features a physical "one-click start / pause" button on its side, 12mm in diameter and 2mm raised, with clear tactile feedback to prevent accidental touches. The button is connected to a microswitch with a lifespan of at least 100,000 cycles.

[0031] The wristband also integrates an MPU6050 six-axis gyroscope (detecting hand tilt angle, with a detection range of ±30° and an accuracy of ±0.5°) and an ADXL345 three-axis accelerometer (detecting hand swing amplitude, with a range of ±16g). All sensors have a uniform sampling frequency of 50Hz and are connected to the main control chip (nRF52832) via an I2C bus.

[0032] The communication module uses the nRF52832 Bluetooth 5.0 chip, supporting the Bluetooth Low Energy (BLE) protocol. The wristband pairs directly with the Thunderbird VR glasses without requiring an additional base station. The data packet format is: [Header(2B)] + [Timestamp(4B)] + [GyroX(2B)] + [GyroY(2B)] + [GyroZ(2B)] + [Pressure(2B)] + [AccelX(2B)] + [AccelY(2B)] + [AccelZ(2B)] + [CRC(1B)], with a total length of 19 bytes. With a 20ms connection interval and a 247-byte MTU configuration, the theoretical transmission latency is approximately 40-60ms. Including protocol stack processing time, the total latency is controlled within 80ms, meeting the design requirement of <100ms. It has a built-in 800mAh lithium polymer battery, providing approximately 5 hours of continuous use and 72 hours of standby time under typical usage scenarios (15mA for sensor + 30mA average for intermittent vibration). It uses a magnetic charging interface with a charging current of 500mA and a full charge time of approximately 2 hours.

[0033] The haptic feedback module uses a miniature eccentric rotating mass (ERM) vibration motor (model: 0820, rated voltage 3V), which achieves 1-3Hz pulsed vibration via PWM drive. The vibration intensity is adjusted via duty cycle: weak (30% duty cycle, approximately 0.3N vibration force), medium (60% duty cycle, approximately 0.6N vibration force), and strong (90% duty cycle, approximately 1.0N vibration force). The remote processing terminal includes a data processing and synchronization layer and a VR scene interaction layer. Figure 3 This is a flowchart illustrating the motion mapping and data synchronization process of this invention. The software system is developed based on the Unity 3D engine, written in C#, and runs on the Android system of the Thunderbird VR glasses. The data processing and synchronization layer includes a motion mapping module and a data synchronization module.

[0034] The motion mapping module maps the rotation angle of the wristband knob (0°-90°) to the "grab / release" action in the VR scene, the pressure (0-5N) to "object weight perception", and the hand tilt angle (±30°) to "viewpoint offset".

[0035] The core algorithm of the motion mapping module is as follows: the knob rotation angle θ (0°-90°) is mapped to the degree of object grasping in the VR scene, specifically using piecewise linear mapping: When 0°≤θ<30°, GrabValue = k1×θ, where k1=0.02, corresponding to micro-motion recognition in passive guidance mode; high gain ensures that micro-motion (1°-2°) can produce obvious visual feedback (grasp accuracy 2%-4%), enhancing patients' training confidence; When 30°≤θ<60°, GrabValue = 0.6 + k2×(θ-30), where k2=0.013, corresponding to progressive grasping in active assist mode; medium gain matches the force-length linear range of the tendon in the middle of finger flexion, progressively improving the grasping degree (60%-99%). When 60°≤θ≤90°, GrabValue = 1.0, which corresponds to full grasping in the autonomous training mode; when the full grasping state is reached, angle changes beyond 60° do not change the grasping degree, but are recorded as activity range data for evaluation.

[0036] The pressure intensity F (0-5N) is mapped to the perceived weight of the object using logarithmic compression mapping, Weight = 200g×ln(F+1) / ln(6), where F is the pressure intensity (unit: N), and Weight is the perceived weight of the object (unit: g), with a mapping range of 0-1000g.

[0037] When F=0.1N, Weight≈54g (a light touch can be felt); - When F=1N, Weight≈286g (equivalent to picking up an apple); - When F=3N, Weight≈644g (equivalent to picking up a glass of water); - When F=5N, Weight=1000g (equivalent to picking up a bottle of water).

[0038] This logarithmic mapping conforms to the Weber-Fechner law, providing high resolution (distinguishing differences of 0.05N) in the light pressure range (0.1N-1N) and avoiding rapid saturation in the heavy pressure range (3N-5N), enabling patients to obtain fine force feedback at different stages of rehabilitation.

[0039] The hand tilt angle α (±30°) is mapped to the view offset using the formula: ViewOffset = α × 1.5°. Here, α is the hand tilt angle (in °), and ViewOffset is the view offset angle (in °), with a ±45° view offset limit set. Hand tilts exceeding ±30° do not increase the view offset, preventing dizziness.

[0040] The data synchronization module adopts a lightweight SDK design, with an SDK size of no more than 5MB, and is embedded in the system layer of the Thunderbird VR system.

[0041] The data synchronization module adopts a timestamp alignment mechanism. The main control chip nRF52832 records a local timestamp (1ms accuracy) every time the sensor collects data (50Hz, every 20ms) and sends it to the VR end with the data packet. After receiving the data, the VR end stores it in a circular buffer queue with a depth of 5 frames (covering a 100ms time window) and dynamically adjusts the reading position according to the network latency. The least squares method is used to predict the network transmission latency and dynamically adjust the buffer time to ensure that the total latency from sensor acquisition to VR presentation does not exceed 100ms.

[0042] Actual test data shows that with a Bluetooth 5.0 BLE connection interval of 20ms and an MTU of 247B, the average transmission latency is about 45ms. With VR rendering time (about 16ms@60fps) and buffer compensation, the total latency is stable in the range of 80-95ms.

[0043] The VR scene interaction layer offers three types of training content: fine motor training scenarios, cognitive rehabilitation scenarios, and passive assistance scenarios, along with an assessment and feedback module. Users generate motion input by rotating knobs, pressing buttons, and tilting their hands. After being collected by sensors, processed, and transmitted via the SDK, the motion is mapped into the VR scene. Simultaneously, the haptic feedback module provides vibration guidance, forming a complete closed-loop feedback. Figure 4 This is a schematic diagram of the VR training scenario of the present invention.

[0044] The fine motor skills training scenarios adopt a nostalgic theme, including two scenarios: "operating an old-fashioned sewing machine" and "tuning an old-fashioned radio." In the sewing machine scenario, rotating the knob simulates pedaling, and the sewing process is displayed on the screen simultaneously, triggering nostalgic sound effects (the "click-clack" sound of an old-fashioned sewing machine). In the radio scenario, rotating the knob adjusts the channel, and the screen displays the band change, playing classic old songs or storytelling excerpts.

[0045] The cognitive rehabilitation scenario employs a "fruit and vegetable sorting game" design. Patients control the movement of a basket in VR by tilting their wristbands and press knobs to "grab" falling fruits and vegetables. When a correct grab is made, the screen displays an increase in score and plays encouraging sound effects; when an incorrect grab is made, the wristband vibrates to alert the patient, and the screen displays a correct sorting prompt. This scenario simultaneously trains hand-eye coordination and cognitive judgment abilities.

[0046] The passive assistive scenario is designed for patients with extremely weak finger dexterity. In the "following the butterfly" scenario, vibration pulses guide passive finger movement. The wristband has a built-in miniature vibration motor that provides pulsed vibrations according to the rhythm of the VR scene (such as "following the butterfly"). The vibration frequency is adjustable from 1 to 3 Hz, guiding the fingers to passively follow the movement. The vibration intensity has three levels: weak (0.3N), medium (0.6N), and strong (1.0N), with a vibration duration of 0.5 seconds and an interval of 1.5 seconds, forming a regular tactile guidance rhythm to stimulate the fingers to passively follow the movement. Patients can choose according to their own comfort level.

[0047] Based on the patient's finger mobility assessment results, the system automatically matches passive guidance mode, active assistance mode, or self-training mode to achieve a personalized rehabilitation training plan. Figure 5 This is a flowchart of the hierarchical training mode of the present invention.

[0048] When the system is started, the patient’s finger mobility is assessed first. The assessment indicators include: (1) Finger range of motion (ROM): guide the patient to flex and extend their fingers to the maximum extent three times, record the maximum angle of the rotary encoder, and take the average value; (2) Peak pressure: guide the patient to press the knob with the maximum force three times, record the peak value of the FSR sensor, and take the average value; (3) Cognitive level: assess the cognitive status through simple questions and answers in the VR scene (such as “Please select the red fruit”).

[0049] Based on the assessment results, the system automatically matches the training mode: patients with ROM < 30° or pressure < 1N enter the passive guidance mode; patients with 30° ≤ ROM < 60° or 1N ≤ pressure < 3N enter the active assistance mode; and patients with ROM ≥ 60° and pressure ≥ 3N enter the autonomous training mode.

[0050] The passive guidance mode is suitable for severely disabled elderly people (such as those in the early post-stroke period). The bracelet's vibration frequency is synchronized with the rhythm of the VR scene, stimulating the fingers to awaken their sense of touch. In passive guidance mode, moving targets (such as butterflies or petals) are presented in the VR scene. The patient simply follows the visual target, and the vibration motor pulses at a frequency of 1-3Hz. The vibration is synchronized with the VR event of the target contacting the palm, guiding the fingers to passively flex and extend. For example, in the "petal falling" scene, the bracelet vibrates when the petal touches the palm, guiding the fingers to grasp. The vibration lasts for 0.5 seconds, with an interval of 1.5 seconds, forming a regular guidance rhythm.

[0051] Adjust the vibration intensity based on the vibration response delay (the interval from the start of vibration to the slight movement of the finger) from the first three training sessions. Increase the intensity by one level if the delay is >2s, and decrease it by one level if the delay is <0.5s.

[0052] The active assistance mode is suitable for elderly people with mild disabilities. Motion guidance lines are set in the VR scene; for example, a dotted line guides the finger path when grasping a teacup. The patient can complete the action by rotating a knob along the dotted line. The resistance of the knob is adjustable (0.5N-2N), initially 0.5N, increasing by 0.1N every 5 standard movements to gradually increase finger strength. When the movement deviates from the guidance line beyond a threshold, the guidance line in the VR scene turns red and flashes as a warning.

[0053] The self-training mode is suitable for elderly people in the rehabilitation phase, allowing them to freely explore VR scenes and automatically generate training reports. In this mode, patients can freely explore VR scenes (such as "virtual garden planting"). They complete the entire process of "loosening soil, sowing seeds, and watering" by operating a knob, with the system recording operation data in real time. After training, the system automatically generates a training report, with evaluation indicators including: operational accuracy (success rate of grasping), reaction time (from the appearance of the target to the completion of grasping), and strength control stability (standard deviation of pressing force). The system automatically adjusts the task complexity based on performance, such as increasing the number of objects to be manipulated simultaneously or shortening the reaction time requirement.

[0054] Furthermore, the present invention includes a security protection mechanism.

[0055] The wristband has an "overload protection" function: when the pressure exceeds 5N, the system will automatically record and issue a warning sound; when continuous vibration exceeds 30 seconds, the training will automatically pause to avoid muscle fatigue in the elderly.

[0056] In VR scenes, set an "anti-motion sickness mode": the viewing angle movement speed should not exceed 30° / s to reduce the risk of motion sickness; the frame rate should be kept above 60fps to avoid screen stuttering and discomfort; and a gradual transition should be used when switching scenes, with a transition time of no less than 0.5 seconds.

[0057] The system also features an "emergency pause" function: patients can press the start / pause button on the side at any time to immediately interrupt training, freeze the VR screen on the current frame, and stop the wristband from vibrating. Pressing the button again will resume training, or pressing and holding for 3 seconds will exit the system.

[0058] The system records the following data in real time and generates a recovery trend report, including: ROM change curve: Record the maximum flexion and extension angle for each training session and calculate the 7-day moving average improvement rate; Based on the distribution of force intensity: statistically analyze the distribution frequency of different force ranges and calculate the peak force growth curve; Tactile feedback response: Record the delay time from vibration trigger to finger micro-movement to assess the recovery of neural conduction function; Training adherence: Statistics on training frequency, duration, and completion rate are used to generate a adherence score.

[0059] Predicting the recovery trend over the next 7 days using exponential smoothing: α=0.3 is the smoothing coefficient, which helps doctors adjust rehabilitation plans.

[0060] Figure 6 This is a schematic diagram illustrating the working principle of the invention. The system collects real-time micro-movement data of the fingers through a lightweight rehabilitation bracelet worn on the patient's wrist. After motion mapping and real-time synchronization processing, it drives the VR glasses to present the corresponding visual scene on the one hand, and generates tactile guidance feedback to the patient based on the scene parameters on the other hand, thus forming a closed-loop training system consisting of "sensor acquisition - data processing - visual / tactile dual feedback". Its core lies in achieving data-driven rehabilitation assessment and immersive closed-loop training through the combination of lightweight hardware and age-appropriate scenes. Figure 7 Images depicting actual VR rehabilitation training scenarios for elderly patients. Figure 8 This is the interface for the recovery trend report.

[0061] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A hand training device based on tactile-visual dual synchronization, characterized in that, The device includes a data acquisition terminal and a remote processing terminal. The remote processing terminal is mounted on the VR glasses, and includes: The data acquisition layer is used to acquire the rotation angle, pressure intensity, and hand tilt angle uploaded by the acquisition terminal. The data processing and synchronization layer includes a motion mapping module and a data synchronization module. The motion mapping module is used to map the collected rotation angle to the grab / release action in the VR scene, the pressure intensity to the object weight perception, and the hand tilt angle to the viewpoint offset. The VR scene interaction layer is used to build different scenes and implement the actions mapped by the action mapping module in the VR scene.

2. The apparatus according to claim 1, characterized in that, The data acquisition terminal includes: Bracelet; The interactive module, located on the wristband, includes at least three rotatable and pressable bionic joint knobs for simulating finger flexion and extension movements; The sensor module, located inside the wristband, includes a gyroscope, a pressure sensor, and an accelerometer, used to collect hand movement data; The communication module, located inside the wristband, is used to pair with a remote processing terminal; The feedback module, located inside the wristband, includes a miniature vibration motor for providing tactile guidance feedback.

3. The apparatus according to claim 2, characterized in that: The bionic joint knob has a rotation angle range of 0°-90° with an accuracy of ±1°; a pressing force detection range of 0.1-5N with an accuracy of ±0.1N; and a hand tilt angle detection range of ±30° with an accuracy of ±0.5°.

4. The apparatus according to claim 3, characterized in that: The VR scene interaction layer also includes an assessment feedback module for assessing the patient's finger movement ability; Patients with a finger range of motion <30° or a pressure intensity <1N enter passive guidance mode; patients with a finger range of motion ≤30° and a pressure intensity <60° or a pressure intensity ≤1N enter active assistance mode; patients with a finger range of motion ≥60° and a pressure intensity ≥3N enter self-training mode.

5. The apparatus according to claim 4, characterized in that: In active assistance mode, motion-assisting guide lines are overlaid in the VR scene.

6. The apparatus according to claim 5, characterized in that: In passive guidance mode, the wristband's built-in vibration motor provides pulsed vibrations, guiding the fingers to passively follow the movement according to the rhythm of the VR scene.

7. The apparatus according to claim 6, characterized in that: The motion mapping module maps the acquired rotation angles to grab / release actions in the VR scene. The mapping formula is as follows: GrabValue = ; θ is the rotation angle of the knob, GrabValue is the intensity of the grab / release action, k1=0.02, k2=0.

013.

8. The apparatus according to claim 7, characterized in that: The pressure applied is mapped to the perceived weight of the object, and the mapping formula is: Weight = 200g×ln(F+1) / ln(6); Weight is the perceived weight of the object, and F is the pressure applied.

9. The apparatus according to claim 8, characterized in that: The hand tilt angle is mapped to the view offset, and the mapping formula is: ViewOffset = α × 1.5°; ViewOffset is the view offset angle, and α is the hand tilt angle.

10. A hand training method based on haptic-visual dual synchronization, characterized in that, The hand training device based on tactile-visual dual synchronization as described in any one of claims 1-9 is implemented by comprising the following steps: The patient's finger dexterity is assessed through an interactive module; the assessment includes measuring the maximum angle of finger flexion and extension, measuring the maximum pressure applied, and assessing cognitive function through question-and-answer sessions. The training mode is automatically matched based on the evaluation results; Patients with a maximum finger flexion / extension angle <30° or a pressing force <1N enter the passive guidance mode; patients with a maximum finger flexion / extension angle of 30° ≤ 60° or a pressing force of 1N ≤ 3N enter the active assistance mode; and patients with a maximum finger flexion / extension angle ≥60° and a pressing force ≥3N enter the self-training mode.

Images