Virtual reality augmented feedback processing method and system for vestibular rehabilitation training

Abstract

The application discloses a virtual reality augmented feedback processing method and system for vestibular rehabilitation training. It relates to the field of vestibular rehabilitation training, and utilizes a real-time acquisition module to acquire real-time acquisition data of a VR rehabilitation training instrument device and generate virtual reality environment characteristics of the real-time acquisition data; generates vestibular rehabilitation training environment characteristics according to the virtual reality environment characteristics of the real-time acquisition data in combination with a vestibular rehabilitation training type; performs multistage analysis and dynamic adjustment based on the vestibular rehabilitation training environment characteristics, generates a virtual reality data augmented feedback processing result, and obtains the virtual reality data augmented feedback processing result through an iterative loop updating positive feedback processing mode, thereby providing real-time and intuitive training feedback for a patient, helping the patient to timely adjust a training state, and also being capable of dynamically optimizing a training scheme and a training difficulty, and significantly improving real-time performance, accuracy and effectiveness of the vestibular rehabilitation training, and effectively shortening a patient rehabilitation period.

Description

Technical Field

[0001] This invention relates to the field of vestibular rehabilitation training technology, and in particular to a virtual reality augmented feedback processing method and system for vestibular rehabilitation training. Background Technology

[0002] Vestibular dysfunction can cause symptoms such as balance disorders and dizziness, severely impacting patients' quality of life. Current clinical vestibular rehabilitation training largely relies on traditional physical methods, which suffer from limitations such as limited training scenarios, low patient participation, and difficulty in adjusting training plans in real-time based on individual training data. While some VR rehabilitation training devices have emerged with the application of virtual reality technology in rehabilitation, existing VR vestibular rehabilitation methods generally suffer from insufficient collection and utilization of real-time patient training data. They only provide virtual scenarios that remain almost fixed during training, failing to accurately extract training environment characteristics suitable for the specific type of vestibular rehabilitation training. They cannot dynamically adjust based on the patient's real-time performance. Furthermore, patient training results can only be subjectively assessed based on patient complaints or observed movements, such as eye-tracking data or gaze stability tests. This subjective assessment fails to convert the patient's subjective feelings into analyzable data, resulting in inaccurate evaluations. The lack of an effective feedback mechanism based on real-time data makes it difficult to dynamically optimize the training process based on the patient's training status, leading to poor training effects and prolonged rehabilitation periods. In addition, although existing technologies, such as patent application number 202310597414.6, also involve conventional vestibular training methods such as gait training and gaze stabilization training, they all involve single-item training, and the evaluation results are based on the user's completion rate during this training, with fixed scoring methods. They cannot distinguish the user's individual differences, and the training process cannot form deep fusion and closed-loop feedback of multimodal data. Summary of the Invention

[0003] Therefore, the purpose of this invention is to provide a virtual reality enhanced feedback processing method and system for vestibular rehabilitation training. By integrating virtual reality technology with multimodal biosensing technology, an intelligent vestibular rehabilitation platform integrating "objective assessment, personalized adaptive training and quantitative efficacy tracking" is constructed to solve the problems of poor compliance, subjective assessment, rigid plans and limited scenarios in traditional rehabilitation training, and ultimately realize the intelligent, precise and efficient vestibular rehabilitation.

[0004] To achieve the above objectives, the present invention provides a virtual reality augmented feedback processing method for vestibular rehabilitation training, comprising the following steps:

[0005] S1. Generate the current training scenario based on the preset training type selected by the user;

[0006] S2. Real-time collection of raw eye-tracking data, head posture data, posture data, and plantar pressure distribution data of the user during training in the current training scenario;

[0007] S201. Evaluate the training results in the current training scenario using the raw data collected in real time;

[0008] S202. Accumulate raw data collected in real time over a period of time to form sequence data, use the sequence data for prediction, and adaptively adjust the environmental parameters of the current training scenario based on the prediction results; including:

[0009] S2021. Merge all the original data in the sequence data to generate fused features;

[0010] S2022. Input the fusion features into the eye coordination prediction model and the posture coordination prediction model respectively, and generate an assessment index that characterizes the patient's vestibular function status in real time based on the output results of the eye coordination prediction model and the posture coordination prediction model.

[0011] S2023. Input the evaluation indicators into the adaptive rehabilitation decision-making model, which generates environmental adjustment instructions based on the preset training difficulty level and real-time evaluation indicators.

[0012] S2024. According to the environment adjustment instruction, dynamically adjust the environmental parameters of the current training scenario to adaptively match the patient's real-time rehabilitation status.

[0013] S3. Based on the training type switched by the user, obtain the user's feedback data and training time for completing all training sessions, and obtain the final training completion evaluation result.

[0014] More preferably, in S202, the process of fusing all the original data in the sequence data to generate fused features includes the following steps:

[0015] Eye movement velocity is extracted from raw eye movement data, head angular velocity is extracted from raw head posture data, and vestibular-ocular reflex (VOR) gain is calculated based on the head angular velocity and eye movement velocity.

[0016] Based on the original data of plantar pressure distribution, calculate the trajectory of the pressure center and the area of ​​the center of gravity swing;

[0017] The body's center of gravity swings along the coronal and sagittal planes based on the original posture data, and the amplitude and velocity of the swing trajectory are extracted.

[0018] The calculated vestibulo-ocular reflex (VOR) gain, pressure center trajectory, center of gravity swing area, and amplitude and velocity of the swing trajectory are dimensionally stitched together to obtain the fused features.

[0019] More preferably, the eye coordination prediction model and the posture coordination prediction model are classification or regression models trained based on historical data;

[0020] The eye coordination prediction model is used to output a gaze stability score based on the vestibular-ocular reflex (VOR) gain and eye movement velocity in the fusion features. When the VOR gain is higher than a first preset threshold and the eye movement velocity is lower than a second preset threshold, the gaze stability is considered good.

[0021] The posture coordination prediction model is used to output a posture stability score based on the pressure center trajectory, center of gravity swing area, and amplitude and speed of the body center of gravity swing trajectory in the fused features. When the center of gravity swing area is lower than the third preset threshold, the swing amplitude is lower than the fourth preset threshold, and the swing speed is lower than the fifth preset threshold, the posture stability is judged to be good.

[0022] More preferably, in S2023, the adaptive rehabilitation decision-making model generates environmental adjustment instructions based on a preset training difficulty level and real-time evaluation indicators, including:

[0023] When both gaze stability and postural stability are good, an environmental adjustment instruction that increases the difficulty level is generated according to the preset difficulty level of the rehabilitation logic.

[0024] If either gaze stability or attitude stability is not deemed optimal, a stability compensation adjustment strategy is implemented, including:

[0025] Obtain the stability limits of the center of gravity in multiple directions based on real-time plantar pressure distribution data;

[0026] Calculate the degree to which the current center of gravity position achieves the stability limit in each direction;

[0027] Based on the achievement level, an environment adjustment instruction to increase the difficulty level is generated.

[0028] More preferably, the preset training types mentioned in S1 include: static balance training, gaze stabilization training, posture control training, balance training during movement, and scenario simulation training;

[0029] When users select static balance training: they control virtual objects by moving their head or shifting their center of gravity to complete the task of avoiding obstacles;

[0030] When the user selects gaze stabilization training: the user completes the tracking of a virtual moving target or the gazing at a stationary target through head movements and / or eye movements;

[0031] When the user selects posture control training: the patient follows the virtual target and performs the specified static posture or dynamic swaying movement.

[0032] When the user selects balance training during exercise: the virtual target is driven forward by the user stepping in place, and the moving target is captured by the hand controller;

[0033] When a user selects a scenario simulation training: the user can simulate walking, turning, and grabbing objects in a complex everyday environment by turning, stepping in place, and using a hand controller to grab virtual targets.

[0034] More preferably, the step of obtaining the user's feedback data and training time after completing all training, and obtaining the final training completion evaluation result, includes:

[0035] Obtain feedback data from users after completing all training sessions, and calculate the estimated values ​​of visual focus stability, center of gravity stability, limb coordination, and head sway amplitude in a single training task.

[0036] Count the number of times the user completes the task during the training period from start to finish;

[0037] The number of task completions and training feedback data are input into the training completion evaluation model for calculation.

[0038] The training completion evaluation model is expressed by the following formula:

[0039] E(L) = C * D(L,X) * 100;

[0040] Where C represents the training completion metric function, and D(L,X) represents the difficulty adaptation factor;

[0041] C= ;

[0042] Where R is the task completion rate, R = (number of completions / preset number of completions); and These are adjustable weight parameters; For the accuracy of the action;

[0043]

[0044] in, The actual parameters of the patient's i-th movement (e.g., head rotation angle, body center of gravity position). Let σ be the target parameter, and σ be the standard deviation of the allowable error.

[0045] S represents the time series stability score. dt;

[0046] in, These are real-time performance indicators for VOR gain, gaze focus stability estimate, center of gravity stability estimate, limb coordination estimate, and head sway amplitude in a single training task. This represents the mean of each of the above performance metrics over the training duration.

[0047] The difficulty adaptation factor D(L,X) is expressed by the following formula:

[0048] D(L,X)=γ(L)×Φ(X,L)

[0049] Where γ(L) is the difficulty baseline function, γ(L) = 1 + kγ(L-1); kγ is the difficulty gain coefficient, and L is the difficulty level. When kγ = 0.15, it means that for each level of difficulty, the baseline requirement increases by 15%.

[0050] Φ(X,L) is the difficulty adaptation evaluation function, Φ(X,L)=

[0051] Where X is the patient's actual feature vector. Φ is the expected performance vector at difficulty level L, where δ is the tolerance parameter. The closer the training performance is to the expectation, the closer Φ is to 1.

[0052] The final evaluation result is obtained as E(L) = C*D(L,X)*100; E(L) is a score on a 100-point scale, and E(L) is an absolute score bound to the current training difficulty L; the difficulty L is the core influencing parameter, which directly affects the evaluation benchmark and efficiency expectation, ensuring that the evaluation standard is adaptively adjusted with the difficulty.

[0053] More preferably, the estimated value of the gaze focus stability is calculated based on the variance of the gaze point coordinates;

[0054] The estimated value of the center of gravity stability is calculated based on the original data of the plantar pressure distribution, the position of the center of gravity, and the area enclosed by the pressure center trajectory and the line connecting the start and end points of the pressure center trajectory.

[0055] The estimated limb coordination value is calculated based on the similarity between the swing trajectory of the body's center of gravity in the coronal and sagittal planes and the ideal trajectory.

[0056] The head sway amplitude is calculated by calculating the standard deviation of the head yaw angle during training.

[0057] More preferably, the calculation of the center of gravity stability estimate includes the following steps:

[0058] Receive a byte array from the pressure sensor plate and convert it to an integer variable;

[0059] The total pressure value, weight, is calculated as the sum of the values ​​from the four pressure sensors.

[0060] Calculate the X-axis force component weightX = upper right pressure sensor data + lower right pressure sensor data;

[0061] Calculate the Y-axis component force weightY = upper left pressure sensor data + upper right pressure sensor data;

[0062] Based on the preset total length, the pressure center coordinates (x, y) are calculated using the following formula:

[0063] x= (weightX / weight * totalLength) - (totalLength / 2)

[0064] y= (weightY / weight * totalLength) - (totalLength / 2)

[0065] The trajectory of the pressure center is traced based on the coordinates of the pressure center; the offset value of the pressure center trajectory from the original pressure center is calculated as the estimate of the center of gravity stability.

[0066] The present invention also provides a virtual reality augmented feedback processing system for vestibular rehabilitation training, comprising the following steps for implementing the above-described virtual reality augmented feedback processing method for vestibular rehabilitation training:

[0067] The multimodal data acquisition module collects raw eye-tracking data, head posture data, posture data, and plantar pressure distribution data from the user in the current training scenario in real time.

[0068] The VR interaction module generates the current training scene based on the preset training type selected by the user, and controls the interactive display of the target in the virtual scene based on the user's feedback data during training.

[0069] The data processing and analysis module evaluates the training results in the current training scenario using real-time collected raw data; it accumulates raw data over a period of time into sequence data, and then fuses all the raw data in the sequence data to generate fused features.

[0070] The fusion features are input into the eye coordination prediction model and the posture coordination prediction model respectively. Based on the output results of the eye coordination prediction model and the posture coordination prediction model, an assessment index characterizing the patient's vestibular function status is generated in real time.

[0071] The evaluation indicators are input into the adaptive rehabilitation decision-making model, which generates environmental adjustment instructions based on the preset training difficulty level and real-time evaluation indicators.

[0072] The dynamic scene control module dynamically adjusts the environmental parameters of the current training scene according to the environmental adjustment instructions in order to adaptively match the patient's real-time rehabilitation status.

[0073] Based on the training type switched by the user, obtain the user's feedback data and training time for completing all training sessions, and obtain the final training completion evaluation result.

[0074] Preferably, the multimodal data acquisition module includes at least a VR headset, a pressure sensor plate, motion sensors worn on the ankle, knee, and hip joints, and a handheld controller.

[0075] The virtual reality augmented feedback processing method and system for vestibular rehabilitation training disclosed in this application have at least the following advantages compared to the prior art:

[0076] This application constructs a closed loop of real-time data acquisition → feature calculation → model decision-making → scene adjustment, enabling real-time monitoring of the patient's physiological state (such as VOR gain and center of gravity sway) and dynamically adjusting visual interference and task difficulty in the VR environment accordingly. When the patient performs well, the system automatically increases the challenge; when signs of imbalance or dizziness appear, the system immediately reduces the difficulty or provides assistance to ensure safety. This data-driven adaptive capability avoids the drawbacks of a fixed, one-size-fits-all approach, achieving truly personalized training, thereby improving rehabilitation efficiency while minimizing training risks.

[0077] Another innovation of this application lies in the deep integration of four-dimensional data on eye movement, head movement, posture, and stress, from which multiple objective quantitative indicators (such as VOR gain and eight-direction stability limit) directly corresponding to clinical vestibular function are extracted, including gaze stability, posture control ability, and dynamic balance ability. While existing technologies can accurately estimate the trajectory of the center of pressure, their assessment dimensions are singular. This application, through comprehensive assessment, not only provides doctors with accurate and scientific assessment basis, but also enables prediction based on multi-dimensional data and adaptive adjustment of training levels based on prediction results, making the assessment results closer to the user's actual response and achieving the integration of assessment and treatment.

[0078] The key advantage of this application lies in its deep integration of the medical principles of vestibular rehabilitation with the gamified interaction of VR. For example, VOR training is transformed into a challenge of "head-eye reverse tracking of a swimming fish," and balance training is transformed into a game of "controlling a race car to avoid obstacles by shifting the center of gravity." This design is not merely a change in form, but rather internalizes rehabilitation movements into the core operational mechanism for completing game tasks, allowing patients to unconsciously complete high-intensity, repetitive rehabilitation training in a highly immersive state. This "edutainment" approach effectively distracts patients from discomfort, significantly reduces their psychological burden, and fundamentally solves the biggest pain point of poor adherence in traditional rehabilitation, ensuring the smooth completion of the rehabilitation course. Attached Figure Description

[0079] Figure 1 This is a flowchart of a virtual reality augmented feedback processing method for vestibular rehabilitation training provided by the present invention. Detailed Implementation

[0080] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0081] like Figure 1 As shown, one embodiment of the present invention provides a virtual reality augmented feedback processing method for vestibular rehabilitation training, comprising the following steps:

[0082] S1. Generate the current training scenario based on the preset training type selected by the user;

[0083] The preset training types include: static balance training, gaze stabilization training, posture control training, balance training during movement, and scenario simulation training;

[0084] When the user selects static balance training: control the virtual object by moving the head or shifting the body's center of gravity to complete the task of avoiding obstacles; after entering the training, the perspective changes to the perspective of a forest road, a car is moving forward at a constant speed, and the patient needs to turn the head left and right to make the car on the road move left and right to avoid obstacles in front.

[0085] When the user selects gaze stabilization training: They track a virtual moving target or gaze at a stationary target through head movements and / or eye movements. After entering training, the user selects to observe the movement direction of a small fish. Gazing stabilization training includes three training modes:

[0086] Mode 1 (Eye-tracking, head-staying): The patient is required to keep their head still and use only their eyes to track a virtual target (such as a small fish) moving left and right or up and down on the screen.

[0087] Mode 2 (Head Movement, Eyes Still): The patient is required to keep their eyes fixed on a stationary virtual target and then actively move their head left and right or up and down according to the system prompts.

[0088] Mode 3 (Head-Eye Reverse Movement): This requires the patient's eyes to track a moving virtual target while their head moves in the opposite direction to the target's movement. This is the most challenging training for VOR (Voice-Oriented Reaction) function.

[0089] When a user selects posture control training: guided by a virtual target, the patient performs a specified static posture or dynamic swaying movement; by training the stability of the ankle, knee, and hip joints, the user maintains postural stability and body balance.

[0090] Movement imitation mode: Patients follow virtual therapists or video demonstrations, performing corresponding body postures. The training movement library includes:

[0091] Static postures: standing on one leg, standing with legs apart, standing with legs together.

[0092] Dynamic posture: swaying back and forth, swaying left and right.

[0093] Environment selection: Before training, you can choose to stand on a hard floor or a soft mat to increase the challenge of proprioception.

[0094] When the user selects balance training during exercise: the virtual target is propelled forward by the user stepping in place, and the moving target is captured using a hand controller; the system will display a penguin walking capture task: the patient moves the virtual view forward by stepping in place. At the same time, the hand controller (which transforms into a net) is used to catch butterflies flying towards them.

[0095] When a user selects a scenario simulation training: the user can simulate walking, turning, and grabbing objects in a complex everyday environment by turning, stepping in place, and using a hand controller to grab virtual targets.

[0096] For example, a street walking task: patients walk in place on a virtual street according to voice prompts, heading towards a target location (such as a supermarket).

[0097] Supermarket Shopping Task: After entering the virtual supermarket, find and grab the designated items according to the prompts, and place them on the checkout counter. Environmental Interaction Task: Complete other functional tasks within the scene, such as picking up litter and throwing it into the trash can.

[0098] S2. Real-time collection of raw eye-tracking data, head posture data, body posture data, and plantar pressure distribution data of the user during training in the current training scenario; including:

[0099] S201. Evaluate the training results in the current training scenario using real-time collected raw data. It should be noted that this evaluation based on real-time data involves interacting with the virtual reality scene using the collected data. For example, during static balance training: using raw head posture data, determine the user's head movement direction and speed, and then convert this into the turning direction and speed of the race car in the virtual scene to determine whether obstacle avoidance is possible. This evaluation is used to provide feedback on the user's real-time actions in the virtual reality environment during interaction. The user can directly see whether obstacle avoidance has occurred or whether the target has been controlled as required. This is a basic technology of virtual reality interaction and can be achieved using existing technologies, so it will not be elaborated further here.

[0100] S202. Accumulate raw data collected in real time over a period of time to form sequence data, use the sequence data for prediction, and adaptively adjust the environmental parameters of the current training scenario based on the prediction results; including:

[0101] S2021. Merge all original data in the sequence data to generate fused features; the fused features here are only fused along the data dimension, including:

[0102] Eye movement velocity is extracted from raw eye movement data, and head angular velocity is extracted from raw head posture data. The vestibular-ocular reflex (VOR) gain is calculated based on the head angular velocity and eye movement velocity; that is, VOR gain = eye movement velocity / head movement velocity.

[0103] Based on the original data of plantar pressure distribution, calculate the trajectory of the pressure center and the area of ​​the center of gravity swing;

[0104] Receive a byte array from the pressure sensor plate and convert it to an integer variable;

[0105] The total pressure value, weight, is calculated as the sum of the values ​​from the four pressure sensors.

[0106] Calculate the X-axis force component weightX = upper right pressure sensor data + lower right pressure sensor data;

[0107] Calculate the Y-axis component force weightY = upper left pressure sensor data + upper right pressure sensor data;

[0108] Based on the preset total length, the pressure center coordinates (x, y) are calculated using the following formula:

[0109] x= (weightX / weight * totalLength) - (totalLength / 2)

[0110] y= (weightY / weight * totalLength) - (totalLength / 2)

[0111] The trajectory of the pressure center is recorded based on the coordinates of the pressure center.

[0112] The area enclosed by the line connecting the starting point to the ending point of the pressure center trajectory is denoted as the center of gravity swing area.

[0113] Based on the raw posture data, the swing trajectory of the body's center of gravity in the coronal and sagittal planes is calculated, and the amplitude and velocity of the swing trajectory are extracted. Since the raw posture data is collected from motion sensors worn by the user in the ankle, knee, and hip joints, data from these three body parts needs to be collected. According to common human biomechanical models, the coordinates of the center of gravity are determined (overall center of gravity = (mass of each segment × coordinates of the center of gravity of each segment) / total mass). The coordinates of the center of gravity are projected into the coronal and sagittal planes to obtain the swing trajectory of the body's center of gravity in the coronal and sagittal planes. The amplitude (difference between the maximum and minimum values) and velocity of the swing trajectory can then be obtained from the swing trajectory.

[0114] The calculated vestibulo-ocular reflex (VOR) gain, pressure center trajectory, center of gravity swing area, and amplitude and velocity of the swing trajectory are dimensionally stitched together to obtain the fused features.

[0115] S2022. The fused features are input into the eye coordination prediction model and the posture coordination prediction model, respectively. Based on the output results of the eye coordination prediction model and the posture coordination prediction model, assessment indicators characterizing the patient's vestibular function status are generated in real time, including:

[0116] The eye coordination prediction model and the posture coordination prediction model are classification or regression models trained based on historical data;

[0117] The eye coordination prediction model is used to output a gaze stability score based on the vestibular-ocular reflex (VOR) gain and eye movement velocity in the fusion features. When the VOR gain is higher than a first preset threshold and the eye movement velocity is lower than a second preset threshold, the gaze stability is considered good.

[0118] The posture coordination prediction model is used to output a posture stability score based on the pressure center trajectory, center of gravity swing area, and amplitude and speed of the body center of gravity swing trajectory in the fused features. When the center of gravity swing area is lower than the third preset threshold, the swing amplitude is lower than the fourth preset threshold, and the swing speed is lower than the fifth preset threshold, the posture stability is judged to be good.

[0119] In this application, an LSTM model or a vector machine can be used for classification.

[0120] That is, when VOR gain > first preset threshold and eye movement speed < second preset threshold, Stability≈1; otherwise, the value output by the regression model is any number between 0 and 1. When VOR gain ≤ first threshold and eye movement speed ≥ second threshold, Stability≈0.

[0121] The input features in the posture coordination prediction model include: the trajectory of the center of pressure, the area of ​​the body's center of gravity swing, and the amplitude and speed of the swing trajectory. Similarly, when the swing area is small, the amplitude of the body's center of gravity swing is low, and the speed of the body's center of gravity swing is slow, the posture stability Posture approaches 1; conversely, when the swing area is large, the amplitude is high, and the speed is fast, Posture approaches 0. Other forms can also be used, and existing technologies can be used for such methods, which will not be elaborated on here.

[0122] S2023. Input the evaluation indicators into the adaptive rehabilitation decision-making model, which generates environmental adjustment instructions based on the preset training difficulty level and real-time evaluation indicators; including:

[0123] When both gaze stability and postural stability are good, an environmental adjustment instruction that increases the difficulty level is generated according to the preset difficulty level of the rehabilitation logic.

[0124] If either gaze stability or attitude stability is not deemed optimal, a stability compensation adjustment strategy is implemented, including:

[0125] Obtain the stability limits of the center of gravity in multiple directions based on real-time plantar pressure distribution data;

[0126] Calculate the degree to which the current center of gravity position achieves the stability limit in each direction;

[0127] Based on the achievement level, an environment adjustment instruction to increase the difficulty level is generated.

[0128] It should be noted that, in order to accurately assess the user's stability, the user's plantar pressure distribution data is converted into a polar coordinate graph to generate a center of gravity stability limit profile in eight directions: 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°.

[0129] If the patient can easily keep their center of gravity within the limit range (e.g., within 50% of the target range), the system will increase the difficulty (e.g., narrow the VR balance beam or introduce swaying).

[0130] If the patient's center of gravity continues to approach or reach (e.g., 80%-100%) their personal limit, the system maintains or reduces the difficulty to ensure safety and build confidence.

[0131] If the center of gravity frequently exceeds the historical limit (>100%), it indicates progress. The system will update and expand the limit value in that direction to set new and higher goals for the next round of training.

[0132] Obtain the stability limits of the center of gravity in multiple directions based on real-time plantar pressure distribution data; calculate the degree to which the current center of gravity position achieves the stability limits in each direction; including:

[0133] Define the personal stability limit profile; that is, define a boundary point in each of the above 8 directions; connect these boundary points in sequence to form a convex polygon, which is the personal stability limit profile.

[0134] Calculate the projected distance from the current centroid to each contour edge: convert the current centroid to polar coordinates. Rp Determine the directional range where the current center of gravity is located. θp∈(θ1,θ2)θ1,θ2 Let each represent a two adjacent boundary point, and use linear interpolation to obtain the theoretical limit distance in the current direction. Rmax(θp) :

[0135] Rmax (θp) = ri + (θp-θ1) / (θ2-θ1) * (ri + 1-ri);

[0136] in, ri ri+1 is the maximum stable distance in this direction, and ri+1 is the stable distance in the direction of the next boundary point;

[0137] The formula for calculating the degree of achievement can be expressed as: D=Rp / Rmax(θp)*100%.

[0138] S2024. According to the environment adjustment instruction, dynamically adjust the environmental parameters of the current training scenario to adaptively match the patient's real-time rehabilitation status.

[0139] For example, in static balance training, environmental parameters such as increasing the swaying of the support surface or the speed at which obstacles appear can be adjusted to create interference.

[0140] When the patient performs well in gaze stabilization training, increase the speed and complexity of the background visual flow to distract them and challenge their gaze system. Simultaneously, increase the speed of the target object's movement. When the patient struggles, use a static, solid-color background and slow down or reduce the size of the target's movement.

[0141] When patients perform well in posture control training, ask them to maintain postures such as standing on one leg for longer periods, or gently sway a virtual perspective to simulate external interference. When they struggle, shorten the duration of maintenance and provide simplified backgrounds as visual aids.

[0142] When performing well in balance training during movement, increase the number of targets and make their flight paths more unpredictable (such as flipping up and down). When performing poorly, reduce the number of targets, make them fly in a straight line, and reduce the precision required for capture.

[0143] Scenario simulation training; when performance is excellent, the complexity of the task is increased, such as increasing the number of items to pick up or decreasing the size of the items to pick up.

[0144] S3. Based on the training type switched by the user, obtain the user's feedback data and training time for completing all training sessions, and obtain the final training completion evaluation result.

[0145] More preferably, the step of obtaining the user's feedback data and training time after completing all training, and obtaining the final training completion evaluation result, includes:

[0146] Obtain feedback data from users after completing all training sessions, and calculate the estimated values ​​of visual focus stability, center of gravity stability, limb coordination, and head sway amplitude in a single training task.

[0147] The estimated value of the gaze focus stability is calculated based on the variance of the gaze point coordinates;

[0148] The estimated value of the center of gravity stability is calculated based on the original data of the plantar pressure distribution, the position of the center of gravity, and the area enclosed by the pressure center trajectory and the line connecting the start and end points of the pressure center trajectory.

[0149] The estimated limb coordination value is calculated based on the similarity between the swing trajectory of the body's center of gravity in the coronal and sagittal planes and the ideal trajectory.

[0150] The head sway amplitude is calculated by calculating the standard deviation of the head yaw angle during training.

[0151] More preferably, the calculation of the center of gravity stability includes calculating the offset value of the pressure center trajectory from the original pressure center based on the pressure center trajectory obtained in S2021, as an estimate of the center of gravity stability.

[0152] Obtain feedback data from users after completing all training sessions, and calculate the estimated values ​​of visual focus stability, center of gravity stability, limb coordination, and head sway amplitude in a single training task.

[0153] Count the number of times the user completes the task during the training period from start to finish;

[0154] The number of task completions and training feedback data are input into the training completion evaluation model for calculation.

[0155] The training completion evaluation model includes:

[0156] A percentage score is constructed using the formula E(L)=C*D(L,X)*100. E(L) is an absolute score bound to the current training difficulty L. Difficulty L is the core influencing parameter, which directly affects the evaluation benchmark and efficiency expectation, ensuring that the evaluation criteria are adaptively adjusted with the difficulty.

[0157] Where C represents the training completion metric function, and D(L,X) represents the difficulty adaptation factor;

[0158] C= ;

[0159] Where R is the task completion rate, R = (number of completions / preset number of completions); and These are adjustable weight parameters; For the accuracy of the action;

[0160]

[0161] in, The actual parameters of the patient's i-th movement (e.g., head rotation angle, body center of gravity position). Let σ be the target parameter, and σ be the standard deviation of the allowable error.

[0162] S represents the time series stability score. dt

[0163] in, These are real-time performance indicators for VOR gain, gaze focus stability estimate, center of gravity stability estimate, limb coordination estimate, and head sway amplitude in a single training task. S represents the mean of each of the above performance metrics over the training period; S is the score calculated when the user selects each training item. If the user selects more than two items, the average value is calculated based on the number of items selected.

[0164] The difficulty adaptation factor is expressed by the following formula:

[0165] D(L,X)=γ(L)×Φ(X,L)

[0166] γ(L) is the difficulty baseline function, γ(L) = 1 + kγ(L-1); kγ is the difficulty gain coefficient, and L is the difficulty level. When kγ = 0.15, it means that for each level of difficulty, the baseline requirement increases by 15%.

[0167] Φ(X,L) is the difficulty adaptation evaluation function; Φ(X,L)=

[0168] X is the patient's actual feature vector (e.g., average VOR gain, area of ​​center of gravity swing, etc.). Φ is the expected performance vector at difficulty level L, where δ is the tolerance parameter. The closer the training performance is to the expectation, the closer Φ is to 1.

[0169] The final evaluation result is E(L) = C*D(L,X)*100.

[0170] The present invention also provides a virtual reality augmented feedback processing system for vestibular rehabilitation training, comprising the following steps for implementing the above-described virtual reality augmented feedback processing method for vestibular rehabilitation training:

[0171] The multimodal data acquisition module collects raw eye-tracking data, head posture data, posture data, and plantar pressure distribution data from the user in the current training scenario in real time.

[0172] The VR interaction module generates the current training scene based on the preset training type selected by the user, and controls the interactive display of the target in the virtual scene based on the user's feedback data during training.

[0173] The data processing and analysis module evaluates the training results in the current training scenario using real-time collected raw data; it accumulates real-time collected raw data into a sequence over a period of time, and fuses all raw data in the sequence to generate fused features.

[0174] The fusion features are input into the eye coordination prediction model and the posture coordination prediction model respectively. Based on the output results of the eye coordination prediction model and the posture coordination prediction model, an assessment index characterizing the patient's vestibular function status is generated in real time.

[0175] The evaluation indicators are input into the adaptive rehabilitation decision-making model, which generates environmental adjustment instructions based on the preset training difficulty level and real-time evaluation indicators.

[0176] The dynamic scene control module dynamically adjusts the environmental parameters of the current training scene according to the environmental adjustment instructions in order to adaptively match the patient's real-time rehabilitation status.

[0177] Based on the training type switched by the user, obtain the user's feedback data and training time for completing all training sessions, and obtain the final training completion evaluation result.

[0178] The multimodal data acquisition module includes at least a VR headset, a pressure sensor plate, motion sensors worn on the ankle, knee, and hip joints, and a handheld controller.

[0179] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A virtual reality augmented feedback processing method for vestibular rehabilitation training, characterized in that, Includes the following steps: S1. Generate the current training scenario based on the preset training type selected by the user; S2. Real-time collection of raw eye-tracking data, head posture data, posture data, and plantar pressure distribution data of the user during training in the current training scenario; S201. Evaluate the training results in the current training scenario using the raw data collected in real time; S202. Accumulate raw data collected in real time over a period of time to form sequence data, use the sequence data for prediction, and adaptively adjust the environmental parameters of the current training scenario based on the prediction results; including: S2021. Merge all the original data in the sequence data to generate fused features; S2022. Input the fusion features into the eye coordination prediction model and the posture coordination prediction model respectively, and generate an assessment index that characterizes the patient's vestibular function status in real time based on the output results of the eye coordination prediction model and the posture coordination prediction model. S2023. Input the evaluation indicators into the adaptive rehabilitation decision-making model, which generates environmental adjustment instructions based on the preset training difficulty level and real-time evaluation indicators. S2024. According to the environment adjustment instruction, dynamically adjust the environmental parameters of the current training scenario to adaptively match the patient's real-time rehabilitation status. S3. Based on the training type switched by the user, obtain the user's feedback data and training time for completing all training sessions, and obtain the final training completion evaluation result; including: Obtain feedback data from users after completing all training sessions, and calculate the estimated values ​​of visual focus stability, center of gravity stability, limb coordination, and head sway amplitude in a single training task. Count the number of times the user completes the task during the training period from start to finish; The number of task completions and training feedback data are input into the training completion evaluation model for calculation. The training completion evaluation model is expressed by the following formula: E(L) = C * D(L,X) * 100; Where C represents the training completion metric function, and D(L,X) represents the difficulty adaptation factor; C= ； Where R is the task completion rate, R = (number of completions / preset number of completions); and These are adjustable weight parameters; For the accuracy of the action; in, The actual parameters of the patient's i-th action. Let σ be the target parameter, and σ be the standard deviation of the allowable error. S represents the time series stability score. dt in, These are real-time performance indicators for VOR gain, gaze focus stability estimate, center of gravity stability estimate, limb coordination estimate, and head sway amplitude in a single training task. This represents the mean of each of the above performance metrics over the training duration. The difficulty adaptation factor D(L,X) is expressed by the following formula: D(L,X)=γ(L)×Φ(X,L) Where γ(L) is the difficulty baseline function, γ(L) = 1 + kγ(L-1); kγ is the difficulty gain coefficient, and L is the difficulty level. When kγ = 0.15, it means that for each level of difficulty, the baseline requirement increases by 15%. Φ(X,L) is the difficulty adaptation evaluation function, Φ(X,L)= Where X is the patient's actual feature vector. Φ is the expected performance vector at difficulty level L, where δ is the tolerance parameter. The closer the training performance is to the expectation, the closer Φ is to 1. The final evaluation result is obtained as E(L) = C*D(L,X)*100, where E(L) is a score out of 100. E(L) is an absolute score that is tied to the current training difficulty L. Difficulty L is a core influencing parameter that directly affects the evaluation benchmark and efficiency expectation, ensuring that the evaluation criteria are adaptively adjusted with the difficulty.

2. The virtual reality augmented feedback processing method for vestibular rehabilitation training according to claim 1, characterized in that, In S202, the process of fusing all the original data in the sequence data to generate fused features includes the following steps: Eye movement velocity is extracted from raw eye movement data, head angular velocity is extracted from raw head posture data, and vestibular-ocular reflex (VOR) gain is calculated based on the head angular velocity and eye movement velocity. Based on the original data of plantar pressure distribution, calculate the trajectory of the pressure center and the area of ​​the center of gravity swing; The body's center of gravity swings along the coronal and sagittal planes based on the original posture data, and the amplitude and velocity of the swing trajectory are extracted. The calculated vestibulo-ocular reflex (VOR) gain, pressure center trajectory, center of gravity swing area, and amplitude and velocity of the swing trajectory are dimensionally stitched together to obtain the fused features.

3. The virtual reality augmented feedback processing method for vestibular rehabilitation training according to claim 1, characterized in that, The eye coordination prediction model and the posture coordination prediction model are classification or regression models trained based on historical data. The eye coordination prediction model is used to output a gaze stability score based on the vestibular-ocular reflex (VOR) gain and eye movement velocity in the fusion features. When the VOR gain is higher than a first preset threshold and the eye movement velocity is lower than a second preset threshold, the gaze stability is considered good. The posture coordination prediction model is used to output a posture stability score based on the pressure center trajectory, center of gravity swing area, and amplitude and velocity of the body center of gravity swing trajectory in the fusion features. When the center of gravity swing area is lower than a third preset threshold, the swing amplitude is lower than a fourth preset threshold, and the swing velocity is lower than a fifth preset threshold, the posture stability is considered good.

4. The virtual reality augmented feedback processing method for vestibular rehabilitation training according to claim 3, characterized in that, In S023, the adaptive rehabilitation decision-making model generates environmental adjustment instructions based on preset training difficulty levels and real-time evaluation indicators, including: When both gaze stability and postural stability are good, an environmental adjustment instruction that increases the difficulty level is generated according to the preset difficulty level of the rehabilitation logic. If either gaze stability or posture stability is not deemed satisfactory, a stability compensation adjustment strategy is executed, including: obtaining the stability limits of the center of gravity in multiple directions calculated based on real-time plantar pressure distribution data; calculating the degree of achievement of the current center of gravity position relative to the stability limits in each direction; and generating an environmental adjustment instruction that increases the difficulty level based on the degree of achievement.

5. The virtual reality augmented feedback processing method for vestibular rehabilitation training according to claim 1, characterized in that, The preset training types mentioned in S1 include: static balance training, gaze stabilization training, posture control training, balance training during movement, and scenario simulation training; When users select static balance training: they control virtual objects by moving their head or shifting their center of gravity to complete the task of avoiding obstacles; When the user selects gaze stabilization training: the user completes the tracking of a virtual moving target or the gazing at a stationary target through head movements and / or eye movements; When the user selects posture control training: the patient follows the virtual target and performs the specified static posture or dynamic swaying movement. When the user selects balance training during exercise: the virtual target is driven forward by the user stepping in place, and the moving target is captured by the hand controller; When a user selects a scenario simulation training: the user can simulate walking, turning, and grabbing objects in a complex everyday environment by turning, stepping in place, and using a hand controller to grab virtual targets.

6. The virtual reality augmented feedback processing method for vestibular rehabilitation training according to claim 5, characterized in that, The estimated stability of the gaze focus is calculated based on the variance of the gaze point coordinates; The estimated value of the center of gravity stability is calculated based on the original data of the plantar pressure distribution, the position of the center of gravity, and the area enclosed by the pressure center trajectory and the line connecting the start and end points of the pressure center trajectory. The estimated limb coordination value is calculated based on the similarity between the swing trajectory of the body's center of gravity in the coronal and sagittal planes and the ideal trajectory. The head sway amplitude is calculated by calculating the standard deviation of the head yaw angle during training.

7. The virtual reality augmented feedback processing method for vestibular rehabilitation training according to claim 6, characterized in that, The calculation of the center of gravity stability estimate includes the following steps: Receive a byte array from the pressure sensor plate and convert it to an integer variable; The total pressure value, weight, is calculated as the sum of the values ​​from the four pressure sensors. Calculate the X-axis force component weightX = upper right pressure sensor data + lower right pressure sensor data; Calculate the Y-axis component force weightY = upper left pressure sensor data + upper right pressure sensor data; Based on the preset total length, the pressure center coordinates (x, y) are calculated using the following formula: x= (weightX / weight * totalLength) - (totalLength / 2) y= (weightY / weight * totalLength) - (totalLength / 2) The trajectory of the pressure center is recorded based on the coordinates of the pressure center; the offset value of the pressure center trajectory from the original pressure center is calculated as the estimated value of the center of gravity stability.

8. A virtual reality augmented feedback processing system for vestibular rehabilitation training, characterized in that, Steps for implementing the virtual reality augmented feedback processing method for vestibular rehabilitation training according to any one of claims 1-7: include, The multimodal data acquisition module collects raw eye-tracking data, head posture data, posture data, and plantar pressure distribution data from the user in the current training scenario in real time. The VR interaction module generates the current training scene based on the preset training type selected by the user, and controls the interactive display of the target in the virtual scene based on the user's feedback data during training. The data processing and analysis module evaluates the training results in the current training scenario using real-time collected raw data. The accumulated real-time collected raw data is a sequence of data over a period of time. All raw data in the sequence data are fused to generate fused features: The fusion features are input into the eye coordination prediction model and the posture coordination prediction model respectively. Based on the output results of the eye coordination prediction model and the posture coordination prediction model, an assessment index characterizing the patient's vestibular function status is generated in real time. The evaluation indicators are input into the adaptive rehabilitation decision-making model, which generates environmental adjustment instructions based on the preset training difficulty level and real-time evaluation indicators. The dynamic scene control module dynamically adjusts the environmental parameters of the current training scene according to the environmental adjustment instructions in order to adaptively match the patient's real-time rehabilitation status. Based on the training type switched by the user, obtain the user's feedback data and training time for completing all training sessions, and obtain the final training completion evaluation result.

9. The virtual reality augmented feedback processing system for vestibular rehabilitation training according to claim 8, characterized in that, The multimodal data acquisition module includes at least a VR headset, a pressure sensor plate, motion sensors worn on the ankle, knee, and hip joints, and a handheld controller.

Images