A head-mounted vestibular rehabilitation training system based on VR technology

Abstract

The head-mounted vestibular rehabilitation training system based on VR technology of the present invention comprises: a head-mounted VR device is provided with left and right eye movement tracking cameras and a posture sensor, and a control host comprises: a patient information management module manages patient information; a training program management module manages the patient's vestibular rehabilitation training program; a visual target movement parameter setting module is used to set the moving speed, acceleration and moving path length of the visual target image of each training; the control module retrieves the vestibular rehabilitation training program of the target patient, controls the head-mounted VR device to display the visual target image and moves it according to the visual target movement parameters and training method set in the i-th training session, prompts the target patient to track according to the training mode and tracking mode of the i-th training session, i is selected from 1 to N, obtains the pupil movement video image and / or head movement of each training session, analyzes the eye movement tracking and / or head tracking of each training session, and evaluates the vestibular rehabilitation training effect of the target patient in this training.

Description

Technical Field

[0001] The present invention relates to the field of medical rehabilitation technology, and in particular to a head-mounted vestibular rehabilitation training system based on the combination of VR (virtual reality) technology and eye tracking technology. Background Art

[0002] Vestibular rehabilitation therapy is a training-based treatment for patients with vestibular dysfunction, which aims to improve their ability to coordinate and control balance through vestibular sense, vision and proprioception. Existing vestibular rehabilitation training methods mainly include gaze exercises, balance exercises, maintenance exercises, etc. However, these methods have problems such as low training interest and poor results. Summary of the invention

[0003] In view of the problems and shortcomings of the prior art, the present invention provides a head-mounted vestibular rehabilitation training system based on VR technology.

[0004] The present invention solves the above technical problems through the following technical solutions:

[0005] The present invention provides a head-mounted vestibular rehabilitation training system based on VR technology, which is characterized in that it comprises a head-mounted VR device and a control host, the head-mounted VR device is provided with a left eye tracking camera, a right eye tracking camera and a posture sensor, and the control host comprises a patient information management module, a training program management module, a visual target movement parameter setting module and a control module;

[0006] The patient information management module is used to manage patient information;

[0007] The training program management module is used to manage the vestibular rehabilitation training program of the patient, and the vestibular rehabilitation training program includes at least one training session with N divided training sessions having a sequential relationship and a training mode and a tracking mode adopted for each divided training session;

[0008] The visual target movement parameter setting module is used to set the moving speed, acceleration and moving path length of the visual target image in each training;

[0009] The control module is used to analyze the target patient based on the input patient information, retrieve the vestibular rehabilitation training program of the target patient, control the head-mounted VR device worn by the target patient to display the visual target image and move according to the visual target movement parameters and training method set in the i-th training session, prompt the target patient to track according to the training mode and tracking mode of the i-th training session to perform vestibular rehabilitation training, and retrieve the left eye movement tracking camera to collect the video image of the left eye pupil movement in the i-th training session, or the right eye movement tracking camera to collect the video image of the right eye pupil movement in the i-th training session, and / or the posture sensor to collect the movement of the head in the i-th training session based on the tracking mode of the i-th training session, where i is selected from 1 to N in sequence, and i and N are both positive integers;

[0010] The control module is also used to obtain pupil movement video images and / or head movement conditions of the target patient in each training session, analyze the eye tracking conditions and / or head tracking conditions of the target patient's eyes on the visual target in each training session, and evaluate the vestibular rehabilitation training effect of the target patient.

[0011] The positive and progressive effects of the present invention are:

[0012] The present invention provides a head-mounted vestibular rehabilitation training system, which uses a head-mounted VR device and eye tracking technology to provide users with specific visual stimulation to promote compensation or strengthening of the vestibular system. The system displays visual target images with different moving paths and speeds in virtual reality, and patients train by turning their heads and tracking these visual target images with their eyes, thereby achieving the effect of improving vestibular function.

[0013] The present invention provides rehabilitation training for rehabilitation trainees, so that they can perform vestibular rehabilitation training in a happy mood, reducing the trouble caused by the side effects of vestibular dysfunction drug dependence. The present invention collects and analyzes the trainee's eye movements and postures during the training process, more accurately grasps the trainee's training status and results, evaluates the vestibular rehabilitation training effect of the trainee, and provides an objective basis for the next stage of training. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a structural block diagram of a head-mounted vestibular rehabilitation training system based on VR technology according to a preferred embodiment of the present invention.

[0015] Figure 2 This is a moving trajectory diagram of the visual target image in each training mode during rehabilitation training according to a preferred embodiment of the present invention.

[0016] Figure 3 This is a diagram of the forward left and right rotation training mode of a preferred embodiment of the present invention.

[0017] Figure 4This is a diagram of a vertical up and down rotation training pattern according to a preferred embodiment of the present invention.

[0018] Figure 5 This is a diagram of the left front up and down rotation training mode of a preferred embodiment of the present invention.

[0019] Figure 6 This is a diagram of the right front up and down rotation training mode of a preferred embodiment of the present invention.

[0020] Figure 7 This is a diagram of the upper left-right rotation training mode of a preferred embodiment of the present invention.

[0021] Figure 8 The following is a diagram of a left-right rotation training mode according to a preferred embodiment of the present invention.

[0022] Fig. 9 This is a clockwise circular rotation training pattern diagram of a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0024] like Figure 1 As shown, this embodiment provides a head-mounted vestibular rehabilitation training system based on VR technology, which includes a head-mounted VR device 10 and a control host 20. The head-mounted VR device 10 is provided with a left eye tracking camera 11, a right eye tracking camera 12 and a posture sensor 13. The control host 20 includes a patient information management module 21, a training program management module 22, a target image selection module 23, a target display mode selection module 24, a target movement parameter setting module 25 and a control module 26.

[0025] The patient information management module 21 is used to manage patient information, including entering patient information, modifying patient information, querying patient information, and reviewing patient training information (such as training information of each patient training session).

[0026] The training program management module 22 is used to edit, store, modify and delete the vestibular rehabilitation training program of the patient. The vestibular rehabilitation training program includes at least one training session with N training sessions in a sequential order and a training mode and a tracking mode used in each training session.

[0027] like Figure 2-9As shown, the training modes include front left rotation training, front right rotation training, vertical upward rotation training, vertical downward rotation training, left front upward rotation training, left front downward rotation training, right front upward rotation training, right front downward rotation training, upper left rotation training, upper right rotation training, lower left rotation training, lower right rotation training, circular clockwise rotation training, circular counterclockwise rotation training, etc.

[0028] The tracking modes include single eye tracking mode and head-eye tracking mode. The single eye tracking mode refers to the left eye tracking mode and the right eye tracking mode. The head-eye tracking mode refers to the left eye + head tracking mode and the right eye + head tracking mode.

[0029] The target image selection module 23 is used for the user to select the target image for each training. The target image includes pattern, size, color and shape. The user can select the pattern, size, color and shape of the target image. If the user does not select, the default target image is selected by default. The optional setting of the target image can increase the interest of the patient's rehabilitation training process.

[0030] The target display mode selection module 24 is used for the user to select the display mode of the target image for each training, and the display modes include continuous display mode and hidden display mode. The user can select the display mode of the target image. If the user does not select, the default display mode is selected by default. The optional display mode of the target image can increase the interest of the patient's rehabilitation training process.

[0031] The visual target movement parameter setting module 25 is used for the user to set the moving speed, acceleration and moving path length of the visual target image for each training. The user can customize the moving speed, acceleration and moving path length of the visual target image to meet different vestibular rehabilitation training needs.

[0032] The control module 26 is used to analyze the target patient based on the input patient information, retrieve the vestibular rehabilitation training plan for the target patient, control the head-mounted VR device worn by the target patient to display the target image and move according to the target movement parameters and training method set in the i-th training session, prompt the target patient to track according to the training mode and tracking mode of the i-th training session to perform vestibular rehabilitation training, and based on the tracking mode of the i-th training session, retrieve the left eye tracking camera 11 to capture the video image of the left eye pupil movement in the i-th training session, or the right eye tracking camera 12 to capture the video image of the right eye pupil movement in the i-th training session, and / or the posture sensor 13 to capture the movement of the head in the i-th training session, where i is selected from 1 to N in sequence, and i and N are both positive integers.

[0033] For example: The training method of the first training session is the forward left turn training, and the tracking method is the single eye tracking mode. Figure 2, the visual target image moves from point 0 to point 1, the left eye of the target patient follows the movement of the visual target image, but the head does not move, and the left eye movement tracking camera 11 collects the video image of the left eye pupil movement during the movement of the left eye; the training method of the second training session is the forward left rotation training, and the tracking method is the head eye movement tracking mode, see Figure 2 , the visual target image moves from point 0 to point 1, the left eye and head of the target patient follow the visual target image, during the movement, the left eye tracking camera 11 collects the video image of the left eye pupil movement, and the posture sensor 13 collects the movement of the head; the training method of the third training session is the forward right rotation training, and the tracking method is the single eye tracking mode, see Figure 2 , the visual target image moves from point 0 to point 2, and the right eye of the target patient follows the movement of the visual target image, but the head does not move. During the movement of the right eye, the right eye tracking camera 12 collects video images of the right eye pupil movement.

[0034] The control module 26 is also used to obtain the pupil movement video images and / or head movement conditions of the target patient in each training session of the training, analyze the eye tracking conditions and / or head tracking conditions of the target patient's eyes on the visual target in each training session of the training, evaluate the vestibular rehabilitation training effect of the target patient and generate a rehabilitation training report.

[0035] Specifically, the control module 26 is further used to obtain the image center movement trajectory of the visual target image in the i-th training session, use the recognition algorithm to perform center recognition and positioning of the pupil in the collected pupil movement video image of the i-th training session, and analyze the pupil center movement trajectory, calculate the eye tracking overlap between the pupil center movement trajectory and the image center movement trajectory in the i-th training session, and record the eye tracking overlap in the i-th training session when the eye tracking overlap reaches a first preset value, otherwise, record the eye tracking non-overlap in the i-th training session, and when there is also head movement collected in the i-th training session, analyze the head movement trajectory based on the head movement situation, calculate the head tracking overlap between the head movement trajectory and the image center movement trajectory in the i-th training session, and record the head tracking overlap in the i-th training session when the head tracking overlap reaches a second preset value, otherwise, record the head tracking non-overlap in the i-th training session, i is selected from 1 to N in sequence, based on which the eye tracking overlap and the head tracking overlap in each training session in the training session can be recorded.

[0036] The control module 26 is also used to calculate the vestibular-ocular rehabilitation training rate of the target patient in this training = the number of times the recorded eye movement tracking overlaps / (the number of times the recorded eye movement tracking overlaps + the number of times the recorded eye movement tracking does not overlap), and determine whether the vestibular-ocular rehabilitation training rate is greater than a first threshold. If so, the information that the vestibular-ocular pathway rehabilitation training of the target patient is good is evaluated and output; if not, the information that the vestibular-ocular pathway of the target patient still needs to continue rehabilitation training is evaluated and output.

[0037] The control module 26 is also used to calculate the vestibular-spinal cord rehabilitation training rate of the target patient in this training = the number of recorded head tracking overlaps / (the number of recorded head tracking overlaps + the number of recorded head tracking non-overlaps), and to determine whether the vestibular-spinal cord rehabilitation training rate is greater than a second threshold value. If so, the information that the vestibular-spinal cord pathway rehabilitation training of the target patient is good is evaluated and output; if not, the information that the vestibular-spinal cord pathway of the target patient still needs to continue rehabilitation training is evaluated and output.

[0038] The control module 26 is also used to perform grayscale conversion, cropping, and pupil center positioning on each frame of the eyeball image in the collected pupil movement video image of the i-th training session, so as to obtain the change of the pupil center coordinates of the target patient in the i-th training session in a time sequence, analyze the number of times the change value between the latter pupil center coordinate and the former pupil center coordinate is greater than a set threshold in the i-th training session as the number of nystagmus and the duration of each time the change value of the former pupil center coordinate is greater than the set threshold as the duration of nystagmus, compare the number of nystagmus and the duration of each nystagmus of the i-th training session in this training session with the same training session as the i-th training session in the previous training session and give a comparison result, based on which the comparison results of each training session in this training session with the same training session in the previous training session are obtained, and i is selected from 1 to N in sequence.

[0039] If the number of nystagmus in the i-th training session in this training is less than that in the previous training session that is the same as the i-th training session, and the duration of each nystagmus in the i-th training session in this training is less than that in the previous training session that is the same as the i-th training session, it indicates that the effect of the i-th training session in this training is better than that of the previous training session.

[0040] The control module 26 is also used to determine whether the vestibular-ocular rehabilitation training rate in this training is greater than a first threshold, whether the total number of nystagmus in this training is less than the total number of nystagmus in the previous training, and whether the total duration of nystagmus in this training is less than the total duration of nystagmus in the previous training. If all of the above are true, the information that the vestibular-ocular pathway rehabilitation training of the target patient is good is evaluated and output; otherwise, the information that the vestibular-ocular pathway of the target patient still needs to continue rehabilitation training is evaluated and output.

[0041] Among them, the total number of nystagmus = the cumulative sum of the number of nystagmus in each training session, and the total duration of nystagmus = the cumulative sum of the duration of nystagmus in each training session.

[0042] Although the specific embodiments of the present invention are described above, those skilled in the art should understand that these are only examples, and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. A head-mounted vestibular rehabilitation training system based on VR technology, characterized in that: It includes a head-mounted VR device and a control host, the head-mounted VR device is provided with a left eye tracking camera, a right eye tracking camera and a posture sensor, and the control host includes a patient information management module, a training program management module, a visual target movement parameter setting module and a control module; The patient information management module is used to manage patient information; The training program management module is used to manage the vestibular rehabilitation training program of the patient, and the vestibular rehabilitation training program includes at least one training session with N divided training sessions having a sequential relationship and a training mode and a tracking mode adopted for each divided training session; The visual target movement parameter setting module is used to set the moving speed, acceleration and moving path length of the visual target image in each training; The control module is used to analyze the target patient based on the input patient information, retrieve the vestibular rehabilitation training program of the target patient, control the head-mounted VR device worn by the target patient to display the visual target image and move according to the visual target movement parameters and training method set in the i-th training session, prompt the target patient to track according to the training mode and tracking mode of the i-th training session to perform vestibular rehabilitation training, and retrieve the left eye movement tracking camera to collect the video image of the left eye pupil movement in the i-th training session, or the right eye movement tracking camera to collect the video image of the right eye pupil movement in the i-th training session, and / or the posture sensor to collect the movement of the head in the i-th training session based on the tracking mode of the i-th training session, where i is selected from 1 to N in sequence, and i and N are both positive integers; The control module is also used to obtain pupil movement video images and / or head movement conditions of the target patient in each training session, analyze the eye tracking conditions and / or head tracking conditions of the target patient's eyeballs to the visual target in each training session, and evaluate the vestibular rehabilitation training effect of the target patient; The control module is also used to obtain the image center movement trajectory of the visual target image in the i-th training session, use the recognition algorithm to perform center recognition and positioning of the pupil in the pupil movement video image collected in the i-th training session, and analyze the pupil center movement trajectory, calculate the eye tracking overlap between the pupil center movement trajectory and the image center movement trajectory in the i-th training session, and record the eye movement tracking overlap in the i-th training session when the eye movement tracking overlap reaches a first preset value, otherwise record the eye movement tracking non-overlap in the i-th training session, and analyze the head movement trajectory based on the head movement situation when there is also a head movement situation collected in the i-th training session, calculate the head tracking overlap between the head movement trajectory and the image center movement trajectory in the i-th training session, and record the head tracking overlap in the i-th training session when the head tracking overlap reaches a second preset value, otherwise record the head tracking non-overlap in the i-th training session, and based on this, record the eye movement tracking overlap and the head tracking overlap in each training session; The control module is also used to calculate the vestibular-ocular movement rehabilitation training rate of the target patient in this training = the number of times the recorded eye movement tracking overlaps / (the number of times the recorded eye movement tracking overlaps + the number of times the recorded eye movement tracking does not overlap), and determine whether the vestibular-ocular movement rehabilitation training rate is greater than a first threshold value, and if so, evaluate and output information that the vestibular-ocular movement pathway rehabilitation training of the target patient is good, and if not, evaluate and output information that the vestibular-ocular movement pathway of the target patient still needs to continue rehabilitation training; The control module is also used to calculate the vestibular-spinal cord rehabilitation training rate of the target patient in this training = the number of recorded head tracking overlaps / (the number of recorded head tracking overlaps + the number of recorded head tracking non-overlaps), and determine whether the vestibular-spinal cord rehabilitation training rate is greater than a second threshold. If so, the control module evaluates and outputs information that the vestibular-spinal cord pathway rehabilitation training of the target patient is good; if not, the control module evaluates and outputs information that the vestibular-spinal cord pathway of the target patient still needs to continue rehabilitation training.

2. The head-mounted vestibular rehabilitation training system based on VR technology as claimed in claim 1, characterized in that: The control module is also used to perform grayscale conversion, cropping, and pupil center positioning on each frame of the eyeball image in the collected pupil movement video image of the i-th training session, so as to obtain the change of the pupil center coordinates of the target patient in the i-th training session in a time sequence, analyze the number of times in the i-th training session when the change value of the latter pupil center coordinate and the former pupil center coordinate is greater than a set threshold as the number of nystagmus and the duration of each time when the change value of the former pupil center coordinate is greater than the set threshold as the duration of nystagmus, compare the number of nystagmus and the duration of each nystagmus of the i-th training session in this training session with the same training session as the i-th training session in the previous training session and give a comparison result, based on which the comparison result of each training session in this training session with the same training session in the previous training session is obtained.

3. The head-mounted vestibular rehabilitation training system based on VR technology as claimed in claim 2, characterized in that: The control module is further used to determine whether the vestibular-ocular rehabilitation training rate in the current training is greater than a first threshold, whether the total number of nystagmus in the current training is less than the total number of nystagmus in the previous training, and whether the total duration of nystagmus in the current training is less than the total duration of nystagmus in the previous training, and if all of them are yes, then evaluate and output information that the vestibular-ocular pathway rehabilitation training of the target patient is good; otherwise, evaluate and output information that the vestibular-ocular pathway of the target patient still needs to continue rehabilitation training; Among them, the total number of nystagmus = the cumulative sum of the number of nystagmus in each training session, and the total duration of nystagmus = the cumulative sum of the duration of nystagmus in each training session.

4. The head-mounted vestibular rehabilitation training system based on VR technology as claimed in claim 1, characterized in that: The system also includes a target image selection module, which is used to select a target image for each training. The target image includes a pattern, size, color and shape.

5. The head-mounted vestibular rehabilitation training system based on VR technology as claimed in claim 1, characterized in that: The system also includes a visual target display mode selection module, which is used to select a display mode of the visual target image for each training, and the display modes include a continuous display mode and a hidden display mode.

6. The head-mounted vestibular rehabilitation training system based on VR technology according to claim 1, characterized in that: The training modes include front left rotation training, front right rotation training, vertical upward rotation training, vertical downward rotation training, left front upward rotation training, left front downward rotation training, right front upward rotation training, right front downward rotation training, upper left rotation training, upper right rotation training, lower left rotation training, lower right rotation training, circular clockwise rotation training, and circular counterclockwise rotation training.

7. The head-mounted vestibular rehabilitation training system based on VR technology according to claim 1, characterized in that: The tracking modes include a single eye tracking mode and a head-eye tracking mode.

Images