Method, device and system for evaluating virtual avatar in virtual reality rehabilitation scene
By designing test tasks and evaluating physical representations in virtual reality rehabilitation scenarios, the problem of difficult to evaluate the effect of virtual avatars is solved, and effective testing and rehabilitation training effects of virtual avatars are improved.
Patent Information
- Application Number
- CN202410983810.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-07-22
AI Technical Summary
The existing virtual reality rehabilitation training methods lack accurate evaluation of the role and effect of virtual avatars in rehabilitation training, making it difficult to optimize virtual reality rehabilitation training.
By designing test tasks in virtual reality rehabilitation scenarios, selecting appropriate body representation types (such as touch, position, vestibular, and weight-bearing) according to rehabilitation training goals, sensory stimulation is performed on the user under test and collecting behavioral feedback data in real time, generating Body characterization evaluation indicators, thereby evaluating the evaluation results of virtual avatars.
Effective testing and evaluation of virtual avatars in virtual reality rehabilitation scenarios has been achieved, multi-channel stimulation design optimization of rehabilitation training has been improved, and the role of virtual avatars in rehabilitation training has been improved.
Smart Images

Figure CN118919017A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of virtual reality technology, and in particular to a method, device and system for evaluating a virtual avatar in a virtual reality rehabilitation scenario. Background Art
[0002] Virtual reality (VR) technology can simulate various life scenes in virtual reality scenes and realize the free separation and integration of multi-channel signals. For people with rehabilitation training needs such as stroke patients, the use of virtual reality technology can assist patients in more efficient, convenient and safe rehabilitation training. Rehabilitation training based on virtual reality can be carried out by the trainee controlling the actions of the virtual avatar (AvAtar) in virtual reality, where the virtual avatar is also called a virtual digital person, that is, a digital character image in virtual reality. Compared with traditional rehabilitation training methods, rehabilitation training based on virtual avatars in virtual reality scenes is a rehabilitation training method that is increasingly attracting attention. At present, there are many rehabilitation training methods based on virtual reality, such as a virtual reality rehabilitation training method with quantitative evaluation function, which can realize personalized selection of virtual scenes and functional quantitative evaluation according to the user's interest level. For example, a rehabilitation training method that integrates virtual reality games and intention recognition can obtain the user's training progress, determine the adjustment plan of the current level, and adjust the current training level to the target training level based on the adjustment plan to continue rehabilitation training.
[0003] However, no matter which existing virtual reality-based rehabilitation training method is used, it is selected or adjusted according to the user's interest level or training progress after the virtual reality rehabilitation scene has been put into use. There is a lack of evaluation of the effect of the virtual avatar in the virtual reality rehabilitation scene during the virtual avatar design stage. The virtual avatar is an important part of the virtual scene. If the role of the virtual avatar in rehabilitation training cannot be accurately evaluated, it will be difficult to effectively optimize the virtual reality-based rehabilitation training, thereby making it difficult to enhance the role of the virtual avatar in rehabilitation training.
[0004] Therefore, there is an urgent need for an evaluation method for virtual avatars in virtual reality rehabilitation scenarios and corresponding devices and systems. Summary of the invention
[0005] In view of this, the embodiments of the present application provide a method, device and system for evaluating a virtual avatar in a virtual reality rehabilitation scenario to eliminate or improve one or more defects existing in the prior art.
[0006] One aspect of the present application provides a method for evaluating a virtual avatar in a virtual reality rehabilitation scenario, comprising:
[0007] A test task generating step, selecting a virtual reality target evaluation scene and a body representation type containing a virtual avatar according to the rehabilitation training target, and generating a test task corresponding to the rehabilitation training target based on the selected target evaluation scene and body representation type;
[0008] a body representation evaluation step, after the user under test performs a rehabilitation training task based on the rehabilitation training goal in a virtual reality rehabilitation scene containing the virtual avatar, performing a test task corresponding to the rehabilitation training goal within a predetermined time, performing sensory stimulation matching the body representation type corresponding to the test task on the user under test during the execution of the test task, collecting behavior feedback data of the user under test in real time, and generating a body representation evaluation index based on the behavior feedback data using a body representation evaluation method matching the body representation type corresponding to the test task;
[0009] An evaluation result for the virtual avatar is obtained based on multiple body representation evaluation indicators generated in the body representation evaluation step by respectively executing multiple test tasks generated according to multiple rehabilitation training goals, wherein the multiple test tasks correspond to different body representation types.
[0010] In some embodiments of the present application, the body representation type in each test task is one of the following body representation types: touch, position sense, vestibular sense, and weight sense;
[0011] Accordingly, the body representation evaluation method in each body representation evaluation step is an evaluation method for the sense of touch, the sense of position, the vestibular sense or the sense of weight.
[0012] In some embodiments of the present application, the predetermined time is 20 seconds to 2 minutes.
[0013] In some embodiments of the present application, if the body representation type in the current test task is tactile, a sensory stimulation matching the body representation type corresponding to the test task is performed on the user under test during the execution of the test task, and behavioral feedback data of the user under test is collected in real time, and a body representation evaluation index is generated based on the behavioral feedback data using a body representation evaluation method matching the body representation type corresponding to the test task, including:
[0014] Randomly displaying a visual cue mark in the virtual avatar area corresponding to the target body area in the target evaluation scene of the current test task, and randomly vibrating and stimulating the target vibration position in the target body area of the user under test, so that the user under test judges multiple times whether his or her target vibration position is the same as the position of the visual cue mark, and according to each judgment result, using a preset virtual reality interaction device to send corresponding tactile feedback information respectively;
[0015] The tactile feedback information emitted by the user under test is collected in real time each time, and a tactile evaluation score serving as a body representation evaluation index is calculated based on the time interval between the collected tactile feedback information and the triggering time of the random vibration stimulation corresponding to the tactile feedback information.
[0016] In some embodiments of the present application, if the body representation type in the current test task is position sense, the body representation evaluation step further includes:
[0017] Displaying the virtual avatar of the user under test in the target evaluation scene of the current test task, wherein a fixed horizontal position difference is provided between the current real body midline position of the user under test and the virtual avatar midline position of the virtual avatar in the target evaluation scene;
[0018] Receiving initial horizontal position information sent by the tested user controlling the virtual reality interactive device, wherein the initial horizontal position information is the coordinate information in the horizontal direction corresponding to the initial position of the body midline of the virtual avatar currently in the target evaluation scene fed back by the tested user;
[0019] Determining whether the initial horizontal position information is less than or equal to the horizontal position difference, and if so, determining that the initial positioning of the position sense has been completed;
[0020] Correspondingly, in the process of executing the test task, the user under test is subjected to sensory stimulation matching the body representation type corresponding to the test task and the behavior feedback data of the user under test is collected in real time, and a body representation evaluation index is generated based on the behavior feedback data using a body representation evaluation method matching the body representation type corresponding to the test task, including:
[0021] Displaying the virtual avatar of the user under test in the target evaluation scene of the current test task, so that the user under test controls the virtual reality interaction device to send multiple times the horizontal position information of the body midline corresponding to the virtual avatar in the target evaluation scene;
[0022] The horizontal position information sent by the tested user as behavioral feedback data each time is collected in real time. If each piece of the horizontal position information is less than or equal to the horizontal position difference, each piece of the horizontal position information is retained, and a position sense evaluation score as a body representation evaluation indicator is calculated based on the retained horizontal position information.
[0023] In some embodiments of the present application, if the body representation type in the current test task is vestibular sense, the body representation evaluation step further includes:
[0024] Displaying the virtual avatar of the user under test in the target evaluation scene of the current test task, wherein a fixed angle is formed between the current real gravity direction of the user under test and the visual gravity direction of the virtual avatar in the target evaluation scene;
[0025] Receiving initial visual gravity direction information sent by the tested user controlling the virtual reality interaction device, wherein the initial visual gravity direction information is angle information corresponding to the initial visual gravity direction of the virtual avatar currently in the target evaluation scene fed back by the tested user;
[0026] Determining whether the initial visual gravity direction information is less than or equal to the angle, and if so, determining that the initial positioning of the vestibular sense has been completed;
[0027] Correspondingly, in the process of executing the test task, the user under test is subjected to sensory stimulation matching the body representation type corresponding to the test task and the behavior feedback data of the user under test is collected in real time, and a body representation evaluation index is generated based on the behavior feedback data using a body representation evaluation method matching the body representation type corresponding to the test task, including:
[0028] Displaying the virtual avatar of the user under test in the target evaluation scene of the current test task, so that the user under test controls the virtual reality interaction device to send out visual gravity direction information corresponding to the virtual avatar in the target evaluation scene multiple times;
[0029] The visual gravity direction information emitted by the measured user each time is collected in real time. If each piece of visual gravity direction information is less than or equal to the angle, each piece of visual gravity direction information is retained, and a vestibular sense evaluation score serving as a body representation evaluation indicator is calculated based on the retained visual gravity direction information.
[0030] In some embodiments of the present application, if the body representation type in the current test task is weight-bearing sensation, a sensory stimulation matching the body representation type corresponding to the test task is performed on the user under test during the execution of the test task, and behavioral feedback data of the user under test is collected in real time, and a body representation evaluation index is generated based on the behavioral feedback data using a body representation evaluation method matching the body representation type corresponding to the test task, including:
[0031] Displaying a first object and a second object of the same shape and weight in a target evaluation scene of a current test task, and the volume of the first object is larger than that of the second object, so that the user under test can pull the first object and the second object to a preset height multiple times in the target evaluation scene based on the virtual reality interaction device;
[0032] The grip strength information, the pulling start time, and the pulling end time of the user pulling the first object and the second object are collected based on the virtual reality interaction device, and a weight-bearing perception evaluation score as a body representation evaluation indicator is obtained based on the collected data.
[0033] In some embodiments of the present application, the plurality of body representation evaluation indicators include: a tactile evaluation score, a position sense evaluation score, a vestibular sense evaluation score, and a weight-bearing sense evaluation score;
[0034] The evaluation results for the virtual avatar are obtained by respectively executing the multiple test tasks generated according to the multiple rehabilitation training objectives and generating the multiple body representation evaluation indicators in the body representation evaluation step, including:
[0035] A scoring data matrix is generated based on the tactile evaluation score, the position sense evaluation score, the vestibular sense evaluation score, and the weight-bearing sense evaluation score, and the scoring data matrix is used as the evaluation result of the virtual avatar in the tested virtual reality rehabilitation scene.
[0036] Another aspect of the present application provides an evaluation device for a virtual avatar in a virtual reality rehabilitation scenario, comprising: a processor, a memory, and a computer program stored in the memory, wherein the processor is used to execute the computer program, and when the computer program is executed, the device implements the steps of the method described above.
[0037] A third aspect of the present application provides an evaluation system for a virtual avatar in a virtual reality rehabilitation scene, comprising: an evaluation device for a virtual avatar in a virtual reality rehabilitation scene, and a virtual reality device, a vibration stimulation device for vibrotactile stimulation, and a virtual reality interactive device, which are respectively connected to the evaluation device for the virtual avatar in the virtual reality rehabilitation scene in communication with each other; wherein the vibration stimulation device for vibrotactile stimulation and the virtual reality interactive device are respectively connected to the virtual reality device in communication with each other;
[0038] The evaluation device for the virtual avatar in the virtual reality rehabilitation scene is used to execute the evaluation method for the virtual avatar in the virtual reality rehabilitation scene according to any one of claims 1 to 7;
[0039] The virtual reality equipment includes: a VR helmet;
[0040] The virtual reality interaction device includes: a VR handle;
[0041] The vibration stimulation device for performing vibration tactile stimulation includes: a vibration tactile module patch or a vibration device arranged on the VR handle.
[0042] A fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for evaluating a virtual avatar in a virtual reality rehabilitation scenario.
[0043] A fifth aspect of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method for evaluating a virtual avatar in the virtual reality rehabilitation scene.
[0044] The present application provides an evaluation method for a virtual avatar in a virtual reality rehabilitation scenario, which obtains evaluation data of the virtual avatar in the virtual reality scenario based on the behavioral feedback data of the tested user in each virtual reality rehabilitation scenario, and can realize the testing and evaluation of the virtual avatar in the virtual reality rehabilitation scenario, and can effectively improve the reliability and effectiveness of the virtual avatar testing process in the virtual reality rehabilitation scenario, and further can optimize the design of a multi-channel stimulation method in the rehabilitation scenario, thereby improving the role of the virtual avatar in rehabilitation training.
[0045] Additional advantages, purposes, and features of the present application will be partially described in the following description, and will become partially apparent to those skilled in the art after studying the following, or may be learned from the practice of the present application. The purposes and other advantages of the present application can be achieved and obtained by the structures specifically pointed out in the specification and the drawings.
[0046] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present application are not limited to the above specific description, and the above and other purposes that can be achieved by the present application will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] The drawings described herein are used to provide a further understanding of the present application, constitute a part of the present application, and do not constitute a limitation of the present application. In the drawings:
[0048] Figure 1 Schematic diagram of the process of evaluating a virtual avatar in a virtual reality rehabilitation scenario in one embodiment of the present application.
[0049] Figure 2 This is a first flow chart of the body representation evaluation step in the method for evaluating a virtual avatar in a virtual reality rehabilitation scenario in one embodiment of the present application.
[0050] Figure 3 This is a second flow chart of the body representation evaluation step in the method for evaluating a virtual avatar in a virtual reality rehabilitation scenario in one embodiment of the present application.
[0051] Figure 4This is a third flow chart of the body representation evaluation step in the method for evaluating a virtual avatar in a virtual reality rehabilitation scenario in one embodiment of the present application.
[0052] Figure 5 This is a fourth flow chart of the body representation evaluation step in the method for evaluating a virtual avatar in a virtual reality rehabilitation scenario in one embodiment of the present application.
[0053] Figure 6 This is a fifth flow chart of the body representation evaluation step in the method for evaluating a virtual avatar in a virtual reality rehabilitation scenario in one embodiment of the present application.
[0054] Figure 7 3 is a flow chart of step 300 in a method for evaluating a virtual avatar in a virtual reality rehabilitation scenario in an embodiment of the present application.
[0055] Figure 8 Schematic diagram of the functions of a method for evaluating a virtual avatar in a virtual reality rehabilitation scenario in one embodiment of the present application.
[0056] Fig. 9 This is an example schematic diagram of a virtual ball in a virtual scene in an embodiment of the present application.
[0057] Fig.10 The figure is a schematic diagram showing an example of the location of the vibrotactile module patch in a real scene in an embodiment of the present application.
[0058] Fig.11 FIG. 4 is a schematic diagram of position sense assessment in one embodiment of the present application.
[0059] Fig.12 Schematic diagram of vestibular sense assessment in one embodiment of the present application.
[0060] Fig.13 Schematic diagram of weight-bearing sensation assessment in one embodiment of the present application.
[0061] Fig.14 Schematic diagram of the structure of a device for evaluating a virtual avatar in a virtual reality rehabilitation scenario in one embodiment of the present application. DETAILED DESCRIPTION
[0062] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the implementation modes and the accompanying drawings. Here, the illustrative implementation modes and descriptions of the present application are used to explain the present application, but are not intended to limit the present application.
[0063] It should also be noted here that in order to avoid obscuring the present application due to unnecessary details, only the structures and / or processing steps closely related to the scheme according to the present application are shown in the accompanying drawings, while other details that are not very relevant to the present application are omitted.
[0064] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.
[0065] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.
[0066] In order to solve the problem that the existing rehabilitation training method based on virtual reality technology lacks an evaluation method for virtual avatars in virtual reality rehabilitation scenes, the embodiments of the present application respectively provide an evaluation method for virtual avatars in virtual reality rehabilitation scenes, an evaluation device for virtual avatars in virtual reality rehabilitation scenes for executing the method, and an evaluation system for virtual avatars in virtual reality rehabilitation scenes, after the tested user performs a rehabilitation training task in a virtual reality rehabilitation scene containing a virtual avatar corresponding to the user's rehabilitation training target, further applies sensory stimulation matching the body representation type set in the test task in a test task under a virtual reality target evaluation scene corresponding to the user's rehabilitation training target to obtain user behavior feedback data, and performs corresponding body representation evaluation based on the user behavior feedback data, and obtains multiple body representation evaluations by using different sensory stimulations to obtain user behavior feedback data corresponding to different stimulations for different test tasks for different rehabilitation training targets, and further generates virtual avatar evaluation results based on multiple body representation evaluation results, thereby realizing the testing and evaluation of virtual avatars in virtual reality rehabilitation scenes. Furthermore, the multi-sensory channel stimulation method in the rehabilitation scene can be optimized and designed based on the evaluation results of the virtual avatar, thereby effectively improving the role of the virtual avatar in rehabilitation training.
[0067] Body representation is the human self-consciousness that integrates multiple sensory signals to produce cognition of the body and environment. Body representation represents the human brain's recoding or mapping of all perceived things, that is, the individual brain forms a body representation matrix through the representation or recoding mapping of the body and its surrounding environment. Body representation is further divided into body image and body diagram.
[0068] Body schema refers to the cognitive management of the distribution of various available functions of currently controllable objects (such as the body, parts of limbs, or equipped tools) by self-consciousness. It is easily affected by sensory input and forms a bottom-up update and reconstruction. For example, a person who has never used scissors can quickly learn this new skill. Body image refers to a relatively fixed top-down conscious cognitive management of the carrying object (body or parts of limbs) formed by self-consciousness over a long period of time. Compared with body schema, it is less susceptible to temporary sensory input. For example, a healthy person will firmly believe that he has two hands, regardless of whether he can see (such as in the dark) or feel (such as temporary limb numbness). The plasticity of the brain allows an interesting phenomenon to occur: the body schema formed by long-term stable sensory input will gradually transform into a relatively fixed body image. For example, the "practice makes perfect", "knowing it like the back of your hand", and "handy" formed by long-term use of specific tools are actually manifestations of this transformation; similarly, the "learned disuse" formed by post-stroke patients in the state of long-term specific limb motor dysfunction - gradually forgetting the long-term stable movement pattern in their body image, is also a manifestation of the transformation of a "disused" body schema to a body image. This transformation based on brain plasticity is the reshaping of body representation. Reshaping body representation is helpful for the recovery of the motor system. In the embodiment of the present invention, the virtual avatar evaluation result is obtained by evaluating the body representation.
[0069] See also Figure 1 The method for evaluating a virtual avatar in a virtual reality rehabilitation scenario provided in an embodiment of the present application comprises the following steps:
[0070] Step S100: selecting a virtual reality target evaluation scene and a body representation type containing a virtual avatar according to a rehabilitation training target, and generating a test task corresponding to the rehabilitation training target based on the selected target evaluation scene and body representation type.
[0071] This step is a test task generation step. The test task can be pre-generated and stored based on the rehabilitation training goal. When the generated test task is subsequently executed, the user under test can be subjected to sensory stimulation matching the body representation type corresponding to the test task in a virtual reality target evaluation scenario and the user's behavior feedback data can be collected in real time. Based on the behavior feedback data, a body representation evaluation method matching the body representation type corresponding to the test task can be used to generate a body representation evaluation index.
[0072] In some optional embodiments of the present invention, the body representation type selected according to the rehabilitation training goal may include, for example, at least one of the following types: touch, position sense, vestibular sense and weight sense. These body representation types are only examples and the present invention is not limited thereto.
[0073] Step S200, a body representation evaluation step, after the user under test performs a rehabilitation training task based on the rehabilitation training goal in a virtual reality rehabilitation scene containing the virtual avatar, a test task corresponding to the rehabilitation training goal is performed within a predetermined time, and in the process of performing the test task, a sensory stimulation matching the body representation type corresponding to the test task is performed on the user under test, and behavioral feedback data of the user under test is collected in real time, and a body representation evaluation index is generated based on the behavioral feedback data using a body representation evaluation method matching the body representation type corresponding to the test task.
[0074] In an embodiment of the present invention, after executing a rehabilitation training task for a specific rehabilitation training goal, a test task generated based on the same rehabilitation training goal is executed within a very short time, such as 10 seconds to 2 minutes, which is equivalent to immediate execution. Among them, the rehabilitation training task is used to induce the reshaping of the user's body representation in a certain virtual reality scene before the body representation evaluation, and then perform various sensory evaluations of the body representation. The reshaping of the body representation facilitates the patient to connect the experience in the virtual reality scene with the real scene, thereby improving the rehabilitation efficiency and increasing the rehabilitation effect. The body representation induced in a specific virtual reality scene can only last for 1-2 minutes after leaving the scene. Therefore, in the embodiment of the present invention, the test task is executed immediately after the rehabilitation training task is executed to evaluate the representation characteristics.
[0075] In an embodiment of the present invention, the rehabilitation training tasks in the virtual reality rehabilitation scenario are usually pre-established in the virtual reality rehabilitation training system, and different rehabilitation training tasks corresponding to different rehabilitation training goals can introduce stimulation of a single sense organ (such as tactile, auditory or visual stimulation, etc.), or introduce multi-sensory joint stimulation, that is, introduce stimulation of multiple sense organs at the same time to perform rehabilitation training tasks. In other words, in the rehabilitation training task, the user under test is subjected to sensory stimulation corresponding to the rehabilitation training goal and matching at least one feature of the multi-sensory channel features (features generated by multiple sensory stimulation channels for sensory stimulation). Since in an embodiment of the present invention, the rehabilitation training tasks performed can be completely training tasks that already exist in the existing virtual rehabilitation training system, they will not be repeated here. Different test tasks corresponding to different rehabilitation training goals can only introduce stimulation of a single sense organ (such as tactile, auditory or visual stimulation, etc.), but can also introduce multi-sensory joint stimulation.
[0076] In an embodiment of the present invention, rehabilitation training tasks and test tasks based on the same rehabilitation training goals may correspond to the same or different virtual reality scenes. For example, the rehabilitation training task may be performed in a virtual reality rehabilitation scene, and the test task may be performed in other virtual scenes. However, in an embodiment of the present invention, the virtual reality rehabilitation scene corresponding to the rehabilitation training task and the virtual scene corresponding to the test task contain the same virtual avatar.
[0077] In the body representation evaluation step, during the execution of the test task, by subjecting the tested user to sensory stimulation matching the body representation type corresponding to the test task and collecting the tested user's behavioral feedback data in real time, a body representation evaluation index can be generated based on the behavioral feedback data using a body representation evaluation method matching the body representation type corresponding to the test task. The test user's behavioral feedback data is generated in a virtual reality scene containing a virtual avatar, and is directly related to the image of the virtual avatar. Therefore, in an embodiment of the present invention, the virtual avatar is evaluated based on the body representation evaluation index obtained based on the test user's behavioral feedback data. As an example, when the body representation types corresponding to different test tasks are sub-packaged into touch, position sense, vestibular sense, and weight-bearing sense, the body representation evaluation method that matches the body representation type considered corresponding to each test can be an existing or new evaluation method for touch, position sense, vestibular sense, and weight-bearing sense, respectively. This will be described below through an example.
[0078] Step S300, obtaining an evaluation result for the virtual avatar based on a plurality of body representation evaluation indicators generated in the body representation evaluation step by respectively executing a plurality of test tasks generated according to a plurality of rehabilitation training objectives, wherein the plurality of test tasks correspond to different body representation types.
[0079] Since in the body representation evaluation step, each test task is performed only for one type of body representation to be evaluated, in order to perform a more comprehensive evaluation of multiple types of body representations, in an embodiment of the present invention, rehabilitation training tasks and test tasks are pre-formed for multiple rehabilitation training targets, and the body representation evaluation step is performed multiple times for different rehabilitation training targets to respectively execute multiple test tasks generated according to the multiple rehabilitation training targets, thereby generating evaluation indicators of multiple body representations of multiple different body representation types, and obtaining evaluation results for the virtual avatar based on the generated body representation evaluation indicators.
[0080] The present invention sequentially executes a plurality of training task-test task combinations for different body representation features until the body representation evaluation steps are completed in each virtual reality evaluation scene for all body representations, thereby obtaining evaluation data of the virtual avatar in the tested virtual reality scene according to the behavior feedback data of the tested user in each virtual reality scene.
[0081] From the above description, it can be seen that the evaluation method of the virtual avatar in the virtual reality rehabilitation scene provided in the embodiment of the present application can realize the testing and evaluation of the virtual avatar in the virtual reality rehabilitation scene, and then can optimize the design of the multi-channel stimulation method in the rehabilitation scene, thereby improving the role of the virtual avatar in rehabilitation training.
[0082] See also Figure 2 In one embodiment of the present invention, if the body representation type in the current test task is touch, but the applied stimulation is a multi-sensory joint stimulation including touch, in step S200, during the execution of the test task, the user under test is subjected to a sensory stimulation matching the body representation type corresponding to the test task and the behavior feedback data of the user under test is collected in real time, and based on the behavior feedback data, a body representation evaluation method matching the body representation type corresponding to the test task is used to generate a body representation evaluation index, including:
[0083] Step S010: In the target evaluation scene of the current test task, vibrotactile stimulation is performed on the target body area of the user under test, and the virtual reality device currently displaying the target evaluation scene and worn by the user under test is controlled to display a preset picture corresponding to the vibrotactile stimulation in the target evaluation scene and output a preset sound corresponding to the vibrotactile stimulation, so as to achieve combined stimulation of touch, vision and hearing for the user under test.
[0084] In an example of step S010, visual and auditory stimulation can be achieved through a VR helmet, and tactile stimulation can be achieved through a vibrating tactile module patch. At this stage, the user being tested (the person being evaluated) can wear a VR helmet on his head, and two identical vibrating tactile module patches are attached to the back of the user's dominant hand. The virtual hand on the same side contains two position prompt points at the corresponding position. These two position prompt points are randomly vibrated and stimulated, and the corresponding positions of the virtual hand will be highlighted. Enter the virtual scene and start 2 minutes of random vibration stimulation. The VR helmet will feedback the vibration sound and picture during the stimulation.
[0085] Step S020: collecting feedback information sent by the user under test in real time each time, and calculating a body representation evaluation index based on the collected feedback information. The body representation evaluation index may be an evaluation parameter for touch, hearing and vision.
[0086] It can be understood that each time the sensory stimulation matching the pre-selected body representation type is performed on the user under test in step S020, the body representation evaluation method used in the body representation evaluation also corresponds to the body representation type. For example, if the body representation type in the test task is touch, the body representation evaluation method used is an evaluation method for touch, which may be an existing evaluation method.
[0087] In order to further improve the comprehensiveness of rehabilitation scene selection and body representation feedback, and to further improve the accuracy and effectiveness of virtual avatar testing in rehabilitation scenes, in the evaluation method of virtual avatars in virtual reality rehabilitation scenes provided in the embodiments of the present application, the body representation types evaluated may include, in addition to touch, position sense, vestibular sense and weight sense, but the present invention is not limited to this, and may also include more or fewer body representation types.
[0088] Correspondingly, if the current target evaluation scenario corresponds to the body representation evaluation method for tactile evaluation, see Figure 3 In the body representation evaluation step of the evaluation method of the virtual avatar in the virtual reality rehabilitation scene, during the execution of the test task, the user under test is subjected to sensory stimulation matching the body representation type corresponding to the test task and the behavior feedback data of the user under test is collected in real time, and based on the behavior feedback data, a body representation evaluation method matching the body representation type corresponding to the test task is used to generate a body representation evaluation index, which may include the following steps:
[0089] Step S021: randomly displaying a visual cue mark in the virtual avatar area corresponding to the target body area in the target evaluation scene of the current test task, and performing random vibration stimulation on the target vibration position in the target body area of the user under test, so that the user under test can judge multiple times whether his or her target vibration position is the same as the position of the visual cue mark, and according to the result of each judgment, use a preset virtual reality interaction device to send out corresponding tactile feedback information respectively.
[0090] Step S022: collecting tactile feedback information sent by the user in real time each time, and calculating a tactile evaluation score as a body representation evaluation index based on the time interval between the collected tactile feedback information and the triggering time of the random vibration stimulation corresponding to the tactile feedback information.
[0091] In one example, a highlighted visual prompt appears randomly at the corresponding part of the virtual hand vibration tactile module patch, and the vibration tactile module patch vibrates randomly. The user determines whether the vibration part is the same as the visual prompt part and provides feedback through the handle. This action is repeated n times.
[0092] If the body representation type in the current test task is position sense, see Figure 4 The step of evaluating the body representation in the evaluation method for the virtual avatar in the virtual reality rehabilitation scene further specifically includes the following contents before generating the body representation evaluation index based on the behavior feedback data using the body representation evaluation method matching the body representation type corresponding to the test task:
[0093] Step S001: Displaying the virtual avatar of the user under test in the target evaluation scene of the current test task, wherein a fixed horizontal position difference exists between the current real body midline position of the user under test and the virtual avatar midline position of the virtual avatar in the target evaluation scene.
[0094] Step S002: receiving initial horizontal position information sent by the tested user controlling the virtual reality interaction device, wherein the initial horizontal position information is the coordinate information in the horizontal direction corresponding to the initial position of the body midline of the virtual avatar currently in the target evaluation scene fed back by the tested user.
[0095] Step S003: Determine whether the initial horizontal position information is less than or equal to the horizontal position difference. If so, determine that the initial positioning of the position sense has been completed.
[0096] Correspondingly, in the process of executing the test task, the user under test is subjected to sensory stimulation matching the body representation type corresponding to the test task and the behavior feedback data of the user under test is collected in real time, and a body representation evaluation index is generated based on the behavior feedback data using a body representation evaluation method matching the body representation type corresponding to the test task, including:
[0097] Step S023: displaying the virtual avatar of the user under test in the target evaluation scene of the current test task, so that the user under test controls the virtual reality interaction device to send multiple times the horizontal position information of the body midline corresponding to the virtual avatar in the target evaluation scene;
[0098] Step S024: Collect in real time the horizontal position information sent by the tested user each time as behavioral feedback data; if each piece of the horizontal position information is less than or equal to the horizontal position difference, retain each piece of the horizontal position information; and calculate a position sense evaluation index as a body representation evaluation index based on the retained horizontal position information, more specifically, a position sense evaluation score.
[0099] In one example, it can be assumed that the real body and the horizontal body always have a fixed horizontal position difference X s , and then initial positioning is required: the user wears a VR helmet and uses the VR handle to feedback the horizontal coordinate information of the initial position of the body midline in the virtual scene, and the initial horizontal position information is defined as X0. If X0>
[0100] X s , it proves that the evaluation is wrong and needs to be re-evaluated and the above steps are repeated; if X0≤X s, then record X0. Then perform sensory stimulation after initial positioning. After sensory stimulation, the user uses the VR handle to feedback the horizontal position coordinate information of the body midline in the virtual scene, and define the i-th horizontal position information as X i , this action is repeated n times, where n is a positive integer greater than 1.
[0101] If the body representation type in the current test task is vestibular, see Figure 5 , the step of evaluating the body representation in the method for evaluating the virtual avatar in the virtual reality rehabilitation scene further includes: generating the body representation evaluation index based on the behavior feedback data by using the body representation evaluation method matching the body representation type corresponding to the test task;
[0102] Step S004: displaying the virtual avatar of the user under test in the target evaluation scene of the current test task, wherein a fixed angle is formed between the current real gravity direction of the user under test and the visual gravity direction of the virtual avatar in the target evaluation scene;
[0103] Step S005: receiving initial visual gravity direction information sent by the tested user controlling the virtual reality interaction device, wherein the initial visual gravity direction information is angle information corresponding to the initial visual gravity direction of the virtual avatar currently in the target evaluation scene fed back by the tested user;
[0104] Step S006: Determine whether the initial visual gravity direction information is less than or equal to the angle. If so, it is determined that the initial positioning of the vestibular sense has been completed.
[0105] Correspondingly, in the process of executing the test task, the user under test is subjected to sensory stimulation matching the body representation type corresponding to the test task and the behavior feedback data of the user under test is collected in real time, and a body representation evaluation index is generated based on the behavior feedback data using a body representation evaluation method matching the body representation type corresponding to the test task, including:
[0106] Step S025: displaying the virtual avatar of the user under test in the target evaluation scene of the current test task, so that the user under test controls the virtual reality interaction device to send out visual gravity direction information corresponding to the virtual avatar in the target evaluation scene multiple times;
[0107] Step S026: Collect the visual gravity direction information emitted by the tested user each time in real time. If each piece of visual gravity direction information is less than or equal to the angle, retain each piece of visual gravity direction information, and calculate the vestibular sense evaluation score as a body representation evaluation indicator based on the retained visual gravity direction information.
[0108] In one example, the visual gravity direction in the virtual scene and the real gravity direction can be set to have a fixed angle θ s Then, in the initial positioning stage, the user wears a VR helmet and uses the VR handle to feedback the direction of gravity in the virtual scene, defining the initial visual gravity direction information as θ0. s , it proves that the evaluation is wrong and needs to be re-evaluated and the above steps are repeated; if θ0≤θ s , then record θ0. After the initial positioning, multi-sensory joint stimulation is performed. After the multi-sensory joint stimulation is completed, the vestibular sense formal assessment begins: the user wears a VR helmet and uses the VR handle to control the frame stick instrument to feedback the gravity direction felt by the person being assessed. The visual gravity direction information fed back for the i-th time is θ i , the action is repeated n times. If θ i >θ0, an evaluation error occurs and the above steps need to be repeated.
[0109] If the body representation type in the current test task is weight perception, see Figure 6 The body representation evaluation step in the evaluation method of the virtual avatar in the virtual reality rehabilitation scene also includes:
[0110] Step S027: Displaying a first object and a second object of the same shape and weight in the target evaluation scene of the current test task, and the volume of the first object is larger than that of the second object, so that the user under test can pull the first object and the second object to a preset height multiple times in the target evaluation scene based on the virtual reality interaction device.
[0111] Step S028: Based on the virtual reality interactive device, the grip strength information, pulling start time and pulling end time of the user each time pulling the first object and the second object are collected, and based on the collected data, a weight perception evaluation score as a body representation evaluation indicator is obtained.
[0112] In order to further improve the effectiveness and accuracy of evaluating the rehabilitation effectiveness of the tested virtual reality rehabilitation scene group, in the embodiment of the evaluation method of the virtual avatar in the virtual reality rehabilitation scene provided in the embodiment of the present application, see Figure 7 Step S300 of the method for evaluating a virtual avatar in a virtual reality rehabilitation scene specifically includes the following contents:
[0113] Step S310: respectively determining the judgment values corresponding to each piece of tactile feedback information for indicating whether the judgment result of the feedback action display is accurate, and determining the tactile evaluation score corresponding to the virtual reality scene based on the tactile test according to the time interval between each piece of tactile feedback information and the judgment value.
[0114] Specifically, the calculation formula of the tactile evaluation score p1 is as follows:
[0115]
[0116] Among them, RT i (Reaction Time) is the time interval between the time when the tactile feedback information sent by the tested user for the i-th time and the triggering time of the random vibration stimulation corresponding to the tactile feedback information; CCE represents the cross-modal consistency effect; r represents the judgment function, wherein -1 and 1 both represent the judgment value; n in the calculation formula in one or more embodiments of the present application represents the corresponding number of times the user sends the behavior feedback data, which can be referred to as the number of repetitions;
[0117] Among them, the judgment function r is as follows:
[0118]
[0119] Step S320: determining a position sense evaluation score corresponding to the virtual reality scene based on the position sense test according to each horizontal position information and the horizontal position difference.
[0120] Specifically, the calculation formula for the position sense evaluation score p2 is as follows:
[0121]
[0122] Step S330: Determine the vestibular sense evaluation score corresponding to the virtual reality scene based on the vestibular sense test according to each visual gravity direction information and angle.
[0123] Specifically, the vestibular sense evaluation score p3 is calculated as follows:
[0124]
[0125] Step S340: determining a weight perception evaluation score corresponding to the virtual reality scene based on the weight perception test according to the grip strength information, the pulling start time, and the pulling end time of the tested user each time when the user pulls the first object and the second object respectively.
[0126] Specifically, the calculation formula of the weight-bearing perception evaluation score p4 is as follows:
[0127]
[0128] The grip force information when pulling the large object (i.e., the first object) for the i-th time is defined as F i , the grip force information when pulling the small object (i.e., the second object) for the i-th time is f i ; The starting time of the i-th pulling action is t is, the end time of the i-th pulling of the small block is t ie ; The starting time of the i-th pulling of the large object is T is , the end time of the i-th pulling of the large object is T ie .
[0129] Step S350: Generate a scoring data matrix as rehabilitation effectiveness evaluation data corresponding to the virtual avatar evaluation result data in the scene based on the tactile evaluation score, the position sense evaluation score, the vestibular sense evaluation score and the weight-bearing sense evaluation score.
[0130] Among them, the scoring data matrix P = [p1, p2, p3, p4].
[0131] In order to further illustrate the evaluation method of the virtual avatar in the virtual reality rehabilitation scene provided by the above embodiment, the present application also provides a specific application example of the evaluation method of the virtual avatar in the virtual reality rehabilitation scene, in which the technical fields of artificial intelligence medicine and neurology, rehabilitation engineering and virtual reality are involved, see Figure 8 The multi-sensory joint stimulation module may include three stimulation channels: vision, touch, and hearing. The evaluation phase is performed in the order of touch evaluation, position evaluation, vestibular evaluation, and weight-bearing evaluation, which is specifically embodied in a scoring data matrix P, which includes a touch evaluation score p1, a position evaluation score p2, a vestibular evaluation score p3, and a weight-bearing grip evaluation score p4. The body representation information of the person being evaluated is P = [p1, p2, p3, p4]. The overall execution process of this application example is: the user completes the virtual reality rehabilitation scene rehabilitation training task (at least including tactile stimulation) associated with the tactile assessment, performs the tactile assessment, completes the virtual reality rehabilitation scene rehabilitation training task (at least including positional stimulation) associated with the positional assessment, performs the positional assessment, completes the virtual reality rehabilitation scene rehabilitation training task (at least including vestibular stimulation) associated with the vestibular assessment, performs the vestibular assessment, completes the virtual reality rehabilitation scene rehabilitation training task (at least including weight-bearing stimulation) associated with the weight-bearing assessment, performs the weight-bearing assessment, and obtains the scoring data matrix. The specific instructions are as follows:
[0132] (1) Rehabilitation training tasks in various virtual reality rehabilitation scenarios involving virtual avatars
[0133] The virtual reality rehabilitation scene corresponding to each rehabilitation training task includes a virtual avatar. Each training task can only take the stimulation corresponding to the type of body representation to be evaluated in the immediately associated body representation evaluation process, or can take multi-sensory joint stimulation. Multi-sensory joint stimulation may include the use of multi-sensory stimulation features provided by some or all of the sensory channels such as vision, touch, and hearing to provide part or all of the stimulation of vision, touch, and hearing. The visual and auditory stimulation can be achieved through a VR helmet, and the tactile stimulation can be achieved through a vibrating tactile module patch. In this stage, the user wears a VR helmet on the head, and two identical vibrating tactile module patches are attached to the back of the dominant hand of the person being evaluated. The corresponding position of the virtual hand on the same side contains two position prompt points, and the two position prompt points are randomly vibrated. Stimulation, at the same time, the corresponding position of the virtual hand will be highlighted. Enter the virtual scene, start 2 minutes of random vibration stimulation, and the VR helmet will feedback the vibration sound and picture during the stimulation.
[0134] Examples of virtual balls in virtual scenes are Fig. 9 As shown, the location of the vibrotactile module patch in the real scene is shown as follows: Fig.10 As shown. The person being evaluated refers to the user being tested mentioned in the embodiments of the present application, and may also be referred to as the user for short. The virtual scene is the abbreviation of the virtual reality rehabilitation scene mentioned in the embodiments of the present application, but is not limited to the virtual reality rehabilitation scene, and may also be other virtual reality scenes.
[0135] Among them, the vibration tactile module patch can also be replaced by the vibration system on the VR handle; the vibration tactile module patch and its corresponding position in the virtual scene can be replaced to other positions (such as lower arm, upper arm, shoulder, etc.); the small ball touching the back of the hand in the virtual scene can be replaced by a brush touching the back of the hand or another task in the virtual scene touching the back of the user's hand.
[0136] (2) Tactile evaluation
[0137] Before the tactile assessment, a virtual reality rehabilitation scenario rehabilitation training task involving a virtual avatar is required. In the virtual reality scenario, the corresponding part of the virtual hand vibration tactile module patch randomly appears a highlighted visual prompt, and the vibration tactile module patch vibrates randomly. The user determines whether the vibration part is the same as the visual prompt part and gives feedback through the handle. This action is repeated n times. Define the reaction time RT i (Reaction Time) is the time interval between the moment when the user makes the i-th judgment and the moment when the tactile sensation occurs, and then the tactile evaluation score p1 is calculated according to the aforementioned calculation formula of the tactile evaluation score p1 of the cross-modal consistency effect.
[0138] (3) Position sense assessment
[0139] During the position sense assessment process, there is always a fixed horizontal position difference X between the real body and the horizontal body. s The position sense evaluation process requires initial positioning: the user wears a VR helmet and uses the VR handle to provide feedback on the horizontal coordinate information of the initial position of the body midline in the virtual scene. The initial horizontal position information is defined as X0. If X0>X s , it proves that the evaluation is wrong and needs to be re-evaluated and the above steps of this process are repeated; if X0≤X s , then record X0. After the initial positioning, a virtual reality rehabilitation scene rehabilitation training task involving a virtual avatar associated with position sense assessment is performed.
[0140] After completing the virtual reality rehabilitation training task associated with position sense assessment, the user uses the VR handle to feedback the horizontal position coordinate information of the body midline in the virtual scene, and the i-th horizontal position information is defined as X i , the action is repeated n times. i >X0, the evaluation is wrong and the above steps need to be repeated; if X i ≤X0, the position sense evaluation score p2 is calculated based on the calculation formula of the position sense evaluation score p2.
[0141] Examples of applications of position sense assessment include Fig.11 shown.
[0142] (4) Vestibular assessment
[0143] During the vestibular assessment, the visual gravity direction in the virtual scene and the real gravity direction have a fixed angle θ s In the initial positioning stage of the vestibular assessment process, the user wears a VR helmet and uses the VR handle to provide feedback on the direction of gravity felt in the virtual scene. The initial visual gravity direction information is defined as θ0. s , it proves that the evaluation is wrong and needs to be re-evaluated and the above steps are repeated; if θ0≤θ s , then record θ0.
[0144] After the initial positioning, a rehabilitation training task involving a virtual reality rehabilitation scenario involving a virtual avatar was performed in association with the vestibular sense assessment.
[0145] After the rehabilitation training task is completed, the vestibular sense is formally assessed: the user wears a VR helmet and uses the VR handle to control the frame bar to feedback the direction of gravity felt by the person being assessed. The visual gravity direction information fed back for the i-th time is θ i , the action is repeated n times. If θ i >θ0, the evaluation is wrong and the above steps need to be repeated; if θ i≤θ0, the vestibular sense evaluation score p3 is recorded based on the calculation formula of the vestibular sense evaluation score p3.
[0146] Examples of applications of vestibular assessment include Fig.12 shown.
[0147] (5) Weight-bearing sensation assessment
[0148] When performing weight-bearing assessment immediately after completing the virtual reality rehabilitation training task involving a virtual avatar associated with weight-bearing assessment, the user needs to wear a VR helmet and pull two objects of different sizes, shapes and weights to a specified height above the table in the virtual scene. Each object is pulled n times and the average grip strength is fed back through the VR handle. The grip strength information when the i-th large object is pulled is defined as F i , the grip force information when pulling the small object for the i-th time is f i ; The starting time of the i-th pulling action is t is , the end time of the i-th pulling of the small block is t ie ; The starting time of the i-th pulling of the large object is T is , the end time of the i-th pulling of the large object is T ie , this action is repeated n times, and the weight-bearing sense evaluation score p4 is recorded based on the calculation formula of the weight-bearing sense evaluation score p4.
[0149] Examples of applications of weight-bearing perception assessment include Fig.13 shown.
[0150] (6) All recorded data can be fed back to the computer terminal via the Bluetooth of the handle and recorded in Excel. The feedback results include the original data and the scoring data matrix.
[0151] In one example, the tactile assessment process is repeated n≥20 times; the initial position X0 of the position sense assessment process can be in the range of [-25cm, 25cm], and the number of repetitions n≥5; the initial rotation angle θ0 of the vestibular sense can be selected in the range of [-18°, 18°], and the number of repetitions n≥5 of the vestibular sense assessment process; the number of repetitions n≥5 of the weight-bearing sense assessment process.
[0152] That is to say, the embodiment of the present application uses body representation to realize the evaluation of the virtual avatar in the virtual reality scene used for rehabilitation. The body representation evaluation used in the present application includes the evaluation of vision, position sense, vestibular sense and weight sense. Before the evaluation, it is necessary to induce the reshaping of the user's body representation in a certain virtual reality scene, and then the evaluation of each sense of the body representation can be performed.
[0153] The embodiments of the present application can evaluate the virtual avatars of different virtual online scenes. For example, there is currently a virtual scene A containing a virtual avatar, and a new virtual scene B containing a virtual avatar is about to be built. The present application can be used to detect whether scene B is better than scene A. The embodiments of the present application propose for the first time to use body representation to evaluate virtual avatars in virtual reality scenes. In the embodiments of the present application, vestibular sense (frame-bar illusion) and gravity sense (illusion of large and small heavy objects) are used for the first time to quantitatively evaluate body representation. The embodiments of the present application evaluate the constructed virtual scenes for the first time, and the evaluation objects are the virtual avatars in the virtual scenes rather than human users. The embodiments of the present application propose that before conducting the evaluation, rehabilitation training tasks of rehabilitation scenes containing virtual avatars need to be carried out to induce reshaping of body representations.
[0154] The embodiments of the present application can obtain the effectiveness of the virtual avatar in the rehabilitation scene at the neural mechanism level through quantitative evaluation of body representation. The method proposed in the present application can use the body representation matrix score to evaluate the immersion and effectiveness of the virtual reality system, and the evaluation can be used to select virtual avatars in existing virtual scenes and test virtual avatars in new virtual reality rehabilitation scenes.
[0155] The present application also provides an evaluation device for a virtual avatar in a virtual reality rehabilitation scene for executing all or part of the evaluation method for a virtual avatar in a virtual reality rehabilitation scene, see Fig.14 , the evaluation device for the virtual avatar in the virtual reality rehabilitation scene includes:
[0156] The test task generation module 10 is used to select a virtual reality target evaluation scene and a body representation type containing a virtual avatar according to the rehabilitation training target, and generate a test task corresponding to the rehabilitation training target based on the selected target evaluation scene and body representation type.
[0157] The evaluation module 20 performs the test task corresponding to the rehabilitation training target within a predetermined time after the user under test performs the rehabilitation training task based on the rehabilitation training target in the virtual reality rehabilitation scene containing the virtual avatar. In the process of performing the test task, the user under test is subjected to sensory stimulation matching the body representation type corresponding to the test task and the behavior feedback data of the user under test is collected in real time. Based on the behavior feedback data, a body representation evaluation method matching the body representation type corresponding to the test task is used to generate a body representation evaluation index. In an embodiment of the present invention, the evaluation module may further include sub-modules such as a tactile evaluation module, a position sense evaluation module, a vestibular sense evaluation module, and a weight-bearing sense evaluation module, which are used to evaluate the body representation types of tactile, position sense, vestibular sense, and weight-bearing sense, respectively, and obtain corresponding evaluation indicators.
[0158] The virtual avatar evaluation module 30 is used to obtain evaluation results for the virtual avatar based on multiple body representation evaluation indicators generated in the body representation evaluation step by respectively executing multiple test tasks generated according to multiple rehabilitation training goals, wherein the multiple test tasks correspond to different body representation types.
[0159] The embodiment of the device for evaluating a virtual avatar in a virtual reality rehabilitation scenario provided in the present application can be specifically used to execute the processing flow of the embodiment of the method for evaluating a virtual avatar in a virtual reality rehabilitation scenario in the above-mentioned embodiment. Its functions will not be described in detail here, and reference can be made to the detailed description of the embodiment of the method for evaluating a virtual avatar in a virtual reality rehabilitation scenario.
[0160] The part of the evaluation device for the virtual avatar in the virtual reality rehabilitation scene that performs the evaluation of the virtual avatar in the virtual reality rehabilitation scene can be executed in the server or completed in the client device. The specific selection can be based on the processing power of the client device and the limitations of the user's usage scenario. This application is not limited to this. If all operations are completed in the client device, the client device may also include a processor for specific processing of the evaluation of the virtual avatar in the virtual reality rehabilitation scene.
[0161] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.
[0162] From the above description, it can be seen that the evaluation device for virtual avatars in virtual reality rehabilitation scenarios provided in the embodiments of the present application can realize the testing and evaluation of virtual reality rehabilitation scenarios, and can effectively improve the reliability and effectiveness of the virtual avatar testing process in virtual reality rehabilitation scenarios, and then can optimize the design of multi-channel stimulation methods in rehabilitation scenarios, thereby improving the role of virtual avatars in rehabilitation training.
[0163] In one example of the present application, according to the actual situation of stroke patients and based on the evaluation results of the virtual avatar, the research on the body ownership and self-body positioning components of the virtual avatar can be focused on in terms of embodiment.
[0164] The present application also provides an evaluation system for a virtual avatar in a virtual reality rehabilitation scene, which includes the evaluation device for a virtual avatar in a virtual reality rehabilitation scene. The system specifically includes the following contents:
[0165] An evaluation device for a virtual avatar in a virtual reality rehabilitation scene, and a virtual reality device, a vibration stimulation device for vibrotactile stimulation, and a virtual reality interactive device, which are respectively connected to the evaluation device for the virtual avatar in the virtual reality rehabilitation scene in a communication manner; wherein the vibration stimulation device for vibrotactile stimulation and the virtual reality interactive device are respectively connected to the virtual reality device in a communication manner;
[0166] The device for evaluating a virtual avatar in a virtual reality rehabilitation scene is used to execute all or part of the method for evaluating a virtual avatar in a virtual reality rehabilitation scene mentioned in the above embodiment;
[0167] The virtual reality equipment includes: a VR helmet;
[0168] The virtual reality interaction device includes: a VR handle;
[0169] The vibration stimulation device for performing vibration tactile stimulation includes: a vibration tactile module patch or a vibration device arranged on the VR handle.
[0170] An embodiment of the present application also provides an electronic device, which may include a processor and a memory, wherein the processor is used to execute the method for evaluating a virtual avatar in a virtual reality rehabilitation scenario mentioned in the above embodiment, wherein the processor and the memory may be connected via a bus or other means, taking connection via a bus as an example.
[0171] The electronic device may further include a transceiver unit, which may include a receiver and a transmitter.
[0172] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the evaluation method of the virtual avatar in the virtual reality rehabilitation scene are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.
[0173] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements a method for evaluating a virtual avatar in a virtual reality rehabilitation scene.
[0174] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is specifically performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.
[0175] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.
[0176] In the present application, features described and / or illustrated for one embodiment may be used in the same manner or in a similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.
[0177] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the embodiments of the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A method for evaluating a virtual avatar in a virtual reality rehabilitation scenario, characterized in that: include: A test task generating step, selecting a virtual reality target evaluation scene and a body representation type containing a virtual avatar according to the rehabilitation training target, and generating a test task corresponding to the rehabilitation training target based on the selected target evaluation scene and body representation type; a body representation evaluation step, after the user under test performs a rehabilitation training task based on the rehabilitation training target in a virtual reality rehabilitation scene containing the virtual avatar, performing a test task corresponding to the rehabilitation training target within a predetermined time, performing sensory stimulation matching the body representation type corresponding to the test task on the user under test during the execution of the test task, collecting behavior feedback data of the user under test in real time, and generating a body representation evaluation index based on the behavior feedback data using a body representation evaluation method matching the body representation type corresponding to the test task; An evaluation result for the virtual avatar is obtained based on multiple body representation evaluation indicators generated in the body representation evaluation step by respectively executing multiple test tasks generated according to multiple rehabilitation training goals, wherein the multiple test tasks correspond to different body representation types.
2. The method according to claim 1, characterized in that The method further comprises: The type of body representation in each test task was one of the following types of body representation: touch, position sense, vestibular sense, and weight sense; Accordingly, the body representation evaluation method in each body representation evaluation step is an evaluation method for the sense of touch, the sense of position, the vestibular sense or the sense of weight.
3. The method according to claim 2, characterized in that If the body representation type in the current test task is tactile, then during the execution of the test task, the user under test is subjected to sensory stimulation matching the body representation type corresponding to the test task and behavioral feedback data of the user under test is collected in real time, and based on the behavioral feedback data, a body representation evaluation method matching the body representation type corresponding to the test task is used to generate a body representation evaluation index, including: Randomly displaying a visual cue mark in the virtual avatar area corresponding to the target body area in the target evaluation scene of the current test task, and vibrating and stimulating the target vibration position in the target body area of the user under test, so that the user under test judges whether his or her target vibration position is the same as the position of the visual cue mark for multiple times, and according to each judgment result, using a preset virtual reality interaction device to send corresponding tactile feedback information respectively; The tactile feedback information emitted by the user under test is collected in real time each time, and a tactile evaluation score serving as a body representation evaluation index is calculated based on the time interval between the collected tactile feedback information and the triggering time of the random vibration stimulation corresponding to the tactile feedback information.
4. The method according to claim 2, characterized in that: If the body representation type in the current test task is position sense, the body representation evaluation step further includes: Displaying the virtual avatar of the user under test in the target evaluation scene of the current test task, wherein a fixed horizontal position difference is provided between the current real body midline position of the user under test and the virtual avatar midline position of the virtual avatar in the target evaluation scene; Receiving initial horizontal position information sent by the tested user controlling the virtual reality interactive device, wherein the initial horizontal position information is the coordinate information in the horizontal direction corresponding to the initial position of the body midline of the virtual avatar currently in the target evaluation scene fed back by the tested user; Determining whether the initial horizontal position information is less than or equal to the horizontal position difference, and if so, determining that the initial positioning of the position sense has been completed; Correspondingly, in the process of executing the test task, the user under test is subjected to sensory stimulation matching the body representation type corresponding to the test task and the behavior feedback data of the user under test is collected in real time, and a body representation evaluation index is generated based on the behavior feedback data using a body representation evaluation method matching the body representation type corresponding to the test task, including: Displaying the virtual avatar of the user under test in the target evaluation scene of the current test task, so that the user under test controls the virtual reality interaction device to send multiple times the horizontal position information of the body midline corresponding to the virtual avatar in the target evaluation scene; The horizontal position information sent by the tested user as behavioral feedback data each time is collected in real time. If each piece of the horizontal position information is less than or equal to the horizontal position difference, each piece of the horizontal position information is retained, and a position sense evaluation score as a body representation evaluation indicator is calculated based on the retained horizontal position information.
5. The method according to claim 2, characterized in that: If the body representation type in the current test task is vestibular sense, the body representation evaluation step further includes: Displaying the virtual avatar of the user under test in the target evaluation scene of the current test task, wherein a fixed angle is formed between the current real gravity direction of the user under test and the visual gravity direction of the virtual avatar in the target evaluation scene; Receiving initial visual gravity direction information sent by the tested user controlling the virtual reality interaction device, wherein the initial visual gravity direction information is angle information corresponding to the initial visual gravity direction of the virtual avatar currently in the target evaluation scene fed back by the tested user; Determining whether the initial visual gravity direction information is less than or equal to the angle, and if so, determining that the initial positioning of the vestibular sense has been completed; Correspondingly, in the process of executing the test task, the user under test is subjected to sensory stimulation matching the body representation type corresponding to the test task and the behavior feedback data of the user under test is collected in real time, and a body representation evaluation index is generated based on the behavior feedback data using a body representation evaluation method matching the body representation type corresponding to the test task, including: Displaying the virtual avatar of the user under test in the target evaluation scene of the current test task, so that the user under test controls the virtual reality interaction device to send out visual gravity direction information corresponding to the virtual avatar in the target evaluation scene multiple times; The visual gravity direction information emitted by the measured user each time is collected in real time. If each piece of visual gravity direction information is less than or equal to the angle, each piece of visual gravity direction information is retained, and a vestibular sense evaluation score serving as a body representation evaluation indicator is calculated based on the retained visual gravity direction information.
6. The method according to claim 2, characterized in that If the body representation type in the current test task is weight-bearing sensation, a sensory stimulation matching the body representation type corresponding to the test task is performed on the user under test during the execution of the test task, and behavioral feedback data of the user under test is collected in real time. Based on the behavioral feedback data, a body representation evaluation method matching the body representation type corresponding to the test task is used to generate a body representation evaluation index, including: Displaying a first object and a second object of the same shape and weight in a target evaluation scene of a current test task, and the volume of the first object is larger than that of the second object, so that the user under test can pull the first object and the second object to a preset height multiple times in the target evaluation scene based on the virtual reality interaction device; The grip strength information, the pulling start time, and the pulling end time of the user pulling the first object and the second object are collected based on the virtual reality interaction device, and a weight-bearing perception evaluation score as a body representation evaluation indicator is obtained based on the collected data.
7. The method according to any one of claims 2 to 6, characterized in that: The multiple body representation evaluation indicators include: a tactile evaluation score, a position sense evaluation score, a vestibular sense evaluation score, and a weight-bearing sense evaluation score; The evaluation results for the virtual avatar are obtained by respectively executing the multiple test tasks generated according to the multiple rehabilitation training objectives and generating the multiple body representation evaluation indicators in the body representation evaluation step, including: A scoring data matrix is generated based on the tactile evaluation score, the position sense evaluation score, the vestibular sense evaluation score, and the weight-bearing sense evaluation score, and the scoring data matrix is used as the evaluation result of the virtual avatar in the tested virtual reality rehabilitation scene.
8. A device for evaluating a virtual avatar in a virtual reality rehabilitation scene, comprising a processor, a memory, and a computer program stored in the memory, characterized in that: The processor is used to execute the computer program. When the computer program is executed, the device implements the steps of the method according to any one of claims 1 to 7.
9. A virtual avatar evaluation system in a virtual reality rehabilitation scenario, characterized in that: include: An evaluation device for a virtual avatar in a virtual reality rehabilitation scene, and a virtual reality device, a vibration stimulation device for vibrotactile stimulation, and a virtual reality interactive device, which are respectively connected to the evaluation device for the virtual avatar in the virtual reality rehabilitation scene in communication with each other; wherein the vibration stimulation device for vibrotactile stimulation and the virtual reality interactive device are respectively connected to the virtual reality device in communication with each other; The evaluation device for the virtual avatar in the virtual reality rehabilitation scene is used to execute the evaluation method for the virtual avatar in the virtual reality rehabilitation scene according to any one of claims 1 to 7; The virtual reality equipment includes: a VR helmet; The virtual reality interaction device includes: a VR handle; The vibration stimulation device for performing vibration tactile stimulation includes: a vibration tactile module patch or a vibration device arranged on the VR handle.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Hearing and speech rehabilitation device and method, electronic device and storage medium
CN110782962A
Parkinson's disease mild cognitive impairment diagnosis and rehabilitation system and method based on VR
CN113160930A
Fitness scheme recommendation method and device based on strength fitness mirror
CN115881271A
Rehabilitation training system based on virtual reality
CN115985461A
Parkinson's disease dyskinesia assessment method and rehabilitation integrated device
CN117174248A
Cited By
Home-based care autonomous intelligent robot system and construction method
CN119188785A
Rehabilitation training method and device for adjusting cortical muscle coherence based on body feeling of remolding tool
CN119580944A
Complex constraint-considered multi-chip mounter CT optimization method and system, and storage medium
CN120974780A
Multi-chip CT optimization method and system considering complex constraints, storage medium
CN120974780B
Virtual reality enhanced feedback processing method and system for vestibular rehabilitation training
CN121411620A
