Evaluation Method, Device and System for Virtual Avatar in Virtual Reality Rehabilitation Scenario

Abstract

The present application provides a method, device and system for evaluating a virtual avatar in a virtual reality rehabilitation scenario. The method includes: selecting a virtual reality target evaluation scenario containing a virtual avatar and a type of body representation according to a rehabilitation training target, and generating a corresponding test task; a body representation evaluation step, after the user to be tested performs a rehabilitation training task in a virtual reality rehabilitation scenario containing the virtual avatar based on the rehabilitation training target, performing the test task corresponding to the rehabilitation training target within a predetermined time, performing sensory stimulation on the user to be tested and collecting real-time behavioral feedback data of the user to be tested, and generating a body representation evaluation index based on the behavioral feedback data by using a body representation evaluation method matching the type of body representation corresponding to the test task; obtaining an evaluation result for the virtual avatar based on a plurality of body representation evaluation indexes generated in the body representation evaluation step by respectively performing a plurality of test tasks generated according to a plurality of rehabilitation training targets.

Description

Technical Field

[0001] This application relates to the field of virtual reality technology, and particularly to an evaluation method, device, and system for virtual avatars in virtual reality rehabilitation scenarios. Background Art

[0002] Virtual reality (VR) technology can simulate various life scenarios in a virtual reality scene and achieve the free separation and integration of multi-channel signals. For people with rehabilitation training needs such as stroke patients, using virtual reality technology can assist patients in performing more efficient, convenient, and highly secure rehabilitation training. Based on virtual reality rehabilitation training, rehabilitation training can be carried out by the trainee controlling the actions of a virtual avatar (AvAtar) in virtual reality. Among them, the virtual avatar is also called a virtual digital human, that is, a digital human image in virtual reality. Compared with traditional rehabilitation training methods, rehabilitation training based on virtual avatars in virtual reality scenarios is a rehabilitation training method that has received increasing attention. Currently, there are already many virtual reality-based rehabilitation training methods. For example, a virtual reality rehabilitation training method with a quantitative evaluation function can achieve personalized selection of virtual scenarios and functional quantitative evaluation through the user's degree of interest. Another example is a rehabilitation training method that combines virtual reality games and intention recognition, which can obtain the user's training progress, determine the adjustment plan for the current level, and adjust the current training level to the target training level based on the adjustment plan to continue rehabilitation training.

[0003] However, no matter which existing virtual reality-based rehabilitation training method is used, it is selected or adjusted through the user's degree of interest or training progress after the virtual reality rehabilitation scenario has been put into use. There is a lack of evaluation of the effect of the virtual avatar during the virtual avatar design stage in the virtual reality rehabilitation scenario. The virtual avatar is an important part of the virtual scene. If the role of the virtual avatar in rehabilitation training cannot be accurately evaluated, it will be difficult to effectively optimize virtual reality-based rehabilitation training, and thus it will be difficult to enhance the role of the virtual avatar in rehabilitation training.

[0004] Therefore, there is an urgent need for an evaluation method for virtual avatars in virtual reality rehabilitation scenarios, as well as corresponding devices and systems. Summary of the Invention

[0005] In view of this, the embodiments of the present application provide an evaluation method, device, and system for virtual avatars in virtual reality rehabilitation scenarios to eliminate or improve one or more defects existing in the prior art.

[0006] One aspect of the present application provides an evaluation method for virtual avatars in virtual reality rehabilitation scenarios, including:

[0007] Test task generation step: Select a virtual reality target evaluation scenario with a virtual avatar and a type of body representation according to the rehabilitation training goal, and generate a test task corresponding to the rehabilitation training goal based on the selected target evaluation scenario and type of body representation;

[0008] Body representation evaluation step: After the user to be tested performs a rehabilitation training task in a virtual reality rehabilitation scenario with the virtual avatar based on the rehabilitation training goal, execute the test task corresponding to the rehabilitation training goal within a predetermined time. During the execution of the test task, perform sensory stimulation matching the type of body representation corresponding to the test task on the user to be tested and collect the user's behavioral feedback data in real time, and generate a body representation evaluation index based on the behavioral feedback data using a body representation evaluation method matching the type of body representation corresponding to the test task;

[0009] Obtain an evaluation result for the virtual avatar based on multiple body representation evaluation indexes generated in the body representation evaluation step by respectively executing multiple test tasks generated according to multiple rehabilitation training goals, wherein the multiple test tasks correspond to different types of body representation.

[0010] In some embodiments of the present application, the type of body representation in each test task is one of the following types of body representation: tactile sense, position sense, vestibular sense, and weight-bearing sense;

[0011] Correspondingly, the body representation evaluation method in each body representation evaluation step is an evaluation method for the tactile sense, the position sense, the vestibular sense, or the weight-bearing sense.

[0012] In some embodiments of the present application, the predetermined time is 20 seconds to 2 minutes.

[0013] In some embodiments of the present application, if the type of body representation in the current test task is the tactile sense, then during the execution of the test task, perform sensory stimulation matching the type of body representation corresponding to the test task on the user to be tested and collect the user's behavioral feedback data in real time, and generate a body representation evaluation index based on the behavioral feedback data using a body representation evaluation method matching the type of body representation corresponding to the test task, including:

[0014] Randomly display visual cue signs in the virtual avatar area corresponding to the target body area in the target evaluation scenario of the current test task, and perform random vibration stimulation on the target vibration position in the target body area of the user to be tested, so that the user to be tested repeatedly judges whether the target vibration position of himself / herself is the same as the position where the visual cue sign is located, and according to the judgment result of each time, use a preset virtual reality interaction device to respectively send corresponding tactile feedback information;

[0015] Collect the tactile feedback information sent by the user to be measured in real time, and calculate the tactile evaluation score as the body representation evaluation index based on the time interval between the time of the collected tactile feedback information and the trigger time of the corresponding random vibration stimulus of the tactile feedback information.

[0016] In some embodiments of the present application, if the body representation type in the current test task is proprioception, the body representation evaluation step further includes:

[0017] Display the virtual avatar of the user to be measured in the target evaluation scenario of the current test task, wherein there is a fixed horizontal position difference between the current real body midline position of the user to be measured and the virtual avatar midline position of the virtual avatar in the target evaluation scenario;

[0018] Receive the initial horizontal position information sent by the user to be measured by controlling the virtual reality interaction device, where the initial horizontal position information is the coordinate information corresponding to the initial position of the body midline of the virtual avatar currently in the target evaluation scenario as feedback by the user to be measured in the horizontal direction;

[0019] Judge whether the initial horizontal position information is less than or equal to the horizontal position difference. If so, it is determined that the initial positioning for the proprioception is completed;

[0020] Correspondingly, during the execution of the test task, perform sensory stimulation matching the body representation type corresponding to the test task on the user to be measured and collect the behavioral feedback data of the user to be measured in real time, and generate a body representation evaluation index based on the behavioral feedback data using a body representation evaluation method matching the body representation type corresponding to the test task, including:

[0021] Display the virtual avatar of the user to be measured in the target evaluation scenario of the current test task, so that the user to be measured controls the horizontal position information of the current body midline corresponding to the virtual avatar sent by the virtual reality interaction device multiple times;

[0022] Collect the horizontal position information as the behavioral feedback data sent by the user to be measured each time in real time. If each horizontal position information is less than or equal to the horizontal position difference, retain each horizontal position information, and calculate the proprioception evaluation score as the body representation evaluation index based on the retained horizontal position information.

[0023] In some embodiments of the present application, if the body representation type in the current test task is vestibular sense, the body representation evaluation step further includes:

[0024] Display the virtual avatar of the user under test in the target evaluation scenario of the current test task, where there is a fixed included angle between the current true gravity direction of the user under test and the visual gravity direction of the virtual avatar in the target evaluation scenario;

[0025] Receive the initial visual gravity direction information sent by the user under test controlling the virtual reality interaction device, where the initial visual gravity direction information is the angle information corresponding to the initial visual gravity direction of the virtual avatar of the user under test in the target evaluation scenario feedback by the user under test;

[0026] Determine whether the initial visual gravity direction information is less than or equal to the included angle. If so, determine that the initial positioning for the vestibular sense has been completed;

[0027] Correspondingly, during the execution of the test task, perform sensory stimuli matching the body representation type corresponding to the test task on the user under test and collect the behavioral feedback data of the user under test in real time, and generate body representation evaluation indicators based on the behavioral feedback data using the body representation evaluation method matching the body representation type corresponding to the test task, including:

[0028] Display the virtual avatar of the user under test in the target evaluation scenario of the current test task, so that the user under test controls the visual gravity direction information corresponding to the virtual avatar in the target evaluation scenario sent by the virtual reality interaction device multiple times;

[0029] Collect the visual gravity direction information sent by the user under test each time in real time. If each visual gravity direction information is less than or equal to the included angle, retain each visual gravity direction information, and calculate the vestibular sense evaluation score as the body representation evaluation indicator based on the retained visual gravity direction information.

[0030] In some embodiments of the present application, if the body representation type in the current test task is the sense of load, during the execution of the test task, perform sensory stimuli matching the body representation type corresponding to the test task on the user under test and collect the behavioral feedback data of the user under test in real time, and generate body representation evaluation indicators based on the behavioral feedback data using the body representation evaluation method matching the body representation type corresponding to the test task, including:

[0031] Display a first object and a second object with the same shape and weight in the target evaluation scenario of the current test task, and the volume of the first object is greater than that of the second object, so that the user under test pulls the first object and the second object to a preset height in the target evaluation scenario based on the virtual reality interaction device respectively multiple times;

[0032] Based on the virtual reality interaction device, the grip force information, the pulling start time, and the pulling end time when the user under test pulls the first object and the second object respectively are collected, and a weight perception evaluation score as a body representation evaluation index is obtained based on the collected data.

[0033] In some embodiments of the present application, the multiple body representation evaluation indexes include: tactile evaluation score, position sense evaluation score, vestibular sense evaluation score, and weight perception evaluation score;

[0034] The evaluation result for the virtual avatar obtained based on the multiple body representation evaluation indexes generated in the body representation evaluation step by respectively executing multiple test tasks generated according to multiple rehabilitation training goals includes:

[0035] A scoring data matrix is generated based on the tactile evaluation score, position sense evaluation score, vestibular sense evaluation score, and weight perception evaluation score, and the scoring data matrix is used as the evaluation result of the virtual avatar in the virtual reality rehabilitation scenario under test.

[0036] Another aspect of the present application provides an evaluation device for a virtual avatar in a virtual reality rehabilitation scenario, including: a processor, a memory, and a computer program stored on the memory, where the processor is used to execute the computer program, and when the computer program is executed, the device implements the steps of the method described above.

[0037] A third aspect of the present application provides an evaluation system for a virtual avatar in a virtual reality rehabilitation scenario, including: an evaluation device for a virtual avatar in a virtual reality rehabilitation scenario, and a virtual reality device, a vibration stimulation device for performing vibration tactile stimulation, and a virtual reality interaction device that are respectively communicatively connected to the evaluation device for a virtual avatar in a virtual reality rehabilitation scenario; wherein, the vibration stimulation device for performing vibration tactile stimulation and the virtual reality interaction device are respectively communicatively connected to the virtual reality device;

[0038] The evaluation device for a virtual avatar in a virtual reality rehabilitation scenario is used to execute the evaluation method for a virtual avatar in the virtual reality rehabilitation scenario described above;

[0039] The virtual reality device includes: a VR helmet;

[0040] The virtual reality interaction device includes: a VR handle;

[0041] The vibration stimulation device for performing vibration tactile stimulation includes: a vibration tactile module patch or a vibration device provided on the VR handle.

[0042] The fourth aspect of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the evaluation method of the virtual avatar in the virtual reality rehabilitation scenario is implemented.

[0043] The fifth aspect of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, the evaluation method of the virtual avatar in the virtual reality rehabilitation scenario is implemented.

[0044] The evaluation method of the virtual avatar in the virtual reality rehabilitation scenario provided by the present application can obtain the evaluation data of the virtual avatar in the virtual reality scenario according to the behavior feedback data of the user to be tested in each virtual reality rehabilitation scenario, can realize the test and evaluation of the virtual avatar in the virtual reality rehabilitation scenario, and can effectively improve the reliability and effectiveness of the test process of the virtual avatar in the virtual reality rehabilitation scenario. Furthermore, it can optimize the design of the multi-channel stimulation method in the rehabilitation scenario, thereby improving the role of the virtual avatar in the rehabilitation training.

[0045] The additional advantages, objectives, and features of the present application will be partially described below, and will become partially obvious to those of ordinary skill in the art after studying the following text, or can be learned according to the practice of the present application. The objectives and other advantages of the present application can be achieved and obtained through the structure specifically pointed out in the specification and the drawings.

[0046] Those skilled in the art will understand that the objectives and advantages that can be achieved by the present application are not limited to the above specific descriptions, and the above and other objectives that the present application can achieve will be more clearly understood according to the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application, but do not limit the present application. In the drawings:

[0048] Figure 1 It is a schematic flow chart of the evaluation method of the virtual avatar in the virtual reality rehabilitation scenario in an embodiment of the present application.

[0049] Figure 2 It is the first schematic flow chart of the body representation evaluation step in the evaluation method of the virtual avatar in the virtual reality rehabilitation scenario in an embodiment of the present application.

[0050] Figure 3 It is the second schematic flow chart of the body representation evaluation step in the evaluation method of the virtual avatar in the virtual reality rehabilitation scenario in an embodiment of the present application.

[0051] Figure 4It is the third process schematic diagram of the body representation evaluation step in the evaluation method of the virtual avatar in the virtual reality rehabilitation scenario in an embodiment of the present application.

[0052] Figure 5 It is the fourth process schematic diagram of the body representation evaluation step in the evaluation method of the virtual avatar in the virtual reality rehabilitation scenario in an embodiment of the present application.

[0053] Figure 6 It is the fifth process schematic diagram of the body representation evaluation step in the evaluation method of the virtual avatar in the virtual reality rehabilitation scenario in an embodiment of the present application.

[0054] Figure 7 It is the process schematic diagram of step 300 in the evaluation method of the virtual avatar in the virtual reality rehabilitation scenario in an embodiment of the present application.

[0055] Figure 8 It is the function schematic diagram of the evaluation method of the virtual avatar in the virtual reality rehabilitation scenario in an embodiment of the present application.

[0056] Figure 9 It is the example schematic diagram of the virtual ball in the virtual scene in an embodiment of the present application.

[0057] Figure 10 It is the example schematic diagram of the patch position of the vibrotactile module in the real scene in an embodiment of the present application.

[0058] Figure 11 It is the schematic diagram of the position sense evaluation in an embodiment of the present application.

[0059] Figure 12 It is the schematic diagram of the vestibular sense evaluation in an embodiment of the present application.

[0060] Figure 13 It is the schematic diagram of the load sense evaluation in an embodiment of the present application.

[0061] Figure 14 It is the structural schematic diagram of the evaluation device of the virtual avatar in the virtual reality rehabilitation scenario in an embodiment of the present application. Detailed implementation manners

[0062] To make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below in combination with the implementation manners and the drawings. Herein, the illustrative implementation manners of the present application and their descriptions are used to explain the present application, but do not limit the present application.

[0063] Here, it should also be noted that in order to avoid obscuring the present application with unnecessary details, only the structures and / or processing steps closely related to the solution according to the present application are shown in the drawings, while other details less relevant to the present application are omitted.

[0064] It should be emphasized that the term "comprising / including" when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0065] Hereinafter, embodiments of the present application will be described with reference to the drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0066] To solve the problem that the existing rehabilitation training method based on virtual reality technology lacks an evaluation method for virtual avatars in virtual reality rehabilitation scenarios, embodiments of the present application respectively provide an evaluation method for virtual avatars in virtual reality rehabilitation scenarios, an evaluation device for virtual avatars in virtual reality rehabilitation scenarios for executing the method, and an evaluation system for virtual avatars in virtual reality rehabilitation scenarios. After the user to be tested performs a rehabilitation training task in a virtual reality rehabilitation scenario containing a virtual avatar corresponding to the user's rehabilitation training goal, further, in the test task in the virtual reality target evaluation scenario corresponding to the user's rehabilitation training goal, sensory stimuli matching the type of body representation set in the test task are applied to obtain user behavior feedback data, and a corresponding body representation evaluation is performed based on the user behavior feedback data. By using different sensory stimuli for different test tasks for different rehabilitation training goals to obtain user behavior feedback data corresponding to different stimuli to obtain multiple body representation evaluations, and further generating a virtual avatar evaluation result based on the results of multiple body representation evaluations, so as to realize the test and evaluation of virtual avatars in virtual reality rehabilitation scenarios. Further, the multi-sensory channel stimulation method in the rehabilitation scenario can be optimized based on the evaluation result of the virtual avatar, so as to effectively improve the role of the virtual avatar in rehabilitation training.

[0067] Body representation is the self-awareness of humans that generates the cognition of the body and the environment through the integration of various sensory signals. Body representation represents the re-encoding or mapping of all perceived things by the human brain, that is, the individual brain forms a body representation matrix through the representation or re-encoding mapping of the body and its surrounding environment. Body representation is further divided into body image and body schema.

[0068] The body schema refers to the cognitive management of the distribution of various available functions that self-awareness has for currently controllable objects (such as the body, parts of the limbs, or tools of equipment). It is easily affected by sensory input and undergoes bottom-up updates and reconstructions. For example, a person who has never used scissors can quickly learn this new skill. The body image refers to a relatively fixed top-down conscious cognitive management of the carrier object (the body or parts of the limbs) formed by self-awareness over a long period. It is less susceptible to temporary sensory input compared to the body schema. For example, a healthy person firmly believes that they have two hands, whether they can currently see them (such as in the dark) or feel them (such as temporary limb numbness). The plasticity of the brain allows an interesting phenomenon to occur: the body schema formed by long-term stable sensory input will gradually transform into a relatively fixed body image. For example, the so-called "practice makes perfect," "know like the palm of one's hand," and "handle with ease" formed by long-term use of specific tools are actually manifestations of this transformation. Similarly, the "learned disuse" formed by patients after stroke in a state of long-term specific limb motor dysfunction - gradually forgetting the movement patterns that have long been stable in their body image - is also a manifestation of the transformation of a "disused" body schema into a body image. This transformation based on brain plasticity is the remodeling of body representation. Remodeling body representation is helpful for the recovery of the motor system. In the embodiments of the present invention, the virtual avatar evaluation result is obtained by evaluating the body representation.

[0069] See Figure 1 , the evaluation method of the virtual avatar in the virtual reality rehabilitation scenario provided by the embodiments of the present application includes the following steps:

[0070] Step S100: Select a virtual reality target evaluation scenario containing a virtual avatar and a type of body representation according to the rehabilitation training goal, and generate a test task corresponding to the rehabilitation training goal based on the selected target evaluation scenario and type of body representation.

[0071] This step is the test task generation step. The test task can be pre-generated and stored based on the rehabilitation training goal. When the generated test task is subsequently executed, sensory stimuli matching the type of body representation corresponding to the test task can be applied to the user being tested in the virtual reality target evaluation scenario, and the behavioral feedback data of the user being tested can be collected in real time. Based on the behavioral feedback data, a body representation evaluation index is generated using a body representation evaluation method matching the type of body representation corresponding to the test task.

[0072] In some optional embodiments of the present invention, the type of body representation selected according to the rehabilitation training goal may, for example, include at least one of the following types: tactile sense, proprioception, vestibular sense, and weight-bearing sense. These types of body representation are only examples, and the present invention is not limited thereto.

[0073] Step S200, the physical representation evaluation step. After the user under test performs a rehabilitation training task in a virtual reality rehabilitation scenario containing the virtual avatar based on the rehabilitation training goal, a test task corresponding to the rehabilitation training goal is performed within a predetermined time. During the execution of the test task, sensory stimulation matching the physical representation type corresponding to the test task is applied to the user under test, and the behavioral feedback data of the user under test is collected in real time. Based on the behavioral feedback data, a physical representation evaluation index is generated using a physical representation evaluation method matching the physical representation type corresponding to the test task.

[0074] In the embodiment of the present invention, after performing a rehabilitation training task for a specific rehabilitation training goal, a test task generated based on the same rehabilitation training goal is performed within a very short time, such as within 10 seconds to 2 minutes, which is equivalent to immediate execution. Among them, the rehabilitation training task is used to induce the reshaping of the user's physical representation in a certain virtual reality scenario before the physical representation evaluation, and then the sensory evaluation of the physical representation is carried out. The reshaping of the physical representation facilitates the patient to connect the experience in the virtual reality scenario with the real scenario, thereby improving the rehabilitation efficiency and enhancing the rehabilitation effect. The physical representation induced in a specific virtual reality scenario can only be maintained for 1 - 2 minutes after leaving the scenario. Therefore, in the embodiment of the present invention, the test task is immediately performed after the rehabilitation training task to evaluate the representation characteristics.

[0075] In the embodiment of the present invention, the rehabilitation training tasks in the virtual reality rehabilitation scenario are usually pre - established in the virtual reality rehabilitation training system. Different rehabilitation training tasks corresponding to different rehabilitation training goals can introduce single - sensory stimulation (such as tactile, auditory, or visual stimulation, etc.), or multi - sensory combined stimulation, that is, multiple sensory stimulations are introduced simultaneously to perform the rehabilitation training task. In other words, in the rehabilitation training task, the user under test is subjected to sensory stimulation corresponding to the rehabilitation training goal and matching at least one feature in the multi - sensory channel characteristics (characteristics for sensory stimulation generated by multiple sensory stimulation channels). Since in the embodiment of the present invention, the rehabilitation training tasks performed can completely be the training tasks existing in the existing virtual rehabilitation training system, they will not be elaborated here. Different test tasks corresponding to different rehabilitation training goals can introduce only single - sensory stimulation (such as tactile, auditory, or visual stimulation, etc.), but can also introduce multi - sensory combined stimulation.

[0076] In the embodiment of the present invention, the rehabilitation training task and the test task based on the same rehabilitation training goal can correspond to the same or different virtual reality scenarios. For example, the rehabilitation training task can be performed in a virtual reality rehabilitation scenario, and the test task can be performed in other virtual scenarios. However, in the embodiment of the present invention, the virtual reality rehabilitation scenario corresponding to the rehabilitation training task and the virtual scenario corresponding to the test task contain the same virtual avatar.

[0077] In the physical representation evaluation step, during the execution of the test task, by applying sensory stimuli corresponding to the type of physical representation of the test task to the user being tested and collecting the behavioral feedback data of the user being tested in real time, physical representation evaluation metrics can be generated based on the behavioral feedback data using a physical representation evaluation method that matches the type of physical representation corresponding to the test task. The behavioral feedback data of the test user is generated in a virtual reality scenario containing an avatar and is directly related to the image of the avatar. Therefore, in the embodiments of the present invention, the avatar is evaluated based on the physical representation evaluation metrics obtained from the behavioral feedback data of the test user. As an example, when the types of physical representations corresponding to different test tasks are divided into tactile, position sense, vestibular sense, and weight-bearing sense, the physical representation evaluation methods that match the types of physical representations corresponding to each test task can be existing or new evaluation methods for tactile, position sense, vestibular sense, and weight-bearing sense respectively. This will be described through examples below.

[0078] Step S300, obtain the evaluation result for the avatar based on multiple physical representation evaluation metrics generated in the physical representation evaluation step by respectively executing multiple test tasks generated according to multiple rehabilitation training goals, where the multiple test tasks correspond to different types of physical representations.

[0079] Since in the physical representation evaluation step, each execution of a test task only evaluates one type of physical representation, in order to more comprehensively evaluate multiple types of physical representations, in the embodiments of the present invention, rehabilitation training tasks and test tasks are pre-formed for multiple rehabilitation training goals, and the physical representation evaluation step is executed multiple times for different rehabilitation training goals to respectively execute multiple test tasks generated according to multiple rehabilitation training goals, and thereby generate evaluation metrics for multiple physical representations of multiple different types of physical representations, and obtain the evaluation result for the avatar based on the generated physical representation evaluation metrics.

[0080] The present invention sequentially executes multiple training task - test task combinations for different physical representation characteristics until the physical representation evaluation step is completed for all physical representations in each virtual reality evaluation scenario, so as to obtain the evaluation data of the avatar in the virtual reality scenario to be tested according to the behavioral feedback data of the user being tested in each virtual reality scenario.

[0081] From the above description, it can be seen that the method for evaluating the avatar in the virtual reality rehabilitation scenario provided by the embodiments of the present application can implement the testing and evaluation of the avatar in the virtual reality rehabilitation scenario, and further can optimize the design of the multi-channel stimulation method in the rehabilitation scenario, thereby improving the role of the avatar in rehabilitation training.

[0082] See Figure 2 In an embodiment of the present invention, when the body representation type in the current test task is touch, but the applied stimulus is a multi-sensory combined stimulus including touch, in step S200, during the execution of the test task, the user to be tested is subjected to sensory stimuli matching the body representation type corresponding to the test task, and the behavioral feedback data of the user to be tested is collected in real time. Based on the behavioral feedback data, a body representation evaluation index is generated by using a body representation evaluation method matching the body representation type corresponding to the test task, including:

[0083] Step S010: In the target evaluation scenario of the current test task, vibratory tactile stimulation is applied to the target body area of the user to be tested, and the virtual reality device currently displaying the target evaluation scenario worn by the user to be tested is controlled to display the picture corresponding to the preset vibratory tactile stimulation and output the sound corresponding to the preset vibratory tactile stimulation in the target evaluation scenario, so as to realize the combined stimulation of touch, vision and audition for the user to be tested.

[0084] In an example of step S010, the visual and auditory stimuli can be realized through a VR helmet, and the tactile stimulus can be realized through a vibratory tactile module patch. At this stage, the user to be tested (the person to be evaluated) can wear the VR helmet on the head, and two identical vibratory tactile module patches are attached to the back of the user's dominant hand. Two position cue points are included at the corresponding positions of the virtual hand on the same side, and random vibratory stimulation is applied to these two position cue points respectively. At the same time, the corresponding positions of the virtual hand will be highlighted. Enter the virtual scene and start 2 minutes of random vibratory stimulation. The VR helmet feeds back the vibratory sound and picture during the stimulation.

[0085] Step S020: The feedback information sent by the user to be tested each time is collected in real time, and a body representation evaluation index is calculated based on the collected feedback information. The body representation evaluation index can be an evaluation parameter for touch, audition and vision.

[0086] It can be understood that each time the user to be tested is subjected to sensory stimuli matching the pre-selected body representation type in step S020, the body representation evaluation method adopted in the body representation evaluation is also corresponding to the body representation type. For example, when the body representation type in the test task is touch, the body representation evaluation method adopted is an evaluation method for touch, and this evaluation method can be an existing one.

[0087] In order to further improve the comprehensiveness of rehabilitation scenario selection and body representation feedback, so as to further improve the accuracy and effectiveness of virtual avatar testing in the rehabilitation scenario, in the method for evaluating a virtual avatar in the virtual reality rehabilitation scenario provided in the embodiments of the present application, the types of body representations to be evaluated may include, in addition to tactile sense, position sense, vestibular sense, and weight-bearing sense, but the present invention is not limited thereto, and may also be more or fewer types of body representations.

[0088] Correspondingly, if the body representation evaluation method corresponding to the current target evaluation scenario is an evaluation for tactile sense, then refer to Figure 3 , in the body representation evaluation step of the method for evaluating a virtual avatar in the virtual reality rehabilitation scenario, during the execution of the test task, sensory stimuli matching the type of body representation corresponding to the test task are applied to the user to be tested, and the behavioral feedback data of the user to be tested is collected in real time, and based on the behavioral feedback data, a body representation evaluation index is generated by using a body representation evaluation method matching the type of body representation corresponding to the test task, which may include the following steps:

[0089] Step S021: Randomly display a visual cue mark in the virtual avatar area corresponding to the target body area in the target evaluation scenario of the current test task, and randomly vibrate the target vibration position in the target body area of the user to be tested, so that the user to be tested judges multiple times whether the target vibration position of himself / herself is the same as the position where the visual cue mark is located, and according to the judgment result each time, corresponding tactile feedback information is respectively sent out by using a preset virtual reality interaction device.

[0090] Step S022: Collect in real time each time the user to be tested sends out tactile feedback information, and calculate a tactile evaluation score as the body representation evaluation index based on the time interval between the time of the collected tactile feedback information and the triggering time of the random vibration stimulus corresponding to the tactile feedback information.

[0091] In an example, a highlighted visual cue randomly appears at the corresponding part of the virtual hand vibration tactile module patch, the vibration tactile module patch vibrates randomly, the user judges whether the vibration part is the same as the visual cue part, and gives feedback through the handle, and this action is repeated n times.

[0092] If the type of body representation in the current test task is position sense, then refer to Figure 4 , before generating a body representation evaluation index by using a body representation evaluation method matching the type of body representation corresponding to the test task in the body representation evaluation step of the method for evaluating a virtual avatar in the virtual reality rehabilitation scenario, the following specific content is further included:

[0093] Step S001: Display the virtual avatar of the user under test in the target evaluation scenario of the current test task, where there is a fixed horizontal position difference between the current real body midline position of the user under test and the virtual avatar midline position of the virtual avatar in the target evaluation scenario.

[0094] Step S002: Receive the initial horizontal position information sent by the user under test controlling the virtual reality interaction device, where the initial horizontal position information is the coordinate information in the horizontal direction corresponding to the initial position of the body midline of the virtual avatar currently in the target evaluation scenario as feedback by the user under test.

[0095] Step S003: Determine whether the initial horizontal position information is less than or equal to the horizontal position difference. If so, it is determined that the initial positioning for the position sense has been completed.

[0096] Correspondingly, during the execution of the test task, perform sensory stimulation matching the body representation type corresponding to the test task on the user under test and collect the behavioral feedback data of the user under test in real time, and generate body representation evaluation indicators based on the behavioral feedback data using a body representation evaluation method matching the body representation type corresponding to the test task, including:

[0097] Step S023: Display the virtual avatar of the user under test in the target evaluation scenario of the current test task, so that the user under test controls the horizontal position information of the current body midline corresponding to the virtual avatar sent by the virtual reality interaction device multiple times;

[0098] Step S024: Collect the horizontal position information as the behavioral feedback data sent by the user under test each time in real time. If each of the horizontal position information is less than or equal to the horizontal position difference, retain each of the horizontal position information, and calculate the position sense evaluation indicator as the body representation evaluation indicator based on the retained horizontal position information. More specifically, such as the position sense evaluation score.

[0099] In an example, it can be first set that there is always a fixed horizontal position difference X between the real body and the horizontal body s , and then the initial positioning needs to be carried out first: The user wears a VR helmet and gives feedback on the horizontal direction coordinate information of the initial position of the body midline felt through the VR handle in the virtual scene, and defines the initial horizontal position information as X 0 . If X 0 > X s , it proves that the evaluation is incorrect and needs to be re-evaluated. Repeat the above steps; if X 0 ≤ X s , then record X 0。After initial positioning, sensory stimulation is carried out. After the sensory stimulation, the user provides feedback on the horizontal position coordinate information of the midline of the body in the virtual scene through the VR handle, and defines the horizontal position information of the i-th time as X i , and this action is repeated n times, where n is a positive integer greater than 1.

[0100] If the body representation type in the current test task is the vestibular sense, then refer to Figure 5 , before generating the body representation evaluation index by using the body representation evaluation method matching the body representation type corresponding to the test task based on the behavioral feedback data in the body representation evaluation step of the evaluation method of the virtual avatar in the virtual reality rehabilitation scene, it also includes:

[0101] Step S004: Display the virtual avatar of the user to be tested in the target evaluation scene of the current test task, where there is a fixed included angle between the current true gravity direction of the user to be tested and the visual gravity direction of the virtual avatar in the target evaluation scene;

[0102] Step S005: Receive the initial visual gravity direction information sent by the user to be tested by controlling the virtual reality interaction device, where the initial visual gravity direction information is the angle information corresponding to the initial visual gravity direction of the virtual avatar currently in the target evaluation scene feedback by the user to be tested;

[0103] Step S006: Determine whether the initial visual gravity direction information is less than or equal to the included angle. If so, it is determined that the initial positioning for the vestibular sense is completed.

[0104] Correspondingly, during the execution of the test task, sensory stimulation matching the body representation type corresponding to the test task is carried out on the user to be tested, and the behavioral feedback data of the user to be tested is collected in real time, and the body representation evaluation index is generated by using the body representation evaluation method matching the body representation type corresponding to the test task based on the behavioral feedback data, including:

[0105] Step S025: Display the virtual avatar of the user to be tested in the target evaluation scene of the current test task, so that the user to be tested controls the virtual reality interaction device to send the visual gravity direction information corresponding to the virtual avatar currently in the target evaluation scene multiple times;

[0106] Step S026: Collect the visual gravity direction information sent by the user to be tested each time in real time. If all the visual gravity direction information is less than or equal to the included angle, then retain all the visual gravity direction information, and calculate the vestibular sense evaluation score as the body representation evaluation index based on the retained visual gravity direction information.

[0107] In one example, a fixed angle θ can be set in advance between the visual gravity direction in the virtual scenario and the real gravity direction. s Then, in the initial positioning stage, the user wears a VR headset and feeds back the felt gravity direction in the virtual scenario through the VR handle. Define the initial visual gravity direction information as θ. 0 If θ 0 > θ s , it proves that the evaluation is incorrect and needs to be re-evaluated. Repeat the above steps; if θ 0 ≤ θ s , then record θ 0 . After the initial positioning, multi-sensory combined stimulation is performed. After the multi-sensory combined stimulation ends, enter the formal vestibular evaluation: The user wears a VR headset and controls the frame bar instrument through the VR handle to feedback the gravity direction felt by the evaluated person. The visual gravity direction information feedback for the i-th time is θ i , and this action is repeated n times. If θ i > θ 0 , the evaluation is incorrect and the above steps need to be repeated.

[0108] If the body representation type in the current test task is the sense of weight bearing, refer to Figure 6 , the body representation evaluation steps in the evaluation method of the virtual avatar in the virtual reality rehabilitation scenario further include:

[0109] Step S027: Display a first object and a second object with the same shape and weight in the target evaluation scenario of the current test task, and the volume of the first object is greater than that of the second object, so that the tested user pulls the first object and the second object to a preset height in the target evaluation scenario respectively multiple times based on the virtual reality interaction device.

[0110] Step S028: Collect the grip force information, the pulling start time, and the pulling end time of the tested user when pulling the first object and the second object respectively each time based on the virtual reality interaction device, and obtain the weight-bearing sense evaluation score as the body representation evaluation index based on the collected data.

[0111] In order to further improve the effectiveness and accuracy of the rehabilitation effectiveness evaluation of the tested virtual reality rehabilitation scenario group, in the embodiment of the evaluation method of the virtual avatar in the virtual reality rehabilitation scenario provided in the embodiments of the present application, refer to Figure 7 , step S300 in the evaluation method of the virtual avatar in the virtual reality rehabilitation scenario specifically includes the following content:

[0112] Step S310: Determine the judgment values respectively corresponding to each haptic feedback information for indicating whether the judgment result of the feedback action display is accurate, and determine the haptic evaluation score corresponding to the virtual reality scene based on the haptic test according to the time interval and judgment values among the haptic feedback information.

[0113] Specifically, the haptic evaluation score p 1 is calculated as follows:

[0114]

[0115] where RT i (Reaction Time) is the time interval between the time of the i-th haptic feedback information sent by the user under test and the triggering time of the random vibration stimulus corresponding to this haptic feedback information; CCE represents the cross-modal congruency effect; r represents the judgment function, where -1 and 1 both represent judgment values; in the calculation formula in one or more embodiments of the present application, n represents the corresponding number of times the user sends behavior feedback data, which can be simply referred to as the number of repetitions;

[0116] where the judgment function r is as follows:

[0117]

[0118] Step S320: Determine the position sense evaluation score corresponding to the virtual reality scene based on the position sense test according to each horizontal position information and the horizontal position difference.

[0119] Specifically, the position sense evaluation score p 2 is calculated as follows:

[0120]

[0121] Step S330: Determine the vestibular sense evaluation score corresponding to the virtual reality scene based on the vestibular sense test according to each visual gravity direction information and the included angle.

[0122] Specifically, the vestibular sense evaluation score p 3 is calculated as follows:

[0123]

[0124] Step S340: Determine the load sense evaluation score corresponding to the virtual reality scene based on the load sense test according to the grip force information, the pulling start time and the pulling end time when the user under test pulls the first object and the second object respectively each time.

[0125] Specifically, the load sense evaluation score p 4The calculation formula is as follows:

[0126]

[0127] Among them, the grip force information when pulling the large object (i.e., the first object) for the i-th time is defined as F i , and the grip force information when pulling the small object (i.e., the second object) for the i-th time is defined as f i ; the start time of the action of pulling the small object for the i-th time is t is , and the end time of the action of pulling the small object for the i-th time is t ie ; the start time of the action of pulling the large object for the i-th time is T is , and the end time of the action of pulling the large object for the i-th time is T ie .

[0128] Step S350: Generate a scoring data matrix of rehabilitation effectiveness evaluation data corresponding to the evaluation result data of the virtual avatar in the scene based on the tactile evaluation score, proprioceptive evaluation score, vestibular evaluation score, and weight-bearing sense evaluation score.

[0129] Among them, the scoring data matrix P = [p 1 , p 2 , p 3 , p 4 .

[0130] To further illustrate the evaluation method of the virtual avatar in the virtual reality rehabilitation scene provided in the above embodiments, the present application also provides a specific application example of the evaluation method of the virtual avatar in the virtual reality rehabilitation scene. In this example, it involves technical fields such as artificial intelligence medicine and neurology, rehabilitation engineering, and virtual reality. See Figure 8 , the multisensory combined stimulation module may include three stimulation channels: vision, touch, and hearing. The evaluation stage is executed in the order of tactile evaluation, proprioceptive evaluation, vestibular evaluation, and weight-bearing sense evaluation. Specifically, it is embodied as a scoring data matrix P, which includes the tactile evaluation score p 1 , the proprioceptive evaluation score p 2 , the vestibular evaluation score p 3 , and the weight-bearing sense grip force evaluation score p 4 . The body representation information of the evaluated person is P = [p 1 , p 2 , p 3 , p 4. The overall execution process of this application example is as follows: The user completes a virtual reality rehabilitation scenario rehabilitation training task (including at least tactile stimulation) associated with tactile evaluation, conducts tactile evaluation, completes a virtual reality rehabilitation scenario rehabilitation training task (including at least proprioceptive stimulation) associated with proprioceptive evaluation, conducts proprioceptive evaluation, completes a virtual reality rehabilitation scenario rehabilitation training task (including at least vestibular stimulation) associated with vestibular evaluation, conducts vestibular evaluation, completes a virtual reality rehabilitation scenario rehabilitation training task (including at least weight-bearing sensation stimulation) associated with weight-bearing sensation evaluation, conducts weight-bearing sensation evaluation, and obtains a scoring data matrix. The specific description is as follows:

[0131] (1) Each virtual reality rehabilitation scenario rehabilitation training task including a virtual avatar

[0132] The virtual reality rehabilitation scenario corresponding to each rehabilitation training task includes a virtual avatar. In each training task, only the stimuli corresponding to the type of body representation to be evaluated in the immediately following associated body representation evaluation process can be adopted, or multisensory combined stimuli can be adopted. The multisensory combined stimuli can include some or all of the visual, tactile, auditory, and other stimuli provided by using the multisensory stimulus characteristics provided by some or all of the visual, tactile, auditory, and other sensory channels. The visual and auditory stimuli can be realized through a VR headset, and the tactile stimulus can be realized through a vibrotactile module patch. In this stage, the user wears the VR headset on the head, and two identical vibrotactile module patches are attached to the back of the user's dominant hand. Two position cue points are included at the corresponding positions of the virtual hand on the same side, and random vibration stimuli are applied to these two position cue points respectively. At the same time, the corresponding positions of the virtual hand will be highlighted. Enter the virtual scene and start 2 minutes of random vibration stimulation. The VR headset feedbacks the vibration sound and picture during the stimulation.

[0133] An example of a virtual ball in the virtual scene is as Figure 9 shown, and an example of the position of the vibrotactile module patch in the real scene is as Figure 10 shown. The evaluated person refers to the user to be measured mentioned in the embodiments of this application, and can also be simply referred to as the user. The virtual scene is the abbreviation of the virtual reality rehabilitation scenario mentioned in the embodiments of this application, but is not limited to the virtual reality rehabilitation scenario, and can also be other virtual reality scenarios.

[0134] Among them, the vibrotactile module patch can also be replaced by the vibration system on the VR handle; the position of the vibrotactile module patch and its corresponding position in the virtual scene can be replaced to other positions (such as the lower arm, upper arm, shoulder, etc.); the touch of the back of the hand by the ball in the virtual scene can be replaced by the touch of the back of the hand by a brush or the touch of the back of the hand by another task in the virtual scene.

[0135] (2) Tactile assessment

[0136] Before the tactile assessment, a rehabilitation training task of a virtual reality rehabilitation scenario including an avatar associated with the tactile assessment needs to be carried out. In the virtual reality scenario, a highlighted visual cue randomly appears at the corresponding position of the virtual hand vibration tactile module patch, and the vibration tactile module patch vibrates randomly. The user judges whether the vibration position is the same as the visual cue position and gives feedback through the handle. This action is repeated n times. Define the reaction time RT i (Reaction Time) as the time interval between the moment when the user makes the i-th judgment and the moment when the touch occurs, and then calculate the tactile evaluation score p1 according to the calculation formula of the aforementioned tactile evaluation score p1 of the cross-modal consistency effect.

[0137] (3) Proprioception assessment

[0138] During the proprioception assessment, there is always a fixed horizontal position difference X between the real body and the horizontal body s . The proprioception assessment process needs to first perform an initial positioning: the user wears a VR helmet and gives feedback on the horizontal direction coordinate information of the initial position of the body midline felt in the virtual scene through the VR handle, and define the initial horizontal position information as X 0 . If X 0 > X s , it proves that the assessment is incorrect and needs to be re-assessed. Repeat the above steps of this process; if X 0 ≤X s , then record X 0 . After performing the initial positioning, a rehabilitation training task of a virtual reality rehabilitation scenario including an avatar associated with the proprioception assessment is carried out.

[0139] After the rehabilitation training task of the virtual reality rehabilitation scenario associated with the proprioception assessment is completed. The user gives feedback on the horizontal direction position coordinate information of the body midline in the virtual scene through the VR handle, and define the i-th horizontal position information as X i , and this action is repeated n times. If X i > X 0 , the assessment is incorrect and the above steps need to be repeated; if X i ≤X 0 , then calculate the proprioception assessment score p 2 based on the calculation formula of the proprioception evaluation score p 2 .

[0140] Among them, an application example of the proprioception assessment is as Figure 11 shown.

[0141] (4) Vestibular assessment

[0142] During the vestibular sense evaluation process, there is a fixed included angle θ between the visual gravity direction and the real gravity direction in the virtual scene. s In the initial positioning stage of the vestibular sense evaluation process, the user wears a VR helmet and feeds back the felt gravity direction through the VR handle in the virtual scene. Define the initial visual gravity direction information as θ. 0 If θ 0 > θ s , it proves that the evaluation is incorrect and needs to be re-evaluated. Repeat the above steps; if θ 0 ≤ θ s , then record θ 0 .

[0143] After the initial positioning, a virtual reality rehabilitation scene rehabilitation training task including a virtual avatar associated with the vestibular sense evaluation is carried out.

[0144] After the completion of this rehabilitation training task, enter the formal vestibular sense evaluation: The user wears a VR helmet and controls the frame bar instrument through the VR handle to feedback the gravity direction felt by the evaluated person. The visual gravity direction information of the i-th feedback is θ i , and this action is repeated n times. If θ i > θ 0 , then the evaluation is incorrect and the above steps need to be repeated; if θ i ≤ θ 0 , then record the vestibular sense evaluation score p 3 based on the calculation formula of the vestibular sense evaluation score p 3 .

[0145] Among them, an application example of the vestibular sense evaluation is as Figure 12 shown.

[0146] (5) Proprioceptive sense of weight evaluation

[0147] When performing the proprioceptive sense of weight evaluation immediately after the completion of a virtual reality rehabilitation training task including a virtual avatar associated with the proprioceptive sense of weight evaluation, the user needs to wear a VR helmet and pull two objects of different sizes but the same shape and weight to a specified height above the table in the virtual scene. Each object is pulled n times and the average grip force situation is fed back through the VR handle. Define the grip force information when pulling the large object for the i-th time as F i , and the grip force information when pulling the small object for the i-th time as f i ; the start time of the i-th action of pulling the small object is t is , and the end time of the i-th action of pulling the small object is t ie ; the start time of the i-th action of pulling the large object is T is , and the end time of the i-th action of pulling the large object is T ie， This action is repeated n times, and record the proprioceptive sense of weight evaluation score p 4 based on the calculation formula of the proprioceptive sense of weight evaluation score p4 。

[0148] Among them, application examples of load sensation assessment are as Figure 13 shown.

[0149] (6) All the recorded data can be fed back to the computer terminal via the Bluetooth of the handle and recorded in Excel. The feedback results include the original data and the scoring data matrix.

[0150] Among them, in one example, the number of repetitions n of the tactile assessment process is n≥20; the initial position X 0 of the position sense assessment process can take a range of [-25 cm, 25 cm], and the number of repetitions n≥5; the initial rotation angle θ 0 of the vestibular sense can be selected in the range of [-18°, 18°], and the number of repetitions n of the vestibular sense assessment process is n≥5; the number of repetitions n of the load sensation assessment process is n≥5.

[0151] That is to say, the embodiments of the present application use body representations to implement the assessment of virtual avatars in virtual reality scenarios for rehabilitation. The body representation assessments used in the present application include assessments of vision, position sense, vestibular sense, and load sensation. Before the assessment, it is necessary to first induce the remodeling of the user's body representation in a certain virtual reality scenario, and then the sensory assessments of the body representation can be carried out.

[0152] The embodiments of the present application can assess virtual avatars in different virtual online scenarios. For example, there is currently a virtual scenario A containing a virtual avatar, and a new virtual scenario B containing a virtual avatar is to be built. The present application can be used to detect whether scenario B is better than scenario A. The embodiments of the present application first propose to use body representations to evaluate virtual avatars in virtual reality scenarios. In the embodiments of the present application, the vestibular sense (frame of reference illusion) and the gravity sense (illusion of large and small weights) are first used to quantitatively evaluate the body representation. The embodiments of the present application first evaluate the built virtual scenarios, and the evaluation object is the virtual avatar in the virtual scenario rather than the human user. The embodiments of the present application propose that before the assessment, it is necessary to first perform a rehabilitation training task in a rehabilitation scenario containing a virtual avatar to induce the remodeling of the body representation.

[0153] The embodiments of the present application can obtain the effectiveness of virtual avatars in rehabilitation scenarios at the neural mechanism level through the quantitative assessment of body representations. The method proposed by the present application can use the body representation matrix score to evaluate the immersion and effectiveness of virtual reality systems. This assessment can be used for the selection of virtual avatars in existing virtual scenarios and the testing of virtual avatars in new virtual reality rehabilitation scenarios.

[0154] The present application also provides an evaluation device for a virtual avatar in a virtual reality rehabilitation scenario, which is used to execute all or part of the evaluation method for a virtual avatar in a virtual reality rehabilitation scenario. Refer to Figure 14 , the evaluation device for a virtual avatar in a virtual reality rehabilitation scenario includes:

[0155] A test task generation module 10, configured to select a virtual reality target evaluation scenario containing a virtual avatar and a body representation type according to a rehabilitation training target, and generate a test task corresponding to the rehabilitation training target based on the selected target evaluation scenario and body representation type.

[0156] An evaluation module 20, after the user to be tested executes a rehabilitation training task in a virtual reality rehabilitation scenario containing the virtual avatar based on the rehabilitation training target, executes the test task corresponding to the rehabilitation training target within a predetermined time, performs sensory stimulation matching the body representation type corresponding to the test task on the user to be tested during the execution of the test task, and collects the behavioral feedback data of the user to be tested in real time, and generates a body representation evaluation index based on the behavioral feedback data by using a body representation evaluation method matching the body representation type corresponding to the test task. In an embodiment of the present invention, the evaluation module may further include sub-modules such as a tactile evaluation module, a position sense evaluation module, a vestibular sense evaluation module, and a weight sense evaluation module, which are used to evaluate body representation types such as tactile sense, position sense, vestibular sense, and weight sense respectively, and obtain corresponding evaluation indexes.

[0157] An evaluation module 30 of the virtual avatar, configured to obtain an evaluation result for the virtual avatar based on a plurality of body representation evaluation indexes generated in the body representation evaluation step by respectively executing a plurality of test tasks generated according to a plurality of rehabilitation training targets, wherein the plurality of test tasks correspond to different body representation types.

[0158] The embodiment of the evaluation device for a virtual avatar in a virtual reality rehabilitation scenario provided by the present application can specifically be used to execute the processing flow of the embodiment of the evaluation method for a virtual avatar in a virtual reality rehabilitation scenario in the above embodiment, and its functions will not be described in detail here, and reference may be made to the detailed description of the embodiment of the evaluation method for a virtual avatar in a virtual reality rehabilitation scenario above.

[0159] The part of the evaluation device for a virtual avatar in a virtual reality rehabilitation scenario for evaluating a virtual avatar in a virtual reality rehabilitation scenario can be executed in a server or completed in a client device. Specifically, it can be selected according to the processing capacity of the client device and the limitations of the user usage scenario, etc. The present application does not make any limitations in this regard. If all operations are completed in the client device, the client device may further include a processor for specific processing of evaluating a virtual avatar in a virtual reality rehabilitation scenario.

[0160] The above-mentioned client device may have a communication module (i.e., communication unit), which can communicate and connect with a remote server to achieve data transmission with the server. The server may include a server on the side of the task scheduling center, and in other implementation scenarios, it may also include a server of an intermediate platform, such as a server of a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster composed of multiple servers, or a server structure of a distributed device.

[0161] As can be seen from the above description, the evaluation device for virtual avatars in the virtual reality rehabilitation scenario provided by the embodiments of the present application can realize the test and evaluation of the virtual reality rehabilitation scenario, effectively improve the reliability and effectiveness of the virtual avatar test process in the virtual reality rehabilitation scenario, and further optimize the design of multi-channel stimulation methods in the rehabilitation scenario, thereby improving the role of the virtual avatar in rehabilitation training.

[0162] In an example of the present application, according to the actual situation of stroke patients, based on the evaluation results of virtual avatars, the research on the sense of body ownership and self-body localization components of virtual avatars can be focused on in terms of embodiment sense.

[0163] The present application also provides an evaluation system for virtual avatars in a virtual reality rehabilitation scenario, which includes the above-mentioned evaluation device for virtual avatars in a virtual reality rehabilitation scenario. The system specifically includes the following:

[0164] An evaluation device for virtual avatars in a virtual reality rehabilitation scenario, and a virtual reality device, a vibration stimulation device for performing vibrotactile stimulation, and a virtual reality interaction device that are respectively communicatively connected to the evaluation device for virtual avatars in a virtual reality rehabilitation scenario; wherein, the vibration stimulation device for performing vibrotactile stimulation and the virtual reality interaction device are respectively communicatively connected to the virtual reality device;

[0165] The evaluation device for virtual avatars in a virtual reality rehabilitation scenario is used to execute all or part of the content of the evaluation method for virtual avatars in a virtual reality rehabilitation scenario mentioned in the foregoing embodiments;

[0166] The virtual reality device includes: a VR helmet;

[0167] The virtual reality interaction device includes: a VR handle;

[0168] The vibration stimulation device for performing vibrotactile stimulation includes: a vibrotactile module patch or a vibration device provided on the VR handle.

[0169] An embodiment of the present application also provides an electronic device, which may include a processor and a memory. The processor is configured to execute the evaluation method of the virtual avatar in the virtual reality rehabilitation scenario mentioned in the foregoing embodiment. The processor and the memory may be connected through a bus or other means. Taking the connection through the bus as an example.

[0170] The electronic device may further include a transceiver unit, and the transceiver unit may include a receiver and a transmitter.

[0171] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the foregoing evaluation method of the virtual avatar in the virtual reality rehabilitation scenario are implemented. The computer-readable storage medium may be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0172] An embodiment of the present application also provides a computer program product, including a computer program, which implements the evaluation method of the virtual avatar in the virtual reality rehabilitation scenario when executed by a processor.

[0173] Those of ordinary skill in the art should understand that the various exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Specifically, whether to execute in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application. When implemented in hardware, it may be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment may be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave on a transmission medium or a communication link.

[0174] It should be clear that the present application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present application.

[0175] In this application, features described and / or illustrated for one embodiment can be used in the same or a similar manner in one or more other embodiments, and / or combined with the features of other embodiments or replace the features of other embodiments.

[0176] The above are only the preferred embodiments of this application and are not intended to limit this application. For those skilled in the art, various changes and modifications can be made to the embodiments of this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included within the protection scope of this application.

Claims

1. A method for evaluating a virtual avatar in a virtual reality rehabilitation scenario, characterized in that: include: A test task generating step, selecting a virtual reality target evaluation scene and a body representation type containing a virtual avatar of the user under test according to the rehabilitation training target, and generating a test task corresponding to the rehabilitation training target based on the selected target evaluation scene and body representation type; a body representation evaluation step, after the user under test performs a rehabilitation training task based on the rehabilitation training target in a virtual reality rehabilitation scene containing the virtual avatar, performing a test task corresponding to the rehabilitation training target within a predetermined time, performing sensory stimulation matching the body representation type corresponding to the test task on the user under test during the execution of the test task, collecting behavior feedback data of the user under test in real time, and generating a body representation evaluation index based on the behavior feedback data using a body representation evaluation method matching the body representation type corresponding to the test task; Obtaining an evaluation result for the virtual avatar based on a plurality of body representation evaluation indicators generated in the body representation evaluation step by respectively executing a plurality of test tasks generated according to a plurality of rehabilitation training objectives, wherein the plurality of test tasks correspond to different body representation types; Among them, the body representation type in each test task is one of the following body representation types: touch, position sense, vestibular sense, and weight sense; Accordingly, the body representation evaluation method in each body representation evaluation step is an evaluation method for the sense of touch, the sense of position, the vestibular sense or the sense of weight.

2. The method according to claim 1, characterized in that The sensory stimulation that matches the body representation type corresponding to the test task is performed on the user under test during the execution of the test task, including: only the sensory stimulation that matches the body representation type corresponding to the test task is performed on the user under test during the execution of the test task; or the multi-sensory combined stimulation is performed on the user under test during the execution of the test task, and the multi-sensory combined stimulation includes sensory stimulation that matches the body representation type corresponding to the test task.

3. The method according to claim 1, characterized in that If the body representation type in the current test task is tactile, then during the execution of the test task, the user under test is subjected to sensory stimulation matching the body representation type corresponding to the test task and behavioral feedback data of the user under test is collected in real time, and based on the behavioral feedback data, a body representation evaluation method matching the body representation type corresponding to the test task is used to generate a body representation evaluation index, including: Randomly displaying a visual cue mark in the virtual avatar area corresponding to the target body area in the target evaluation scene of the current test task, and vibrating and stimulating the target vibration position in the target body area of ​​the user under test, so that the user under test judges whether his or her target vibration position is the same as the position of the visual cue mark for multiple times, and according to each judgment result, using a preset virtual reality interaction device to send corresponding tactile feedback information respectively; The tactile feedback information emitted by the user under test is collected in real time each time, and a tactile evaluation score serving as a body representation evaluation index is calculated based on the time interval between the collected tactile feedback information and the triggering time of the random vibration stimulation corresponding to the tactile feedback information.

4. The method according to claim 1, characterized in that: If the body representation type in the current test task is position sense, the body representation evaluation step further includes: Displaying the virtual avatar of the user under test in the target evaluation scene of the current test task, wherein a fixed horizontal position difference is provided between the current real body midline position of the user under test and the virtual avatar midline position of the virtual avatar in the target evaluation scene; Receiving initial horizontal position information sent by the tested user controlling the virtual reality interactive device, wherein the initial horizontal position information is the coordinate information in the horizontal direction corresponding to the initial position of the body midline of the virtual avatar currently in the target evaluation scene fed back by the tested user; Determining whether the initial horizontal position information is less than or equal to the horizontal position difference, and if so, determining that the initial positioning of the position sense has been completed; Correspondingly, in the process of executing the test task, the user under test is subjected to sensory stimulation matching the body representation type corresponding to the test task and the behavior feedback data of the user under test is collected in real time, and a body representation evaluation index is generated based on the behavior feedback data using a body representation evaluation method matching the body representation type corresponding to the test task, including: Displaying the virtual avatar of the user under test in the target evaluation scene of the current test task, so that the user under test controls the virtual reality interaction device to send multiple times the horizontal position information of the body midline corresponding to the virtual avatar in the target evaluation scene; The horizontal position information sent by the tested user as behavioral feedback data each time is collected in real time. If each piece of the horizontal position information is less than or equal to the horizontal position difference, each piece of the horizontal position information is retained, and a position sense evaluation score as a body representation evaluation indicator is calculated based on the retained horizontal position information.

5. The method according to claim 1, characterized in that: If the body representation type in the current test task is vestibular sense, the body representation evaluation step further includes: Displaying the virtual avatar of the user under test in the target evaluation scene of the current test task, wherein a fixed angle is formed between the current real gravity direction of the user under test and the visual gravity direction of the virtual avatar in the target evaluation scene; Receiving initial visual gravity direction information sent by the tested user controlling the virtual reality interaction device, wherein the initial visual gravity direction information is angle information corresponding to the initial visual gravity direction of the virtual avatar currently in the target evaluation scene fed back by the tested user; Determining whether the initial visual gravity direction information is less than or equal to the angle, and if so, determining that the initial positioning of the vestibular sense has been completed; Correspondingly, in the process of executing the test task, the user under test is subjected to sensory stimulation matching the body representation type corresponding to the test task and the behavior feedback data of the user under test is collected in real time, and a body representation evaluation index is generated based on the behavior feedback data using a body representation evaluation method matching the body representation type corresponding to the test task, including: Displaying the virtual avatar of the user under test in the target evaluation scene of the current test task, so that the user under test controls the virtual reality interaction device to send out visual gravity direction information corresponding to the virtual avatar in the target evaluation scene multiple times; The visual gravity direction information emitted by the measured user each time is collected in real time. If each piece of visual gravity direction information is less than or equal to the angle, each piece of visual gravity direction information is retained, and a vestibular sense evaluation score serving as a body representation evaluation indicator is calculated based on the retained visual gravity direction information.

6. The method according to claim 1, characterized in that If the body representation type in the current test task is weight-bearing sensation, a sensory stimulation matching the body representation type corresponding to the test task is performed on the user under test during the execution of the test task, and behavioral feedback data of the user under test is collected in real time. Based on the behavioral feedback data, a body representation evaluation method matching the body representation type corresponding to the test task is used to generate a body representation evaluation index, including: Displaying a first object and a second object of the same shape and weight in a target evaluation scene of a current test task, and the volume of the first object is larger than that of the second object, so that the user under test can pull the first object and the second object to a preset height in the target evaluation scene multiple times based on a virtual reality interaction device; The grip strength information, the pulling start time, and the pulling end time of the user pulling the first object and the second object are collected based on the virtual reality interaction device, and a weight-bearing perception evaluation score as a body representation evaluation indicator is obtained based on the collected data.

7. The method according to any one of claims 1 to 6, characterized in that: The multiple body representation evaluation indicators include: a tactile evaluation score, a position sense evaluation score, a vestibular sense evaluation score, and a weight-bearing sense evaluation score; The evaluation results for the virtual avatar are obtained by respectively executing the multiple test tasks generated according to the multiple rehabilitation training objectives and generating the multiple body representation evaluation indicators in the body representation evaluation step, including: A scoring data matrix is ​​generated based on the tactile evaluation score, the position sense evaluation score, the vestibular sense evaluation score and the weight-bearing sense evaluation score, and the scoring data matrix is ​​used as the evaluation result of the virtual avatar in the tested virtual reality rehabilitation scene.

8. A device for evaluating a virtual avatar in a virtual reality rehabilitation scene, comprising a processor, a memory, and a computer program stored in the memory, characterized in that: The processor is used to execute the computer program. When the computer program is executed, the device implements the steps of the method according to any one of claims 1 to 7.

9. A virtual avatar evaluation system in a virtual reality rehabilitation scenario, characterized in that: include: An evaluation device for a virtual avatar in a virtual reality rehabilitation scene, and a virtual reality device, a vibration stimulation device for vibrotactile stimulation, and a virtual reality interactive device, which are respectively connected to the evaluation device for the virtual avatar in the virtual reality rehabilitation scene in communication with each other; wherein the vibration stimulation device for vibrotactile stimulation and the virtual reality interactive device are respectively connected to the virtual reality device in communication with each other; The evaluation device for the virtual avatar in the virtual reality rehabilitation scene is used to execute the evaluation method for the virtual avatar in the virtual reality rehabilitation scene according to any one of claims 1 to 7; The virtual reality equipment includes: a VR helmet; The virtual reality interaction device includes: a VR handle; The vibration stimulation device for performing vibration tactile stimulation includes: a vibration tactile module patch or a vibration device arranged on the VR handle.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images