Rehabilitation support device, method and program

Abstract

To control a load to be applied to a cognitive function of a user in rehabilitation.SOLUTION: A rehabilitation support device includes a detection section for detecting three-dimensional rehabilitation movements of a user, a display control section for producing an avatar object being movable in response to the detected rehabilitation movements and a target object to be visually recognized by the user, in a three-dimensional virtual space and displaying them on a display, a notification section for causing production of the target object to be notified of, and a cognitive load control section for applying a cognitive load to the user by producing the target object after the lapse of a prescribed time from the notification of the production of the target object.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a rehabilitation support device, a method and a program thereof. [Background technology]

[0002] In the above technical field, Patent Document 1 discloses a system for supporting rehabilitation. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-228957 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technique described in the above document cannot control the load on the user's cognitive function during rehabilitation.

[0005] An object of the present invention is to provide a technique for solving the above-mentioned problems. [Means for solving the problem]

[0006] In order to achieve the above object, a rehabilitation support device according to the present invention comprises: A detection unit for detecting a three-dimensional rehabilitation movement of a user; a display control unit that generates, in a three-dimensional virtual space, an avatar object that moves in accordance with the detected rehabilitation movement and a target object that is to be visually recognized by the user, and displays them on a display; Equipped with The display control unit further Represents the trigger for the appearance of Displaying a trigger object on the display; The type of the trigger object , at least one of number and sizeThis is a rehabilitation support device that quantitatively adjusts the cognitive load to be applied to the user by adjusting the above.

[0007] In order to achieve the above object, a rehabilitation support method according to the present invention comprises: A detection step in which a detection unit detects a three-dimensional rehabilitation motion of a user; a display control step of generating, in a three-dimensional virtual space, an avatar object that moves in response to the detected rehabilitation motion and a target object that is to be visually recognized by the user, and displaying the generated avatar object and target object on a display by a display control unit; Including, In the display control step, the target object Represents the trigger for the appearance of Displaying a trigger object on the display; The type of the trigger object , at least one of number and size This is a rehabilitation support method for quantitatively adjusting the cognitive load to be applied to the user by adjusting the above.

[0008] In order to achieve the above object, the rehabilitation support program according to the present invention comprises: a detection step of detecting a three-dimensional rehabilitation movement of a user; a display control step of generating, in a three-dimensional virtual space, an avatar object that moves in accordance with the detected rehabilitation movement and a target object that is to be visually recognized by the user, and displaying the generated avatar object and target object on a display; A rehabilitation support program for causing a computer to execute the following: In the display control step, the target object Represents the trigger for the appearance of Displaying a trigger object on the display; The type of the trigger object , at least one of number and size This is a rehabilitation support program that quantitatively adjusts the cognitive load to be imposed on the user by adjusting the above. Effect of the Invention

[0009] According to the present invention, it is possible to control the load on the cognitive function of a user during rehabilitation. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration of a rehabilitation support device according to a first embodiment. [Figure 2A] FIG. 11 is a block diagram showing the configuration of a rehabilitation support device according to a second embodiment. [Figure 2B] FIG. 11 is a diagram illustrating a task database of the rehabilitation support device according to the second embodiment. [Diagram 3] 13 is a diagram showing an example of a display screen of the rehabilitation support device according to the second embodiment. FIG. [Figure 4] 13 is a diagram showing an example of a display screen of the rehabilitation support device according to the second embodiment. FIG. [Diagram 5] 13 is a diagram showing an example of a display screen of the rehabilitation support device according to the second embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. However, the components described in the following embodiment are merely examples, and are not intended to limit the technical scope of the present invention.

[0012] [First embodiment] A rehabilitation support device 100 according to a first embodiment of the present invention will be described with reference to FIG.

[0013] The rehabilitation support device 100 includes a detection unit 101, a display control unit 102, a notification unit 103, and a cognitive load control unit 104.

[0014] The detection unit 101 detects the head orientation and three-dimensional rehabilitation movement of a user 130 wearing a head mounted display 120.

[0015] The display control unit 102 generates, in the three-dimensional virtual space 150, an avatar object 151 that moves according to the detected rehabilitation movement and a target object 152 that is visually recognized by the user 130. Then, the display control unit 102 causes the avatar object 151 and the target object 152 to be displayed on the head mounted display 120 according to the head orientation of the user 130 detected by the detection unit 101.

[0016] The notification unit 103 notifies the occurrence of a target object 152. For example, in FIG. 1, a radar screen image for notifying the occurrence position of the target object 122 is displayed as the notification image 153. The radar screen image indicates the relative direction of the position of the target object 152 with respect to a reference direction in the virtual space (here, initially set to be the front direction of a user sitting upright in a chair). The radar screen image further indicates the relative direction of the head of the user 130 with respect to the reference direction.

[0017] The cognitive load control unit 104 generates the target object 152 a predetermined time after the timing of notification of the generation of the target object 152, thereby applying a cognitive load to the user 130.

[0018] As described above, according to this embodiment, the user must continuously memorize the rehabilitation action that he / she should perform from the time when the target object is notified to be generated until the target object actually occurs, and it is possible to quantitatively adjust and control the cognitive load that the user's brain must process. For example, it is possible to impose a cognitive load with a higher complexity of information to be memorized by adjusting the type, number, size, spatial extent, position, amount, etc. of information contained in the notification image and notification sound.

[0019] [Second embodiment] A rehabilitation support system 200 according to a second embodiment of the present invention will be described with reference to Fig. 2. Fig. 2 is a diagram for explaining the configuration of the rehabilitation support system 200 according to this embodiment.

[0020] As shown in FIG. 2, the rehabilitation support system 200 includes a rehabilitation support device 210, two base stations 231 and 232, a head-mounted display 233, and two controllers 234 and 235. A user 220 performs rehabilitation movements by twisting the upper body and stretching the arms in accordance with the display on the head-mounted display 233 while sitting on a chair 221. In this embodiment, the rehabilitation is performed while sitting on a chair, but the present invention is not limited to this. The rehabilitation may be performed while standing, walking, running, or performing other specific movements. The controller may be held or attached to a body part other than the hand, such as the foot or the trunk.

[0021] The two base stations 231, 232 detect the movement of the head mounted display 233 and the movements of the controllers 234, 235, and send them to the rehabilitation support device 210. The rehabilitation support device 210 performs display control of the head mounted display 233 while evaluating the rehabilitation movement of the user 220. The head mounted display 233 may be of a non-transparent type, a video see-through type, or an optical see-through type. In this embodiment, a virtual space of VR (Virtual Reality) is presented to the user, but the real space and the virtual space may be superimposed and displayed as in AR (Augmented Reality), or real information may be reflected in the virtual space as in MR (Mixed Reality).

[0022] In this embodiment, the controllers 234, 235 held by the user 220 and the base stations 231, 232 are shown as examples of sensors for detecting the position or movement of the user's hands or head, but the present invention is not limited to this. Cameras (including depth sensors) for detecting the position or movement of the user's hands by image recognition processing, sensors for detecting the position of the user's hands by temperature, wristwatch-type wearable terminals worn on the user's arms, motion capture, and the like are also applicable to the present invention.

[0023] The rehabilitation support device 210 includes a motion detection unit 211 , display control units 212 and 213 , an evaluation unit 214 , an update unit 215 , a task set database 216 , and an operation unit 217 .

[0024] The motion detection unit 211 acquires the positions of the controllers 234, 235 held by the user 220 via the base stations 231, 232, and detects the rehabilitation motion of the user 220 from a change in the position of the user's 220's hands.

[0025] The display control unit 212 generates an avatar object 241 that moves according to the detected rehabilitation motion and a target object 242 that indicates a target of the rehabilitation motion in the virtual space. Then, images of the avatar object 241 and the target object 242 are displayed on the display screen 240 according to the orientation and position of the head mounted display 233 detected by the motion detection unit 211. The images of the avatar object 241 and the target object 242 are displayed superimposed on a background image 243. Here, the avatar object 241 has the same shape as the controllers 234 and 235, but is not limited to this, and the size, shape, and color may be different between the left and right. The avatar object 241 moves in the display screen 240 in accordance with the movements of the controllers 234 and 235. The controllers 234 and 235 are provided with one or more buttons, and are configured so that various settings can be performed by operating the buttons. The background image 243 is cut out from a virtual space including a horizon 244 and a ground surface image 245.

[0026] The display control unit 212 gradually changes the display position and size of the target object 242 so that it appears as if it is descending downward from above the user 220. The user 220 moves the controllers 234, 235 to move the avatar object 241 on the screen closer to the target object 242. Note that the movement direction of the target object may be, for example, from the floor surface direction to an upward direction, or may include movement in the depth direction in addition to the vertical movement.

[0027] When the shortest distance between a sensor object (not shown here) included in the avatar object 241 and the target object 242 falls within a predetermined range, the goal is achieved and the target object 242 disappears. At this time, if the shortest distance between the sensor object included in the avatar object 241 and the target object 242 falls below a first threshold, the goal is completely achieved and the message "Bravo!" is displayed, and a corresponding sound is output as feedback. At the same time, the controllers 234, 235 may be vibrated.

[0028] On the other hand, if the shortest distance between the sensor object included in the avatar object 241 and the target object 242 is equal to or greater than the first threshold and equal to or less than the second threshold, the message "Well done" is displayed to indicate that the target has been achieved, and a corresponding voice is output as feedback. At the same time, the controllers 234 and 235 may be vibrated.

[0029] The movement of the user 220 moving the controllers 234, 235 in this way is a rehabilitation movement, and the display of a target object that prompts the user 220 to perform one rehabilitation movement is called a task. Information (task data) that represents one task includes the target object's appearance direction, shape, appearance position (distance from the user), appearance interval (time interval), movement speed (falling or rising, etc.), size, color, which controller (left or right) should be used to acquire the target object, the number of target objects appearing simultaneously, the size of the sensor object, etc.

[0030] That is, the display control unit 212 can change the distance in the depth direction from the user 220 to the drop position of the target object 242 (for example, in three stages) according to the task data. For example, the distance can be changed to drop right next to the user 220 or to a position that the user 220 cannot reach unless he or she leans forward significantly. This makes it possible to control the exercise load given to the user.

[0031] The display control unit 213 displays the radar screen image 250 on the display screen 240 of the head mounted display 233. The radar screen image 250 is an announcement image for announcing the occurrence of the target object 152. The radar screen image 250 announces in which direction the position of the target object 242 to be generated is relative to a reference direction in the virtual space (here, initially set to be the front direction of the user sitting upright in a chair). The radar screen image 250 further announces how far the position of the target object 242 to be generated is from the user 220. Note that the announcement image is not limited to the radar screen image, and may be announced by characters, arrows, symbols, illustrations, types of light or color, strength, blinking, etc. Also, the announcement method is not limited to the image, and may be announced by voice, vibration, or any combination of voice, vibration, and image.

[0032] The display control unit 213 displays the radar screen image 250 in the center (for example, within a range of -50 degrees to 50 degrees) of the display screen 240 of the head mounted display 233 regardless of the direction of the head of the user 220. However, the display part is not limited to the center, and may be any place such as the four corners or the top, bottom, left, or right ends of the screen.

[0033] The radar screen image 250 includes a head image 251 representing the user's head as seen from above, a block image 252 obtained by dividing the periphery of the head image 251 into a plurality of blocks, and a fan-shaped image 253 as a visual field area image showing the user's visual field area. A target position image showing the position of a target object is shown by which block of the block image 252 is colored, blinks, or lights up. This allows the user 220 to know whether the target object is on the left side or the right side of the direction in which the user is facing. Note that in this embodiment, the block image 252 is fixed and the fan-shaped image 253 moves, but the present invention is not limited to this. The fan-shaped image 253 and the head image 251 may be fixed, and the block image 252 may be moved according to the direction of the head. Specifically, if the head faces left, the block image 252 may rotate to the right.

[0034] The evaluation unit 214 evaluates the rehabilitation movement of the user 220 according to the quantity and quality of the tasks accomplished by the user, and adds points. The quality of the accomplished task is determined as "good" or "great", that is, how close the avatar object was to the target object. The evaluation unit 214 assigns different points to each accomplished task (high points for distant objects and low points for close objects).

[0035] The update unit 215 updates the target tasks according to the accumulated points. For example, the target tasks may be updated using a task achievement rate (number of goals achieved / number of tasks).

[0036] The task set database 216 stores a set of a plurality of tasks. The task set database 216 stores a task set that determines in what order such a plurality of tasks should be presented to the user.

[0037] For example, a task set may be stored as a template for each hospital, or a history of executed task sets may be stored for each user. The rehabilitation support device 210 may be configured to be able to communicate with other rehabilitation support devices via the Internet. In this case, one task set can be executed by the same user at multiple locations, and various templates can be shared among multiple users at different locations.

[0038] The operation unit 217 is provided for operating the display control in the display control unit 212 and the display control unit 213 .

[0039] The cognitive load control unit 218 generates the target object 242 after a predetermined time has elapsed since the timing of notifying the user 220 of the generation of the target object 242, thereby imposing a cognitive load on the user 220. In other words, the user must continuously remember the action he or she should perform from the time when the user knows that the target object will be generated until the time when the target object actually occurs, and this "requirement to remember" becomes a cognitive load for the user. The cognitive load control unit 218 may control the cognitive load by changing the predetermined time from the time when the user is notified of the generation of the target object 242 until the time when the target object 152 is generated.

[0040] Furthermore, the cognitive load control unit 218 may control the cognitive load by changing the time "until the target object 152 approaches within reach of the user 220" instead of "until the timing at which the target object 152 is generated."

[0041] The cognitive load control unit 218 may impose a cognitive load on the user 220 by causing the head mounted display 233 to display a background image 243 other than the target object 242 .

[0042] 2B is a diagram showing a task table 260 stored in the task set database 216. In the task table 260, time (task occurrence timing) 221, task interval 222, task position 223, task angle 224, and task distance (strength) 225 are stored, each linked to a task ID. In addition, in the task table 260, speed 226 of a target object, criterion 227 for a perfect judgment (admirable evaluation), criterion 228 for a good judgment (superb evaluation), size 229 of a sensor object, and the like are stored, each linked to a task ID. In addition to these, a delay time (predetermined time) from notification of task occurrence to task occurrence may be set for each task.

[0043] 3 to 5 are diagrams showing an example of a display on the display screen 240 according to this embodiment. In Fig. 3 and Fig. 4, an image of a person representing a farmer is displayed as a trigger object 302 that triggers the appearance of a target object in a background image 301 representing a field. That is, the display control unit 213 displays the trigger object 302 as a notification image for notifying the appearance of the target object 152.

[0044] A predetermined time after the trigger object 302 throws the potato-shaped target object 303 upward, a large potato-shaped target object 403 appears on the screen as shown in Fig. 4. The task is accomplished by moving the colander-shaped avatar object 402 to catch the falling target object 403. The left and right avatar objects 402 move on the screen in conjunction with the movements of the controllers 234, 235.

[0045] The cognitive load control unit 218 generates a target object 403 a predetermined time after the trigger object 302 throws the target object 303 upward and notifies the user 220 of the generation of the target object 303, thereby imposing a cognitive load on the user 220. Note that the generation of the target object may be notified at the same timing by a radar chart type notification image in conjunction with the movement of the trigger object 302, or a voice notification may be combined.

[0046] In this way, the cognitive load control unit 218 applies a cognitive load to the user by displaying a background image with a large amount of information, as in Figures 3 and 4, instead of a background of only the horizon as in Figure 2. In other words, it makes it difficult for the user in rehabilitation to remember that the target object 401 has appeared and the position where the target object 401 will fall, and applies a cognitive load closer to that required in real life to the user in rehabilitation.

[0047] In particular, the cognitive load control unit 218 applies a cognitive load to the user 220 by changing at least a part of the background image 301 over time. In the example of FIG. 3, for example, clouds 304 may be moved, plants 305 may be swayed, or animals (not shown) unrelated to the target object may appear in the background image 301. This can prevent the user 220 from concentrating on the target object 303, making it more difficult for the user 220 to remember the position where the target object 303 will fall. More technically, it can be said that the cognitive load is controlled by providing an environment that makes it difficult to concentrate on the target object by displaying information unrelated to the task in the background image, and intentionally inducing attention disorders (more specifically, selective attention disorders, distributed attention disorders, shifting attention disorders, and sustained attention disorders) to make it difficult to remember.

[0048] FIG. 5 is a diagram showing another example of display on the display screen 240 according to the present embodiment. In FIG. 5, a trigger object 502 representing a monkey and a target object 503 representing an apple are displayed in a background image 501 such as a forest. The trigger object 502 representing a monkey drops the target object 503 representing an apple from a tree, and the avatar object 504 representing a colander moves to catch the target object 503 approaching the user, thereby accomplishing the task. Here, too, the cognitive load control unit 218 applies a cognitive load to the user 220 while inducing attention disorders by starting the drop of the target object 503 after a predetermined time has elapsed since the trigger object 502 shook the tree to notify the user of the occurrence of the target object 503.

[0049] Furthermore, the cognitive load control unit 218 can impose a stronger cognitive load on the user 220 by having at least two target objects 503 exist simultaneously in the three-dimensional virtual space and display them on the display screen 240. In other words, the cognitive load control unit 218 generates at least two target objects 503 at different positions in the left-right direction in the three-dimensional virtual space. In other words, the user 220 must remember at which positions the trigger object 502 shook the tree for a plurality of positions, which imposes a stronger cognitive load on the user 220.

[0050] In particular, at least two target objects 503 are generated at multiple positions in different directions (left and right directions in FIG. 5) relative to the moving direction of the target object 503 (falling direction in FIG. 5), so that a greater cognitive load can be applied. In other words, the user 220 must move the controllers 234, 235 while taking into consideration the vertical movement, the difference in the horizontal generation position, and the difference in the falling position in the depth direction, so that the spatial cognitive ability is also tested. In this way, by adjusting the type, number, size, spatial spread, position, amount, etc. of information included in the notification image and notification sound including the trigger object in addition to changing the predetermined time of the task, it becomes possible to quantitatively adjust and control the complexity of the information to be stored, that is, the cognitive load that the brain must process information for the user.

[0051] The evaluation unit 214 evaluates the user's cognitive ability based on whether the avatar object reaches the three-dimensional target position represented by the target object in a timely manner, the time interval and number of target objects from the notification of their occurrence to their occurrence, and the degree of attention disorder in the background image, etc.

[0052] As described above, according to this embodiment, the load on the cognitive function of the user during rehabilitation can be effectively controlled. Specifically, the user is not asked to remember something, but is naturally tested on memory and cognitive processing ability through a very low hurdle task of receiving a ball, a potato, or an apple, so the psychological burden is small and the user is unlikely to reject or reject the rehabilitation, or experience a depressive reaction due to failure. The rehabilitation operation according to this embodiment can restore so-called cognitive functions and higher brain functions (disorders), including memory, cognitive processing ability, spatial cognitive ability, and attention function (disorder), in addition to motor ability, and can lead to a comfortable daily life.

[0053] In the above embodiment, a head-mounted display is used, but any glasses-type display may be used. Also, instead of a head-mounted display, a hologram or a large television-type display may be placed around the user to encourage the user to perform rehabilitation movements.

[0054] [Other embodiments] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the technical scope of the present invention. In addition, systems or devices that combine the separate features included in each embodiment in any way are also included in the technical scope of the present invention.

[0055] The present invention may be applied to a system consisting of multiple devices, or to a single device. Furthermore, the present invention is also applicable to a case where an information processing program for implementing the functions of the embodiments is supplied to a system or device and executed by a built-in processor. In order to implement the functions of the present invention with a computer, a program installed on a computer, a medium storing the program, a server for downloading the program, and a processor for executing the program are also included in the technical scope of the present invention. In particular, at least a non-transitory computer readable medium storing a program for causing a computer to execute the processing steps included in the above-mentioned embodiments is included in the technical scope of the present invention.

Claims

1. A detection unit for detecting a three-dimensional rehabilitation movement of a user; a display control unit that generates, in a three-dimensional virtual space, an avatar object that moves in response to the detected rehabilitation movement and a target object that is visually recognized by the user, and displays them on a display; Equipped with The display control unit further causes a trigger object representing a trigger for the appearance of the target object to be displayed on the display; A rehabilitation support device that quantitatively adjusts the cognitive load to be applied to the user by adjusting at least one of the type, number, and size of the trigger object.

2. The rehabilitation support device according to claim 1 , wherein the cognitive load is controlled by changing a time from when the trigger object notifies the user of the occurrence of the target object until the occurrence of the target object.

3. The rehabilitation support device according to claim 1 , wherein a cognitive load is applied to the user by displaying a background image other than the target object and the trigger object on the display.

4. The rehabilitation support device according to claim 3 , wherein at least a part of the background image is changed over time to impose a cognitive load on the user.

5. a detection step in which a detection unit detects a three-dimensional rehabilitation motion of a user; a display control step of generating, in a three-dimensional virtual space, an avatar object that moves in response to the detected rehabilitation motion and a target object that is to be visually recognized by the user, and displaying the generated avatar object and target object on a display by a display control unit; Including, The display control step further includes displaying a trigger object on the display, the trigger object representing a trigger for the appearance of the target object; A rehabilitation support method for quantitatively adjusting the cognitive load to be applied to the user by adjusting at least one of the type, number, and size of the trigger object.

6. a detection step of detecting a three-dimensional rehabilitation movement of a user; a display control step of generating, in a three-dimensional virtual space, an avatar object that moves in accordance with the detected rehabilitation movement and a target object that is to be visually recognized by the user, and displaying the generated avatar object and target object on a display; A rehabilitation support program for causing a computer to execute the following: The display control step further includes displaying a trigger object on the display, the trigger object representing a trigger for the appearance of the target object; A rehabilitation support program that quantitatively adjusts the cognitive load to be applied to the user by adjusting at least one of the type, number, and size of the trigger object.

Images