Information processing system, information processing method, and information processing program

Abstract

To effectively improve the user's athletic ability. [Solution] An information processing system is provided that includes a request unit that requests the user to perform a body movement that causes an object or part of the user's body that moves in conjunction with the user's movements to overlap with a target object, and an evaluation unit that calculates a disorientation evaluation value indicating the degree of swaying of the user's body during the body movement. By having the user use such an information processing system, it becomes possible to effectively improve the user's motor skills.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and an information processing program.

Background Art

[0002] In the above technical field, Patent Document 1 discloses a system for hemiplegic patients caused by stroke or the like.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the technology described in the above document, there is no index as to when the user should proceed to the next step when making certain movements or when not making certain movements.

[0005] An object of the present invention is to provide a technology for solving the above problems.

Means for Solving the Problems

[0006] To achieve the above object, an information processing system according to the present invention a body movement that overlaps an object that moves along with the user's movement or a part of the user's body and <千]] It is presented in a way that the user can see it in a three-dimensional space. a target object, a request unit that requests the user to perform; An acquisition unit that detects and acquires the user's body movements, From the user's body movements acquired by the aforementioned acquisition unit, during the body movement, the body of the user Degree of ataxic tremor an evaluation unit that calculates a dyscoordination evaluation value indicating ataxia and an information processing system including. is an information processing system provided with.

[0007] To achieve the above objective, the information processing method according to the present invention is The requesting unit requests an object or part of the user's body that moves in conjunction with the user's actions, It is presented in a way that the user can see it in a three-dimensional space. A request step that requests the user to perform a body movement that overlaps with the target object, The acquisition step involves the acquisition unit detecting and acquiring the user's body movements, ataxia The evaluation department, From the user's body movements acquired in the aforementioned acquisition step, The user's body during the aforementioned physical movement Degree of ataxic tremor An evaluation step to calculate a disorder evaluation value that indicates the disorder, of include It is an information processing method.

[0008] To achieve the above objective, the program according to the present invention An object or part of the user's body that moves in conjunction with the user's actions, It is presented in a way that the user can see it in a three-dimensional space. A request step that requests the user to perform a body movement that overlaps with the target object, The acquisition step involves detecting and acquiring the user's body movements, From the user's body movements acquired in the aforementioned acquisition step, The user's body during the aforementioned physical movement Degree of ataxic tremor An evaluation step to calculate a disorder evaluation value that indicates the disorder, It is an information processing program that causes a computer to execute something. [Effects of the Invention]

[0009] According to the present invention, it is possible to effectively improve the user's athletic ability. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing the configuration of the information processing system according to the first embodiment. [Figure 2] This figure shows the configuration and usage method of the capability improvement support system according to the second embodiment. [Figure 3] This figure illustrates a method for evaluating dysfunction in the ability improvement support system according to the second embodiment. [Figure 4]This is a diagram showing an example of the operation panel screen of the ability improvement support system according to the second embodiment. [Figure 5] This is a diagram showing an example of the operation panel screen of the ability improvement support system according to the second embodiment. [Figure 6] This is a diagram showing an example of the operation panel screen of the ability improvement support system according to the second embodiment. [Figure 7] This is a diagram showing an example of the operation panel screen of the ability improvement support system according to the second embodiment. [Figure 8] This is a diagram showing an example of the task data table of the ability improvement support system according to the second embodiment. [Figure 9] This is a diagram showing an example of the display screen of the ability improvement support system according to the second embodiment. [Figure 10] This is a diagram showing an example of the display screen of the ability improvement support system according to the second embodiment. [Figure 11] This is a diagram showing an example of the display screen of the ability improvement support system according to the second embodiment. [Figure 12] This is a diagram showing an example of the display screen of the ability improvement support system according to the second embodiment.

Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present invention will be exemplarily described in detail with reference to the drawings. However, the components described in the following embodiments are merely examples, and the technical scope of the present invention is not intended to be limited thereto.

[0012] [First Embodiment] The information processing system 100 as the first embodiment of the present invention will be described with reference to FIG. 1. The information processing system 100 is a system that effectively evaluates ataxia.

[0013] As shown in Figure 1, the information processing system 100 includes a request unit 101 and an evaluation unit 102. The request unit 101 requests the user 110 to perform a body movement that superimposes an object or part of the body 111 that moves in conjunction with the user 110's movements with a target object 112. The evaluation unit 102 calculates a disorientation evaluation value 121 that indicates the degree of shaking of the user 110's body during the movement.

[0014] With the above configuration, the user can make malfunctions during operation apparent. Although Figure 1 illustrates an example in which the target object 112 is displayed on a head-mounted display for convenience, the present invention is not limited to this. The target object 112 may also be displayed on a flat display placed in front of the user 110, or it may be displayed three-dimensionally as a hologram or the like.

[0015] [Second Embodiment] A second embodiment of the ability improvement support system 200 of the present invention will be described with reference to Figure 2. The ability improvement support system 200 is an example of a system that supports the improvement of motor ability in rehabilitation and the like, but the present invention is not limited thereto. That is, the present invention should not be limited to the concept of "rehabilitation," and treatment systems that actively treat physical diseases such as paralysis, ataxia, upper limb dysfunction, and speech and swallowing disorders are also included in the present invention. Furthermore, movement verification systems, ability improvement and enhancement systems for improving the motor ability of healthy individuals who do not require any treatment or rehabilitation are also included in the present invention.

[0016] Figure 2 is a diagram illustrating the configuration of the ability improvement support system 200 according to this embodiment. The ability improvement support system 200 according to this embodiment improves at least one of the following functions: physical functions such as upper limb function, walking function, trunk function and balance function, articulation and swallowing function, and sensory functions (including the inner ear, vestibular system and touch, temperature and pain sensation, proprioception and proprioception).

[0017] As shown in Figure 2, the ability improvement support system 200 comprises an information processing device 210, two base stations 231 and 232, a head-mounted display 233 as a blocking unit, and two controllers 234 and 235. The user 220 sits in a chair 221, wears the head-mounted display 233, and moves their body in accordance with the display on the head-mounted display 233. In this embodiment, the explanation is based on the premise of verifying actions performed while sitting in a chair, but the present invention is not limited thereto, and the actions may be performed standing, walking, on a bed, lying supine or prone, running, or while performing other specific actions. In this embodiment, the user 220 holds the controllers 234 and 235 with both hands and detects their position and angle, but the present invention is not limited thereto, and it may be done with one hand, or sensors may be held or attached to body parts other than the hands, such as the feet or torso, to detect their position and angle.

[0018] The two base stations 231 and 232 detect the position and movement of the head-mounted display 233, as well as the position and movement of the controllers 234 and 235, and send this information to the information processing device 210. The information processing device 210 controls the display on the head-mounted display 233 based on its movement. Specifically, it changes the position and orientation of the viewpoint in the virtual space according to the position and orientation of the head-mounted display 233 and determines the image to be displayed on the head-mounted display 233. It also evaluates the user's actions based on the movements of the controllers 234 and 235.

[0019] The head-mounted display 233 may be an opaque type, a video see-through type, an optical see-through type, or a glasses type. In this embodiment, a virtual space of VR (Virtual Reality) is presented to the user, but the real space and virtual space may be superimposed, as in AR (Augmented Reality), or real information may be reflected in the virtual space, as in MR (Mixed Reality), or other XR technology or hologram technology may be used as alternative means.

[0020] The important point here is that by wearing the head-mounted display 233, the user 220 cannot directly see all or part of their own body. Normally, in real space, humans visually perceive their body movements and unconsciously make fine adjustments in their brains to move towards a target. In other words, because the user 220 cannot directly see their own hands in real space, the user's brain cannot make accurate corrections to hand movements based on visual information as it normally would. As a result, ataxia is more likely to become apparent. In this case, for example, a projection mapping system that displays a background on all or part of the user's body may be used as a visual information blocking device. Alternatively, a contact lens type display that displays a virtual space may be used as a blocking device.

[0021] In this embodiment, controllers 234 and 235 held in the hand of user 220, and base stations 231 and 232 are shown as examples of sensors for detecting the position or movement of the user's hands or head, but the present invention is not limited thereto. Cameras (including depth sensors) for detecting the position or movement of the user's hands themselves by image recognition processing, sensors for detecting the position of the user's hands by temperature, wristwatch-type wearable terminals worn on the user's arm, and motion capture devices can also be applied to the present invention in conjunction with the motion detection unit 211. In other words, one embodiment involves using a three-dimensional tracking device or motion analysis device such as Kinect®, or attaching markers to the body.

[0022] The information processing device 210 includes an action detection unit 211, display control units 212 and 213, a feedback unit 214, a task set database 216, a setting unit 217, an action guidance unit 218, and an evaluation unit 219.

[0023] The motion detection unit 211 acquires the position and angle of the controllers 234 and 235 and the head-mounted display 233 held by the user 220 via the base stations 231 and 232, and detects the user 220's movements based on changes in the position of the user 220's hands and head.

[0024] The display control unit 212 generates and displays target objects 242a and 242b in virtual space to prompt the user 220 to perform three-dimensional body movements. In particular, the display control unit 212 generates target object 242a for three-dimensional movement of a part of the left side of the user 220's body, and target object 242b for movement of a part of the right side of the user 220's body, in virtual space. In other words, the display control unit 212 functions as a request unit that requests the user 220 to perform body movements that superimpose the target objects 242a and 242b with objects or parts of the body 234 and 235 that move in conjunction with the user 220's movements. The display control unit 212 further displays a visual aid image 246 to assist in the viewing of the target objects 242a and 242b.

[0025] The display control unit 213 displays the radar screen image 250 on the display screen 240 of the head-mounted display 233. The radar screen image 250 is a notification image for notifying the occurrence of a target object 152. The radar screen image 250 indicates the relative direction of the positions of the occurring target objects 242a and 242b with respect to a reference direction in the virtual space (which is initially set to be the direction in front of the user sitting upright in a chair). The radar screen image 250 also indicates how far the positions of the occurring target objects 242a and 242b are from the user 220. Note that the notification image is not limited to a radar screen image, and may be notified by characters, arrows, symbols, illustrations, types and intensity of light and color, flashing, etc. Also, the notification method is not limited to images, and may be done by sound, vibration, or a combination of sound, vibration, and images. The display control unit 213 displays the radar screen image 250 in the central part of the display screen 240 of the head-mounted display 233 (for example, within the range of -50 to 50 degrees), regardless of the orientation of the user's head 220. However, the display area is not limited to the center; it can be any location on the screen, such as the four corners, top edge, bottom edge, left edge, or right edge. The patient can estimate the difficulty of the next movement to be performed from the position, angle, and number of target objects displayed on the radar screen, and if a more difficult movement is predicted for the patient, the ataxia becomes more apparent. In other words, the display control unit 212 also functions as a control unit for controlling the difficulty of the movements required of the patient.

[0026] The radar screen image 250 includes a head image 251 representing the user's head as seen from above, a block image 252 dividing the area around the head image 251 into multiple blocks, and a fan-shaped image 253 as a field of view image indicating the user's field of view. The target position image, which indicates the location of the target object, is indicated by which block of the block image 252 is colored, blinking, or lit up. This allows the user 220 to know whether the target object is to the left or to the right of the direction they are facing. In this embodiment, the block image 252 is fixed and the fan-shaped image 253 moves, but the present invention is not limited to this, and the fan-shaped image 253 and head image 251 may be fixed while the block image 252 moves according to the direction of the head. Specifically, if the head is turned to the left, the block image 252 may rotate to the right.

[0027] The feedback unit 214 preferably changes the message type according to the evaluation of the user's actions via the display control unit 212. For example, if the sensor object touches the center of the target objects 242a and 242b, it displays "Excellent," and if the sensor object touches only the area around the center of the target objects 242a and 242b, it displays "Superb." Note that when the avatar object 241a touches the visual aid image 246, the target object 242a does not disappear, and the task is not fully completed (it is an incomplete completion, a certain score is awarded, and it is evaluated as "Superb"). The task is only fully completed (Excellent evaluation) when the avatar objects 241a and 241b touch the target objects 242a and 242b.

[0028] Here, the invention requires reaching a target object as a virtual target displayed on a head-mounted display, but the invention is not limited to this, and the target may be an object located in the real world. The target may be an object suspended by two or more wires, or an object levitated in the air by ultrasonic vibrations. The target may also be an object displayed three-dimensionally as a hologram. The shape of the target object is not limited to a sphere, but may be triangular, square, or dish-shaped. It may also be the shape of some character. The concept of the invention also includes systems that use optical see-through or smart glasses type head-mounted displays to partially block the user's vision, making the user's own movements invisible, while still allowing the target to be seen.

[0029] The display control unit 212 can move the target objects 242a and 242b generated in the virtual space. In this case, the head-mounted display 233 will display them so that their position and size gradually change (for example, they will gradually become larger and then smaller). The direction of movement of the target objects may be, for example, upward from the floor to above the head. Furthermore, the target objects may move in any three-dimensional way, including not only vertical movement but also movement in the depth direction and left-right direction, as well as random movement. Alternatively, depending on the user's stage, they may remain fixed at a specific coordinate position without moving.

[0030] Furthermore, the display control unit 212 generates avatar objects 241a and 241b in the virtual space that move in response to detected user movements. Here, avatar object 241a is an object that moves in accordance with the controller 234 operated by the left hand and represents the position of a part of the user's left side. Avatar object 241b is an object that moves in accordance with the controller 235 operated by the right hand and represents the position of a part of the user's right side. The avatar objects are not limited to objects that move in accordance with the position of the controllers. They may be virtual objects that move in accordance with changes in the position of a part of the user's body using the three-dimensional body tracking technology described above, or they may be avatars that represent a part of the body itself. The object operated by the user is not limited to the controllers shown in the diagram, but may be something like a baseball bat or a golf club. The display control unit 212 requests the user to perform a body movement that superimposes the avatar objects 241a and 241b onto the target objects 242a and 242b.

[0031] Images of avatar objects 241a, 241b and target objects 242a, 242b are displayed on the display screen 240 according to the orientation and position of the head-mounted display 233 detected by the motion detection unit 211. The images of avatar objects 241a, 241b and target objects 242a, 242b are superimposed on the background image 243. Here, avatar objects 241a, 241b have the same shape as controllers 234, 235, but are not limited to this, and may be in the shape of hands. Furthermore, the size, shape, and color may be different for the left and right. Controllers 234, 235 are provided with one or more buttons, and are configured so that various settings, including initial settings such as origin setting, can be made by operating the buttons, but the button function may be disabled, or the buttons themselves may not be provided, and all settings may be executed using a separate external control unit. The background image 243 is extracted from a virtual space that includes the horizon 244 and ground surface objects 245. Depending on the movement of the gaze (position and orientation of the head-mounted display), the appearance of the background image 243, avatar objects 241a, 241b, and target objects 242a, 242b changes.

[0032] The display control unit 212 generates an avatar object indicating the position of a part of the body on the right side, and a target object indicating the target position of the avatar object to its right, using the same color or shape. The display control unit 212 also generates an avatar object indicating the position of a part of the body on the left side, and a target object indicating the target position of the avatar object to its left, using the same color or shape. Furthermore, it is desirable that the color or shape of the left avatar object and the left target object differ from the color or shape of the right avatar object and the right target object. However, even if they are on the same side, the avatar object and target object do not necessarily need to be set to the same color or shape, nor is there a necessity to differentiate the colors between the right and left sides. These settings can be adjusted according to the user's preference.

[0033] Here, as an example, avatar objects 241a and 241b are color-coded, for example, blue and red, and target objects 242a and 242b are also color-coded blue and red. The task is completed by bringing the blue avatar object 241a into contact with the blue target object 242a. Similarly, the task is completed by bringing the red avatar object 241b into contact with the red target object 242b. In other words, the task will not be completed by bringing avatar objects of different colors into contact. Here, we have used blue and red as an example, but for colorblind users, other color coding (such as yellow and green) may be used, or shapes, linguistic notations such as "left" and "right," "L (Left)" and "R (Right)," or symbols such as stars, triangles, and circles may be used, or different patterns (stripes) may be used to distinguish them, as shown for 242a and 242b in the diagram.

[0034] The motion guidance unit 218 guides the user so that movements of a part of the user's right side of the body and movements of a part of the user's left side of the body are performed alternately. Specifically, the motion guidance unit 218 guides the user's movements by displaying letters, colors, or shapes representing right and left, by outputting voices to distinguish between "right" and "left", by using tactile stimulation through vibrations of controllers held in the user's left and right hands, or by touching or tapping the shoulders or arms.

[0035] When avatar objects 241a and 241b collide with target objects 242a and 242b, the display control unit 212 makes the target objects 242a and 242b disappear, and the feedback unit 214, recognizing that the target action has been achieved, displays a message or a visual stimulus (visual effect) such as a flash of light or blinking light to provide visual feedback for the purpose of notifying or informing the user of the achievement of the target action. If the sensor center of avatar objects 241a and 241b reaches the center of target objects 242a and 242b, "Excellent!" is displayed, and even if it reaches the surrounding area, "Very good!" is displayed for each task. In this way, the user receives feedback on the achievement of each requested physical action. The feedback unit 214 only needs to stimulate one or more of the user's five senses to provide feedback. If feedback is provided by stimulating two or more of the user's five senses, it is possible to more effectively improve the user's cognitive ability, motor ability, or both.

[0036] In other words, the feedback unit 214 notifies the user 220 of the success of each action performed on target object 242a and target object 242b. Various methods of notification are possible. The user may be informed of the degree of success of the target action by temporarily displaying words such as "Excellent" or "Well done" on the display screen, or the user may be informed of the degree of success of the target action by auditory stimulation using similar voices or sound effects. Furthermore, the user may be notified of the degree of success of the target action by simultaneously vibrating only the controller 234 or 235 that performed the movement to make contact with the target object. Alternatively, the notification of success may be graded, for example, complete success, incomplete success, or failure to achieve, to inform the user of the degree or degree of success.

[0037] For example, when the shortest distance between the sensor object included in avatar object 241 and target objects 242a and 242b falls within a predetermined range, the goal is achieved, and target objects 242a and 242b disappear. At this time, if the shortest distance between the sensor object included in avatar objects 241a and 241b (for example, a spherical object containing the center point of the tip of avatar objects 241a and 241b) and target objects 242a and 242b falls below a threshold, the goal is considered fully achieved, and feedback is provided by displaying, for example, "Well done," and simultaneously outputting a corresponding sound. This type of feedback that stimulates multiple different senses is called multi-channel biofeedback or multi-sensory biofeedback function. Simultaneously, controllers 234 and 235 may be vibrated, or stimuli may be applied to the sense of smell or taste. In other words, the feedback unit 214 can stimulate any of the five senses—sight, hearing, touch, taste, and smell—as feedback. Any two of these sensory stimuli may be combined, three or more may be combined, or all of them may be stimulated. In situations where point estimation is required, providing feedback each time a requested action is achieved can trigger the brain to correct and improve imbalances, promoting the reconstruction of neural circuits in the brain. Furthermore, safety features may be included to prevent side effects caused by repeated and continuous occurrence of specific stimuli, such as preventing the generation of a target object in the same location more than a certain number of times. For example, a function could be considered to prevent photosensitive seizures caused by repeated light stimulation (the so-called Pokémon shock), noise-induced hearing loss that can occur when high-frequency auditory stimulation in the 2000 to 4000 Hz range continues for a long period without interruption, and hand-arm vibration syndrome caused by continuous and continuous vibration of the controller, by preventing the generation of an object in the same location more than 21 times.

[0038] The improvement in these motor symptoms progresses more efficiently, or the improvement is maintained over the long term, the more sensory stimulation is involved during the feedback process. For example, symptoms that previously took several days to improve with two sensory feedbacks can improve in hours with two sensory feedbacks compared to one. Similarly, symptoms that previously took several hours to improve with three sensory feedbacks can improve in minutes with three sensory feedbacks compared to two. Furthermore, symptoms that improved with one sensory feedback but quickly reverted to their original state can maintain their improvement for several days with two sensory feedbacks. Or, symptoms that improved with two sensory feedbacks but reverted to their original state in a few days can maintain their improvement for several weeks with three sensory feedbacks. To express the characteristics of this treatment, it is also referred to in medical settings as "brain reprogramming therapy (BRT)," "brain re-wiring therapy (BRT)," Neuro Reorganization Therapy, Brain Reorganization Therapy, Somato-Cognitive Coordination Therapy (SCCT), or "motor coordination therapy (MCT)." Furthermore, this treatment method is thought to be an intervention on the brain's coordination motor area (Somato Cognitive Action Network, SCAN), and it has been reported that this method, despite being performed in a seated position, produces cross-modal improvements in motor function, including upper limb function such as finger dexterity, walking function, and articulation / swallowing function. The changes in the motor pathways during this process have been visualized using diffusion tensor tractography.

[0039] The operation may be evaluated in three or more stages depending on how much the distance between the sensor object and the target objects 242a and 242b has decreased. Furthermore, two or more target objects may be simultaneously generated and displayed in the virtual space.

[0040] If the shortest distance between the sensor object in avatar objects 241a and 241b and target objects 242a and 242b is less than or equal to the first threshold, the system should display "Bravo!" and simultaneously output the corresponding voice "Bravo!" as feedback to indicate that the goal has been fully achieved. If the shortest distance between the sensor object in avatar objects 241a and 241b and target objects 242a and 242b is between the first threshold and the second threshold, the system should display "Excellent!" and simultaneously output the corresponding voice "Excellent!" as feedback to indicate that the goal has been achieved. Note that the output voice does not have to be the same as the message; for example, a non-verbal sound effect such as "Piroon" may be used. As the distance required for contact between the sensor object and the target object is reduced, more precise movements are required, so the brain needs to request more accurate and refined movements from the body as motor commands. As mentioned above, this image formation by the brain is called feedforward. In other words, the smaller the distance between the sensor object required to complete the action and the target object, the stronger the feedforward force needed, which allows for continuous adjustment of the user's motor, cognitive, and sensory load levels.

[0041] The feedback unit 214 provides feedback to a user who virtually touches the target objects 242a and 242b, stimulating two or more of the five senses (sight, hearing, touch, taste, and smell) at the moment the sensor object enters a predetermined distance from the center of the target objects 242a and 242b, or at approximately the same time as the sensor object makes contact with the target objects 242a and 242b (this is called real-time multi-channel biofeedback or immediate multi-signal biofeedback). The delay between these timings and the feedback is highly effective if, for example, it is within 1 second, and the effect is greater the closer the interval between the user's action timing and the feedback timing (the smaller the delay). The feedback unit 214 provides feedback that stimulates the user's vision with an image that says "Bravo!", while simultaneously providing feedback that stimulates the user's hearing with sound or sound effects output from a speaker. Notification of task completion in relation to the five senses can be combined in any way, depending on the type of action, etc.

[0042] Furthermore, the feedback unit 214 may simultaneously output feedback that stimulates the user 220's vision with an image of "Bravo!", feedback that stimulates the user 220's hearing with sound output from the speaker, and feedback that stimulates the user 220's touch with vibrations from the controller 234. Alternatively, the feedback unit 214 may simultaneously output only two types of feedback: feedback that stimulates the user 220's vision with an image of "Bravo!", and feedback that stimulates the user 220's touch with vibrations from the controller 234. Or, the feedback unit 214 may simultaneously output only two types of feedback: feedback that stimulates the user 220's hearing with the sound of "Bravo!", and feedback that stimulates the user 220's touch with vibrations from the controller 234.

[0043] A task or issue is a display of a target object that prompts a single action that user 220 should perform. Information representing a single task (task data) includes the direction in which the target object appears (90 degrees to the right, 45 degrees to the right, front, 45 degrees to the left, 90 degrees to the left relative to the front of the chair), the distance to the target object, its shape (size), its appearance position (distance and angle from the user), the appearance interval (time interval), its movement speed (such as falling or rising), its size, color, which controller (left or right) should acquire it, the number of target objects that appear simultaneously, and the size of the sensor object. In other words, the task data includes information on at least one of the virtual space attributes of target objects 242a and 242b: movement speed, number of displays, size, display position, and display interval. The depth distance from user 220 to the falling position of target objects 242a and 242b may be set continuously, or it may be set to one of three stages, for example. For example, the drop can be adjusted so that it falls right next to user 220, or so that user 220 has to lean forward significantly to reach it. This allows for control over the physical load on the user, as well as the load on their spatial cognitive ability or spatial awareness.

[0044] The task set database 216 stores sets of multiple tasks. A task represents a single requested action that a user should perform. Specifically, it stores information representing a single task, such as where, at what speed, and what size a target object was made to appear, and what size the avatar object was at that time. The task set database 216 stores task sets that determine the order in which such multiple tasks are provided to the user. For example, task sets may be stored as templates for each hospital, or a history of executed task sets may be stored for each user. The information processing device 210 may be configured to communicate with other information processing devices via the internet, in which case a single task set can be executed by the same user in multiple locations, and various templates can be shared among multiple users in different locations.

[0045] The size of the target objects 242a and 242b, as well as the size of their surrounding areas, can be set using the setting unit 217. The size of the sensor object can also be set using the setting unit 217.

[0046] The setting unit 217 can be configured to alternate between generating the left target object 242a and the right target object 242b, and the setting unit 217 sends instructions to the display control unit 213 according to the configuration.

[0047] The setting unit 217 can set the generation position in the depth direction of the left target object 242a and the generation position in the depth direction of the right target object 242b in the virtual space.

[0048] By controlling both the timing of target object generation and the vertical movement speed of those target objects, the system ensures that the user's movements are always alternating between left and right. In other words, the simplest method is to not generate left and right target objects simultaneously; instead, the left target object is generated only after reaching the right target object is detected. However, this is not the only possible control method. For example, multiple target objects can be generated simultaneously, and their movement speeds can be changed to decrease in the order of right → left → right → left to encourage alternating left and right movements. For instance, after reaching the right object is detected, the right and left objects can be generated simultaneously, or the right object can be generated first, increasing its speed to make it overtake the left object. This can place a high load on the brain. The requesting unit or control unit should control the user's movements to be alternating between left and right.

[0049] On the other hand, if there is paralysis only on the right side, or if you want to actively rehabilitate the left upper body, and you want the right target object to reach downwards and the left target object to reach relatively upwards, you can generate the right target object at a slow speed, and then, after a short interval, generate the left target object at a faster speed. In other words, you control at least one of the timing and speed of the target object generation according to the vertical position of the body where you want to perform the requested movement (according to the position of the target object you want to reach). When requesting movement of the knees or lower body, you control the timing and speed of the target object generation so that it reaches at a low position. Also, if you want to increase the cognitive load, you can generate them in the order of left → right → right, but move only the first target object slowly, and move the second and third at a fast speed. As a result, it becomes possible to reach in the order of right, left, right.

[0050] By rewired in this way, it becomes possible to improve symptoms of conditions that are difficult to treat with modern medicine, such as ataxia, chronic pain disorders, and restless legs syndrome (abnormalities in the basal ganglia).

[0051] Furthermore, the setting unit 217 sets at least one of the following attributes of the target objects 242a and 242b in the virtual space: movement speed, number of displays, shape, color, size, display position (coordinate information, etc.), and display interval. The setting unit 217 may also set a delay time from the timing of notification of the occurrence of target objects 242a and 242b to the timing of the target objects 242a and 242b being generated, thereby controlling the cognitive load imposed on the user 220. In other words, the user must continuously remember the actions they should take from the time they learn the location in the virtual space where the target object will be generated (the position indicating which direction the head-mounted display should be pointed to be displayed) via the radar screen image 250, etc., until the target object actually appears, and this "memory time" constitutes the cognitive load for the user. Alternatively, the setting unit 217 may control the cognitive load by changing the time "until the target object approaches within the user's reach" rather than "until the target object is generated." The setting unit 217 may impose a cognitive load on the user 220 by displaying a somewhat complex image on the head-mounted display 233 as a background image 243 other than the target objects 242a and 242b. When changing the cognitive load, it is desirable to notify the user in advance that the cognitive load will be increased or decreased. The notification method may be visual using characters or symbols, by voice, or by touching a part of the body, such as tapping the shoulder, elbow, arm, or leg.

[0052] The head-mounted display 233 functions as a shield that obscures part or all of the user's body movements from the user's eyes. Compared to when the actual body movements are directly visible, the burden on the user's brain increases when part or all of the body is obscured and not visible. This is because the brain needs to perform a function (so-called point estimation) in which it observes the position (point) of the controller in the virtual space and estimates the position (point) of the controller in the real space, even though the user's own body (and the controller in the real space) is not visible. In this case, the brain needs to strongly imagine the state of the body without relying on vision, which is medically described as requiring strong feedforward. A brain that is required to perform strong feedforward finds it more difficult to coordinate the body, leading to the manifestation of coordination disorders and an increased likelihood of ataxia. Furthermore, the strong feedforward requiring point estimation makes it possible to induce contraction of the body's deep muscles through stimulation via specific neural pathways such as the pyramidal tract.

[0053] Therefore, for example, when moving the right arm to reach the avatar object 241b of the right controller 235 towards the target object 242b on the right side, a dysfunction may occur. The evaluation unit 219 detects such dysfunctions that are not necessary for the requested operation and uses them as an indicator for improving the user's operation. Specifically, the evaluation unit 219 calculates a dysfunction evaluation value (dysfunction score) that indicates the degree of body sway of the user 220 during the operation.

[0054] For example, as shown in Figure 3, suppose user 220 performs an action of reaching a target object 242 (fixed in virtual space) displayed on the head-mounted display 233 with the controller 235. At this time, the measurement of ataxia begins when the avatar object 241 approaches the target object 242 to the evaluation start distance 304. The position of the avatar object 241 at the moment it approaches the target object 242 to the predetermined evaluation start distance 304 is used as the start coordinate, and a straight line is drawn between it and the coordinates of the target object 242. In other words, the evaluation unit 219 starts calculating the ataxia evaluation value when the distance between the controller's avatar object 241 and the center of the target object 242 reaches the preset evaluation start distance 304.

[0055] This straight line is the ideal motion line 301, but in the case of a user with ataxia, the avatar object 241 cannot be moved along the ideal motion line 301 and deviates in any direction, such as up, down, left, or right, as shown by the dotted line 302. The evaluation unit 219 calculates a distress evaluation value that indicates the degree to which the avatar object 241 has deviated from the straight line toward the center of the target object 242. At the same time, the orientation (direction), tilt from the vertical line, and position on the horizontal plane (deviation and deviation of the position as viewed from above) of the head-mounted display 233 are used as further distress evaluation values.

[0056] The evaluation unit 219 projects (plots) the position of the avatar object 241 onto a bullseye 303, which is a circle centered on the ideal motion line 301, and calculates a hand ataxia evaluation value based on the area of ​​movement drawn by the movement (shake) of the avatar object 241 (the integral value of the distance moved away from the ideal motion line 301). In other words, the evaluation unit calculates the ataxia evaluation value based on the area of ​​movement drawn by the movement of the coordinates projected onto a circle centered on the center of the target object, from the point when the distance between the object (controller) or part of the body and the center of the target object reaches a predetermined distance until the object or part of the body and the target object overlap. Specifically, the ataxia evaluation value is the ratio of the area of ​​movement to the set area. Note that this ataxia evaluation value is not limited to a ratio to the set area; it may also be scored using the absolute value of the area, or it may be evaluated by taking time information into account, or it may be evaluated by taking acceleration into account, or it may be the volume of a cylinder or the like centered on the ideal motion line.

[0057] The evaluation unit 219 can also detect at least one of the following from changes in the position, tilt, and direction of the head-mounted display 233: twisting of the user's head and torso, horizontal movement, and tilt, and calculate ataxia evaluation values ​​based on these. This information may be obtained not only from the positional information of physical devices such as the head-mounted display, but also by applying body tracking technologies, including markerless tracking. The position of the head-mounted display 233 may also be evaluated using a bullseye method. Specifically, the ataxia evaluation value is defined as the ratio of the area of ​​movement of the head-mounted display 233 on the horizontal plane to a set area.

[0058] Figures 4 to 6 show the operation panel 400 displayed by the setting unit 217 for operator use. The intuitive operation panel 400 allows users to set task parameters (distance to the target object, its height, direction, size, movement speed, generation interval, sensor object size, visual aid image size, type and presence of background information, type and presence of sound effects and BGM). Users can use manual mode (setting point estimation parameters for each task and operating for each task), template mode (point estimation parameters are pre-set for multiple task sets), or an automatic mode (auto mode) where the device automatically generates tasks, or a combination of these. The operation panel 400 also allows users to access basic user information, various cognitive and motor function evaluation indices, check test results, create templates, and set and instruct auto mode. It is also possible to erase a target object once it has been generated. This is useful when the operator has generated a target object at a position or speed that is too burdensome for the patient.

[0059] The display showing the operation panel 400 may be an external display connected to the information processing device 210, or a display built into the information processing device 210; the device is not limited. The operation panel 400 includes a user field of view display area 401, various parameter setting areas 421 and 422, a score display area 403, a recenter button 406, and a BGM control button 407.

[0060] The user field of view display area 401 displays the image actually shown on the head-mounted display 233. The parameter setting area 421 is an area for setting multiple parameters that define the task. In the parameter setting area 421, the size and speed of the left and right target objects, the size of the visual aid image, and the size of the sensor area of ​​the avatar object can be set. In the example in Figure 4, the sensor size is 1.7 cm for both the left and right. This element also quantitatively changes the accuracy of feedforward, that is, the accuracy of information processing for the brain, and makes it possible to control the therapeutic approach to physical function (upper limb function, walking function, trunk function, balance function), sensory function (including inner ear, vestibular system, touch, temperature and pain sensation, proprioception, and proprioception), and the degree to which ataxia manifests.

[0061] The input area 422 is an area for setting the position of the next target object to appear, and it has the shape of an enlarged radar screen image 250. When an operation is performed by clicking or tapping any of the multiple blocks in the input area 422, or when a corresponding operation is performed by the input device, a target object is generated at the position in the virtual space corresponding to the position of the specified block. These settings quantitatively change the accuracy of the feedforward, that is, the accuracy of the information processing performed by the user's brain. This makes it possible to control the therapeutic approach to physical functions (upper limb function, walking function, trunk function, balance function), sensory functions (including the inner ear, vestibular system, touch, temperature and pain sensation, proprioception, and proprioception), and the degree to which ataxia manifests.

[0062] The operator control panel 400 has three tabs 411 to 413. When tab 411 is selected, the score display area 403 displays the total number of times each task was performed at each location, as well as the score indicating how many times that task was completed. The score may be displayed as a fraction, a percentage, or a combination of these.

[0063] The recenter button 406 is a button that receives a reconstruction instruction from the operator to reconstruct the virtual space to match the position of the user 220. When the recenter button 406 is operated, the display control unit 212 reconstructs the virtual space with the position of the head-mounted display 233 at that moment as the origin and the orientation of the head-mounted display 233 at that moment as the reference direction. By operating the BGM control button 407, background sound during task execution can be turned on or off. By eliminating background sound, the workload of the task can be reduced, allowing the user to concentrate on their body movements. Generally, in users who experience ataxia, eliminating background sound tends to alleviate the ataxia during the task. Conversely, background sound can be turned on for users who do not experience ataxia to intentionally induce it.

[0064] The display control unit 213 may also display an information bar 410 on the display screen 240 of the head-mounted display 233 in addition to the radar screen image 250. The information bar 410 displays the player name, task completion status, elapsed time since the start of play, etc.

[0065] Figure 5 shows the operation panel 400 when tab 412 is selected. When tab 412 is selected, an ataxia evaluation area 501 representing the degree of hand ataxia is displayed. The ataxia evaluation area 501 displays setting areas 511 for the evaluation start distance (judgment area) for each left and right hand, a scale setting area 512 for setting the size of the bullseye 303, an area 513 showing the trajectory of the avatar object (more precisely, the center of the sensor object) in the bullseye 303, and an evaluation score 514. If the value entered in the scale setting area 512 is small (for example, 10), even very small hand tremors can be reflected in the score and evaluated. Conversely, if the value entered in the scale setting area is large (for example, 30), relatively large hand tremors can be reflected in the score and evaluated. Depending on the operator's operation, a record button 515 is displayed to record the image of the displayed ataxia evaluation area 501.

[0066] In other words, the ataxia evaluation area 501 displays the ataxia evaluation score as an ataxia evaluation value and the basis for its calculation. That is, the setting unit 217 functions as a display unit that shows the ataxia evaluation value and the basis for its calculation to the operator. This allows the operator to recognize the user's ataxia changes with greater accuracy and set the parameters of the requested action accordingly. If it is determined that the ataxia has decreased below a certain threshold, the load of the requested action can be increased. If it is determined that the ataxia has increased above a certain threshold, or if it is determined that the increased state continues (the ataxia has not improved), the load of the requested action can be decreased.

[0067] As the user's motor skills improve, the size of both the target and the sensor is reduced to increase the accuracy of point estimation, thereby making the compensatory ataxia more pronounced. By repeatedly requesting the movement in this state, or by guiding the user to perform the correct movement more easily through verbal prompts or physical contact, motor learning progresses efficiently, promoting the reconstruction of neural pathways in the brain, and thus alleviating the ataxia. This results in a rewired state of overall body alignment, and the ataxia is reduced and improved. If we liken the brain to a CPU, if the information processing process that was previously 80% dedicated to ataxia is streamlined to 20% of its 100% computing power, more complex movements become possible, and motor skills improve. Alternatively, if the CPU capacity available for cognitive processing increases, cognitive abilities also improve. In this technology, this concept is applied in the process of increasing the cognitive load, thereby increasing the information processing load on the brain and inducing the manifestation of ataxic symptoms.

[0068] Whether or not ataxia occurs is influenced by factors such as the size, angle, height, distance, direction and presence of movement of the target object, speed of movement, color, shape, visibility, size, color, shape, and visibility of devices and indicators used for point estimation such as controllers, the presence or absence of background information, and the presence or absence of background sound. The goal of this system is to induce and then suppress ataxia. In other words, ataxia serves as an indicator of improved motor skills.

[0069] The setting unit 217 controls the load of the requested action according to the degree of dysfunction in the body part the user is performing the action on. The setting unit 217 controls the size of the target object, the distance from the user, the angle, the speed, the presence or absence of a background, the presence or absence of music, and the size of the visual aid image as point estimation parameters. For example, the presence or absence of music can significantly change the degree of dysfunction. The type of music also affects the severity of the dysfunction. For example, compared to the chirping of birds (which can have a significant impact on people with poor cognitive abilities), the repetition of rhythmic music increases the amount of work the brain has to do to process the information in the background, i.e., the cognitive load, resulting in a greater degree of dysfunction. Similarly, the complexity of the background information is also strongly related to the amount of work the brain has to do to process the information in the background, i.e., the cognitive load. For example, the cognitive load is greater when there is some kind of background or scenery than when there is no three-dimensional space, and the more moving animals or objects there are, and the wider the area of ​​the screen those moving animals or objects cover, the greater the cognitive load and the more pronounced the dysfunction becomes. In other words, it is known that the more we try to improve the accuracy of point estimation, or the higher the load on the brain's background processing, such as sound and background information, the less brain capacity is available for point estimation, leading to more pronounced ataxia.

[0070] Figure 6 shows the control panel 400 when tab 413 is selected. When tab 412 is selected, the ataxia assessment area 601 is displayed to show the degree of ataxia in the head and trunk.

[0071] Specifically, the system visualizes ataxia-related movements of the head and trunk based on the position, tilt, and orientation of the head-mounted display 233 during the requested operation, as detected by base stations 231 and 232. In other words, it calculates an ataxia assessment value based on at least one of the following: twisting, horizontal movement, and tilting of the user's head and trunk during physical movement.

[0072] The ataxia assessment area 601 includes a display area 611 showing trunk (head) tilt, a display area 612 showing head (trunk) twist, a bullseye display area 613 showing the head position as viewed from above, a scale setting area 614, and a score display area 615. The value entered in the scale setting area 614 (20 in this case) indicates the maximum value of the bullseye displayed in the bullseye display area 613. By making the value entered in the scale setting area smaller (for example, 10), very small head movements can be evaluated as larger (i.e., as relatively large ataxia). Conversely, by making the value entered in the scale setting area larger (for example, 30), relatively large body sway, head movement, etc., can be evaluated as large ataxia. Trunk (head) tilt and head (trunk) twist are evaluated absolute, while deviations in the head position as viewed from above are evaluated relative. As a result, the ataxia assessment area 601 displays five types of ataxia assessment values: left angle and right angle, which indicate trunk sway; left twist and right twist, which indicate facial twisting; and a score indicating head position instability.

[0073] Furthermore, when the operator presses the record button 616, the image of the displayed ataxia evaluation area 601 is recorded. The display in display areas 611 to 613 is made possible by multiple sensors (generally a 6DoF sensor combining a 3-axis gyroscope and a 3-axis accelerometer) built into the head-mounted display 233. The tilt and motion data acquired by the sensors are processed in real time and reflected in the image displayed in the ataxia evaluation area 601.

[0074] Figure 7 shows an example of an ataxia assessment, displaying ataxia assessment areas 501 and 601 side by side. The scores shown here represent the ratio of the area of ​​movement of the left hand, right hand, or head (trunk) to the set area (the total area determined by the scale).

[0075] As described above, the system blocks the user's direct line of sight, allows them to perform point estimation, detects a disruption in the brain's neural pathways during requested actions via a target object, and repeatedly requests the action while controlling various parameters until the disruption subsides.

[0076] By requesting movements with the goal of recovering from ataxia, it is possible to very effectively improve the user's motor skills (untangle the brain's dysfunction). The ataxia assessment value serves as a guideline for motor function treatment. By stimulating the brain (point estimation requests) and observing the ataxic response, the treatment plan is determined. Effective treatment is possible by actively touching the body part where ataxia is manifesting.

[0077] Furthermore, multiple ataxia assessment values ​​may be combined into a single integrated index. In other words, each ataxia assessment value may be weighted (multiplied by a predetermined coefficient) and integrated (summed up) to perform a total ataxia assessment. In this case, the order of weighting may be changed according to the purpose of the treatment intervention. For example, if the goal is to improve hand ataxia, the hand score will naturally be the most important. On the other hand, if the goal is to improve trunk (gait) ataxia, then weighting should be adjusted to emphasize the bullseye of head ataxia. If the goal is to address ataxia during changes of direction, then tilt may be weighted as the most important factor.

[0078] Figure 8 shows the task table 800 stored in the task set database 216. The task table 800 stores the time (timing of task occurrence) 801, task interval 802, left / right type 803, task angle 804, and task distance (intensity) 805, all linked to the task ID. Furthermore, the task table 800 stores the speed of the target object 806, the standard size for a perfect judgment (excellent evaluation) 807, the standard size for a good judgment (splendid evaluation) 808, the size of the sensor object 809, the task completion result 810, and the ataxia score 711, all linked to the task ID.

[0079] Figures 9 to 12 show an example of the display in the head-mounted display 233 according to this embodiment. Adding such background information, or adding background sound as one method of implementation, both increase the information processing load on the brain and exacerbate ataxia. In Figure 9, in a background image 901 representing a townscape from the Edo period, an image 900 showing an inro (small container) 911 as the target object is displayed. Furthermore, below the inro 911, a senryobako (a box of gold coins) 913 is displayed as an item that the user must protect, and a ninja 915 is gradually approaching from the background. The speed of the ninja 915 is set in the setting area 421 of the operation panel 400 (here, speed is synonymous with the time limit). A circle 912 is displayed on the inro 911 as a visual aid image. The task is completed if the inro 911 is touched with a sensor object (the tip center of the avatar objects 241a and 241b) before the ninja 915 reaches the senryobako 913. There are two types of circles 912, red and blue. The task is to make contact with the inro 911 surrounded by the red circle 912 by manipulating the red avatar object 241b on the right, which corresponds to the controller 235 held in the right hand. On the other hand, the task is to make contact with the inro 911 surrounded by the blue circle 912 by manipulating the red avatar object 241a on the left, which corresponds to the controller 234 held in the left hand.

[0080] The Inro 911 is displayed at a position (depth and angle) set in the setting area 422 of the control panel 400. The Inro 911 does not change position until the user touches the avatar objects 241a and 241b in the virtual space. In other words, it is a target object fixed in space (also called a horizontal task because it requires the body to be stretched horizontally). Such a fixed target object is very effective as rehabilitation for conditions such as cerebellar ataxia, diplopia (including extraocular muscle ataxia), and inner ear dysfunction. That is, for patients who have forgotten how to move their bodies, feedforward allows a limited image of body movement to be imprinted on the brain, leading to the reconstruction of neural circuits in the brain. The exercise intensity can be changed by increasing the depth of the Inro 911. Furthermore, by combining this with multi-channel biofeedback, motor ability, physical function, and sensory function can be greatly improved. In addition, such horizontal tasks can also improve chronic pain by promoting the reorganization of the cerebral cortex. Alternatively, it can help restore sensory function disorders such as decreased proprioception due to nerve damage in cancer patients who have taken anticancer drugs, or sequelae symptoms associated with COVID-19 infection. The location where the target object will appear can be revealed in advance, and hints can be given to reduce cognitive load. Tactile information by touching the body, repeated verbal information, or a combination thereof are more effective in reducing cognitive load than verbal information. Verbal information can also be made simpler by giving more concise, imperative-like instructions to reduce cognitive load, or it can be made in the form of more complex instructions in the form of questions, such as "Because it's blue? (Take it with your right hand)," or it can be made in the form of verbal information that includes cognitive tasks such as calculations, such as "When you say a number divisible by 2, take it with your right hand." In addition to the horizontal position and depth from which the Inro 911 appears, the height can also be set.

[0081] Figures 10 and 11 show an example of a screen for performing a task in which the target object moves vertically (vertical task, or falling task). In Figure 10, in the background image 1001 representing a field, an image of a person representing a farmer is displayed as the trigger object 1002, which triggers the appearance of the target object. In other words, the display control unit 213 displays the trigger object 1002 as a notification image to notify the appearance of the target object 1003. After a predetermined time has passed since the trigger object 1002 threw the potato-shaped target object 1003 upwards, a large potato-shaped target object 1103 appears on the screen as shown in Figure 11. The task is completed by moving the basket-shaped avatar object 1102 to catch the falling target object 703. The left and right avatar objects 1102 move on the screen in conjunction with the movements of the controllers 234 and 235.

[0082] The setting unit 217 can adjust the cognitive load on the user by setting a delay time from the moment the trigger object 1002 throws the target object 1003 upward and notifies the occurrence of the target object 1003 until the target object 1103 is generated. The longer the delay time, the longer the period for which the memory is retained, increasing the information processing load on the brain and creating an environment that is more likely to induce ataxic symptoms. In addition, the occurrence of the target object may be notified on the radar screen image 250 at the same timing in conjunction with the movement of the trigger object 1002, or an audio notification may be combined.

[0083] Thus, the setting unit 217 can impose a cognitive load on the user not only on tasks with a background consisting only of a horizon, as shown in Figure 2, but also on tasks with a background containing a large amount of information, as shown in Figures 9 to 11. In other words, by making it difficult for the user to remember that the target object 903 appeared and where the target object 1103 fell, it is possible to provide the user with a therapeutic environment that is closer to the cognitive load required in real life and to create an environment that is more likely to induce ataxic symptoms.

[0084] In particular, the settings unit 217 changes the task mode and alters at least a portion of the background image over time, thereby imposing a cognitive load on the user 220 to process the background image in their brain. In the example in Figure 10, for example, the clouds 1004 may be moved, the plants 1005 may be shaken, or an animal (not shown) unrelated to the target object may appear in the background image 1001. This can hinder the user 220 from concentrating on the target object 1003, making it more difficult to remember the location where the target object 1103 is expected to fall. More technically, by displaying information unrelated to the task in the background image, an environment is created that makes it difficult to concentrate on the target object, and cognitive load is controlled by intentionally inducing attention deficits (more specifically, selective attention deficit, allocated attention deficit, shifting attention deficit, and sustained attention deficit) to make memory difficult.

[0085] Figure 12 shows another example of the display on the display screen according to this embodiment (vertical task, or falling task). In Figure 12, a trigger object 1202 representing a monkey and a target object 1203 representing an apple are displayed in a forest-like background image 1201. The trigger object 1202 representing the monkey drops the target object 1203 representing the apple from the tree, and the task is completed when the target object 1203 approaches the user and is caught by an avatar object 1204 representing a sieve. Here as well, the setting unit 217 induces attention impairment and imposes a cognitive load on the user 220 by causing the target object 1203 to fall after a predetermined time has elapsed from the moment the trigger object 1202 shakes the tree and notifies the user of the occurrence of the target object 1203.

[0086] Furthermore, the setting unit 217 can impose a significantly higher cognitive load on the user 220 by simultaneously having at least 2 to 5 target objects 1203 in the three-dimensional virtual space and displaying them on the display screen 240. In other words, the setting unit 217 generates at least 2 target objects 1203 at different positions in the left-right direction within the three-dimensional virtual space.

[0087] In particular, if at least two target objects 1203 are generated at multiple positions in different directions (left-right in Figure 12) relative to the direction of movement of the target object 1203 (downward direction in Figure 12), an even greater cognitive load can be imposed. In other words, the user 220 must move the controllers 234 and 235 while considering the vertical movement, the difference in generation positions in the left-right direction, and even the difference in the falling position in the depth direction, thus testing their spatial cognitive abilities. In this way, in addition to changing the predetermined time of the task, by adjusting the type, number, size, spatial extent, position, and amount of information contained in the notification image and notification sound including the trigger object, it becomes possible to quantitatively adjust and control the complexity of the information to be remembered, that is, the cognitive processing load that the user's brain must process.

[0088] The required accuracy for point estimation differs depending on which of the various modes (display screens) shown in Figures 2, 9, to 12 the user is instructed to execute. For example, compared to Figure 2, where there is no background, the required accuracy for point estimation increases in the order of Figure 9 → Figure 10 → Figure 12. Also, as mentioned above, the required accuracy for point estimation changes depending on the background sound, which affects the ataxia.

[0089] [Other embodiments] Although the present invention has been described above with reference to embodiments, the present invention is not limited to the above embodiments. Various modifications to the structure and details of the present invention can be made, as can be understood by those skilled in the art within the technical scope of the present invention. Furthermore, any system or apparatus that combines the separate features included in each embodiment is also within the technical scope of the present invention.

[0090] Furthermore, the present invention may be applied to a system composed of multiple devices or to a single device. Moreover, the present invention is also applicable when an information processing program that realizes the functions of the embodiment is supplied to a system or device and executed by a built-in processor. The technical scope of the present invention includes programs installed on a computer to realize the functions of the present invention on a computer, or a medium storing such a program, a server that downloads such a program, and a processor that executes such a program. In particular, at least a non-transitory computer-readable medium storing a program that causes a computer to execute the processing steps included in the embodiments described above is included in the technical scope of the present invention.

Claims

1. A request unit that requests the user to perform a body movement such that an object or part of the user's body that moves in conjunction with the user's movement overlaps with a target object presented in three-dimensional space so that the user can see it, An acquisition unit that detects and acquires the user's body movements, A disorder evaluation unit calculates a disorder evaluation value indicating the degree of disordered shaking of the user's body during the body movement from the user's body movements acquired by the acquisition unit, An information processing system equipped with [the following features].

2. The information processing system according to claim 1, wherein the disorder evaluation unit calculates a disorder evaluation value indicating the degree to which the object or part of the body has moved away from the straight line connecting the object or part of the body and the center of the target object.

3. The information processing system according to claim 1, wherein the disorder evaluation unit starts calculating the disorder evaluation value when the distance between the object or part of the body and the center of the target object becomes a predetermined distance.

4. The information processing system according to claim 1, wherein the ataxia evaluation unit calculates a ataxia evaluation value based on the area of ​​movement drawn by the movement of the coordinates obtained by projecting the object or part of the body onto a circle centered on the center of the target object, from the point when the distance between the object or part of the body and the center of the target object becomes a predetermined distance until the object or part of the body and the target object overlap.

5. The information processing system according to claim 4, wherein the ratio of the moving area to the set area is defined as the disorder evaluation value.

6. The information processing system according to claim 1, wherein the disorder evaluation unit calculates a disorder evaluation value that represents the integral value of the distance the object or part of the body moves away from the straight line toward the center of the target object from the point in time when the distance between the object or part of the body and the center of the target object becomes a predetermined distance until the object or part of the body and the target object overlap.

7. The information processing system according to claim 1, wherein the ataxia evaluation unit calculates an ataxia evaluation value based on at least one of the twisting, horizontal movement, and tilting of the user's head or trunk during the physical movement.

8. The information processing system according to claim 1, further comprising a display unit that displays a screen for the operator showing the aforementioned disorder evaluation value and the basis for its calculation.

9. The information processing system according to claim 1, which includes a multisensory biofeedback function that notifies the achievement of a requested action by stimulating at least two of the five senses when the requested action is achieved.

10. The information processing system according to claim 1, wherein the request unit controls the user's body movements so that they alternate between left and right.

11. The information processing system according to claim 1, wherein the request unit has a function to control the cognitive processing load.

12. The information processing system according to claim 1, further comprising a control unit that controls the difficulty of the physical movement by changing at least one of the following: the size of the target object, the distance from the user, the angle, the speed of movement, the presence or absence of a background, the presence or absence of music, and the size of the visual aid image.

13. The information processing system according to claim 1, further comprising a head-mounted display for displaying the target object in a virtual space.

14. The information processing system according to claim 1, further comprising a shielding part that shields the user from seeing part or all of the user's physical movements.

15. The information processing system according to claim 1, further comprising a head-mounted display that displays the target object in a virtual space and shields the user's eyes from seeing part or all of the user's body movements.

16. A request step in which the requesting unit requests the user to perform a bodily movement such that an object or part of the user's body that moves in conjunction with the user's movement overlaps with a target object presented in three-dimensional space so that the user can see it, The acquisition step involves the acquisition unit detecting and acquiring the user's body movements, The ataxia evaluation unit calculates an ataxia evaluation value indicating the degree of ataxic sway of the user's body during the body movement from the user's body movements acquired in the acquisition step, Information processing methods including

17. A request step that requests the user to perform a bodily movement such that an object or part of the user's body that moves in conjunction with the user's movement overlaps with a target object presented in three-dimensional space so that the user can see it, The acquisition step involves detecting and acquiring the user's body movements, An evaluation step which calculates a disorientation evaluation value indicating the degree of disorientation of the user's body during the body movement from the user's body movements acquired in the acquisition step, An information processing program that causes a computer to execute something.

Images