Information processing system, information processing method, and information processing program

Abstract

To set parameters of a user's reaching target quickly while viewing the user's movement.SOLUTION: An information processing device includes a control unit for causing a target object which is a target of a user's reaching to appear in a virtual space, and a setting unit for setting an appearance position of the target object in response to the input from a keypad. Using an information processing device like this, an operator can quickly set parameters of the user's reaching target while viewing the user's movement.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and an information processing program.

Background Art

[0002] In the above technical field, Patent Document 1 discloses a system for use with hemiplegic patients due to stroke or the like.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, with the technology described in the above document, it was not possible to quickly set the target position of the user while observing the user's movement.

[0005] An object of the present invention is to provide a technology for solving the above problems.

Means for Solving the Problems

[0006] To achieve the above object, an apparatus according to the present invention includes a control unit that causes a target object, which is a target to be reached by a user, to appear in a virtual space, and a setting unit that sets the appearance position of the target object in response to an input from a numeric keypad, and is an information processing apparatus provided with these.

[0007] To achieve the above object, a method according to the present invention includes a control step in which a control unit causes a target object, which is a target to be reached by a user, to appear in a virtual space, and A setting step in which a setting unit sets the appearance position of the target object according to an input from a numeric keypad It is an information processing method including

[0008] To achieve the above object, the program according to the present invention A control step of causing a target object that is a target reached by a user to appear in a virtual space A setting step of setting the appearance position of the target object according to an input from a numeric keypad It is an information processing program that causes a computer to execute

Effect of the Invention

[0009] According to the present invention, parameters of a user's reaching target can be quickly set while observing the user's movement.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2A

Figure 2B

Figure 2C

Figure 3A

Figure 3B

Figure 4

Figure 5

Figure 6A

Figure 6B

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Mode for Carrying Out the Invention

[0011] Hereinafter, with reference to the drawings, embodiments of the present invention will be exemplarily described in detail. However, the components described in the following embodiments are merely examples, and are not intended to limit the technical scope of the present invention thereto.

[0012] [First Embodiment] The information processing apparatus 100 according to the first embodiment of the present invention will be described with reference to FIG. 1. The information processing apparatus 100 is an information processing apparatus that requests a reaching operation from the user 110. Note that the reaching operation generally means an action of stretching the hand, but here, for the sake of convenience, any action of moving a specific part of the body toward a specific coordinate in the three-dimensional space will be expressed as a reaching operation. That is, the reaching operation includes not only the so-called upper limbs including the hand and the forearm, but also actions of moving parts such as the shoulder, elbow, lower limbs (knee, foot), and head toward coordinates in the three-dimensional space.

[0013] As shown in FIG. 1, the information processing apparatus 100 includes a control unit 101 that causes target objects 151 and 152, which are targets for the user 110 to reach, to appear in the virtual space 150, and a setting unit 102 that sets the appearance positions of the target objects 151 and 152 in response to an input from the numeric keypad 170.

[0014] According to the above configuration, the target position of the user can be quickly set while observing the user's movement.

[0015] [Second Embodiment] Next, the information processing system 200 according to the second embodiment of the present invention will be described with reference to FIG. 2A. FIG. 2A is a diagram for explaining the configuration of the information processing system 200 according to the present embodiment. The information processing system 200 is a medical device including an information processing apparatus 210 as a computer equipped with rehabilitation support software, a base station 231 as a high-performance VR device, and a head-mounted display 233. The rehabilitation support system 200 according to the present embodiment can improve at least one functional abnormality of physical functions such as upper limb function, walking function, trunk function, and balance function, cognitive functions (including spatial cognition, attention function, and higher brain functions), and sensory functions (including inner ear, vestibular system, tactile sense, temperature and pain sensation, position sense, and deep sensation).

[0016] The information processing system 200 shown here is an example of a medical system that provides an environment for manifesting abnormal joint movements or cooperative movement disorders, and the present invention is not limited to this embodiment. That is, the present invention should not be limited to the concept of "rehabilitation", and treatment systems that actively treat physical diseases such as paralysis and ataxia, as well as cognitive and mental diseases such as dementia and schizophrenia, are also included in the present invention. Furthermore, an operation verification system, an ability improvement and enhancement system, and a sports player-oriented movement ability improvement system for improving the movement ability and cognitive ability of healthy people who do not require any treatment or rehabilitation are also included in the present invention. That is, this software may be more specifically expressed as software that provides a treatment method called somato-cognitive cooperative therapy, and cooperative movement disorders may be more specifically expressed as abnormalities or entanglements of the Somato-Cognitive Action Network (SCAN). Many users with cooperative movement disorders are completely unaware of them.

[0017] As shown in FIG. 2A, the rehabilitation support system 200 includes an information processing device 210, a base station 231, a head-mounted display 233 as a blocking unit, and two controllers 234 and 235.

[0018] The user 220 wears the head-mounted display 233 and sits on the chair 225, and moves the body according to the display of the head-mounted display 233. The user 220 holds or fixes the controllers 243 and 234 in both hands and performs a reaching motion in the seated position according to the display on the head-mounted display 233 or the voice of the operator 280.

[0019] The base station 231, the head-mounted display 233, and the controllers 234 and 235 as high-performance VR devices can accurately measure in millimeters in a three-dimensional space. That is, the postures and positions of the head-mounted display 233 or the controllers 234 and 235 are accurately measured.

[0020] In this embodiment, the description will be given on the premise of verifying the operations performed while sitting on the chair 225. However, the present invention is not limited to this, and it may be performed while standing, on a bed, in a supine position or a prone position, or while performing other specific operations. Also, here the user 220 is holding the controllers 234 and 235 with both hands, but the present invention is not limited to this, and it may be held or worn on body parts other than the hands, such as the feet or the torso.

[0021] In the case of an athlete or the like, while walking or running on a treadmill, the reaching operation in this system may be performed. In this case, the speed of the treadmill may be changed according to the degree of achievement of the required operation.

[0022] Note that the head-mounted display 233 may be of a non-transmissive type, a video see-through type, an optical see-through type, or a glasses type. In this embodiment, a virtual space of VR (Virtual Reality) is presented to the user, but like AR (Augmented Reality), the real space and the virtual space may be superimposed and displayed, or like MR (Mixed Reality), real information may be reflected in the virtual space, or hologram technology may be used as an alternative means.

[0023] What is important here is that by wearing the head-mounted display 233, all or part of the user's own body becomes directly invisible. Usually, in real life (real space), humans unconsciously make fine adjustments to their movements while visually directly recognizing the movements of their bodies and move them to the target (visual recognition correction). In this system, since the user 220 cannot directly see their own hand in the real space, the user's brain cannot accurately perform visual recognition correction on the movement of the hand as usual.

[0024] Therefore, the cooperative movement disorder (entanglement of SCAN) becomes apparent, and abnormal movements (involuntary movements) of joints different from the parts where body movements are performed are likely to be detected. In this embodiment, the head-mounted display 233 is used, but for example, a contact lens type display or a glass type display that displays a virtual space may be adopted. Furthermore, a projection mapping system that displays a background on all or part of the body of the user 220 and makes it look as if all or part of the body is transparent may be adopted. As a device that can easily and completely block the vision of the user 220, a non-transmissive type head-mounted display is the most convenient to use.

[0025] The information processing apparatus 210 includes an operation detection unit 211, a control unit 212, a feedback unit 213, a setting unit 214, and a reaching evaluation unit 215.

[0026] The head-mounted display 233 and the controllers 234, 235 receive an infrared signal from the base station 231 and send it to the operation detection unit 211. The operation detection unit 211 calculates the position and orientation of the head-mounted display 233 and the controllers 234, 235 from the temporal pattern of the infrared signal.

[0027] The control unit 212 controls the display of the head-mounted display 233 based on the position and orientation of the head-mounted display 233. Specifically, the position and orientation of the viewpoint on the virtual space are changed according to the position and orientation of the head-mounted display 233, and the image to be displayed on the head-mounted display 233 is determined. The control unit 212 also causes a target object that the user aims to reach to appear in the virtual space.

[0028] The feedback unit 213 detects and evaluates the rehabilitation movement of the user 220 based on the position and / or movement of the controllers 234, 235 held by the user 220.

[0029] In the present embodiment, as an example of a sensor for detecting the position or movement of a user's hand or head, the controllers 234 and 235 of the type held by the user 220 and the base station 231 are shown, but the present invention is not limited thereto. A camera (including a depth sensor) for detecting the "position or movement of the hand itself" of the user by image recognition processing, a sensor for detecting the position of the user's hand by temperature, a wristwatch-type wearable terminal worn on the user's arm, and motion capture can also be applied to the present invention by being linked with the motion detection unit 211. That is, one embodiment is to use a three-dimensional tracking device such as Kinect (registered trademark) or a motion analysis device, or to wear a marker or the like on the body.

[0030] The control unit 212 causes the target objects 241 and 242 that are the targets for the user 220 to reach to appear in the virtual space 240, and prompts the user 220 with three-dimensional body movements through the display of the head-mounted display 233. In particular, the control unit 212 generates a target object 241 for three-dimensionally rehabilitating a part of the user 220's left body and a target object 242 for rehabilitating a part of the user 220's right body in the virtual space 240. That is, the control unit 212 functions as a request unit that requests the user 220 to reach the target objects 241 and 242.

[0031] Here, reaching a virtual target displayed on the head-mounted display 233 is required, but the present invention is not limited to this, and an object provided in the real world may be used as the target. An object suspended by two or more wires, an object floating in the air by ultrasonic vibration, etc. may be used as the target. Also, an object three-dimensionally displayed as a hologram may be used as the target. The shape of the target object is not limited to a spherical shape, and may be triangular, square, dish-shaped, or any character shape. A system that uses an optical see-through or smart glass type head-mounted display to partially block the user's vision and make the user's own movements invisible while allowing only the target to be visible is also included in the concept of the present invention.

[0032] The control unit 212 generates the target objects 241 and 242 in the virtual space 240 and moves them, for example, from above the user 220 downward. Here, the number of target objects that the control unit 212 can simultaneously display in the virtual space 240 is not limited to two, and three or more target objects may be simultaneously generated and displayed in the virtual space.

[0033] In the head-mounted display 233, the display position and size of the target object 241 gradually change according to the distance between the target object and the viewpoint position in the virtual space (for example, in the case of falling, it gradually becomes larger and then smaller). Note that the moving direction of the target object is not limited to the downward direction, and may be, for example, the upward direction from the floor surface to the overhead direction. The target objects 241 and 242 may be moved so as to come from the back to the front toward the user 220. The target objects 241 and 242 may be moved so as to pass in front of the user 220 from right to left or from left to right. Furthermore, the movements in the vertical direction, depth direction, and horizontal direction may be appropriately combined. For example, three-dimensional movements such as coming while falling or sliding to the right while falling are also conceivable. It is important to cause an operation according to the user's cognitive ability and motor ability. When at least one of the cognitive function and motor function of the user 220 is extremely low, the target object may be fixed at a specific coordinate position without moving the target object to encourage the user's reaching. For users 220 who are almost immobile or immobile due to hemiplegia, cerebral palsy, etc., or bedridden users 220 due to higher brain dysfunction, severe intellectual disability, severe dementia, etc., a controller may be held or fixed to body parts such as the user's hand, elbow, and shoulder, and the target object may be dropped at that position so that the user can touch the target object (this is called intracerebral reaching in the sense of not involving physical movement). In this case, although it may seem at first glance that there is no reaction from the user until the information transmission neural pathways between the brain and the body related to body movements and the information transmission neural pathways related to cognition are connected, by repeatedly stimulating the brain, these body-cognition information transmission neural pathways will be connected and the body will start to move or a cognitive reaction will be obtained.

[0034] The control unit 212 generates avatar objects 243 and 244 that move according to the actions of the user 220 detected by the base station 231 within the virtual space 240. The avatar object 243 here is an object that moves corresponding to the controller 234 operated by the left hand, and represents the position of a part of the user's left body. The avatar object 244 is an object that moves corresponding to the controller 235 operated by the right hand, and represents the position of a part of the user's right body. The control unit 212 requests the user to perform a body movement that superimposes the avatar objects 243 and 244 on the target objects 241 and 242.

[0035] The avatar objects 243 and 244 are not limited to objects that move corresponding to the movements of the controllers 234 and 235. They may be virtual objects that move along with changes in the positions of parts of the user's body using three-dimensional body tracking technology as described above. The objects operated by the user are also not limited to the controllers as shown in the figure, and may be, for example, a racket, bat, or golf club equipped with a position sensor.

[0036] The control unit 212 causes the avatar objects 243 and 244 and the target objects 241 and 242 to be displayed on the display screen 245 of the head-mounted display 233 according to the orientation and position of the head-mounted display 233 detected by the motion detection unit 211. The images of the avatar objects 243 and 244 and the target objects 241 and 242 are superimposed and displayed on the background image. Here, the avatar objects 243 and 244 have the same shape as the controllers 234 and 235, but are not limited to this, and may be in the shape of a hand. Furthermore, the size, shape, and color may be changed between the left and right. The background image is cut out from the virtual space including the horizon 246 and the ground surface object 247. According to the movement of the line of sight (the position and orientation of the head-mounted display), the appearance of the background image, the avatar objects 243 and 244, and the target objects 241 and 242 changes.

[0037] The control unit 212 generates an avatar object 244 indicating the position of a part of the right body and a target object 242 indicating the target position of the avatar object 244 on the right side in the same color system or the same shape. Further, the control unit 212 generates an avatar object 243 indicating the position of a part of the left body and a target object 241 indicating the target position of the avatar object 243 on the left side in the same color system or the same shape. Also, it is desirable to set the color or shape of the left avatar object 243 and the left target object 241 to be different from the color or shape of the right avatar object 244 and the right target object 242.

[0038] The operations of the users 220 moving the controllers 234, 235 are rehabilitation operations, and the display of the target object that prompts one rehabilitation operation that the user 220 should perform is called a task or an assignment. Here, as an example, the avatar objects 243 and 244 are color-coded, for example, in blue and red, and the target objects 241 and 242 are also color-coded in blue and red. Touching the blue avatar object 243 to the blue target object 241 results in task completion. Similarly, touching the red avatar object 244 to the red target object 242 results in task completion. That is, touching avatar objects of different colors does not result in task completion. Although color-coding in blue and red is shown here as an example, for color-blind users, it may be displayed in other color-codings (such as yellow and green), or without using colors, by shapes, or by linguistic notations such as "left", "right", "L (Left)", "R (Right)", or by notations of symbols such as star marks, triangle marks, or circle marks, or by different patterns (stripes) such as 243 and 244 in the figure so that they can be distinguished.

[0039] As information representing one task (task data), it includes the appearance direction of the target object (90 degrees to the right, 45 degrees to the right, straight ahead, 45 degrees to the left, 90 degrees to the left with respect to the front direction of the chair) (※ alternatively, it may be expressed as 0 degrees for directly to the left, 45 degrees for diagonally in front to the left, 90 degrees for straight ahead, 135 degrees for diagonally in front to the right, 180 degrees for directly to the right like a protractor), the distance to the target object (such as short distance, middle distance, long distance), the size of the target object, the appearance interval (time) of the target object, the moving speed of the target object, the number of target objects appearing simultaneously, the size of the sensor object, etc. The depth direction distance from the user 220 to the falling positions of the target objects 241 and 242 can also be set in, for example, three levels. For example, it can be changed to drop right next to the user 220, drop at a position that the user 220 cannot reach without a large forward lunge, or drop at an intermediate position. Thereby, the exercise load imposed on the user, and the load on the spatial perception ability or the spatial grasping ability can be controlled.

[0040] [Feedback] When the avatar objects 243 and 244 collide with the target objects 241 and 242, the control unit 212 eliminates the target objects 241 and 242, and the feedback unit 213 performs visual feedback using visual effects such as displaying a message for the purpose of notifying or informing that the target action has been achieved, or bursting the target object, assuming that the target action has been achieved. If the avatar objects 243 and 244 reach the centers of the target objects 241 and 242, "Well done" is displayed, and if they reach the peripheral area, "Great" is displayed for each task. Thereby, the achievement of each required physical action is fed back to the user while distinguishing the achievement accuracy. The feedback unit 213 may perform feedback by stimulating one or more of the five senses of the user. If feedback is performed by stimulating two or more of the five senses of the user, the cognitive ability or motor ability of the user, or both, can be effectively improved.

[0041] That is, the feedback unit 213 notifies the user 220 of the achievement of each rehabilitation operation for the target objects 241 and 242 respectively. Various methods can be considered for the notification here. As described above, characters such as "Great (Perfect or perfect)" and "Wonderful (Good or good)" can be temporarily displayed in the display screen 245 to inform the user of the achievement of the target operation. Similarly, auditory stimuli can be used to teach the user about the achievement of the target operation through similar voices or sound effects. Furthermore, among the controllers 234 and 235, only the controller that has moved to contact the target object can be vibrated simultaneously to notify the user of the achievement of the target operation through tactile stimuli. Alternatively, the achievement notification may be graded, for example, as complete achievement, incomplete achievement, or inability to achieve, to notify the degree of achievement, or taste stimuli and olfactory stimuli may be used.

[0042] More specifically, when the shortest distance between the sensor objects (spherical objects that define the position and area of the avatar objects) included in the avatar objects 243 and 244 and the target objects 241 and 242 is within a predetermined range, the target is achieved. Then, the target objects 241 and 242 disappear. The sensor object is, for example, a spherical object that includes the center point of the tip of the avatar objects 243 and 244. The sizes of the target objects 241 and 242 and the sizes of the surrounding parts can be set by the setting unit 214. Also, the size of the sensor object can be set by the setting unit 214.

[0043] The feedback unit 213 preferably changes the message type according to the evaluation of the rehabilitation operation via the control unit 212. For example, if the sensor object touches the center of the target objects 241 and 242, "Great" is displayed, and if the sensor object touches only the surrounding part of the center of the target objects 241 and 242, "Wonderful" is displayed, etc.

[0044] For example, on the screen of FIG. 3B, the radius of the target object is adjusted to 10 cm, the radius of the visual assistance object is adjusted to 20 cm, and the radius of the sensor object is adjusted to 2 cm. If a part of the sensor object (a sphere with a radius of 2 cm) overlaps with the target object even slightly, it is a Perfect determination. On the other hand, if the target object is not touched but the visual assistance object is touched, it is a Good determination. However, at this time, even if it enters the Good determination range, as long as the controller is directed towards the Perfect radius direction, the hit determination of the Good determination is not finalized. It becomes a Good determination under the condition that it enters the Good determination range and the sensor object moves away from the center of the Perfect radius. In this embodiment, the size of the visual assistance object is made to match the size of the Good determination area, but the present invention is not limited to this, and the visual assistance object may be larger or smaller than the Good determination area.

[0045] If the shortest distance between the outer edges of the target objects 241 and 242 and the outer edge of the sensor object becomes equal to or less than the first threshold, it is regarded as the complete achievement of the target, and for example, an image of "Great!" is displayed and the corresponding sound is output for feedback. If it is greater than the first threshold but equal to or less than the second threshold, it is regarded as the achievement of the target, and for example, an image of "Well done!" is displayed and the corresponding sound is output for feedback. However, as described above, even if it is equal to or less than the second threshold, if the sensor object is moving in a direction such that it will be equal to or less than the first threshold, the determination that it is equal to or less than the second threshold is not made.

[0046] Such feedback that stimulates a plurality of different senses (here, vision and hearing) is called multi-channel biofeedback, or multi-signal biofeedback, or multi-sensory biofeedback. At the same time, the controllers 234 and 235 may be vibrated to stimulate the sense of touch, or stimulation may be given to the sense of smell or taste.

[0047] That is, the feedback unit 213 can stimulate any one of the five senses of vision, hearing, touch, taste, and smell as feedback. These sensory stimulations may be combined in any two, or three or more, or all of them may be stimulated.

[0048] Note that the output voice does not have to be the same as the message. For example, a non-verbal sound effect such as "peeron" may be used. Note that as the distance required for the contact between the sensor object and the target object is reduced in this way, a more accurate operation is required. Therefore, the brain needs to require the body to perform more accurate and refined operations as movement commands. As described above, this mental imaging of the brain is called feedforward. That is, the smaller the distance between the sensor object and the target object required for the completion of the operation, the more powerful the feedforward is required, and this can continuously adjust the load levels of the user's movement, cognition, and sensation.

[0049] Here, the feedback unit 213 evaluated and notified in two levels of excellent and wonderful. However, the rehabilitation operation may be evaluated in three or more levels according to how much the distance between the sensor object and the target objects 241 and 242 has been reduced.

[0050] The feedback unit 213 provides feedback that stimulates two or more of the five senses (vision, hearing, touch, taste, smell) to the user who virtually touches the target objects 241 and 242. This feedback is provided almost simultaneously with the timing when the sensor object enters within a predetermined distance from the centers of the target objects 241 and 242, or the timing when the sensor object comes into contact with the target objects 241 and 242 (referred to as real-time multi-channel biofeedback, or immediate multi-signal biofeedback, or immediate multi-sensory biofeedback). The delay from those timings to the feedback is highly effective if it is, for example, within 1 second, and the closer the interval between the user's operation timing and the feedback timing (the smaller the delay), the greater the effect. The feedback unit 213 provides feedback that stimulates the user's vision with an image saying "Well done!" and at the same time, or with a delay of a few seconds, provides feedback that stimulates the user's hearing with voice or sound effects output from a speaker. The notification of task achievement for the five-sense stimulation may be combined in any way according to the operation type.

[0051] Furthermore, the feedback unit 213 may simultaneously output visual feedback with an image saying "Well done!", auditory feedback with voice, and tactile feedback with vibration. Also, the feedback unit 213 may simultaneously output only two types of feedback, visual feedback with an image saying "Well done!" and tactile feedback with vibration. Alternatively, the feedback unit 213 may simultaneously output only two types of feedback, auditory feedback with a voice saying "Well done!" and tactile feedback with vibration.

[0052] [Joint-related] Here, although the avatar objects 243 and 244 are visible to the user 220, the actual body movements are not directly visible. As a result, part or all of the body is blocked from view, increasing the burden on the user's brain. When part or all of one's own body (including the controller in the real space) is not visible, the brain has to make an estimation, more specifically in FIG. 2A, to overlay (touch) the position (point) of the sensor object at the tip of the controller in the virtual space and a specific coordinate in the three-dimensional space, more specifically the position of the target object in FIG. 2A. This is called point estimation.

[0053] When point estimation is performed, the brain has to construct a body image (body image construction) in the brain quite strongly without relying on vision. In medicine, this is expressed as "requiring a strong feed-forward". A brain that is required to have a strong feed-forward makes it more difficult to coordinate the body, resulting in the manifestation of coordination disorders and making it easier for abnormal movements of joints different from the parts where body movements are performed to occur. Also, when a strong feed-forward that requires point estimation is performed, it becomes possible to induce the contraction of the deep muscles of the body by stimulation through the pyramidal tract or the like.

[0054] When a large load is placed on the brain, for example, when the avatar image 244 of the right controller 235 reaches the right target object 242, there is a user 220 who unconsciously applies force to the knee joint of the left foot and the lower leg of the left foot moves forward or backward (221). In this specification, the phenomenon in which joints that are not necessary for such required movements move involuntarily and unintentionally is called articular linkage, or an abnormality of the Somato-Cognitive Action Network (SCAN) or an entanglement of the SCAN. The articular linkage can be used as an indicator for improving the movement of the user 220. Note that this articular linkage is manifested even without an anatomical connection called an anatomical train. For example, when the joints of the toes of the foot are moved voluntarily, involuntary movements occur in the temporomandibular joint, hip joint, shoulder joint, etc. This is a phenomenon that is completely different from the phenomenon in which the left hand moves to balance when the body leans to the right, and is a phenomenon that cannot be seen in healthy people.

[0055] Articular linkage is a phenomenon in which joints that should not move actually move, and can also be expressed as involuntary movements that appear in joints other than the intended movement site. Intervention is performed by paying attention to this articular linkage. So-called bug fixing of the information processing network in the brain is performed. According to the applicant, by minimizing the visual information of the user and requiring left-right alternating reaching movements in a sitting position or the like while performing point estimation, and by notifying the degree of movement achievement by multisensory biofeedback when the movement is achieved, it has been empirically found that the articular linkage can be improved immediately. Thereby, the cooperative movement disorder can be treated and improved. In other words, the information processing process between the brain and the body can be organized (untangled). The articular linkage can also be referred to as the entangled somato-cognitive action network as described above. Based on the same principle, it is also an effective treatment method for patients called developmental disorders and hyperactivity. It has also been found that even patients who have had their brains greatly removed by brain tumor surgery can be compensated for by the system according to this embodiment.

[0056] As shown in Figure 2C, which was also mentioned in the non-patent literature (Nature "A somato-cognitive action network alternates with effector regions in motor cortex" by Evan M. Gordon et al. Published online: 19 04 2023), the motor areas responsible for coordinated movement exist across all joints. Looking at this Figure 2C, it can be seen that in the latest neurology, the coronal section diagrams of the primary motor areas of the brain are grouped for each symmetric part (such as the right and left hands, right and left feet, right and left eyes, etc.) (※ actually, they are distributed concentrically in three dimensions), and these motor areas are arranged symmetrically in the brain. Furthermore, it has newly been found that there are nerve regions responsible for the coordination of each body part shown by the puppet between the foot motor area and the hand motor area, and between the hand motor area and the face motor area. Due to the existence of the nerves that coordinate each body part like a puppet, it is considered that the joint linkages during movement become apparent in a specific environment where point estimation is required in a situation where all or part of the body is not visible.

[0057] The inventor actually arrived at the creation of the present invention by observing the rehabilitation movements of actual users even before the publication of the above non-patent literature, and it can be said that its usefulness and logic have been theoretically supported by this paper.

[0058] In Figure 2A, the movement of the knee joint is illustrated as an example of the occurrence of joint linkage 221, but the present invention is not limited to this, and all involuntary movements of joints different from the body part performing the required movement are included in the concept of the present invention. For example, as shown in Figure 2B, by detecting joint planes 371 and 372 and calculating their inclinations, it may be possible to detect the simultaneous occurrence of torsion of the shoulder joint and the trunk (cervical vertebra joint, thoracic vertebra joint) and torsion of the waist (lumbar vertebra joint, hip joint).

[0059] Specifically, joint associations occur as follows: torsion of the cervical facet joints (manifested in the direction of the head), torsion of the thoracic facet joints (manifested in the angle of the shoulders), elevation and abduction of the shoulder joints (the sides open), extension and contraction of the knee joints, hip joints, and ankle joints (the knees and toes move up and down or invert and abduct), extension and contraction of the elbow joints (the arm not required to move is fixed in a flexed position), extension of the temporomandibular joints (the mouth opens), flexion progression of the metacarpophalangeal joints and interphalangeal joints, flexion (claw toe) and progression of the interphalangeal joints of the toes, etc. Such joint movements include not only flexion and extension, but also adduction, abduction, internal rotation, external rotation, circumduction, or combinations thereof, as well as movements not described herein.

[0060] A joint association evaluation unit 219 may be provided that installs a camera 236 to detect and evaluate the position, torsion, inclination, movement speed, acceleration, etc. of each joint of the user 220 during rehabilitation movements. The joint association evaluation unit 219 detects the movement of each joint of the user 220 and evaluates whether joint associations are occurring based on the magnitude, speed, timing, etc. of the movement.

[0061] Specifically, movements that occur even in healthy individuals, such as the swinging of the hands observed during walking, are not judged as joint associations, while movements that are not expected to occur in healthy individuals, such as the movement of both lower limbs, the trunk, and the inclination of the head and neck when reaching out the hands while sitting, are judged as joint associations. Alternatively, the system may define joint associations according to the direction and magnitude of the movement of specific joints.

[0062] The camera 236 photographs the user, and the joint kinematic evaluation unit 219 can detect abnormal torsions and movements of the joint surfaces by comparing with the image in the normal state (before exercise) or with a normal motion model. The joint kinematic evaluation unit 219 may be provided with a database of movements to be judged as joint kinematics and / or a database of movements not to be judged as joint kinematics. Sensors may be attached to the user's joints (e.g., both shoulders, both elbows, wrists, both knees, both ankle joints, etc.) (or by motion capture), and the base station 231 may detect the positions thereof to detect joint kinematics. Of course, the operator may visually recognize the occurrence of joint kinematics and input it into the system.

[0063] The joint kinematic evaluation unit 219 may detect the inclination of the head-mounted display 233 by the base station 231 to detect the joint kinematics of the neck. For example, if the head-mounted display 233 is inclined by 30 degrees, it can be determined that the trunk is 15 degrees (judgment based on the camera image) and the neck is 15 degrees. The inclination of the wrist joint can be determined from the positions and inclinations of the controllers 234 and 235.

[0064] Six joints of the hip joint, knee joint, ankle joint, neck joint, trunk, and elbow joint may be intensively detected. Further, it may be divided into more classifications, and all joint units (a person has 68) may be examined, or conversely, a device may be devised to focus on one or a small number of joints that are most likely to be detected as abnormal. Further, a plurality of evaluation indexes may be converted into one integrated index or parameter such as a total score. That is, weights may be assigned to the joint kinematics of the whole body or important parts and integrated to perform a total joint kinematic evaluation (calculation of a joint kinematic score).

[0065] If the required operation is to overlap a 2-cm sensor on a 10-cm target, the accuracy of point estimation is low. However, for the required operation of overlapping a 1-cm sensor on a 1-cm target, high-accuracy point estimation is required. In this case, if the user uses a blocking part such as a head-mounted display, the visual information of the real world is blocked, resulting in an inconvenient state. Therefore, the cranial nerves for point estimation need to be sharpened, and the associated movement disorder becomes apparent, that is, the joint association appears more prominently. This is one of the mechanisms of joint association occurrence.

[0066] If the rehabilitation level (the user's motor ability or cognitive ability) improves, by reducing the sizes of both the target and the sensor, the required accuracy of point estimation can be increased, and the compensatory joint association is made more prominent and induced. By repeatedly requiring rehabilitation operations in that state, or by guiding the correct operation to be performed more easily through vocalization or physical contact, the joint association will be cured. As a result, the overall body alignment is adjusted, and the SCAN becomes rewired, and the associated movement disorder is alleviated and improved.

[0067] The joint association evaluation unit 219 may display the detected position and size of the joint association on the operator screen 290 or the like. It may further have a comparison unit that compares with past joint associations and a normal operation model, and notify the operator of the comparison result. Thereby, the operator can recognize the change of the joint association more accurately.

[0068] The joint association evaluation unit 219 may generate and display a heat map in a bull's-eye image or the like with the movement as a joint association in terms of distance, acceleration, etc., or may represent the change over time of the joint association and provide information useful for controlling the difficulty of the operation. Specifically, it may visualize how the joint association changes from the start to the end of the required reaching operation, or may compare the sizes of the joint associations of each joint. It becomes easier to determine which joint to pay attention to.

[0069] The operator 280 or the information processing device 210 requests a rehabilitation operation so that the joint connection becomes smaller and causes the user 220 to achieve it.

[0070] For example, if the brain is compared to a CPU, and out of its computing power of 100, the information processing process that was taken up by 80 for body movement processing due to the existence of joint connection is reorganized to 20, more complex operations than those required become possible, and the motor ability is improved. Or, according to the idea that if the CPU itself that avoids cognitive processing improves, the cognitive ability and attention function also improve.

[0071] When it is determined that the joint connection has become smaller than a certain reference, the load of the rehabilitation operation is increased. When it is determined that the joint connection has become larger than a certain reference value or that a state where it has become larger than a predetermined value has continued for a predetermined time or more (the joint connection has not improved), the load of the rehabilitation operation is decreased.

[0072] Note that whether or not joint connection occurs is affected by factors such as the size, angle, height, distance, direction of movement or presence / absence of movement, speed of movement, color, shape, visibility of the target object, the size, color, shape, visibility of the device or index for superposition by point estimation such as a controller, the presence / absence of background information, and the presence / absence of background sound. Causing joint connection and calming it down is the goal of the rehabilitation operation in this system. That is, joint connection becomes the guideline for treatment. During the rehabilitation operation, pressing the user's knee or shoulder may cause the joint connection to disappear. In this way, it is also possible to manually and forcibly correct the fraying of the intracerebral neural circuit.

[0073] In a situation where reaching (point estimation) is performed while blocking vision, feedback is provided each time the required operation is achieved in this way, which triggers the brain to correct and improve the movement disorder. That is, the joint connection and the movement disorder are improved. This improvement progresses more efficiently as the number of sensory stimulations during feedback increases, or the improvement is retained as memory in the long term. For example, compared to one-sensory feedback, symptoms that were improved over several days by performing two-sensory feedback will be improved in units of time. Or compared to two-sensory feedback, symptoms that were improved over several hours by performing three-sensory feedback will be improved in units of minutes. Furthermore, even for symptoms that returned to their original state immediately after improvement during one-sensory feedback, the improvement effect will be maintained for several days by performing two-sensory feedback. Or even for symptoms that returned to their original state within several days after improvement during two-sensory feedback, the improvement effect will be maintained for several weeks by performing three-sensory feedback. To express the characteristics of this treatment, in the medical field, this treatment is also expressed as "brain reprogramming therapy (BRT)", "brain re-wiring therapy (BRT)", or "motor coordination therapy (MCT)", "somat-cognitive coordination therapy (SCCT)", etc.

[0074] The flow of accuracy control for point estimation of joint connection is as follows. (1) Perform rehabilitation movements with default settings. (2) The joint connection is too large (the angle or distance of the crab thigh, shoulder torsion, waist torsion, head torsion, etc. exceeds the threshold). (3) Lower the required difficulty of point estimation (change the point estimation parameters). (4) The joint connection becomes moderately smaller. (5) Repeat the rehabilitation movements alternately left and right. (6) The joint connection disappears. (7) Increase the required accuracy of point estimation (change the point estimation parameters). (8) The joint connection becomes of moderate size. (9) Return to (5).

[0075] In addition, in (5), it may be possible to suppress joint entanglement by touching the user's body or speaking to the user. Also, in (7), it may be possible to apply a load to the user by speaking and achieve the same effect as increasing the required accuracy of point estimation. It has been found that by touching the user's body, the user's joint entanglement can be resolved more quickly.

[0076] As described above, point estimation is performed by blocking the user's direct field of vision, and deep muscles contract through a powerful motion command via a target object to align the whole body. At this time, joint entanglement or a disruption of the cranial nerve called entangled SCAN is detected, and while controlling the required accuracy of point estimation (various parameters), rehabilitation operations are repeated until the joint entanglement subsides.

[0077] When multiple joint entanglements appear, it is necessary to determine which joint entanglement to correct mainly. For example, first correct the largely moving knee (hip joint), and then move on to the next joint (for example, the involuntary movement of the ankle joint causes the sole of the foot to turn inward).

[0078] If rehabilitation operations are performed with the goal of recovering joint entanglement, it is possible to very effectively improve the user's cognitive motor ability (relax the brain's fraying). Joint entanglement serves as a guide for treatment. Stimulate the brain (point estimation requirement) and judge the treatment policy by observing the reaction of joint entanglement. Effective treatment becomes possible by actively contacting the body part where joint entanglement appears.

[0079] [Radar Screen] The control unit 212 causes the radar screen image 250 to be displayed on the display screen 240 of the head-mounted display 233. The radar screen image 250 is a notification image for notifying the occurrence of the target object 152. The radar screen image 250 notifies in which direction the positions of the next-occurring target objects 241 and 242 are relative to the reference direction (usually set to the front direction of the chair 225 by calibration) in the virtual space. The radar screen image 250 further notifies how far the positions of the occurring target objects 241 and 242 are from the user 220. Note that the notification image is not limited to the radar screen image, and may be notified by characters, arrows, symbols, illustrations, types, intensities, blinking, etc. of light and color. Also, the notification method is not limited to images, and may be performed by voice, vibration, or any combination of voice, vibration, and images.

[0080] Regardless of the orientation of the user 220's head, the control unit 212 causes the radar screen image 250 to be displayed in the central portion (for example, within the range of -50 degrees to 50 degrees) of the display screen 240 of the head-mounted display 233. However, the display portion is not limited to the center, and may be any location such as the four corners, upper end, lower end, left end, or right end of the screen. The patient can estimate the difficulty of the next movement operation from the information on the position, angle, and number of the target objects displayed on the radar screen. When a more difficult operation is predicted by the patient, the joint connection becomes more prominent.

[0081] The radar screen image 250 includes a head image 251 representing the user's head as viewed from above, a block image 252 obtained by dividing the periphery of the head image 251 into a plurality of blocks, and a fan-shaped image 253 as a visual field area image indicating the user's visual field area. The target position image indicating the position of the target object is shown by which block of the block image 252 is colored or flashing or lit. Thereby, the user 220 can know whether the target object is on the left side or the right side with respect to the direction in which he is facing. In this embodiment, the block image 252 is fixed and the fan-shaped image 253 moves. However, the present invention is not limited to this, and the block image 252 may be moved according to the direction of the head while the fan-shaped image 253 or the head image 251 is fixed. Specifically, when the head is turned to the left, the block image 252 may be configured to rotate to the right.

[0082] In this example, the coloring position and color of the block image 252 indicate that the next right-hand target object 242 appears at the farthest position to the left with respect to the reference direction in the virtual space. And from the position of the fan-shaped image 253 and the orientation of the head image 251, it can be seen that the user is already facing left.

[0083] [Various setting items, task data] The setting unit 214 can set the user's basic information (ID, name, gender, age, disease, past history, various cognitive function and motor function evaluation indices, various examination results, etc.) and stores it in a searchable manner in the user database. The user ID is also associated with the records of past rehabilitation (facility, date and time, task data) in the reaching evaluation unit 215.

[0084] The setting unit 214 first sets either the manual mode (a mode in which point estimation parameters are set as task data for each individual task), the template mode (a mode in which two or more consecutive pieces of task data set in advance are used as a task data set), or the entrustment mode (auto mode) in which the device automatically generates tasks. The setting unit 214 can create a task data set to be used in the template mode based on an operator's instruction.

[0085] Furthermore, the setting unit 214 can select the type of task from among a horizontal (stationary) task (without background video), a horizontal task (with background video), a falling task (without background video), a falling task (with simple background video), and a falling task (with complex background video) by an operator's operation. In the present embodiment, the above five types of tasks will be described, but the present invention is not limited thereto.

[0086] The horizontal (stationary) task is a task that requires reaching the target object that is stationary or fixed in the three-dimensional space, as will be described later with reference to FIG. 10, with a sensor within a predetermined time (for example, until the interfering object approaches).

[0087] For the horizontal task, patterns with and without background video are prepared. Also, the BGM can be turned on or off for each.

[0088] The falling task (without background video) is a task that requires reaching the target object that has fallen from above with a sensor in a state where the absolute position of the background (horizontal line) does not change within the virtual space, as shown in FIG. 2A.

[0089] The dropping task (simple background image) is a task that requires reaching a target object that has fallen from above with a sensor in a state where there is a background image and a distracting stimulus (here, an image such as a monkey jumping from tree to tree) is arranged mainly only in the upper half of the screen, as will be described later with reference to FIG. 11. Note that the arrangement of the image as a distracting stimulus does not have to be in the upper half of the screen. For example, it may be in the left half, the right half, or the lower half. The position of the stimulus arrangement does not necessarily occupy half of the screen. It may be one-third, one-fourth, or even if the distracting stimulus intensity is weak, it may be arranged in most of the screen, for example, three-fifths. Also, the distracting image stimulus does not have to be limited to animals such as monkeys. For example, it may be something like leaves falling from a tree, and is not limited to an image moving around within the screen.

[0090] The dropping task (complex background image) is a task that requires reaching a target object that has fallen from above with a sensor in a state where the background image is changing complexly or a distracting image stimulus is arranged throughout the screen, as will be described later with reference to FIGS. 12 and 13. Also, in the dropping task (complex background image), prediction of the appearance of the target object and its memory are required, such as a specification in which the period from the occurrence of an event until actual movement is required is extended.

[0091] FIG. 3 is a diagram showing a screen (operation panel) 300 for an operator to operate. In the present embodiment, as an example, the operation panel 300 for the dropping task in the manual mode will be described. In the manual mode, the setting unit 214 causes such an operation panel 300 to be displayed on a display 290 or the like connected to the information processing apparatus 210. In the present embodiment, task creation for rehabilitation support is performed by setting point estimation parameters using an intuitive operation panel 300.

[0092] The display for presenting the operation panel 300 may be a projector or an external display 290 connected to the information processing apparatus 210, or may be a display built in the information processing apparatus 210. The operation panel 300 includes a user visual field area 301, various parameter setting areas 302, a score display area 303, a task history display area 304, a check box 305, a stop button 306, a re-center button 307, and a BGM control button 308. In the example of FIG. 3, the operation panel 300 includes a user image area 309 representing the state of the actual user 220, but the present invention is not limited thereto.

[0093] The user visual field area 301 on the right side of the screen presents an image that is displayed on the head-mounted display 233 and that the user 220 is viewing. Thereby, the visual field of the patient can be confirmed. Although omitted in FIG. 2A, visual recognition assisting objects 341 and 342 for improving the visibility of the target objects are displayed around the target objects 241 and 242. In the case of a user 220 with a low cognitive function, the visual recognition assisting objects 341 and 342 are set to be large to make it easier to find the target objects. In the rehabilitation support system according to the present embodiment, unlike a general game screen or the like, various contrivances are made everywhere to facilitate achievement of such a task.

[0094] Conversely, although the user can see a circle with a radius of 20 cm (visual assistance object), in reality, the task is not considered completely completed unless the user touches a ball with a radius of 10 cm (target object) located at the center of the circle. If the size of the visual assistance object is small, it becomes difficult for the user to find the target object. If the visual assistance object is enlarged, it becomes easier for the user to find the target object. If the target object is enlarged, the allowable margin of sensor reaching deviation increases. If the target object is made smaller, the allowable margin of sensor reaching deviation decreases, and the rehabilitation operation can be evaluated more severely. The radius of the visual assistance object can also be made to match the radius of the target object. These settings quantitatively change the feedforward accuracy, that is, the accuracy of the information processing performed by the user's brain. It becomes possible to control the treatment approach for physical functions (upper limb function, walking function, trunk function, balance function), cognitive functions (including spatial cognition and attention function), and sensory functions (including inner ear, vestibular system, tactile sense, thermoalgesia, position sense, deep sensation), as well as the degree of manifestation of joint-related disorders.

[0095] The various parameter setting area 302 is an area for setting a plurality of parameters that define the task. The various parameter setting area 302 includes a speed setting area 321 for the left hand target object, a radius setting area 322 for the left hand target object, a radius setting area 323 for the visual assistance object of the left hand target object, and a sensor radius setting area 324 for the left hand avatar object. The various parameter setting area 302 includes a speed setting area 325 for the right hand target object, a radius setting area 326 for the right hand target object, a radius setting area 327 for the visual assistance object of the right hand target object, and a sensor radius setting area 328 for the right hand avatar object. The various parameter setting area 302 further includes a target object position input area 329.

[0096] If the sensor size set in the sensor radius setting areas 324 and 328 is large, even if the hand position deviates significantly from the target object, the task is considered to be completed, so the difficulty level of the rehabilitation operation decreases. Conversely, if the sensor size is small, the hand must be accurately moved to the central area (evaluation size) of the target object, so the difficulty level of the rehabilitation operation increases. In the example of FIG. 3A, the sensor size is 2 cm on each of the left and right sides. This element also quantitatively changes the feedforward accuracy, that is, the information processing accuracy for the brain, and enables control of the treatment approach for physical functions (upper limb function, walking function, trunk function, balance function), cognitive functions (including spatial cognition and attention function), and sensory functions (including inner ear, vestibular system, tactile sense, thermoalgesia, proprioception, deep sensation), as well as the degree of manifestation of joint-related problems.

[0097] The target object position input area 329 is an image for setting the position of the target object to be generated (distance from the user and angle from the reference direction), and has a shape obtained by enlarging the radar screen image 250. The input area 329 includes a plurality of blocks with different distances from the user and angles from the reference direction of the target object. When an operation of selecting any one of the plurality of blocks is performed by the PC or the numeric keypad 290, the target object 241 or the target object 242 is generated at a position in the virtual space corresponding to the position of the specified block. Here, the input area 329 is divided into 18 blocks, but the present invention is not limited to this.

[0098] In the example of FIG. 3A, in the virtual space, the task is to timely contact an avatar object 243 including a 2-cm sensor portion with a target object 241 having a radius of 10 cm that falls at a speed of 45 cm / s at a far location on the left side. When the avatar object 243 touches the visual object 341, the target object 241 does not disappear, and the task is not completely achieved (a certain score is entered and a good evaluation is given). The task is completely achieved (perfect evaluation) only when the sensors of the avatar objects 243 and 244 touch the target objects 241 and 242.

[0099] The score display area 303 shows the total number of times of the task for each appearance position of the target object and the number of times the task has been achieved. Here, the score is expressed in fractional form and percentage notation, but it is not limited to this. After the rehabilitation operation, the reaching evaluation unit 215 derives a rehabilitation evaluation point using the value in this score display area 303. The above score may be weighted according to which task among the stationary task, horizontal task, falling task (without background image), falling task (simple background image), and falling task (complex background image) is performed. For example, a weighting coefficient such as 1 times for the stationary task, 1.2 times for the horizontal task, 1.5 times for the falling task (without background image), 2 times for the falling task (simple background image), and 3 times for the falling task (complex background image) may be determined in advance. Then, the score may be multiplied by the weighting coefficient to calculate the point.

[0100] The task history display area 304 displays the total rehabilitation time, the total number of tasks, the total number of left-hand tasks at the 0-degree, 45-degree, and 90-degree positions, and the total number of right-hand tasks at the 0-degree, 45-degree, and 90-degree positions. By checking this task history display area 304, the operator can grasp the degree of load applied to the user.

[0101] In addition, the setting unit 214 can also receive inputs for various parameter setting areas 321 to 329 from the numeric keypad 270, and the operation panel 300 has a check box 305 for setting whether to accept operations on the appearance position of the target object using the numeric keypad 270. Here, since the check box 305 is checked, operations can be performed by the input device.

[0102] The stop button 306 is a button for instructing a temporary stop or termination of a task. These stop instructions are designed such that they cannot be performed from the numeric keypad 270. In the manual mode, clicking on the target object position input area 329 serves as an instruction to start the task, and without such a click, the target object will not appear, i.e., it will be in a temporary stop state. On the other hand, in the template mode, selecting the stop button 306 stops the appearance of the target object. When the end button is selected, the reaching evaluation unit 215 calculates the rehabilitation evaluation points from the scores of the tasks performed up to that point.

[0103] The re-center button 307 is a button for redefining the position of the user 220's head-mounted display to the central position within the virtual space. The BGM control button 308 is a button for turning the background music on and off.

[0104] When the re-center button 307 is operated, the control unit 212 reconstructs the virtual space with the position of the head-mounted display 233 at that moment as the origin and the orientation of the head-mounted display 233 at that moment as the reference direction. By operating the BGM control button 307 to turn off the background music during task execution, the cognitive load of the task can be reduced and the user can concentrate on body movements. Generally, in users with joint pain, the joint pain during the task tends to subside when the background music is turned off. Conversely, for users without joint pain, the background music can be turned on to deliberately cause joint pain.

[0105] In the template settings for the template mode, the generation interval of the target object can be set as a task setting parameter.

[0106] In addition to the radar screen image 250, the control unit 212 may also display an information bar 310 on the display screen 240 of the head-mounted display 233. The information bar 310 displays the player name 311, the task achievement level 312, the elapsed time since the start of play 313, and so on. Furthermore, the information bar 310 is always parallel to the horizon 246 in the virtual space. When the patient's head position and body axis are distorted, the parallel relationship between the radar screen 250 that follows the tilt of the head-mounted display 233 and the information bar 310 is disrupted. For this reason, the patient can visually assist in correcting inner ear information, and the therapist can also easily notice the distortion of the patient's body axis. In FIG. 3, an image when the head position is tilted 20 degrees to the right is illustrated. In this way, the user interface of the head-mounted display 233 is designed to have medical benefits in addition to the elements for the patient to be fully engaged in rehabilitation.

[0107] Also, FIG. 3B is a diagram showing an operation panel 350 as another example. In the operation panel 350, compared with the operation panel 300, it is significantly different in that a joint connection generation button 351 is provided. The operator checks the movement of the user 220 and clicks the joint connection generation button 351 if a joint connection appears. Also, a bull's-eye-shaped heat map 352 may be superimposed and displayed on the user image 309 to show the image area where a joint connection is likely to occur in red.

[0108] The setting unit 214 can set the generation of the left target object 241 and the generation of the right target object 242 to be performed alternately, and the setting unit 214 sends an instruction to the control unit 212 according to the setting. The setting unit 214 can set the generation position of the left target object 241 in the depth direction within the virtual space 240 and the generation position of the right target object 242 in the depth direction within the virtual space 240, respectively.

[0109] By controlling both the generation timing of the target object and the vertical movement speed of the target object, the rehabilitation movement of the user is controlled so that it is always left - right alternating. That is, in the simplest way, the left and right target objects are not generated simultaneously. After reaching the right target object is detected, the left target object is generated. However, it is not limited to this, and various control methods are possible. That is, while generating a plurality of target objects simultaneously, the movement speed can be changed so that it decreases in the order of right → left → right → left to promote left - right alternating movement. For example, after reaching the right object is detected, the right object and the left object can be generated simultaneously, or the right object can be generated first, the speed of the right object can be increased, and an action of overtaking the left object can be made. In this way, a high load can be applied to the brain. The control unit 212 may control the rehabilitation movement of the user so that it is left - right alternating.

[0110] On the other hand, when there is paralysis only on the right side, or when it is desired to actively rehabilitate the left upper body, etc., when it is desired to reach the right target object below and the left target object relatively above, after generating the right target object at a slow speed, the left target object can be generated at a high speed with an interval. That is, at least one of the generation timing and speed of the target object is controlled according to the vertical position of the body for which rehabilitation is to be performed (according to the position of the target object to be reached). In the case of rehabilitation of the knees or lower body, the generation timing and speed of the target object are controlled so that reaching occurs at a low position. Also, when it is desired to increase the cognitive load, it is generated in the order of left → right → right, but only the first target object is moved slowly, and the second and third are moved at high speed. As a result, reaching can be made in the order of right, left, right.

[0111] By performing traffic control in the brain (rewired in the sense of untangling the intertwined SCAN) in this way, it becomes possible to improve the symptoms of diseases that are difficult to treat in modern medicine, such as chronic pain including pain modulation pain and restless leg syndrome (abnormality of the basal ganglia).

[0112] The setting unit 214 may set the delay time from the timing when the generation of the target objects 241 and 242 is notified to the timing when the target objects 241 and 242 are generated, and thereby control the cognitive load imposed on the user 220. That is, the user has to continuously remember the position in the virtual space where the target object is generated (the position indicating in which direction the head-mounted display should be directed to display it), such as on the radar screen image 250, until the actual generation of the target object, and this "memory time" becomes the cognitive load for the user.

[0113] Also, the setting unit 214 may control the cognitive load by changing the time "until the target object 152 approaches the reachable range of the user 220" instead of "until the timing when the target object 152 is generated". The setting unit 214 may impose a cognitive load on the user 220 by displaying a background image 245 other than the target objects 241 and 242 on the head-mounted display 233. When changing the cognitive load, it is desirable to notify the user in advance whether the cognitive load is increased or decreased. The notification method may be visual, using characters or symbols, or by voice, or in a form such as touching a part of the body, for example, tapping the shoulder, elbow, arm, or leg.

[0114] The setting unit 214 controls the difficulty of the body movement according to the movement of joints different from the body part where the user performs the body movement, that is, the degree of joint coupling. The setting unit 214 as a control unit controls the size of the target object, the distance from the user, the angle, the speed, the presence or absence of background, the presence or absence of music, and the size of the visual assistance object as point estimation parameters. For example, the joint coupling may change significantly depending on the presence or absence of music. The size of the joint coupling generated also varies depending on the type of music. For example, compared with the chirping of birds (which has a great impact even on people with poor cognition), the repetition of rhythm music increases the amount of work, that is, the cognitive load, for the brain to process information in the background, so the joint coupling becomes larger. Similarly, the complexity of background information is also greatly related to the amount of work, that is, the cognitive load, for the brain to process information in the background. For example, the cognitive load is greater when there is some background or scenery than when there is no three-dimensional space with nothing, and the greater the presence of moving animals or objects in it, and the wider the range of the screen where the moving animals or objects appear, the greater the cognitive load and the more prominent the joint coupling appears. That is, basically, the more the accuracy of point estimation is improved, or the higher the load of the brain's background processing such as sound and background information, the less the reserve amount of the brain available for point estimation, so it is known that the joint coupling appears strongly.

[0115] [Reaching Evaluation] The reaching evaluation unit 215 evaluates the rehabilitation movement of the user and adds points according to the amount and quality of the tasks achieved by the user 220. Here, the quality of the tasks achieved refers to "amazing" or "wonderful", that is, it includes how close the avatar object can approach the target object. The reaching evaluation unit 215 assigns different points to the achieved tasks (higher points for distant objects and lower points for close objects). The reaching evaluation unit 215 can update the tasks according to the accumulated points. For example, the task (the attributes of the target object) may be updated using the task achievement rate (the number of targets achieved / the number of tasks). The reaching evaluation unit 215 compares the rehabilitation movement detected by the movement detection unit 211 with the target position represented by the target object displayed by the control unit 212 to evaluate the rehabilitation ability of the user 220. Specifically, it is determined whether the avatar objects 243, 244 that move corresponding to the rehabilitation movement detected by the movement detection unit 211 overlap with the target objects 241, 242 by comparing the positions in the three-dimensional virtual space. If these overlap, it is evaluated that one rehabilitation movement has been cleared, and points are added.

[0116] The reaching evaluation unit 215 may update the target task (task data) according to the accumulated points. For example, the target task may be updated using the task achievement rate (the number of targets achieved / the number of tasks).

[0117] FIG. 4 is a diagram showing the history database 400 stored in the reaching evaluation unit 215. The history database 400 includes a task ID, time (the timing of the target object generation), target type (left / right), angle (0 to 180), distance (short, middle, long), target object speed, target object radius, visual assistance object radius, sensor radius, and result (amazing: 2, wonderful: 1, failure: 0).

[0118] That is, by looking at the history database 400, it is possible to find out how many seconds after the start of rehabilitation, what type, radius, and visual recognition radius of the target object fell where at what speed, and whether reaching was achieved as a result of reaching with a sensor of what size. And the occurrence of joint-related disorders at that time may also be accumulated as history. Furthermore, the patient ID is naturally associated with this history data. Video data of the patient may be associated with each history data. Such history data can be used to analyze changes in the user's cognitive function and motor function.

[0119] [Numeric Keypad] Returning to FIG. 2A, the control (parameter setting) of the task given to the user can be executed using the wireless numeric keypad 270. The operator 280 operates the numeric keypad 270 with one hand while checking the state of the user 220, the magnitude of the joint-related disorder 221, and the user's field of view. If assistance with both hands is required, it is also possible to operate the numeric keypad 270 with the feet on the floor. The setting unit 214 sets the appearance positions of the target objects 241, 242 according to the input from the numeric keypad 270.

[0120] The appearance of the numeric keypad 270 is shown in FIG. 5. In this embodiment, a commercially available numeric keypad with the seals for this system pasted on the 1-9 buttons is used. A direction seal representing the roles of the 1, 4, 7, 3, 6, 9 buttons of the numeric keypad and a distance seal representing the roles of the 2, 5, 8 buttons of the numeric keypad are prepared and pasted on the 1-9 buttons of the commercially available numeric keypad, thereby completing the numeric keypad 270 as shown in FIG. 5. That is, the content indicated by those seals is also part of the system according to this embodiment. The seal includes the original numbers of the numeric keypad and the arrows and characters representing the roles of each button as display content. The seal displays the 1, 4, 7 of the numeric keypad and the 3, 6, 9 of the numeric keypad in different colors, which are colors corresponding to the colors of the target objects. Here, for convenience, seals are used, but the content described on the above seals may be directly printed on the numeric keypad, or may be scribed, engraved, or carved using laser printing technology or the like.

[0121] This numeric keypad 270 corresponds to a system that can display a target object in six directions: directly to the left of the user (0 degrees on a protractor with the user as the origin), diagonally forward to the left (45 degrees), directly in front (90 degrees to the left), directly in front (90 degrees to the right), diagonally forward to the right (135 degrees), and directly to the right (180 degrees). The appearance directions of the target object in this embodiment are merely examples, and the present invention is not limited to these angles. Each of the buttons 1, 4, 7, 3, 6, and 9 on the numeric keypad 270 is assigned to one of the above six directions. For example, pressing the button 1 may cause the target object to appear to the left (not limited to 0 degrees, for example, in the range of -10 degrees to 10 degrees), and pressing the button 3 may cause the target object to appear to the right (not limited to 180 degrees, for example, in the range of 170 degrees to 190 degrees). Similarly, pressing the button 7 may cause the target object for left-handed tasks to appear in the front direction (not limited to 90 degrees, for example, in the range of 80 to 100 degrees), and pressing the button 9 may cause the target object for right-handed tasks to appear in the front direction (not limited to 90 degrees, for example, in the range of 80 to 100 degrees). Pressing the button 4 may cause the target object for left-handed tasks to appear in an arbitrary diagonal direction between the front and the left (for example, in the range of 30 degrees to 60 degrees), and pressing the button 6 may cause the target object for right-handed tasks to appear in an arbitrary diagonal direction between the front and the right (for example, in the range of 120 degrees to 150 degrees).

[0122] Furthermore, as a modification of this embodiment, as shown in FIG. 6B, for the case where the operator stands in front of the user and operates the numeric keypad while looking at the user, a key assignment may be used where the up, down, left, and right directions are reversed. Alternatively, a configuration may be adopted where the key assignment in FIG. 6A and the key assignment in FIG. 6B can be selected.

[0123] When each of the buttons 1, 4, and 7 on the numeric keypad 270 is pressed, the control unit 212 and the setting unit 214 cause a target object for the left half of the user's body (here, the left hand, but it may also be the left foot, etc.) to appear. When each of the buttons 3, 6, and 9 on the numeric keypad 270 is pressed, the control unit 212 and the setting unit 214 cause a target object for the right half of the user's body (here, the right hand, but it may also be the right foot, etc.) to appear.

[0124] Each of the buttons 2, 5, and 8 on the numeric keypad 270 is used to set the appearance distance from a reference position set in advance in the virtual space as the appearance position of the target object. Specifically, when the button 2 is pressed, the setting unit 214 sets to make the target object appear at a position close to the user. When the button 8 is pressed, the setting unit 214 sets to make the target object appear at a position far from the user. When the button 5 is pressed, the setting unit 214 sets to make the target object appear at an intermediate position between them. At this time, when the distance buttons 2, 5, and 8 are pressed, the distance information is saved, and it is no longer necessary to specify the distance with the distance buttons 2, 5, and 8 every time after the next time. That is, if 1, 4, 7, 3, 6, or 9 is pressed after pressing 5, tasks at a medium distance will appear for all angles, and the distance information will not be changed unless the button 2 or 8 is pressed.

[0125] The numeric keypad 270 has functions assigned so as to be in a symmetrical arrangement, and the buttons 7 and 9 on the numeric keypad are used to make the target object appear in the front direction, the buttons 4 and 6 are used to make the target object appear in the diagonal direction, and the buttons 1 and 3 are used to make the target object appear in the left and right directions.

[0126] The setting unit 214 may give a parameter cycling selection function such as the speed of the target object to the " / " button 501 of the numeric keypad 270. By repeatedly pressing the " / " button 501, the numerically changeable parameters are switched in the order of the speed setting area 321 of the left-handed target object → the radius setting area 322 of the left-handed target object → the radius setting area 323 of the visual assistance object of the left-handed target object → the sensor radius setting area 324 of the left-handed avatar object → the speed setting area 325 of the right-handed target object → the radius setting area 326 of the right-handed target object → the radius setting area 327 of the visual assistance object of the right-handed target object → the sensor radius setting area 328 of the right-handed avatar object, and then return to the speed setting area 321 of the left-handed target object.

[0127] Furthermore, the "+" button 503 and "-" button 502 of the numeric keypad 270 may be provided with a function to increase or decrease the parameter value. In this case, it is also preferable that the amount of increase or decrease when the "+" or "-" button is pressed once varies depending on the type of parameter to be changed. For example, the speed setting increases or decreases by 5 cm per second, while the information related to the radius of the object increases or decreases by 1 cm each time. Further, when the " / " button 501 and "*" button 504 of the numeric keypad 270 are pressed simultaneously, the on / off state of the check box 305 is switched. That is, based on the input from the numeric keypad, it is switched whether or not to set the appearance position of the target object.

[0128] Note that two numeric keypads may be connected to the information processing device and operated by two operators. Similar to a training vehicle, it becomes possible for an instructor to operate the numeric keypad while giving instructions.

[0129] FIG. 6 is a diagram that clearly explains the operation method of the numeric keypad 270. As is clear here, the numeric keypad 270 does not implement a function to end the task. This is to avoid task interruption or task termination due to incorrect operation.

[0130] [Flow of processing] FIG. 7 is a flowchart showing the flow of processing in the information processing device 210. In step S701, as calibration processing, the target position of the rehabilitation operation is initialized according to the user 220. Specifically, first, the user 220 is asked to perform an operation to obtain the operable range as calibration, and after setting that range as the initial value, the target is initialized according to the user. This initial value may be set according to the user's operable range, or may be determined by a therapist providing the treatment, taking into account the user's body awareness function and the target treatment effect.

[0131] FIG. 8 is a diagram showing an example of a calibration screen 800. On the calibration screen 800, similar to the operation panel 300, in the user's visual field area 801 on the right side of the screen, an image that is displayed on the head-mounted display 233 and that the user 220 is viewing is shown. On the head-mounted display 233, an avatar object (in this example, the avatar object 244 on the right hand side) on the side for which calibration is to be performed is displayed, either on the left or the right.

[0132] Then, for each of the six directions of directly to the user's left, diagonally in front of the left, in front of the left, in front of the right, diagonally in front of the right, and directly to the user's right, the shortest distance (or, in the case of the stationary mode or the horizontal mode, also including height information) and the longest distance for the target object to appear are determined. Therefore, for each of those directions, the nearest position coordinates (distance and height) when the user 220 simply stretches out an arm and the position coordinates (distance and height) when the user bends the torso forward and stretches out the arm as much as possible are obtained. The position at 90% of the position coordinates when the user bends the torso forward and stretches out the arm as much as possible is set as the farthest position coordinates (long coordinates) for the target object to appear. For a person with a bent waist, it is preferable to set the nearest position coordinates (reference position) higher. Also, for a patient with a problem in the foot such as the knee, setting the nearest position coordinates (reference position) lower is preferable in terms of enhancing the therapeutic effect on the physical disability.

[0133] The progress display area 801 is an area indicating the progress of the calibration. For positions for which coordinates have already been acquired, a circle is drawn in the progress display area 801. The S surrounded by the circle represents the shortest distance (Short coordinates), and the L surrounded by the circle represents the longest distance (long coordinates). The acquisition timing of each coordinate may be performed by the user in response to a button operation provided on the controller, but when a check box 802 for disabling the user operation is checked, only the operator can input coordinates. On the numeric keypad, coordinate acquisition is operated by the "Enter" button 505.

[0134] When the nearest position coordinates are input, a wall 811 appears on the head-mounted display 233. By pushing the wall 811 forward, a wireframe 812 appears, and the wall 811 moves away (Fig. 9), and the position coordinates are obtained with the trunk tilted forward and the arms fully extended. At this time, a sound of "gugugugu" is emitted and the controller vibrates. That is, for all of vision, hearing, and touch, the user is given the feeling of pushing the wall. In this way, high affordance is realized so that the user can intuitively perform calibration. In addition, on the calibration screen 800, a measurement value display area 803, a decision button 804, a return button 805, a manual input area 806, an angle skip button 807, etc. are provided. On the numeric keypad, the decision is operated by the "Enter" button 505, the return is by the "-" button 502, and the skip is by the "+" button 503.

[0135] Returning to Fig. 7, next, in step S703, mode selection is performed. That is, the user is set to either the manual mode (a mode in which point estimation parameters are set as task data for each task), the template mode (a mode in which two or more consecutive task data set in advance are used as a task data set), or the entrusted mode (auto mode) in which the device automatically generates tasks. Further, here, the operator selects the type of task from among a horizontal (stationary) task (with or without a background image), a falling task (without a background image), a falling task (with a simple background image), and a falling task (with a complex background image).

[0136] Next, in step S704, task data is set. In the case of the template mode, reading is performed from the template (task data set).

[0137] In step S705, in the manual mode, when any of the buttons 1, 4, and 7 of the numeric keypad 270 is pressed, the control unit 212 and the setting unit 214 cause a target object for the user's left hand (or left half of the body) to appear. When any of the buttons 3, 6, and 9 of the numeric keypad 270 is pressed, the control unit 212 and the setting unit 214 cause a target object for the user's right hand (or right half of the body) to appear. Then, a task (display of the target object by the control unit 212 and evaluation of achievement by the avatar object) is started.

[0138] Until it is determined that the task ends in step S707, the processes of steps S704 and S705 are repeated.

[0139] When it is determined that the task ends in step S707, the process proceeds to step S709, where the achievement score of the task is obtained and points are calculated.

[0140] According to the present embodiment, while assisting the user with one hand, the user can perform a numeric keypad operation with the other hand to generate a task. In rehabilitation, it is most important to match the pace of the patient. By using the numeric keypad, the user can concentrate on the patient's movements while operating the system, and furthermore, it becomes possible to assist.

[0141] [Examples of Other Task Screens] FIGS. 10 to 13 are diagrams showing other examples of the display on the head-mounted display 233 according to the present embodiment.

[0142] FIG. 10 is an example of a screen that requests a user to perform a horizontal task. In a background image 1001 representing the streetscape of the Edo period, an image showing a medicine box 1011 as a target object is displayed. Further, below the medicine box 1011, a thousand-tael box 1013 is displayed as an item to be protected by the user, and a ninja 1015 is configured to gradually approach from the back. The speed of the ninja 1015 is the speed set in the input area 323 of the operation panel 300 (the speed here is synonymous with the time limit). Note that when the speed is set to 0, the ninja does not appear in the first place. A circle 1012 is displayed on the medicine box 1011 as a visual recognition assistance object. When the medicine box 1011 is touched with a sensor object (the tip center of the avatar objects 243 and 244) before the ninja 1015 reaches the thousand-tael box 1013, the task is completed. Two types of circles 1012 are prepared, red and blue. For the medicine box 1011 surrounded by the red circle 1012, the task is to operate and touch the red avatar object 244 on the right corresponding to the controller 2310 held in the right hand. On the other hand, for the medicine box 1011 surrounded by the blue circle 1012, the task is to operate and touch the red avatar object 243 on the left corresponding to the controller 234 held in the left hand.

[0143] The seal cage 1011 is displayed at the position (depth and angle) set in the input area 324 of the operation panel 300. The seal cage 1011 does not change its position until the user touches the avatar objects 243, 244 in the virtual space. That is, it is a target object fixed in space (referred to as a horizontal task because it requires stretching the body horizontally). Such a fixed target object is very effective as a rehabilitation or treatment method for diseases such as cerebellar ataxia, diplopia, and inner ear dysfunction, and for rehabilitation or treatment methods for orthopedic diseases of the lower extremities such as the hip joint, knee joint, and ankle joint. That is, for patients who have forgotten how to move their bodies, it is possible to imprint a limited image of body movement on the brain by feedforward and align the alignment around the joints. By increasing the distance in the depth direction of the seal cage 1011, the exercise intensity can be changed. Furthermore, by combining multi-channel biofeedback, motor ability, physical function, cognitive function, and sensory function are greatly improved. Also, according to such a horizontal task, chronic pain can be improved by promoting the reorganization of the cerebral cortex. Alternatively, it is also possible to recover peripheral neuropathy including cognitive impairment called chemo brain, sensory dysfunction in which the proprioception of cancer patients who have taken anticancer drugs decreases due to nerve damage, and sequela symptoms associated with novel coronavirus (COVID-19) infection. It may be possible to give hints in advance about where the target object will appear to reduce the cognitive load. Tactile information obtained by touching the body, multiple repetitions of verbal information, or a combination thereof is also effective in reducing the cognitive load, rather than verbal information. The method of verbal information may also reduce the cognitive load by giving simpler instructions that are more straightforward and closer to the imperative form, or may take a more complex form of instruction in the form of a question, such as "Because it's blue? (Take it with your right hand)", or verbal information may be given in a form that includes a cognitive task such as calculation, such as "When you say a number divisible by 2, take it with your right hand". Note that not only the horizontal position and depth at which the seal cage 1011 is generated but also the height may be configured to be settable.

[0144] FIG. 11 is an example of a screen that requests a user to perform a dropping task (simple background video). In FIG. 11, in a background image 1101 like a forest, a trigger object 1102 representing a monkey and a target object 1103 representing an apple are displayed. The task is completed when the trigger object 1102 representing the monkey drops the target object 1103 representing the apple from the tree and the avatar object 1104 representing the monkey catches the target object 1103 that has approached the user. Here too, the control unit 212 starts the fall of the target object 1103 after a predetermined time has elapsed from the timing when the trigger object 1102 shakes the tree and notifies the occurrence of the target object 1103, thereby imposing a cognitive load on the user 220 while causing an attention disorder.

[0145] Furthermore, the control unit 212 can impose an extremely strong cognitive load on the user 220 by simultaneously having at least 2 to 5 target objects 1103 exist in the three-dimensional virtual space. In other words, the control unit 212 generates at least 2 target objects 1103 at different positions in the left-right direction within the three-dimensional virtual space.

[0146] In particular, if at least 2 target objects 1103 are generated at a plurality of positions in a direction different from the moving direction of the target object 1103 (the dropping direction in FIG. 11, i.e., the left-right direction in FIG. 11), an even greater cognitive load can be imposed. That is, the user 220 has to move the controllers 234 and 235 considering the movement in the vertical direction, the difference in the generation positions in the left-right direction, and further the difference in the dropping positions in the depth direction, which means that the spatial cognitive ability is also being tested. In this way, in addition to changing the predetermined time of the task, by adjusting the types, numbers, sizes, spatial spreads, positions, amounts, etc. of the information included in the notification image and notification voice including the trigger object, it becomes possible to quantitatively adjust and control the complexity of the information to be retained in memory, that is, the cognitive load that the brain should process for the user.

[0147] 12 and 13 are diagrams showing examples of screens for requesting a user to perform a falling task (complex background image). In a background image 1201 showing a field, an image of a person showing a farmer is displayed as a trigger object 1202 which triggers the appearance of a target object. That is, the control unit 212 displays the trigger object 1202 as an announcement image for announcing the appearance of a target object 1203. A predetermined time after the trigger object 1202 throws the potato-shaped target object 1203 upward, the large potato-shaped target object 1203 appears from the screen as shown in FIG. 13. The task is accomplished by moving the sieve-shaped avatar object 1202 to catch the falling target object 1203. The left and right avatar objects 1202 move on the screen in conjunction with the movements of the controllers 234 and 235.

[0148] The setting unit 214 can adjust the cognitive impact on the user by setting a delay time from the timing when the trigger object 1202 throws the target object 1203 upward and notifies the generation of the target object 1203 to the generation of the target object 1303. The longer the delay time, the longer the period during which memory is retained, and the greater the load on the brain's information processing. Note that the generation of the target object may be notified at the same timing by a radar chart-type notification image 250 in conjunction with the movement of the trigger object 1202, or a voice notification may be combined.

[0149] In this way, the setting unit 214 can impose a cognitive load on the user not only in a task with a background of only the horizon 246 as in Fig. 2A, but also in a task with a background with a large amount of information as in Fig. 12 and Fig. 13. In other words, by making it difficult for the user to remember that the target object 1203 has appeared and the position where the target object 1303 is expected to fall, a cognitive load closer to that required in real life is imparted to the rehabilitation user.

[0150] In particular, the control unit 212 imposes a cognitive load on the user 220 to process the background image in the brain by changing at least a part of the background image 245 over time. In the example of FIG. 12, for example, among the background images 1201, the clouds 1204 may be moved, the vegetation 1205 may be swayed, or an animal (not shown) not related to the target object may appear. Thereby, it is possible to prevent the user 220 from concentrating on the target object 1203 and to make it difficult to remember the position where the target object 1203 will fall. More specifically, by displaying information unrelated to the task in the background image, an environment in which it is difficult to concentrate on the target object is prepared, and it can be said that the memory is made difficult and the cognitive load is controlled by intentionally causing an attention disorder (more specifically, selective attention disorder, divided attention disorder, switching attention disorder, sustained attention disorder).

[0151] The reaching evaluation unit 215 evaluates the user's cognitive ability by using information such as whether the avatar object has accurately reached the three-dimensional target position represented by the target object in a timely manner, the time interval and number from the generation notification of the target object to its generation, and the degree of the load that causes an attention disorder in the background image.

[0152] Among the various modes (display screens) shown in FIGS. 2A, 10 to 13, the required accuracy of point estimation also varies depending on which mode is to be executed for the user. For example, the required accuracy of point estimation increases in the order of FIGS. 10 → 11 → 12 compared to the case where there is no background as in FIG. 2A. Also, as described above, the required accuracy of point estimation also changes depending on the background sound and is affected by the joint connection.

[0153] [Other Embodiments] As described above, the present invention has been described with reference to the embodiments, but the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the technical scope of the present invention. Also, a system or device in which the separate features included in each embodiment are combined in any manner is also included in the technical scope of the present invention.

[0154] In addition, the present invention may be applied to a system composed of a plurality of devices or to a single device. Further, the present invention is also applicable when an information processing program for realizing the functions of the embodiment is supplied to a system or a device and executed by a built-in processor. In order to realize the functions of the present invention by a computer, a program installed in the computer, a medium storing the program, a server for downloading the program, and a processor for executing the program are all included in the technical scope of the present invention. In particular, at least a non-transitory computer readable medium storing a program for causing a computer to execute the processing steps included in the above-described embodiment is included in the technical scope of the present invention.

Claims

1. A control unit that causes a target object that a user aims to reach to appear in a virtual space, and a setting unit that sets the appearance position of the target object according to an input from a numeric keypad. An information processing apparatus comprising the same.

2. The setting unit includes, as parameters for the appearance position of the target object, an appearance distance from a reference position set in advance in the virtual space and an appearance direction with respect to a reference direction set in advance in the virtual space, uses the buttons “2”, “5”, and “8” of the numeric keypad as switches for the appearance distance, and uses the buttons “1”, “4”, “7”, “3”, “6”, and “9” of the numeric keypad as switches for the appearance direction. The information processing apparatus according to claim 1.

3. When the button “2” of the numeric keypad is pressed, the setting unit sets to cause the target object to appear at a position close to the user. When the button “8” is pressed, the setting unit sets to cause the target object to appear at a position far from the user. When the button “5” is pressed, the setting unit sets to cause the target object to appear at an intermediate position therebetween. The information processing apparatus according to claim 2.

4. The setting unit is configured to cause the target object for the left half of the user to appear when the buttons “1”, “4”, and “7” of the numeric keypad are pressed. The control unit causes the target object for the left half of the user to appear triggered by the pressing of the buttons “1”, “4”, and “7” of the numeric keypad, The setting unit is configured to cause the target object for the right half of the user to appear when the buttons “3”, “6”, and “9” of the numeric keypad are pressed. The control unit causes the target object for the right half of the user to appear triggered by the pressing of the buttons “3”, “6”, and “9” of the numeric keypad. The information processing apparatus according to claim 1.

5. The control unit, causes the target object to appear in the lateral direction of the user when the buttons “1” and “3” of the numeric keypad are pressed, causes the target object to appear in the front direction of the user when the buttons “7” and “9” of the numeric keypad are pressed, and causes the target object to appear in a diagonal direction between the front direction and the lateral direction of the user when the buttons “4” and “6” of the numeric keypad are pressed. The information processing apparatus according to claim 1.

6. The first notation or label representing the roles of the buttons “1”, “4”, “7”, “3”, “6”, and “9” of the numeric keypad, or a sticker, The second notation or label representing the roles of the buttons “2”, “5”, and “8” of the numeric keypad, or a sticker, The information processing apparatus according to claim 2, further comprising the same.

7. The information processing apparatus according to claim 5, wherein the notation or label includes, as display contents, the original numbers of the numeric keypad and arrows and characters representing the roles of the respective buttons.

8. The notation or label displays the “1”, “4”, “7” of the numeric keypad and the “3”, “6”, “9” of the numeric keypad in different colors, The information processing apparatus according to claim 5, wherein the color is a color corresponding to the color of the target object.

9. The information processing apparatus according to any one of claims 1 to 7, wherein the “ / ” button of the numeric keypad has a task parameter selection function, and the “+” and “-” buttons of the numeric keypad have a function of increasing and decreasing parameter values.

10. A control step in which a control unit causes a target object, which is a target to be reached by a user, to appear in a virtual space, A setting step in which a setting unit sets the appearance position of the target object according to an input from the numeric keypad, An information processing method including the above.

11. A control step of causing a target object, which is a target to be reached by a user, to appear in a virtual space, A setting step of setting the appearance position of the target object according to an input from the numeric keypad, An information processing program for causing a computer to execute the above.

Images