Gait evaluation support method, gait evaluation support device, and gait evaluation support program

Abstract

Provided is a gait evaluation support method executed by a computer, comprising: causing a display device to display a virtual reality (VR) environment simulating a living environment; acquiring data corresponding to walking action of a subject wearing the display device via a sensor and recording the acquired data; analyzing a gait state of the subject from the recorded data; and visualizing the result of the analysis so that the gait state of the subject can be evaluated.

Description

Gait assessment support method, gait assessment support device, and gait assessment support program

[0001] The present invention relates to a gait evaluation support method, a gait evaluation support device, and a gait evaluation support program.

[0002] Traditionally, assessment of gait disorders has been performed by visual examination by a physician, and therefore the validity of the assessment has depended on the physician's experience, making it difficult to assess objectively.

[0003] A rehabilitation system for higher brain dysfunction that utilizes virtual reality has also been proposed (Patent Document 1). This system includes an image processing device that runs an application for presenting rehabilitation problems to a patient using images that utilize virtual reality, augmented reality, or mixed reality, and stores a history of the patient's solving the problems as rehabilitation history information, a therapist terminal that receives the rehabilitation history information from the image processing device, a server that stores the rehabilitation history information sent from the therapist terminal, and a doctor terminal that receives the rehabilitation history information from the server and displays the progress of the patient's rehabilitation based on the rehabilitation history information.

[0004] Patent No. 6593852

[0005] When using a system to support the evaluation of movement disorders, simply recording the subject's condition using sensor devices to evaluate conventional evaluation items is not considered to be very useful relative to the cost of introducing the system. Therefore, the purpose of this technology is to provide a technology that enables the evaluation of new findings.

[0006] In order to solve the above problems, the following means are adopted: A gait evaluation support method in which a computer executes the following steps: displaying a virtual reality (VR) environment that simulates a living environment on a display device; acquiring and recording data corresponding to the walking movements of a subject wearing the display device via a sensor; analyzing the walking state of the subject from the recorded data; and visualizing the results of the analysis so that the walking state of the subject can be evaluated.

[0007] The disclosed technology can provide a technology that enables evaluation of new findings for movement disorders, including gait disorders.

[0008] FIG. 1 is a diagram illustrating an example of the system configuration. FIG. 2 is a diagram illustrating a VR space. FIG. 3 is a diagram illustrating playback of movements by an avatar. FIG. 4 is a diagram illustrating an example of the device configuration of the system. FIG. 5 is a process flow diagram illustrating an example of evaluation support processing executed by the system. FIG. 6 is a diagram illustrating the definition of a VR space. FIG. 7 is a diagram illustrating an example of a graph representing the movements of a subject. FIG. 8 is a diagram illustrating an example of a graph representing the movements of a subject. FIG. 9 is a diagram illustrating an example of a graph representing the analyzed walking trajectory of a subject in a heat map. FIG. 10 is a diagram illustrating an example of a graph representing the correlation between the number of gait stalls of a subject and the assessment result of a specialist. FIG. 11 is a diagram illustrating an example of a graph representing the scored frequency of stalls calculated using the subject's Q-learning algorithm and FOG severity score. FIG. 12 is a boxplot illustrating an example of the scored stall frequency results of a subject.

[0009] Hereinafter, embodiments will be described with reference to the drawings. The configurations of the embodiments are examples, and the configuration of the invention is not limited to the specific configurations of the disclosed embodiments. When implementing the invention, specific configurations according to the embodiments may be appropriately adopted.

[0010] [Embodiment] Fig. 1 is a diagram showing an example of a system configuration. The system 10 in Fig. 1 includes a medical institution device 1, which is a computer used by a doctor or the like, and a VR (Virtual Reality) device 2, which is worn by a subject and includes a head-mounted display (HMD) for displaying a VR space and a sensor for detecting the subject's movements. The system may further include a motion capture system 3 for recording the subject's movements.

[0011] The medical institution device 1 is a computer, and outputs, for example, information defining a VR space to the VR device 2. The VR device 2 is also a computer, and, for example, renders the VR space and displays it on a display, and detects the movements of the subject wearing the VR device 2 using a sensor and updates the display according to the detected movements. The VR device 2 also outputs information representing the subject's movements to the medical institution device 1. Note that the processing shared by the medical institution device 1 and the VR device 2 is just an example, and information defining the VR space may be stored on the VR device 2. Alternatively, the medical institution device 1 may render the VR space based on sensing data output by a sensor in the VR device 2, and the VR device 2 may function as a thin client that only displays the VR space. The medical institution device 1 may also acquire information representing the subject's movements from a motion capture system 3. The VR space displayed on the VR device 2 may be only the VR space generated by the medical institution device 1 and the VR device 2, or it may be a mixed reality (MR) space in which objects generated by VR are projected and superimposed on a real-life environment. In other words, the virtual reality environment displayed on the VR device 2 can be said to be a three-dimensional space created by XR (Extended Reality / Cross Reality).

[0012] These components may be connected to each other via a wired or wireless network, or may be connected via a network. Data may also be exchanged via a storage medium such as a flash memory. The network may include, for example, an Internet Protocol (IP) network, and devices connected to the network may communicate based on a predetermined communication protocol. Part of the network 5 may be a telephone network (a landline network or a mobile communication network), an ad hoc network, an intranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (Wireless LAN), a wide area network (WAN), or the Internet.

[0013] In one aspect of the present technology, the VR device 2 displays an environment different from the examination room as a VR space, allowing the subject's movements to be observed in the VR space. Conventionally, when examining motor disorders such as gait disorders, there has been a discrepancy between the patient's symptoms in the examination room and in real life. That is, unlike the examination room, the real-life environment contains narrow corridors, doors, various obstacles, and dark places. In such an environment, gait disorders can worsen, resulting in, for example, freezing of gait, which can lead to falls. This is an important clinical finding, but has been difficult to evaluate in the examination room. Therefore, the system according to the present embodiment induces motor dysfunction, including freezing of gait, allowing the evaluation of movements such as walking in a VR space. It is preferable that the VR space be a living environment similar to the subject's real life, but it is not particularly limited as long as it makes the symptoms of motor dysfunction more likely to appear.

[0014] FIG. 2 is a diagram illustrating a VR space. (A) to (C) of FIG. 2 show an example of a VR space displayed on the display of the VR device 2. That is, an example of the VR space as seen from the subject's perspective. (A) to (C) of FIG. 2 are virtual reproductions of a corridor. The VR space shown in (A) of FIG. 2 includes a passage formed by left and right wall surfaces 241 and a floor surface 242, and a door 243 is located at the front end of the passage. Furthermore, a goal 244 is set on the floor surface 242 at the end of the passage, just in front of the door 243. It is preferable that the width of the passage can be set to any size. The VR space shown in (B) of FIG. 2 further includes two obstacles 245 in the VR space shown in (A). The obstacles 245 have the appearance of cardboard boxes, and the two obstacles 245 are stacked one on top of the other, aligned approximately parallel to the wall surface 241. 2C differs from the VR space of (B) in the orientation of the obstacles 245. That is, in (C) of FIG. 2, the orientations of the two obstacles 245 are different from each other, and they are stacked one on top of the other in a disorderly manner, each at an angle relative to the wall surface 241. It is preferable that the obstacles 245 can be placed at any position in the passageway in any orientation.

[0015] In the example of FIG. 2 , the subject walks from a predetermined starting point to a goal 244 in a VR space. That is, as the subject wears the VR device 2 and walks in a real space, such as a consultation room, data corresponding to the subject's movements is acquired via sensors, and the VR device 2 updates the display of the VR space according to the acquired data. The VR device 2 is assumed to be capable of detecting the subject's walking movement using a sensor with, for example, 6 DoF (Degree of Freedom) or more. Furthermore, in the VR space, the subject is asked to pass between a wall 241 and an obstacle 245, which induces a worsening of gait disorders, such as freezing of gait. The shorter the distance between the wall 241 and the obstacle 245, the worsening of the gait disorders is induced. The VR space is useful not only for changing the presence or absence of the obstacle 245 and the distance between the wall 241 and the obstacle 245, but also for allowing the obstacles 245 to be randomly arranged. In other words, when obstacles are arranged in a disorderly manner, the burden on the subject's information processing related to visuospatial cognition increases compared to when they are arranged in an orderly fashion, and it is expected that this will lead to a greater exacerbation of, for example, gait disorders. This is based on the pathological hypothesis of motor perceptual dysfunction, which considers that abnormal processing in the central sensory system affects motor symptoms in neurodegenerative diseases such as Parkinson's disease. In this way, by creating a virtual environment different from the examination room, this embodiment makes it easier to observe the subject's symptoms.

[0016] In another aspect of the present technology, the medical institution device 1 reproduces the subject's movements on an avatar in a virtual three-dimensional space. Conventionally, findings of motor dysfunction are not only recorded in written form in a medical record but also saved as video footage captured by a video camera. The saved footage can be evaluated by a doctor other than the examining physician. The saved footage also allows for comparison of the patient's condition before and after treatment. However, the relative positions of the video camera and the subject can make evaluation difficult. Therefore, the system according to the present embodiment reproduces the subject's movements using an avatar, enabling the patient's symptoms to be observed from multiple perspectives. The term "avatar" refers to a three-dimensional humanoid model that reproduces the subject's movements in a three-dimensional space. The subject's movements may be, for example, movements in the VR space described above, but may also be walking movements recorded by a motion capture system 3 in an examination room by a subject not wearing a VR device 2.

[0017] FIG. 3 is a diagram illustrating the playback of avatar movements. The playback of avatar movements is performed in the medical institution device 1 shown in FIG. 1. In this case, it is preferable to record the movements of almost the entire body of the subject using, for example, a motion capture system 3. The user interface (UI) 14 in FIG. 3 displays images of a virtual three-dimensional space viewed from two viewpoints. The image 141 shown on the left side of FIG. 3 is a bird's-eye view of the three-dimensional space from diagonally above. The three-dimensional space corresponds to the VR space shown in FIG. 2C. That is, the three-dimensional space includes a passage formed by left and right walls 241 and a floor 242. Also, as shown in image 141 in FIG. 3, a start point 246 is set on one side of the passage, and an avatar 247 for playing back the recorded movements of the subject is displayed at the start point 246. Two obstacles 245 are randomly arranged in the passage. The viewpoint in the three-dimensional space can be set at any position. For example, the desired viewpoint may be selected from multiple preset viewpoints, such as a bird's-eye view from above, or a viewpoint that displays the subject from the starting point 246 of the passage, the goal 244, or the wall 241. Furthermore, image 142 shown on the right side of Figure 3 is an image of a three-dimensional space viewed from the viewpoint of an avatar 247. That is, image 142 reproduces where the subject was looking while walking, and similar to the VR environment shown in Figure 2, a door 243 and a goal 244 are positioned. The viewpoint of the avatar (i.e., the subject) can be used to verify the relationship with visual-spatial cognitive disorders, etc. Note that components corresponding to image 141 and Figure 2 are denoted by the same reference numerals and will not be described again. Furthermore, the subject's gaze direction can be roughly reproduced using the head tracking function of the VR device 2, but information on the subject's gaze point may also be acquired using an eye tracker while the subject is walking and displayed on image 142. In this way, the recorded subject's movements can be displayed from multiple viewpoints.

[0018] [Device Configuration] FIG. 4 is a diagram illustrating an example of the device configuration of the system. The medical institution device 1 is a computer such as a PC (Personal Computer) or a server. The medical institution device 1 includes a processor 11, a storage device 12, a communication interface (IF) 13, and a user interface (UI) 14. The processor 11 is an arithmetic processing device such as a CPU (Central Processing Unit) that executes programs to perform various processes according to the present embodiment. The storage device 12 is, for example, a main storage device such as a RAM (Random Access Memory) or a ROM (Read Only Memory), and an auxiliary storage device (secondary storage device) such as a HDD (Hard-Disk Drive), an SSD (Solid State Drive), or a flash memory. The main storage device temporarily stores programs read by the processor and secures a working area for the processor. The auxiliary storage device stores programs executed by the processor and data exchanged with other devices. The communication IF 13 is, for example, a network adapter or communication module that communicates wired or wirelessly and performs data communication based on a predetermined protocol. The UI 14 is a user interface such as a touch panel, a keyboard, a pointing device, a microphone, a speaker, a camera, etc. The UI 14 accepts user operations and outputs information to the user.

[0019] The VR device 2 is a computer such as a VR headset or VR goggles. The VR headset may include an HMD and connect to another computer to display VR content, or may be standalone and display VR content. The VR goggles may function as an HMD by displaying VR content on a smartphone or other device worn by the user. The VR device 2 includes a processor 21, a storage device 22, a communication interface (IF) 23, a UI 24, and a sensor 25. The processor 21, the storage device 22, and the communication IF 23 are generally similar to the processor 11, the storage device 12, and the communication IF 13, respectively. In this embodiment, the processor 21 acquires information defining a VR space from the medical institution device 1, renders it, and displays it on a display that is the UI 24. The UI 24 includes a display. The display and the VR device 2 may, for example, use binocular parallax to display a VR space in three dimensions using a twin lens, or may use a monocular (single-lens) to display a VR space in two dimensions. The sensor 25 is, for example, an inertial measurement unit (IMU) that includes a three-axis acceleration sensor and a three-axis gyro sensor. The IMU may further include a three-axis geomagnetic sensor.

[0020] The motion capture system 3 is, for example, an inertial wearable motion capture system that measures acceleration and angular velocity at multiple locations on the subject's body. The motion capture system 3 includes a processor 31, a communication interface (IF) 32, and a sensor 33. The processor 31 and the communication IF 32 are generally similar to the processor 11 and the communication IF 13, respectively. The processor 31 transmits information representing the subject's movements measured by the sensor 33 to the medical institution device 1 via the communication IF 32. The sensor 33 includes, for example, multiple IMUs and is equipped with a triaxial acceleration sensor and a triaxial gyro sensor. The IMU may also be equipped with a triaxial geomagnetic sensor. The motion capture system 3 may measure the subject's movements using a so-called optical method or other methods.

[0021] 5 is a process flow diagram showing an example of the evaluation support process executed by the system. The evaluation support process is started by an operation of an evaluator in an examination room or the like when examining a patient for movement disorders, for example.

[0022] First, the processor 11 of the medical institution device 1 transmits information defining the VR space to the VR device 2 (FIG. 5: S1). The information defining the VR space is, for example, a three-dimensional model for displaying a corridor. Note that the information defining the VR space is assumed to be set in advance by the user.

[0023] FIG. 6 is a diagram illustrating the definition of a VR space. The upper part of FIG. 6 shows an example of a table storing the definition of a VR space. The table includes attributes such as the length d1 of the passageway, the width w1 of the entire passageway, the distance d2 from the starting point to the obstacle, the distance d3 from the wall farthest from the obstacle to the obstacle or the passable passage width w2, and the angle θ between the obstacle and a virtual plane perpendicular to the wall. These parameters are set for each of two obstacles. The obstacle is, for example, a cardboard box of a predetermined size. However, the type of obstacle other than a cardboard box and the size of the obstacle may also be defined. Furthermore, when both w2 and d3 are set, the size of the obstacle may be changed according to w1, w2, and d3. Furthermore, the number of obstacles is not limited to two and may be zero, one, or three or more. Information indicating the size of each obstacle is registered in the field of each attribute of such a table.

[0024] The lower part of FIG. 6 is a plan view showing an example of a VR space. For example, the length d1 of the passage is the distance from the start point 246 to the goal 244, and is, for example, about 3 meters. The distance d2 is the distance from the start point 246 to the obstacle 245, and is, for example, about 2 meters. The width w1 of the passage is the distance between two wall surfaces 241, and is, for example, about 1 meter. The distance d3 from the wall surface farther from the obstacle to the obstacle is, for example, the distance between the center of the obstacle 245 in the width direction of the passage (floor surface 242) and the wall surface 241 farther from the obstacle 245. The passable passage width w2 is the distance between the obstacle 245 and the wall surface 241 farther from the obstacle 245. The angle θ formed between the obstacle and a virtual plane perpendicular to the wall surface is the magnitude of the angle between a plane perpendicular to the wall surface 241 and the outline of the obstacle 245. In S1 of FIG. 5, information as shown in the upper part of FIG. 6 may be transmitted, or information representing a three-dimensional model as shown in the lower part of FIG. 6 may be transmitted.

[0025] Meanwhile, the processor 21 of the VR device 2 receives data from the medical institution device 1 and displays the VR space on the UI 24 (FIG. 5: S2). In this step, the VR space as shown in FIG. 2 is displayed. The subject wearing the VR device 2 walks from a predetermined starting point 246 to a goal 244 while viewing the image of the VR space.

[0026] The processor 21 of the VR device 2 also acquires sensing data output by the sensor 25 (S3 in FIG. 5) and updates the VR space displayed on the UI 24 based on the acquired data (S4 in FIG. 5). That is, the VR device 2 achieves head tracking and position tracking using a three-axis acceleration sensor and a three-axis gyro sensor. Note that if the system 10 includes a motion capture system 3, the processor 31 of the motion capture system 3 also acquires sensing data from the sensor 33 in parallel and transmits it to the medical institution device 1 via, for example, the communication IF 32.

[0027] Then, the processor 21 of the VR device 2 determines whether to end the process (FIG. 5: S5). For example, if the subject reaches the goal 244 in the VR space, the processor 21 determines that the process should be ended. If it is determined not to end the process (S5: NO), the process returns to S3 and the processor 21 repeats the process.

[0028] On the other hand, if it is determined in S5 that the processing is to be ended (S5: YES), the processor 21 of the VR device 2 transmits the sensing data to the medical institution device 1 via the communication IF 23 (FIG. 5: S6). Note that the processor 21 may be configured to transmit the sensing data sequentially while the subject is walking.

[0029] After S6, the processor 11 of the medical institution device 1 acquires the sensing data from the VR device 2 via the communication IF 13 and stores it in the storage device 12 (FIG. 5: S7). Note that if the system 10 includes a motion capture system 3, the processor 11 also acquires sensing data from the motion capture system 3 as appropriate and stores it in the storage device 12.

[0030] The processor 11 of the medical institution device 1 also uses the acquired data to display information representing the subject's movements on the UI 14 (FIG. 5: S8). In this step, the sensing data measured by the sensor 25 of the VR device 2 may be displayed as a graph. If the system 10 includes a motion capture system 3, the sensing data measured by the sensor 33 of the motion capture system 3 may be displayed as the movements of an avatar in three-dimensional space.

[0031] 7 and 8 are diagrams showing examples of graphs representing the movements of subjects. In the graph in FIG. 7, the horizontal axis represents distance and the vertical axis represents walking speed. A healthy young person (HC(Young)) walks at approximately the same speed from the starting point (0 m) to the goal (d1 m). A healthy elderly person (HC(Old)) has variations in walking speed, but reaches the goal without stopping. However, a Parkinson's disease patient (PD) stops walking around d2 m, where an obstacle is located. FIG. 8(A) shows a graph of a healthy subject. FIG. 8(B) shows a graph of a Parkinson's disease patient. In each graph in FIG. 8, the horizontal axis represents distance, and the vertical axis represents the height of the left and right feet and the elapsed time. In the example in FIG. 8, the healthy subject also walks at approximately the same speed. Furthermore, the height of the healthy subject's feet also indicates that the healthy subject walks without stopping. On the other hand, the graph of the elapsed time of the Parkinson's disease patient shows that he stopped for about 5 seconds at around d2m where the obstacle was located. Also, the foot height of the Parkinson's disease patient shows that he was unable to step forward at around d2m, and that he was freezing his gait.

[0032] The information output by the medical institution device 1 in S8 of FIG. 5 is displayed based on the distance traveled, making it easier to understand the subject's movements near the location of an obstacle. For example, information showing the change in the subject's speed, stride length, foot height, or elapsed time relative to the distance traveled may be displayed. However, FIGS. 7 and 8 are merely examples of information showing the subject's movements, and the information output by the medical institution device 1 is not limited to these. For example, a graph based on elapsed time rather than distance traveled may be output. The number of collisions between the subject and the wall 241 or obstacle 245 in the VR space, or the location of the collisions, may also be output. Recording collisions can help evaluate spatial cognitive ability. The variability (e.g., standard deviation) of the subject's walking speed, stride length, or foot height may also be calculated and output. The variability may indicate the possibility of a rhythm formation disorder. At least one of the location and time when the subject stopped walking may also be output. This method is expected to be clinically applicable as a highly accurate method for evaluating gait disorders. This will facilitate early detection of Parkinson's disease and the provision of appropriate treatment, leading to improved quality of Parkinson's disease care. In terms of drug discovery, it will improve the accuracy of evaluating treatment effectiveness, aiding in development. From a social perspective, the system will enable medical professionals other than physicians to evaluate gait disorders, leading to work style reforms such as reducing doctors' working hours. Furthermore, the established system can be widely applied to neurological and orthopedic disorders that present with gait disorders other than Parkinson's disease.

[0033] Furthermore, in S8 of FIG. 5 , when the subject's movements are displayed as avatar movements in a three-dimensional space, the processor 11 of the medical institution device 1 displays, for example, a screen such as that shown in FIG. 3 on the UI 14. The arrangement of objects in the three-dimensional space shown in FIG. 3 corresponds to the arrangement of objects in the VR environment shown in FIG. 2 . Therefore, by viewing the avatar's movements, the user can understand the location and movements of the subject. In the example of FIG. 3 , the avatar's movements are simultaneously displayed from two different viewpoints, but they may be displayed from a single viewpoint or from three or more viewpoints. Furthermore, by allowing the viewpoint to be switched, blind spots such as shadows of obstacles can be eliminated. Furthermore, the transparency of the wall 241 and the obstacle 245 may be adjustable. Since medical findings can be reconfirmed from various angles, the accuracy of evaluations by users such as doctors can be improved. Furthermore, using the avatar when explaining symptoms to a subject can help the subject understand the condition. Furthermore, the avatar's movements can be anonymized so that the subject's appearance is not revealed, which has the advantage of facilitating the use of video data of cases.

[0034] Note that the process flow diagram of FIG. 5 is a schematic diagram, and the order of the processes may be changed or executed in parallel without departing from the spirit of the present disclosure. Furthermore, information defining the VR space may be stored on the VR device 2 side, and the VR device 2 may operate standalone. Furthermore, sensing data output by the sensor of the VR device 2 may be transmitted to the medical institution device 1 each time it is acquired in S3. That is, the processes of S6 and S7 may be performed between S3 and S4. Furthermore, the medical institution device 1 may render a VR space based on the acquired sensing data and transmit it to the VR device 2. That is, the process of S4 may be performed based on data transmitted by the medical institution device 1.

[0035] The definition of the VR space shown in Figure 6 is an example of a database, and the data structure is not limited to this. For example, the location of an obstacle may be defined by the distance from the side wall closest to the obstacle to the obstacle, or the distance between the goal and the obstacle. Furthermore, the table may be properly normalized to store information separately in multiple tables, or may be denormalized to store additional information in a single table.

[0036] Furthermore, while the VR space (VR environment) preferably mimics a living environment, it is not necessary to reproduce an environment unique to each subject. Furthermore, examples are not limited to those including corridors. A VR space simulating a living environment may be, for example, a space with doors, gates, or other obstacles, a relatively narrow space, a relatively dark space, or a combination of two or more of these elements. Even such spaces can induce motor dysfunction in the subject. It has been pointed out that gait disorders in movement disorders such as Parkinson's disease are easily influenced by the environment. For example, compared to a bright, relatively spacious space such as an examination room, a home environment is likely to be exacerbated by obstacles such as furniture and doors, narrow spaces (e.g., narrow spaces caused by furniture or luggage placed on the floor, or already narrow spaces such as bathrooms), and dark places. Thus, a VR space simulating a living environment may be sufficient as long as it can reproduce an environment that can induce motor dysfunction in the subject. In addition, in order to efficiently induce gait disorders such as freezing and evaluate the symptoms, an unrealistic VR space may be displayed, such as an empty space with only obstacles and courses related to the walking test.

[0037] Note that "disorderly placement" refers to stacking multiple obstacles of the same shape so that their sides are not flush, or stacking obstacles of different shapes. For example, it has been pointed out that one of the mechanisms of gait disorders, such as Parkinson's disease, is related to visual-spatial cognitive dysfunction, and a decrease in spatial resolution is expected. Therefore, the more cluttered the walking space, the greater the burden it can place on the brain to analyze the surrounding space while walking and create a gait plan (e.g., what walking trajectory, speed, and posture to walk in). Since the burden of analyzing the surrounding space while walking and creating a gait plan is thought to be one of the mechanisms of gait disorders, examples include cardboard boxes that are about to collapse or multiple obstacles arranged at an angle to each other. Note that the burden of analyzing the surrounding space while walking and creating a gait plan may be, for example, the presence of graffiti on walls or floors, the presence of transparent objects such as automatic doors, etc.

[0038] FIG. 9 is an example of a graph representing a heat map of the gait trajectory of a Parkinson's disease patient calculated from the center coordinates of the VR device 2. This heat map is based on an analysis of data recorded for a virtual corridor in the VR space shown in the lower part of FIG. 6 , where the length d1 of the corridor is 3.0 m, the width w1 is 1.0 m, and the d2 where a virtual obstacle is located is 2.0 m. In the heat map, for example, the 3.0 m x 1.0 m area of ​​the virtual corridor is divided into 50 x 150 subregions (2 cm x 2 cm) of a predetermined size, and the color becomes darker the longer the patient remains in that area. This division of the subregions can be appropriately set depending on the gait trajectory being analyzed. As shown in the area Z1 circled by a white line, the patient's gait falters near the virtual obstacle located in the virtual corridor. In this way, by representing the gait trajectory calculated from the center coordinates of the VR device 2 as a heat map, it is possible to visualize the gait stagnation of a patient with gait disorders.

[0039] Freezing of gait is a gait disorder that can occur mainly in Parkinson's disease, progressive supranuclear palsy, etc., and is defined as so-called "freezing of gait (FOG)." Freezing of gait is a symptom in which the soles of the feet seem to stick to the floor, making it impossible to move, despite the intention to walk. In other words, focusing on stagnation during walking (a state in which there is no forward or backward movement) and appropriately analyzing the frequency of stagnation in the subject's walking trajectory makes it possible to quantitatively evaluate it.

[0040] FIG. 10 is an example graph showing the correlation between the number of gait stalls and the assessment results of two specialists. To confirm the correlation between the visualized gait stall frequency described in FIG. 9 and the severity of FOG, two specialists assessed the severity of FOG on a four-level scale based on the subject's movements, which were reproduced in a virtual three-dimensional space using an avatar. Specifically, if no FOG was assessed, the assessment level was "0." If the subject moved forward in small, shuffling steps, the assessment level was "1." Similarly, if movement such as leg tremors was observed but no effective forward movement was observed, the assessment level was "2." If foot movement was barely observed, the subject stopped in place and did not move forward at all, the assessment level was "3." The number of gait stalls was calculated based on the number of stalls calculated under baseline conditions, and the number of stalls exceeding a predetermined multiple (e.g., 5 times) of this number was used as a feature for determining the correlation. In FIG. 10, the vertical axis represents the number of portions where the increase is greater than or equal to a predetermined multiple, and the horizontal axis represents the results of the specialist's assessment.

[0041] The correlation coefficient (Pearson's product-moment correlation coefficient) r, which is an index representing the strength and direction of the relationship between two quantitative variables, was calculated between the numerical values ​​of the stagnation points extracted based on the baseline conditions and the FOG severity assessment by a specialist. In the graph shown in Figure 10, the calculated correlation coefficient r was 0.74, confirming a fairly strong correlation. The p-value indicating the significance of the r value in Figure 10 was "p<0.001." In other words, it can be said that the frequency of gait stagnation visualized in Figure 9 shows a strong correlation with the severity of FOG, which is freezing of gait.

[0042] The presence or absence of freezing of gait based on the specialist's assessment may be evaluated using, for example, a machine learning classifier. In machine learning, not only the number of times of stumbling as described above, but also stumbling time, walking speed, etc. may be extracted as features to be used for classification so as to maximize the classifier's assessment performance (e.g., stumbling time around 30 cm of an obstacle on the walking path, etc.).

[0043] FIG. 11 is an example of a graph showing the scored frequency of stagnation calculated using the subject's Q-learning algorithm and FOG severity score. In FIG. 11 , a virtual corridor from the starting point (0 m) to the goal (3.0 m) is divided into subregions every 0.2 m, and the calculated stagnation frequency for each subregion is shown. The subregion divisions may be appropriately set depending on the characteristics (age, height, gender, etc.) of the subject being analyzed. The stagnation frequency analysis is performed by scoring the stagnation frequency based on the center coordinates of the VR device 2 sampled at a predetermined periodic interval (e.g., 20 ms). The scoring utilizes the processing of the Q-learning algorithm, and a predetermined reward (penalty) is assigned based on the relationship between the position at time t and the position at time t+1. Specifically, a reward of "0" is assigned if the position at time t+1 is forward or backward relative to the position at time t, and a reward of "1" is assigned if the subject is stagnant (no forward or backward movement), thereby calculating a total score for each subregion. However, the scoring method is not limited to the Q-learning algorithm process. Any method that can score the frequency of stagnation in the virtual corridor can be set appropriately.

[0044] 11, it can be seen that the scores of healthy elderly people and PD (Parkinson's disease) patients tend to be relatively high near the virtual obstacle. However, the scores of PD patients tend to be higher overall (in all partial regions) compared to the scores of healthy elderly people, and this tendency is particularly pronounced near the virtual obstacle.

[0045] FIG. 12 is a boxplot showing an example of the results of scoring the stagnation frequency near a virtual obstacle. In FIG. 12, normalization was applied using the score values ​​for each subregion obtained under baseline conditions. Here, the baseline conditions refer to a walking trajectory calculated under conditions in which no virtual obstacles were present in the virtual corridor from the starting point (0 m) to the goal (3.0 m). Normalizing the walking trajectory in the virtual corridor where a virtual obstacle was placed using the walking trajectory calculated under the baseline conditions can eliminate individual differences between subjects. Furthermore, the vicinity of the virtual obstacle refers to four subregions (e.g., 1.6 m to 2.4 m) before and after the location where the virtual obstacle was placed (e.g., 2.0 m). HC in FIG. 12 shows a boxplot of the averaged scores for the normalized healthy elderly group, and PD in FIG. 12 shows the averaged scores for the normalized Parkinson's disease patient group.

[0046] Here, the healthy elderly group (HC) and the Parkinson's disease patient group (PD) can be considered to be independent groups with no correlation with each other. Therefore, an unpaired t-test was performed to confirm whether there was a statistical difference between the healthy elderly group (HC) and the Parkinson's disease patient group (PD) for the stagnation frequency results shown in Figure 12. The result of the t-test showed a p-value of 5.54 x 10 ∧ (-8), and a statistically significant difference was confirmed between the healthy elderly group (HC) and the Parkinson's disease patient group (PD).

[0047] In this way, by using the system 10, it is possible to induce freezing of gait (FOG) in the subject by using a virtual obstacle displayed in a virtual reality environment. Furthermore, it was confirmed that the degree of gait disorder can be quantitatively visualized and analyzed by analyzing the walking trajectory of the subject calculated from the center coordinates of the VR device 2.

[0048] [Others] Although the embodiments have been described above, the present disclosure is not limited thereto, and various modifications based on the knowledge of those skilled in the art are possible as long as they do not deviate from the spirit of the claims. For example, at least some of the functions of the medical institution device 1 may be distributed among multiple devices, or multiple devices may provide the same functions in parallel. Furthermore, a system may be provided that simply records the subject's movements using a 6DoF sensor or motion capture system, or that simply displays the recorded data as a graph or avatar's movements.

[0049] The present technology also includes a method and a computer program for executing the above-described processing, and a computer-readable recording medium having the program recorded thereon. The recording medium having the program recorded thereon enables the above-described processing by causing a computer to execute the program.

[0050] Here, a computer-readable recording medium refers to a recording medium that stores information such as data and programs electrically, magnetically, optically, mechanically, or chemically and can be read by a computer. Among such recording media, those that can be removed from a computer include flexible disks, magneto-optical disks, optical disks, magnetic tapes, memory cards, etc. Furthermore, recording media that are fixed to a computer include HDDs, SSDs, ROMs, etc.

[0051] Embodiments of the present invention include the following aspects (hereinafter referred to as appendices). (Appendix 1) A gait assessment support method, in which a computer executes the steps of: causing a display device to display a virtual reality (VR) environment that simulates a living environment; and acquiring and recording data corresponding to the walking movements of a subject wearing the display device via a sensor. (Appendix 2) A gait assessment support device, in which a display device displays a virtual reality (VR) environment that simulates a living environment; and acquiring and recording data corresponding to the walking movements of a subject wearing the display device via a sensor. (Appendix 3) A gait assessment support program, in which a computer executes the steps of: causing a display device to display a virtual reality (VR) environment that simulates a living environment; and acquiring and recording data corresponding to the walking movements of a subject wearing the display device via a sensor. (Supplementary Note 4) A gait evaluation support method executed by a computer, comprising: acquiring and recording, via a sensor, data corresponding to the walking movement of a subject wearing a display device that displays a virtual reality (VR) environment simulating a living environment; and reproducing the walking movement corresponding to the data through the movement of an avatar in a three-dimensional space corresponding to the virtual reality environment, the three-dimensional space being configured to allow a user to change a viewpoint in order to evaluate the walking movement of the subject. (Supplementary Note 5) The gait evaluation support method according to Supplementary Note 4, wherein obstacles can be placed in the virtual reality environment and the three-dimensional space, and the transparency of the obstacles can be changed in the three-dimensional space. (Supplementary Note 6) The gait evaluation support method according to Supplementary Note 4, wherein the virtual reality environment and the three-dimensional space include a passage, and the transparency of walls of the passage can be changed in the three-dimensional space. (Supplementary Note 7) The gait evaluation support method according to Supplementary Note 4, wherein the viewpoint can be changed to that of the avatar in the three-dimensional space. (Supplementary Note 8) The gait evaluation support method according to Supplementary Note 4, wherein the movement of the avatar can be displayed from a plurality of different viewpoints simultaneously. (Supplementary Note 9) The gait evaluation support method according to any one of Supplementary Notes 4 to 8, wherein the sensor is a sensor that realizes motion capture.(Supplementary Note 10) A gait evaluation support device that performs the following steps: acquiring and recording, via a sensor, data corresponding to the walking movement of a subject wearing a display device that displays a virtual reality (VR) environment that simulates a living environment; and reproducing, by means of an avatar's movement, the walking movement corresponding to the data in a three-dimensional space that corresponds to the virtual reality environment and that is configured to allow a user to change the viewpoint in order to evaluate the walking movement of the subject. (Supplementary Note 11) A gait evaluation support program that causes a computer to execute the following steps: acquiring and recording, via a sensor, data corresponding to the walking movement of a subject wearing a display device that displays a virtual reality (VR) environment that simulates a living environment; and reproducing, by means of an avatar's movement, the walking movement corresponding to the data in a three-dimensional space that corresponds to the virtual reality environment and that is configured to allow a user to change the viewpoint in order to evaluate the walking movement of the subject.

[0052] 10: System 1: Medical institution device, 11: Processor, 12: Storage device, 13: Communication interface (IF), 14: User interface (UI) 2: VR device, 21: Processor, 22: Storage device, 23: Communication interface (IF), 24: User interface (UI), 25: Sensor 3: Motion capture system, 31: Processor, 32: Communication interface (IF), 33: Sensor

Claims

1. A gait evaluation support method in which a computer executes the following steps: displaying a virtual reality (VR) environment that simulates a living environment on a display device; acquiring and recording data corresponding to the walking movements of a subject wearing the display device via a sensor; analyzing the walking state of the subject from the recorded data; and visualizing the results of the analysis so that the walking state of the subject can be evaluated.

2. The gait assessment support method according to claim 1, wherein obstacles can be arranged in an aligned or random fashion in the virtual reality environment.

3. The gait assessment support method according to claim 2, wherein the virtual reality environment includes a passageway.

4. The gait assessment support method according to claim 3, wherein at least one of the width of the passage and the arrangement of the obstacles is changeable in the virtual reality environment.

5. The gait evaluation support method according to claim 3, further comprising the step of counting and displaying the number of collisions with the wall of the passage or the obstacle in the virtual reality environment.

6. The gait evaluation support method according to claim 1, wherein the sensor realizes at least one of position tracking and motion capture.

7. The gait evaluation support method according to claim 6, further comprising the computer displaying a graph of the walking speed of the subject, the stride length of the subject, the foot height of the subject, or the elapsed time relative to the distance traveled.

8. The gait evaluation support method according to claim 6, wherein the walking speed, stride length, or rate of change in foot height of the subject, or the position and time when the subject stopped walking, is calculated and displayed.

9. The gait evaluation support method according to claim 1, further comprising the computer reproducing, by the movement of an avatar, walking movements according to the data in a three-dimensional space corresponding to the virtual reality environment, configured so that a user can change the viewpoint in order to evaluate the walking movements of the subject.

10. A gait evaluation support method as described in claim 1, which analyzes and visualizes the walking trajectory of the subject in the virtual reality environment, and evaluates the walking condition of the subject based on the frequency of stagnation in areas divided into partial areas of a predetermined size in the virtual reality environment.

11. A gait evaluation support device that performs the following operations: displays a virtual reality (VR) environment that simulates a living environment on a display device; acquires and records data corresponding to the walking movements of a subject wearing the display device via a sensor; analyzes the walking state of the subject from the recorded data; and visualizes the results of the analysis so that the walking state of the subject can be evaluated.

12. A gait evaluation support program that causes a computer to perform the following steps: displaying a virtual reality (VR) environment that simulates a living environment on a display device; acquiring and recording data corresponding to the walking movements of a subject wearing the display device via a sensor; analyzing the walking state of the subject from the recorded data; and visualizing the results of the analysis so that the walking state of the subject can be evaluated.

Images