Measurement and evaluation system, and measurement and evaluation program

Abstract

Its simple configuration enables highly accurate and quantitative evaluation of movement disorders. [Solution] The measurement and evaluation system comprises: a detection unit that detects movements including finger movements of the user by detecting hand tracking on a head-mounted display worn by the user who is the subject of the test; a generation unit that generates a hand object, a reference point, and a target point; a display control unit that controls the display of the reference point and the target point to alternately switch in accordance with the contact of the fingertips of the hand object, and controls the display of the reference point to a fixed position from the user's viewpoint and the target point to be displayed at a different position; a measurement unit that analyzes the movements detected for each round trip and measures each measurement item of the motor impairment; and an output unit that outputs the measurement results for each measurement item.

Description

Technical Field

[0001] The present disclosure relates to a measurement and evaluation system and a measurement and evaluation program.

Background Art

[0002] Conventionally, there are technologies related to the evaluation of motor disorders and the like.

[0003] For example, there is a technology for effectively supporting rehabilitation related to ataxia (see Patent Document 1). In this technology, it is disclosed that feedback is performed by evaluating whether the user can achieve an operation of touching an avatar object to a target object.

[0004] Also, there is a technology related to an augmented reality system using reflection (see Patent Document 2). In this technology, a technology related to augmented reality content using reflection in a wearable display device is disclosed, and it is disclosed that a so-called nose-finger test for the evaluation of ataxia can be performed.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] Incidentally, both ataxia and Parkinsonian symptoms are conditions that impair smooth movement, significantly reducing the patient's quality of life (QOL). Currently, there are few successful clinical trials demonstrating efficacy in various neurodegenerative diseases that present with ataxia, particularly spinocerebellar degeneration (SCD). One contributing factor has been the subjective nature of conventional severity assessment methods.

[0007] Traditionally, the Scale for the Assessment and Rating of Ataxia (SARA) has been used to assess the severity of ataxia, and the Unified Parkinson's Disease Rating Scale (UPDRS) has been used to assess the severity of Parkinson's disease, a representative disease exhibiting Parkinsonian symptoms. However, both are semi-quantitative assessments that primarily rely on visual observation, and are susceptible to the examiner's skill level and subjectivity, as well as lacking the sensitivity to detect subtle changes. Furthermore, because SCD progresses relatively slowly, it is possible that the effectiveness of investigational drugs cannot be accurately and objectively evaluated in clinical trials conducted over short periods.

[0008] Attempts to quantitatively evaluate ataxia and Parkinson's symptoms have been explored both domestically and internationally using various methods. For example, attempts have been made to quantify the movements of the limbs and trunk using three-dimensional motion analysis with motion capture devices that record human movement in three dimensions, and to quantify these movements using robotic devices. However, these methods have significant limitations, such as requiring large-scale or expensive equipment, or specific environmental facilities, and a system that can be easily used for evaluation in a wide range of clinical settings has not yet been established.

[0009] The objective of this disclosure is to provide a measurement and evaluation system and a measurement and evaluation program that enable highly accurate and quantitative evaluation of movement disorders with a simple configuration. [Means for solving the problem]

[0010] The measurement and evaluation system of this disclosure includes: a detection unit that detects the movements of a user, including the movement of the user's fingers, by detecting hand tracking on a head-mounted display worn by the user, who is a subject of the test; a generation unit that generates a hand object corresponding to the user's movements for a test relating to the evaluation of motor impairment, a base point corresponding to the nose of the subject of the test, and a target point corresponding to the finger of the examiner of the test, in a virtual space generated and displayed on the head-mounted display; a display control unit that controls the display to alternately switch the display of the base point and the target point in accordance with the contact of the fingertip of the hand object, and controls the display to show the base point at a fixed position from the user's viewpoint and the target point at a fixed distance from the user's viewpoint at a different position each time the display is switched; a measurement unit that analyzes the movements detected for each round trip of the hand object between the base point and the target point, and measures the following items as measurement items for motor impairment: a measurement item relating to the distance in the actual distance representing the distance the fingertip moves relative to the shortest distance between the base point and the target point, a measurement item relating to the path distance past the target point, and a measurement item relating to the time of the round trip; and an output unit that outputs the measurement results for each of the measurement items. [Effects of the Invention]

[0011] The measurement and evaluation system and measurement and evaluation program described herein offer the advantage of enabling highly accurate and quantitative evaluation of motor impairments with a simple configuration. [Brief explanation of the drawing]

[0012] [Figure 1] Figure 1 shows an example of a subject wearing a head-mounted display. [Figure 2] Figure 2 shows an example of drawing a base point and a target point in a virtual space. [Figure 3] Figure 3 is a block diagram showing the functional configuration of a head-mounted display. [Figure 4] Figure 4 shows an example of the settings for a hand object. [Figure 5] Figure 5 is an image of measurement items related to distance. [Figure 6] Figure 6 is an image of measurement items related to path distance. [Figure 7] Figure 7 is an image of measurement items related to path distance. [Figure 8] Figure 8 is an image of measurement items related to the round-trip time. [Figure 9] Figure 9 is a graph of the measured values and coefficient of variation of a verification example of measurement items for the difference in distance. [Figure 10] Figure 10 is a graph of the measured values and coefficient of variation of a verification example of measurement items for the ratio of distances. [Figure 11] Figure 11 is a graph of the measured values and coefficient of variation of a verification example of measurement items for path distance. [Figure 12] Figure 12 is a graph of the measured values and coefficient of variation of a verification example of measurement items for the maximum distance. [Figure 13] Figure 13 is a graph of the measured values and coefficient of variation of a verification example of measurement items for the average value of the time required for the forward and return trips. [Figure 14] Figure 14 is a graph of the measured values and coefficient of variation of a verification example of measurement items for the average moving speed of the fingertips. [Figure 15] Figure 15 is a table summarizing the correlation between each measurement item and SARA. [Figure 16] Figure 16 is a table showing the correlation coefficients between each measured value and the SARA total score / SARA upper limb subscore. [Figure 17] Figure 17 is a table showing the correlation coefficients between each coefficient of variation and the SARA total score / SARA upper limb subscore. [Figure 18] Figure 18 is a flowchart showing the flow of the measurement evaluation process.

Mode for Carrying Out the Invention

[0013] An example of an embodiment of the disclosed technology will be described below with reference to the drawings. In each drawing, identical or equivalent components and parts are given the same reference numerals. Furthermore, the dimensional ratios in the drawings are exaggerated for illustrative purposes and may differ from actual ratios.

[0014] This document outlines the embodiments of this disclosure. The objective of the technology of this disclosure is to create clinical biomarkers that accurately and quantitatively reflect the severity of upper limb motor impairment in patients exhibiting ataxia or Parkinsonian symptoms, using a head-mounted display, which is a VR (Virtual Reality) device. Specifically, the nose-finger test used in conventional SARA evaluation methods is evaluated in the virtual space of the head-mounted display, visualizing and quantifying the smoothness of upper limb movement. This allows for accurate evaluation of upper limb motor impairment, including ataxia and Parkinsonian symptoms, enabling accurate evaluation before and after treatment, leading to appropriate treatment selection. Furthermore, for progressive diseases such as SCD, accurate assessment of severity is possible, leading to appropriate advice to patients, such as rehabilitation. It is also considered useful in differentiating diseases that exhibit ataxia or Parkinsonian symptoms. In addition, since evaluation can be performed at the bedside, it is possible to evaluate even patients who are unable to walk, enabling evaluation of a wide range of patients.

[0015] The measurement and evaluation system of this embodiment makes it possible to reproduce in a virtual space the evaluation test for motor impairment, the so-called nose-finger test, which is conducted in a virtual space using a head-mounted display. Furthermore, by measuring in the virtual space, it enables quantitative evaluation using unique measurement items for motor impairment that could not be achieved with conventional tests.

[0016] Figure 1 shows an example of a subject wearing a head-mounted display. The head-mounted display 102 in the measurement and evaluation system 100 is worn on the head of the subject, the user (U), and includes a display, speaker, camera, hand tracking, and virtual space content generation function. The memory area of ​​the head-mounted display 102 stores a measurement and evaluation program for performing the measurement and evaluation of this embodiment. The content generation function can generate various objects for tests related to the evaluation of motor impairment. These objects include hand objects, base points, and target points. In the example in Figure 1, the user (U) performs the test in a seated position, and the base point (B) is displayed on the display for confirmation of the real space. Hand tracking can be achieved, for example, by multiple cameras provided in the head-mounted display 102. The head-mounted display 102 generates a hand object (H1) in the virtual space that corresponds to the user's (U) movements. Various processors can be used as hardware resources for the head-mounted display 102. Examples of processors include the CPU (Central Processing Unit), a general-purpose processor that functions as a hardware resource for executing processing by running software, i.e., programs. Beyond CPUs, other examples include the GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit). The head-mounted display 102 is connected to external devices and displays via wireless communication, enabling data output.

[0017] Figure 2 shows an example of the rendering of a base point and target point in a virtual space. The virtual space generated and displayed by the head-mounted display 102 displays a "base point" and a "target point" for the test on the display screen. The user (U) alternately moves back and forth between the base point (B) and the target point (O) during the test. The base point (B) corresponds to the subject's nose and is drawn as a green sphere, for example. The target point (O) corresponds to the examiner's finger and is drawn as a red fingertip, for example. The base point (B) always appears at a fixed position in front of the subject's eyes, and the target point (O) appears at different positions in a random pattern, but the distance is constant. The base point (B) and target point (O) appear alternately, and feedback is provided by a sound effect when the subject touches them correctly. The target point (O) is also drawn as the fingertip of the examiner's hand object (H2). For the sake of clarity, symbols will be omitted as appropriate in the following explanation.

[0018] The head-mounted display 102 is lightweight, reducing the burden on the subject. Furthermore, it is cable-free and marker-free, allowing for evaluation in any location. In addition, the background of the virtual space display screen has a simple screen layout, creating an environment where the subject can concentrate more on the test.

[0019] Figure 3 is a block diagram showing the functional configuration of the head-mounted display 102. Functionally, the measurement and evaluation function unit 200 includes a detection unit 202, a generation unit 204, a setting unit 206, a display control unit 208, a measurement unit 210, and an output unit 212. Each functional configuration is realized by the processor reading the measurement and evaluation program, expanding it into memory, and executing it.

[0020] The detection unit 202 detects user movements, including finger movements, by detecting hand tracking in the head-mounted display 102.

[0021] The generation unit 204 generates a hand object, a base point, and a target point in the virtual space that correspond to the user's actions.

[0022] The setting unit 206 sets the length of the hand object by measuring the length of the user's arm in the virtual space. Note that the processing of the setting unit 206 can be omitted by obtaining the subject's arm length data in advance.

[0023] Figure 4 shows an example of setting up a hand object. In this embodiment, the length of the user's arm is measured in the virtual space, and a base point and target point are placed according to that length. For example, by displaying navigation such as "Adjust the target according to your arm length" on the virtual space display screen, the user is instructed to extend their arm forward to measure the arm length.

[0024] The display control unit 208 controls the display to alternately switch between the base point and the target point in response to the fingertip contact of the hand object. Furthermore, as described above, the display control unit 208 controls the display to show the base point at a fixed position from the user's viewpoint and the target point at a fixed distance from the user's viewpoint, at a different position each time the display is switched.

[0025] The measurement unit 210 analyzes the motion detected for each round trip of the hand object between the base point and the target point, and measures various measurement items of the motor impairment. The measurement items are (1) distance-related measurement items, (2) path distance past the target point, and (3) round trip time measurement items. (1) Distance-related measurement items are items that measure the actual distance, which represents the distance the fingertip moves relative to the shortest distance between the base point and the target point.

[0026] Here, we will explain the various measurement items measured by the measurement unit 210.

[0027] Figure 5 shows an image of the distance measurement items. In the measurement in the virtual space, the straight line connecting the base point and the target point is defined as the shortest distance (Rs), and the actual distance traveled by the fingertip is measured as the actual distance (Ra). The measurement unit 210 measures the following distance measurement items: (1-1) the difference in distance between the actual distance and the shortest distance between the base point and the target point (actual distance - shortest distance). The measurement unit 210 also measures (1-2) the ratio of the distance between the shortest distance and the actual distance (actual distance / shortest distance).

[0028] Figures 6 and 7 illustrate the measurement items related to path distance. A circle with a diameter of approximately 60 cm centered on the target point is assumed. The measurement unit 210 measures the following items related to path distance: (2-1) the path distance (Dr) from passing the circle to the target point. The measurement unit 210 also measures (2-2) the distance furthest from the target point along the path, i.e., the maximum distance (Dm) from passing the target point to the target point.

[0029] Figure 8 shows an image of the measurement items related to round-trip time. The outward journey is from the base point to the target point, and the return journey is from the target point to the base point. The measurement unit 210 measures the following items related to round-trip time: (3-1) the average time required for the outward and return journeys. The measurement unit 210 also measures (3-2) the average fingertip movement speed for the outward and return journeys.

[0030] Furthermore, the measurement unit 210 also calculates the coefficient of variation for the measurement items (1-1) to (3-2) above and evaluates the variability.

[0031] The output unit 212 outputs the measurement results for each measurement item. The output destination may be the display screen of the head-mounted display 102 or an external display. The output unit 212 also saves the measurement results to the memory area of ​​the head-mounted display 102.

[0032] [Verification Example] As a verification of the measurement and evaluation system 100 of this embodiment, the measured values ​​and coefficients of variation for each measurement item were calculated and statistically evaluated in the three groups shown in Table 1 below. The three groups consisted of subjects from the control group, the ataxia group, and the parkinsonism group. [Table 1]

[0033] Figure 9 shows graphs of measured values ​​and coefficients of variation for an example of the measurement item for distance difference. Figure 10 shows graphs of measured values ​​and coefficients of variation for an example of the measurement item for distance ratio. In both cases, it was confirmed that both the measured values ​​and coefficients of variation were significantly larger in the ataxia group than in the other two groups.

[0034] Figure 11 shows graphs of measured values ​​and coefficients of variation for a verification example of the path distance measurement item. Figure 12 shows graphs of measured values ​​and coefficients of variation for a verification example of the maximum distance measurement item. In both cases, the measured values ​​were confirmed to be significantly larger in the ataxia group than in the other two groups. Furthermore, there was a significant difference in the coefficient of variation between the ataxia group and the parkinsonism group.

[0035] Figure 13 shows graphs of the measured values ​​and coefficient of variation for the verification example of the average time required for the outward and return journeys. Figure 14 shows graphs of the measured values ​​and coefficient of variation for the verification example of the average fingertip movement speed. From the average values, it was confirmed that the ataxia group and the parkinsonism group took significantly longer. The coefficient of variation was confirmed to be larger in the ataxia group than in the other two groups. From the average movement speed, it was confirmed that the parkinsonism group was significantly slower than in the other two groups. The coefficient of variation was confirmed to be significantly larger in the ataxia group than in the other two groups.

[0036] Figure 15 is a table summarizing the correlation between each measurement item and SARA. Figure 16 is a table showing the correlation coefficient between each measured value and the SARA total score / SARA upper limb subscore. Figure 17 is a table showing the correlation coefficient between each coefficient of variation and the SARA total score / SARA upper limb subscore. SARA is one example of a method for assessing the severity of ataxia. The SARA total score and SARA upper limb subscore were calculated for each measured value and coefficient of variation. Significant correlation coefficients were observed for the measured items marked with an asterisk (*) in Figures 17 and 18. Among these, (1-2) the ratio obtained from measured value / shortest distance and (3-1) the average time required for the outward and return journeys showed significant correlations with the SARA total score / SARA upper limb subscore for both measured values ​​and coefficients of variation, suggesting that they are more useful as severity markers in ataxia.

[0037] Furthermore, the evaluation items (1-1) and (1-2) suggest that they may reflect motor breakdown and dysmetria, and that irregularity can also be evaluated by their coefficients of variation. In addition, existing studies have shown that upper limb movements in patients with ataxia tend to be longer and more roundabout than those of healthy individuals.

[0038] The evaluation items in (2-1) and (2-2) suggest the possibility of reflecting measurement impairment and deviation at the final point of arrival. In addition, existing studies have confirmed that there is no difference in fingertip deviation at the final point of arrival at the target point between the ataxic group and the control group in the nose-finger test and pointing task. However, existing studies evaluate the two-dimensional deviation at the point of arrival at the target point, and the evaluation method is different. The method of this embodiment evaluates the path to reach the target point and evaluates the three-dimensional deviation.

[0039] From the evaluation items (3-1) and (3-2), the ataxia group showed no difference in fingertip movement speed compared to the control group, but the variability was greater and it took longer. In addition, existing studies have reported that in patients with ataxia, the variability in movement speed is greater, and the coefficient of variation is a more sensitive indicator than the average speed. Therefore, it was suggested that irregularity (variability) evaluated by the coefficient of variation may be a more sensitive marker. In addition, the parkinsonism group showed slower fingertip movement speed and less variability in speed. Furthermore, existing studies have reported that quantitative motor evaluation of the upper limbs in Parkinson's disease patients reflect bradykinesia, and it was confirmed that the results are consistent with those of this method.

[0040] (Process flow) Next, the operation of the measurement and evaluation system 100 will be explained. Figure 18 is a flowchart showing the flow of the measurement and evaluation process. The processor of the head-mounted display 102 reads the measurement and evaluation program, loads it into the memory area, and executes it. The processor then functions as each part of the measurement and evaluation function unit 200, thereby performing the measurement and evaluation process.

[0041] In step S100, the processor detects the user's movements, including finger movements, by detecting hand tracking in the head-mounted display 102. Note that motion detection continues in the following processes.

[0042] In step S102, the processor generates a hand object, a base point, and a target point in the virtual space that correspond to the user's actions.

[0043] In step S104, the processor sets the length of the hand object by measuring the length of the user's arm in the virtual space.

[0044] In step S106, the processor controls the display of the base point and the target point to alternately switch in response to contact by the fingertip of the hand object.

[0045] In step S108, the processor analyzes the motion detected for each round trip and measures various parameters of the motion impairment.

[0046] In step S110, the processor outputs the measurement results for each measurement item.

[0047] As described above, the measurement and evaluation system 100 according to this embodiment enables highly accurate and quantitative evaluation of movement disorders with a simple configuration.

[0048] This disclosure is not limited to the embodiments described above, and various modifications and applications are possible without departing from the spirit of the invention.

[0049] Furthermore, although the present specification describes an embodiment in which the program is pre-installed, it is also possible to provide the program stored on a computer-readable recording medium. [Explanation of Symbols]

[0050] 100 Measurement and Evaluation Systems 102 Head-mounted displays 200 Measurement and Evaluation Function Unit 202 Detection Unit 204 Generation part 206 Settings Section 208 Display Control Unit 210 Measurement Unit 212 Output section

Claims

1. A detection unit that detects the user's movements, including finger movements, by detecting hand tracking on a head-mounted display worn by the user, who is the subject of the test, A generation unit generates a hand object corresponding to the user's movements for a test related to the evaluation of motor impairment, a base point corresponding to the nose of the test subject, and a target point corresponding to the finger of the examiner in the test, in the virtual space generated and displayed in the head-mounted display. A display control unit controls the display of the base point and the target point to alternately switch in response to the fingertip contact of the hand object, and controls the display of the base point to be at a fixed position from the user's viewpoint and the target point to be at a fixed distance from the user's viewpoint and at a different position each time the display is switched. A measurement unit analyzes the motion detected for each round trip of the hand object between the base point and the target point, and measures the following items as measurement items for movement impairment: a measurement item related to the distance in the actual distance representing the distance the fingertip moves relative to the shortest distance between the base point and the target point, a measurement item related to the path distance past the target point, and a measurement item related to the round trip time. An output unit that outputs the measurement results for each of the aforementioned measurement items, A measurement and evaluation system equipped with the following features.

2. The measurement and evaluation system according to claim 1, further comprising a setting unit that sets the length of the hand object by measuring the length of the user's arm in the virtual space.

3. The measurement unit measures, as measurement items related to distance, the difference in distance between the shortest distance between the base point and the target point and the measured distance representing the distance traveled by the fingertip, and the ratio of the shortest distance and the measured distance. As measurement items related to the aforementioned path distance, the path distance of the fingertip from passing the target point to returning to the target point, and the maximum distance from the point after passing the target point to the target point are measured. The measurement and evaluation system according to claim 1, wherein the measurement items relating to the round trip time include the average time required for the outward and return journeys, and the average fingertip movement speed for the outward and return journeys.

4. On the computer, By detecting hand tracking on a head-mounted display worn by the subject, the system detects the user's movements, including finger movements. In the virtual space generated and displayed in the head-mounted display, a hand object corresponding to the user's movements for the evaluation of motor impairment, a base point corresponding to the nose of the subject of the test, and a target point corresponding to the finger of the examiner of the test are generated. The base point and the target point are controlled to alternately switch their display in response to the fingertip contact of the hand object, the base point is displayed at a fixed position from the user's viewpoint, and the target point is displayed at a fixed distance from the user's viewpoint at a different position each time the display is switched. The motion detected for each round trip of the hand object between the base point and the target point is analyzed, and the following measurement items for motor impairment are measured: a measurement item related to the distance in the actual distance representing the distance the fingertip moves relative to the shortest distance between the base point and the target point; a measurement item related to the path distance past the target point; and a measurement item related to the time of the round trip. Output the measurement results for each of the aforementioned measurement items. A measurement and evaluation program that executes the process.

Images