Finger tapping measurement processing terminal, system, method and computer program

Abstract

To provide a finger tapping measurement processing terminal, system, method, and computer program capable of measurement processing for finger tapping motion inexpensively and usably in a convenient aspect by utilizing widely spread terminals such as smart phones and tablets.SOLUTION: A finger tapping measurement processing terminal 100 which measures finger tapping motion and processes the measurement result includes: a touch panel 103 which is provided to one face of a terminal, is capable of displaying information related to finger tapping motion, and is tapped by one of two fingers of a same hand of a user holding the terminal; finger detection units 101 and 102 which detect motion of the one of the fingers tapping the touch panel 103; and a processing unit which performs a process for measuring, at the finger tapping motion, a distance between two fingers of another of two fingers holding the other face of the terminal located at the opposite side of the touch panel and one finger tapping the touch panel on the basis of detection information detected by the finger detection units.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a finger tapping measurement and processing terminal, a finger tapping measurement and processing system, a method, and a computer program that measure finger tapping movements and process the measurement results. [Background technology]

[0002] As the aging of society progresses, the number of Alzheimer's disease patients is increasing year by year, and if it is detected early, the progression of the disease can be slowed down with medication. However, it can be difficult to distinguish between symptoms of aging, such as forgetfulness, and the disease itself, and in many cases, patients only visit a hospital once their condition has progressed to a severe stage.

[0003] In this situation, screening tests for the early detection of Alzheimer's disease have traditionally included blood tests, olfactory tests, and tests that replicate a doctor's interview on a tablet device, but these have problems with the pain of blood sampling and the length of time required for the test. On the other hand, cognitive function assessments using button presses or one-handed finger movement measurements using a tablet device have also been performed as tests that place less of a burden on the subject, but they have the drawback of not being able to achieve sufficient test accuracy. If a simple screening test could be performed that is highly accurate and places less of a burden on the subject, it could lead to the early detection of Alzheimer's disease, improve the quality of life of patients, and contribute to reducing medical and nursing care costs.

[0004] In recent years, it has become clear that a movement pattern specific to Alzheimer's dementia can be extracted from the opening and closing movement of two fingers (finger tapping movement) using the thumb and index finger of both hands, and it has been confirmed that there is a high correlation with finger movement measurement and dementia tests by general interview. It is said that these are the results of capturing the decline in rhythmic motor function of the fingers of both hands caused by brain atrophy in Alzheimer's dementia by finger tapping movement measurement. In addition, the fingers are said to be the second brain, and many areas of the brain are related to the function of the fingers, and finger movements are said to be related not only to Alzheimer's dementia, but also to dementia such as cerebrovascular and Lewy body dementia, Parkinson's disease, and developmental coordination disorder (such as being unable to skip or jump rope). In other words, it is possible to know the state of the brain from finger tapping movement. Furthermore, finger tapping movement can be used as a "ruler" that indicates the health state of the brain, making it possible to quantify the fine motor function of the fingers, so it can be used in various fields such as healthcare, rehabilitation, and life support.

[0005] As a method for measuring and evaluating finger tapping motion with high accuracy, for example, Patent Document 1 discloses a motor function evaluation system including a motor function measuring device that calculates motor data based on the relative distance between a pair of a transmitter coil and a receiver coil attached to a moving part of a living body, and an evaluation device that evaluates the motor function of a living body based on the motor data received from the motor function measuring device. That is, Patent Document 1 discloses that a magnetic sensor attached to the fingertips converts the change in magnetic force that fluctuates due to the tapping motion of two fingers into an electric signal, measures and quantifies the movement, and captures the characteristic amount that indicates the characteristics of the finger movement, thereby knowing the state of brain function. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2016-49123 A Summary of the Invention [Problem to be solved by the invention]

[0007] The motor function evaluation system disclosed in Patent Document 1 can measure and evaluate finger tapping movements using a magnetic sensor, but it is necessary to attach a magnetic sensor to a finger to measure the finger tapping movement. In addition, it requires expensive dedicated equipment to evaluate and analyze the measurement information detected by the magnetic sensor, which makes it difficult to measure and evaluate finger tapping movements inexpensively.

[0008] The present invention has been made in consideration of the above-mentioned circumstances, and has an object to provide a finger tapping measurement and processing terminal, system, method, and computer program that can measure and process finger tapping movements in a simple, inexpensive, and easy-to-use manner by utilizing widely used terminals such as smartphones and tablets, without the need to wear any special dedicated equipment or requiring expensive dedicated devices for evaluating and analyzing measurement information. [Means for solving the problem]

[0009] In order to solve the above problems, the present invention provides a finger tapping measurement and processing terminal that measures finger tapping movements and processes the measurement results, comprising: a touch panel provided on one surface of the terminal, capable of displaying information related to finger tapping motion, and tapped by one of two fingers of the same hand of a user holding the terminal; a finger detection unit that detects a movement of the one finger tapping on the touch panel; a processing unit that performs processing to measure a distance between the other of the two fingers holding the other surface of the terminal located on the opposite side to the touch panel and the one finger tapping the touch panel based on detection information obtained by the finger detection unit; The present invention is characterized by having the following.

[0010] According to the above configuration, the finger tapping motion can be measured and processed simply by tapping the touch panel of the terminal with a finger due to the presence of the finger detection unit and the processing unit, so that the finger tapping motion can be easily measured and processed without wearing a special dedicated tool or requiring an expensive dedicated evaluation and analysis device. In particular, if a distance measuring sensor or the like that is already mounted as a permanent component on an existing terminal such as a smartphone or tablet can be used as the finger detection unit, and the processing unit can be realized by using the arithmetic circuit of the existing terminal or by newly incorporating it into the terminal as application software, it is possible to measure and process the finger tapping motion in a simple form, at low cost, and with ease, using a terminal such as a widely used smartphone or tablet.

[0011] In the above configuration, the finger detection unit can detect the movement of the fingers using a camera, a distance measuring sensor, or the like. The finger detection unit may be provided on both sides of the terminal corresponding to the fingers of both hands of the user who holds both sides of the terminal, or may be provided on one side of the terminal as a common unit for the fingers of both hands of the user. In addition, when a camera is used as the finger detection unit and the position of the tapping finger is detected from an image captured by the camera, it may be difficult to capture the exact position of the finger due to the shooting angle or shooting distance, so it is preferable to obtain in advance from the image captured by the camera a target distance between two fingers that is a reference for the open position of the fingers in the finger tapping movement as a reference value, and to calibrate the position detection data based on this reference value. In that case, the finger detection unit may specify the reference for the open position of the fingers in the finger tapping movement using a target length jig having a length corresponding to the target distance between two fingers. In addition, it is preferable that such a target length jig has transparency that does not hinder detection by the finger detection unit. In addition, when using the target length jig, if the target length jig is clamped between the surface of the touch panel and the tapping finger, the finger may deform in the clamped state, so it is preferable to use the amount of deformation as well to calibrate the position detection data.

[0012] In the above configuration, the processing unit preferably calculates and analyzes features that lead to an evaluation of the user's brain function based on the measurement result of the inter-finger distance. Furthermore, the processing unit may evaluate the user's brain function and cognitive function from the calculated features (for example, by comparing with data of healthy individuals). Such an evaluation is effective as an early stage screening for dementia and not only helps in detecting dementia, but can also contribute to judging the judgment ability in driving a car, and can also be applied to brain training games, and has a wide range of applications.

[0013] In the above configuration, the touch panel is preferably a touch panel display that displays information by having a touch panel that detects the position of a finger touching its surface. When such a touch panel display is used, the finger detection unit can detect the touch of the tapping finger as a closed position of the finger tapping motion by the touch panel detecting the touch of the tapping finger.

[0014] In the above configuration, it is preferable that the other finger holding the other surface of the terminal is fixed in contact with the other surface of the terminal by the fixing part. In this case, the fixing part may be provided on the terminal side, or may be provided as a mounting tool separate from the terminal.

[0015] The finger tapping measurement processing terminal of the above configuration may be configured as a finger tapping measurement processing system together with a head-mounted information processing device that is attached to the head of a user who uses the terminal and has a display unit that displays real space information and virtual space information in a manner that is visible to the user. In this case, it is preferable that the finger tapping measurement processing terminal and the head-mounted information processing device can communicate with each other via a communication means. By using the communication means, the head-mounted information processing device can receive information to be displayed on the display of the finger tapping measurement processing terminal, for example, and display the received information on the display unit. In this case, as in the previous case, it is preferable that the processing unit of the finger tapping measurement processing terminal can perform a process of calculating and analyzing a feature amount that leads to a brain function evaluation of the user based on the measured inter-finger distance, but it is also possible to have such a calculation and analysis performed by a server device provided remotely. Specifically, for example, such a server device receives the measurement result of the inter-finger distance from the processing unit of the finger tapping measurement processing terminal via the communication means, and calculates and analyzes the feature amount that leads to a brain function evaluation of the user based on the received inter-finger distance measurement result with an arithmetic processing unit. The processing results of the arithmetic processing unit may be stored in a memory unit of the server device, and the data stored in the memory unit may be communicated to the finger tapping measurement processing terminal and the head-mounted information processing device via the communication means.

[0016] Furthermore, in addition to the finger tapping measurement and processing terminal and finger tapping measurement and processing system described above, the present invention also provides a method and computer program for measuring finger tapping movements and processing the measurement results. Effect of the Invention

[0017] According to the present invention, due to the presence of a finger detection unit and a processing unit provided in a terminal with a touch panel, finger tapping motion can be measured and processed simply by tapping the touch panel of the terminal with a finger, so finger tapping motion can be easily measured and processed without the need for wearing any special dedicated equipment or for an expensive dedicated evaluation and analysis device. [Brief description of the drawings]

[0018] [Figure 1] 1 is a perspective view showing a schematic external appearance of a finger tapping measurement processing terminal according to an embodiment of the present invention having a finger detection unit on one side; [Diagram 2] 2 is an enlarged cross-sectional view of a portion of the finger tapping measurement processing terminal of FIG. 1. [Diagram 3] 2 is a diagram for explaining acquisition of a target distance between two fingers when measurement is performed using a camera in the finger tapping measurement processing terminal of FIG. 1. [Figure 4] 13A and 13B are diagrams illustrating how a target distance between two fingers is obtained using a target length jig. [Diagram 5] FIG. 13 is a diagram for additionally explaining how to obtain a target distance between two fingers using a target length jig. [Figure 6] 11A to 11C are diagrams illustrating an operation of acquiring a target distance between two fingers using a finger tapping measurement processing terminal having two displays. [Figure 7] 2 is a diagram showing a measurement method when the finger tapping measurement processing terminal of FIG. 1 uses a distance measuring sensor for measurement. [Figure 8] 1 is a perspective view showing a schematic external appearance of a finger tapping measurement processing terminal having finger detection units on both sides; [Figure 9] 9 is a block diagram showing an example of the configuration of the finger tapping measurement processing terminal shown in FIG. 1 and FIG. 8. [Figure 10] 9 is a flowchart showing an example of the operation of the finger tapping measurement processing terminal shown in FIGS. 1 and 8. [Figure 11] FIG. 2A is a diagram showing an example of measurement data of a finger tapping measurement processing terminal according to an embodiment of the present invention, and FIG. 2B is a diagram showing another example of measurement data of the finger tapping measurement processing terminal according to the embodiment. [Figure 12] 1 is a perspective view showing a schematic external appearance of a finger tapping measurement and processing system according to an embodiment of the present invention; [Figure 13] 13 is a block diagram showing an example of the configuration of a head-mounted information processing device that constitutes the finger tapping measurement processing system of FIG. 12. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The finger tapping measurement processing terminal according to an embodiment of the present invention described below is configured as a terminal capable of measuring and processing finger tapping motions using a touch panel, and may be used in any manner, although it may be connected to a server 1203 via a communication means (network) as shown in Fig. 12 (this will be described later). In this embodiment, the finger tapping measurement processing terminal is configured as a small terminal such as a smartphone, but it may also be in the form of a thin tablet computer, a personal computer, or the like.

[0020] In addition, in the following embodiments, a finger tapping measurement and processing terminal is shown which is equipped with a touch panel and can measure and process finger tapping movements by itself, however, the present invention may be embodied as a system or method which enables measurement and processing of finger tapping movements in cooperation with a separate display, or may be configured as a computer program which enables measurement and processing of such finger tapping movements to be performed by a computer (for example, enabling measurement of finger tapping movements using a terminal having a display on one side and displaying the measurement results on the display) or a computer program product in which such a computer program is stored.

[0021] A specific embodiment of the present invention is shown in FIG. 1, and a finger tapping measurement processing terminal (hereinafter, sometimes simply referred to as a terminal) 100 as an information processing terminal in the figure has a camera 101, a distance measuring sensor 102, and a touch panel 103. The camera 101 and the distance measuring sensor 102 are provided on the edge of one side (the left side of FIG. 1) of the terminal 100, the camera 101 photographs surrounding objects and scenery, and the distance measuring sensor 102 is a sensor that can capture the shape of an object such as a person or an object in three dimensions, and can measure the distance and angle to the object. The touch panel 103 detects the touched position (senses the position of a finger touching its surface), and constitutes a touch panel display 103A. The touch panel display 103A is provided on one side (front surface) of the finger tapping measurement processing terminal 100, and is capable of displaying information related to finger tapping motion, and the touch panel 103 is tapped by one of two fingers (index finger) of the same hand of a user holding the terminal, as described later.

[0022] One side (the side on the left side in FIG. 1) of the finger tapping measurement processing terminal 100 is held by the user's left hand 111, and the other opposing side (the side on the right side in FIG. 1) is held by the user's right hand 121. The thumb 112 of the user's left hand supports the other side (the side opposite the touch panel 103) of the finger tapping measurement processing terminal 100, which is the back side, and the index finger 113 of the user's left hand performs an opening and closing motion of opening and closing in contact with the touch panel 103, i.e., a tapping motion. Similarly, on the other side of the finger tapping measurement processing terminal 100, the thumb 122 of the user's right hand supports the back side of the information processing terminal 100, and the index finger 123 of the user's right hand performs an opening and closing motion of opening and closing in contact with the touch panel 103, i.e., a tapping motion 124. In FIG. 1, the left and right index fingers 113, 123 that open and close are indicated as index fingers 115, 125 when open, and as index fingers 116, 126 when closed and in contact with the touch panel 103.

[0023] In this state, as finger detection unit 800 (see FIG. 9) that detects the movement (opening and closing movement) of index fingers 113, 123 tapping on touch panel 103, specifically, the movement position of index fingers 113, 123, camera 101 can capture the position of index fingers 113, 123 from a captured image, and by calibrating based on a predetermined position image (an image of index fingers open at a target distance described later) captured in advance, it is possible to accurately measure the distances between index fingers 113, 123 during the tapping movement and the surface of hanging finger tapping measurement processing terminal 100 (surface of touch panel 103) over time. In addition, distance measurement sensor 102, as finger detection unit 800, can detect the distance to index fingers 113, 123 and the angle thereof, respectively, and enables highly accurate measurement of the distances between index fingers 113, 123 during the tapping movement and the surface of hanging finger tapping measurement processing terminal 100 over time.

[0024] Therefore, by taking into consideration the thickness of the finger tapping measurement processing terminal 100, it is possible to measure the two-finger interval distance between the index finger 113 during the tapping motion and the left thumb 112 resting on (holding) the back surface of the finger tapping measurement processing terminal 100. Similarly, in the case of the right hand, by taking into consideration the thickness of the finger tapping measurement processing terminal 100, it is possible to measure the two-finger interval distance between the index finger 123 during the tapping motion and the right thumb 122 resting on the back surface of the finger tapping measurement processing terminal 100. This allows the tapping motion between two fingers to be measured in a simple and easy-to-use manner using a widely used and inexpensive information processing terminal without wearing any special equipment. In addition, based on the measured information, it is possible to analyze and evaluate the feature amount leading to the evaluation of the user's brain function by the calculation function in the finger tapping measurement processing terminal 100, and it is possible to realize a convenient test with little burden on the user for early detection of dementia such as Alzheimer's type, cerebrovascular disease, and Lewy body type, Parkinson's disease, and developmental coordination disorder (such as being unable to skip or jump rope). Furthermore, in addition to detecting signs of dementia, the analyzed and evaluated features can be used to quantify and detect the dexterity of the fingers as a "ruler" that indicates the health of the brain, so it can be used as a training or rehabilitation menu to improve brain function. For example, there is a training and rehabilitation menu that encourages the opening and closing of the fingertips in time with sound and evaluates the dexterity of the fingers.

[0025] In the above example, the tapping motion interval distance between the index finger and the surface of the finger tapping measurement processing terminal 100 is measured over time, but the tapping motion between other two fingers, such as the middle finger and thumb, can also be measured. It goes without saying that the tapping motion can be measured even if multiple fingers are opened and closed, and the same action and effect can be obtained. In the following explanation, the tapping motion is mainly performed with the index finger, but the same applies to other fingers.

[0026] Fig. 2 shows a cross section of a part of the finger tapping measurement processing terminal 100 shown in Fig. 1. In Fig. 2, the same parts as those in Fig. 1 are given the same reference numerals as those in Fig. 1, and the description thereof will be omitted. 2 shows a cross section of the peripheral part of the left hand holding the finger tapping measurement processing terminal 100, but the cross section of the peripheral part of the right hand holding the finger tapping measurement processing terminal 100 is similar. In FIG. 2, the finger tapping measurement processing terminal 100 has a touch panel 103, a display unit 201, and a pressure sensor 202 on the front side forming a display, a pressure sensor 203 on the back side, and further has an acceleration sensor 204 and a fixing jig 205 on the edge on the same side as the camera 101. The display unit 201 is made of liquid crystal or the like, and displays display contents via the highly transparent touch panel 103. The pressure sensors 202 and 203 sense the contact pressure (pressure) when the tapping index finger 113 (123) touches the surface of the touch panel 103, and the acceleration sensor 204 is a sensor that detects acceleration, which is a change in speed per unit time, and can capture movement, vibration, impact, and the like. In addition, it is also equipped with sensors that detect not only acceleration but also speed and direction of rotation. Fixing jig 205 is a jig that fixes thumb 112 (142) that holds and supports finger tapping measurement processing terminal 100, and fixes thumb 112 (142) in a state of contact with the back surface of terminal 100.

[0027] The touch panel 103 detects the position and contact area of ​​the index finger 113 (123) and can grasp the positional deviation of the contact. The pressure sensor 202 detects the contact pressure, which is the strength of the contact when the index finger 113 (123) touches, and can reflect it as one of the feature values ​​calculated from the finger tapping motion. In addition, the pressure sensor 203 detects the contact pressure of the thumb 112 (142) that supports and holds the finger tapping measurement processing terminal 100 and touches the terminal 100, and can confirm whether the thumb 112 (142) is in firm contact with the back surface of the terminal 100 and is in a sufficient holding state. In addition, the fixing jig 205 can firmly support and hold the thumb 112 (142) against the terminal 100 without wobbling in a predetermined position. The acceleration sensor 204 detects the magnitude and direction of vibration when the fingers are closed, and can also measure the strength of the force when the fingers return.

[0028] Next, using Figures 3, 4, 5, and 6, we will explain in more detail how the positions of index fingers 113, 123 performing a tapping motion are captured by camera 101 and the distances between index fingers 113, 123 and the surface of the hanging finger tapping measurement processing terminal 100 are measured. FIG. 3 is a cross-sectional view showing a state in which index fingers 113, 123 performing a tapping motion are photographed by camera 101. FIG. 4 is a diagram explaining how index finger 113 is set to a state in which it is opened by a target distance (target distance between two fingers) from the surface of finger tapping measurement processing terminal 100 using target length jig 401, and how index finger 113 in the state of being opened at the target distance is photographed by camera 101. FIG. 5 is a diagram explaining correction when finger 113 (115) is dented (deformed) by the contact surface between target length jig 401 and index finger 113 (115) generated in the state shown in FIG. 4. FIG. 6 is a diagram explaining how index fingers 113, 123 are set to a state in which it is opened by a target distance from the surface of finger tapping measurement processing terminal 100 using the two screens when terminal 100 has two screens, and how index fingers 113, 123 in the state of being opened at the target distance are photographed by camera 101. 3, 4, 5 and 6, the same parts as those in FIGS. 1 and 2 are denoted by the same reference numerals as those in FIGS. 1 and 2, and the description thereof will be omitted.

[0029] 3, camera 101 arranged on one side (left edge in FIG. 3) of finger tapping measurement processing terminal 100 photographs index finger 113 of left hand 111 performing a tapping motion near camera 101, and from the photographed image of index finger 113 (i.e., detection information or position detection data obtained by finger detection unit 800), finger tapping motion measurement analysis processing unit 903 as a processing unit described later in FIG. 9 analyzes and calculates distance 311 moved by opening and closing index finger 113 (distance from the surface of terminal 100, therefore, inter-finger distance). Camera 101 also photographs index finger 123 of right hand 121 performing a tapping motion far from camera 101, and from the photographed image of index finger 123, finger tapping motion measurement analysis processing unit 903 analyzes and calculates distance 321 moved by opening and closing index finger 123. Here, since the camera 101 and the distance measuring sensor 102 need to measure up to the surface of the display to detect finger contact, the angle of view of the camera 101 and the distance measuring sensor 102 may be changed by changing the lens position so that they can capture images closer to the surface.

[0030] However, it is difficult to accurately grasp the distance 311, 321 between the index fingers 113, 123 moving in a tapping motion only from the positions and sizes of the index fingers 113, 123 on the image captured by the camera 101. For example, even if the left and right index fingers 113, 123 open and close over the same distance, the index finger 113 of the left hand 111 closer to the camera 101 appears larger than the index finger 123 farther from the camera 101 on the image captured by the camera 101, and the distance of the opening and closing motion appears longer. In this state, it is difficult to accurately measure the distance of the opening and closing motion of the left and right index fingers 113, 123. Therefore, in this embodiment, before photographing the index fingers 113, 123 and measuring the distance between the index fingers 113, 123 and the surface of the finger tapping measurement processing terminal 100, the index fingers 113, 123 are set in advance to a state where they are opened a predetermined target distance from the surface of the terminal 100, the index fingers 113, 123 in the set state are photographed by the camera 101, and the finger detection unit 800 including the camera 101 identifies and recognizes the target distance between the index fingers 113, 123 and the surface of the terminal 100 on the photographed image, and the identified target distance on the image (target distance between the two fingers) is captured as a length reference (a reference value that serves as a reference for the open position of the fingers in the finger tapping movement). Then, when photographing the index fingers 113, 123 performing the tapping motion and measuring the distance between the index fingers 113, 123 and the surface of the finger tapping measurement processing terminal 100, the finger detection unit 800 calibrates the photographed image of the index fingers 113, 123 based on the length standard captured in advance, so that the finger tapping motion measurement analysis processing unit 903 can accurately calculate and measure the distance between the index fingers 113, 123 and the surface of the terminal 100. Note that there is an optimal distance between two fingers for evaluating brain function through finger tapping motion, and this is captured in advance as a predetermined target distance in this embodiment.

[0031] 4, a specific example of the calibration is to hold a target length jig 401 (having a length corresponding to a target distance between two fingers) having a length equal to a predetermined target distance between the index finger 113 (115) and the surface of the terminal 100 between the open index finger 115 and the surface of the terminal 100, and to photograph the open index finger 115 with the camera 101. Therefore, when the target distance between the index finger 115 and the surface of the terminal 100 on the photographed image is captured as a length reference, and the index finger 113 performing a tapping motion is photographed, the distance between the photographed index finger 113 and the surface of the terminal 100 is calibrated based on the captured length reference, and the distance between the index finger 113 and the surface of the terminal 100 can be measured with high accuracy over time. Then, by taking into account the thickness of terminal 100 in the measured distance between index finger 113 and the surface of terminal 100, the distance between index finger 113 and thumb 112 can be measured with high accuracy over time. Note that, although the case of index finger 115 in an open position on left hand 111 has been described in Fig. 4, it goes without saying that calibration can be performed in the same manner for index finger 125 in an open position on right hand 121.

[0032] In this way, finger detection unit 800 uses target length jig 401 to determine the reference for the open position of the finger tapping movement, but target length jig 401 is formed from a material with a level of transparency that does not obstruct the shooting operation by the camera (does not hinder detection by finger detection unit 800). As a result, even if target length jig 401 is present, camera 101 can shoot the movement of the index finger through target length jig 401, and it is possible to prevent target length jig 401 set for the left hand from interfering with the shooting of the movement of index finger 123 of the right hand, which is far from camera 101.

[0033] In addition, when the target distance between the index finger 113 and the surface of the terminal 100 is measured using the target length jig 401, the index finger 115, which is in an open state and pressing the target length jig 401, becomes flat (deformed) by the pressing force from the original bulge at the contact surface 501 with the target length jig 401, as shown in FIG. 5, and an error occurs in the distance between the index finger 115 and the surface of the terminal 100 by the dent (deformation amount) 502. Therefore, in this embodiment, in order to achieve a predetermined target distance 503 when the index finger 113 is in an open state, the length of the target length jig 401 is increased by the dent 502. This makes it possible to accurately capture the length standard for calibration, and therefore to measure the distance between the index finger and the terminal surface with even greater accuracy over time. That is, in this embodiment, the finger detection unit 800 detects the amount of deformation of the finger that deforms while the target length jig 401 is clamped between the surface of the display and the tapping finger, and calibrates the position detection data based on the detected deformation amount value.

[0034] As another specific example of calibration, as shown in FIG. 6, when the terminal 100 has two screens, a fixed display screen 601 and an opening / closing display screen 602, the opening / closing display screen 602 is opened, and the index finger 113 of the left hand 111 is opened on the fixed display screen 601 to make it look like the index finger 115 in an open state. Then, the index finger 115 is brought to the target height position while looking at the reference "target height position when the index finger is opened" shown in the display content 603 on the opening / closing display screen 602. Thus, the index finger 115 can be set to the position of the distance 604 of the target height from the surface of the terminal 100. In this state, if the index finger 115 is photographed by the camera 101, the target distance between the index finger 115 and the surface of the terminal 100 can be captured as a length reference on the photographed image. Similarly, in the case of the index finger 123 of the right hand 121, the index finger 125 can be brought to the target height position while looking at the reference "target height position when the index finger is opened" shown in the display content 605 on the display screen 602 that opens and closes, and the index finger 125 can be set to the position of the target height distance 606 from the surface of the terminal 100. In this state, if the index finger 125 is photographed by the camera 101, the target distance between the index finger 125 and the surface of the terminal 100 on the photographed image can be captured as a length reference. Therefore, when the index finger performing a tapping motion is photographed, the distance between the photographed index finger and the surface of the terminal 100 can be calibrated based on the previously captured length reference, and as a result, the distance between the index finger and the surface of the terminal 100 can be measured with high accuracy over time, and further, the distance between the index finger and the thumb can be measured with high accuracy over time by adding the thickness of the terminal 100 to the measured distance between the index finger and the surface of the terminal 100.

[0035] Next, a case where the positions of index fingers 113, 123 performing a tapping motion are captured by distance measuring sensor 102, and the distances between index fingers 113, 123 and the surface of hanging finger tapping measurement processing terminal 100 are measured will be described in more detail with reference to Fig. 7. Fig. 7 is a cross-sectional view showing a state where distance measuring sensor 102 detects the distance and angle of index fingers 113, 123 performing a tapping motion. In Fig. 7, the same parts as those in Figs. 1, 2, 3, 4, 5, and 6 are denoted by the same reference numerals as those in Figs. 1, 2, 3, 4, 5, and 6, and the description thereof will be omitted.

[0036] 7, distance measuring sensor 102 arranged on one side (right edge) of terminal 100 detects distance Llc and angle θl to index finger 113 of left hand 111 that is located near distance measuring sensor 102 and performing a tapping motion. Distance Lla between index finger 113 and the surface of terminal 100 can be calculated from the detected distance Llc and angle θl by Llc×sin θl, and thus distance Lla between index finger 113 performing a tapping motion and the surface of terminal 100 can be measured over time. Similarly, distance measuring sensor 102 detects distance Lrc and angle θr to index finger 123 of right hand 121 that is located away from distance measuring sensor 102 and performing a tapping motion. The distance Lra between the index finger 123 and the surface of the terminal 100 can be calculated from the detected distance Lrc and angle θr by Lrc×sinθr, and therefore the distance Lra between the index finger 123 performing the tapping motion and the surface of the terminal 100 can be measured over time. Furthermore, the measured distance between the index finger and the surface of the terminal 100 is taken into account, and the distance between the index finger and the thumb can be measured over time with high accuracy. In addition, by detecting the position where the index fingers 113, 123 touch the touch panel 103 and combining this with the detection of the distance to the fingers by the distance measuring sensor 102, it becomes possible to measure with high accuracy the time elapsed from when the index fingers 113, 123 leave the touch panel 103 until they return and touch it again, and the trajectory of the movement of the index fingers 113, 123 during that time. As described above, in this embodiment, the finger detection unit 800 can detect the position of the tapping finger based on the distance and angle detected by the distance sensor 102, and the finger tapping motion measurement analysis processing unit 903 can measure the distance between the two fingers based on the position detection information.

[0037] Next, a case where the camera 101 and the distance measuring sensor 102 are provided on each of the opposing sides of the terminal 100 (where finger detection units are provided on both sides of the terminal 100) will be described with reference to FIG. Fig. 8 shows finger tapping measurement processing terminal 100 in which camera 101 and distance measurement sensor 102 are provided on one side of terminal 100, and camera 801 and distance measurement sensor 802 are also provided on the other opposing side. Note that in Fig. 8, the same parts as those in Figs. 1, 2, 3, 4, 5, 6, and 7 are given the same reference numerals as those in Figs. 1, 2, 3, 4, 5, 6, and 7, and their description will be omitted.

[0038] 1, when the camera 101 and the distance measuring sensor 102 are provided only on one side of the terminal 100, the left hand 111 is located near the camera 101 and the distance measuring sensor 102, so it is relatively easy to measure the distance between the index finger 113 of the left hand 111 and the surface of the terminal 100 with high accuracy, but the right hand 121 is located far from the camera 101 and the distance measuring sensor 102, so it may be difficult to measure the distance between the index finger 123 of the right hand 121 and the surface of the terminal 100 with high accuracy. In contrast, as shown in FIG. 8, by newly adding a camera 801 and a distance measuring sensor 802 to the other side near the right hand 121, the camera 801 and the distance measuring sensor 802 near the right hand 121 can measure the distance between the index finger 123 of the right hand 121 and the surface of the terminal 100 with high accuracy. That is, the distance between the index finger and the terminal surface of both the left and right hands can be measured with high accuracy by the nearby cameras and distance measuring sensors, and the measurement is not hindered by the index finger of the other hand, so that the distance between the index finger and the terminal surface can be measured with even higher accuracy. That is, according to Fig. 8, the finger detection unit 800 can detect the tapping finger movements of both hands of the user holding the terminal 100 on both sides, and the finger tapping motion measurement analysis processing unit 903 can perform processing to measure the inter-finger distance for each of the user's hands.

[0039] 1 and 8, the operation of the camera 101 and the distance measuring sensor 102 when the camera 101 and the distance measuring sensor 102 are provided on one side or on two sides, top and bottom (or left and right), of the terminal 100 has been described, but the camera 101 and the distance measuring sensor 102 may simultaneously capture and detect the movement of the finger during the tapping motion. As a result, for example, in a situation where two fingers overlap or one finger is not visible when viewed from the camera 101 or the distance measuring sensor 102, the camera 101 and the distance measuring sensor 102, which are positioned offset from each other, can measure the movement of the fingers while complementing each other, and the moving positions of the left and right fingers can be accurately measured without missing any. Of course, it goes without saying that the camera 101 and the distance measuring sensor 102 can be used together even during normal measurement to improve accuracy.

[0040] Fig. 9 shows a block diagram of the configuration of a finger tapping measurement processing terminal 100 according to this embodiment. In Fig. 9, the terminal 100 has a first and second camera 101, 801, a first and second distance measuring sensor 102, 802, a first and second pressure sensor 202, 203, an acceleration sensor 204, a gyro sensor 901, a geomagnetic sensor 902, a finger tapping motion measurement analysis processing unit 903, a control unit 904, a memory unit 910 consisting of a program 911 and information data 916, a touch panel 103, a display unit 201, a microphone 931, a speaker 932, a vibration generating unit 933, a communication unit 934, and a transmitting / receiving antenna 935, and these components are mutually connected via a bus 950, except for the transmitting / receiving antenna 935. In this case, the first and second cameras 101, 801, the first and second distance measuring sensors 102, 802, the first and second pressure sensors 202, 203, and the touch panel 103 constitute a finger detection unit 800 that detects the movement of a finger tapping on the display.

[0041] The first camera 101 is installed on one side of the terminal 100 along the vertical direction (or horizontal direction), and the second camera 801 is installed on the other side of the terminal 100 along the vertical direction (or horizontal direction), and captures the forward visual field state focusing mainly on the user's fingers. In the cases of Fig. 1 and Fig. 3, the first camera 101 captures the index fingers 113 and 123 of the left and right hands performing a tapping motion. In the case of Fig. 8, the first camera 101 captures the index finger 113 of the left hand performing a tapping motion, and the second camera 801 captures the index finger 123 of the left hand performing a tapping motion, and the captured images are captured in the terminal 100.

[0042] The first and second distance measuring sensors 102, 802 are sensors that can capture the shape of a target such as a person or an object in a three-dimensional form. Examples of such sensors include LiDAR (Light Detection and Ranging), which irradiates an object with a laser light such as infrared light and measures the scattered light that bounces back to analyze and detect the distance to a distant target and the state of the target, TOF (Time Of Flight) sensors that measure distance by measuring the reflection time of pulsed light irradiated on a subject for each pixel, and millimeter wave radar that emits millimeter wave radio waves and captures the reflected waves to detect the distance to the target and the state of the target. In the case of Fig. 1 and Fig. 7, the first distance measuring sensor 102 captures the index fingers 113, 123 of the left and right hands that are performing a tapping motion, and detects the distance to each index finger 113, 123 and the angle thereof. In the case of Figure 8, the first distance measuring sensor 102 detects the distance and angle of the index finger 113 of the left hand performing a tapping motion, and the second distance measuring sensor 802 detects the distance and angle of the index finger 123 of the right hand performing a tapping motion, and these can be captured in the terminal 100.

[0043] The first and second pressure-sensitive sensors 202, 203 are sensors that detect pressure (contact pressure) generated by contact. The first pressure-sensitive sensor 202 is provided on the front side of the terminal 100 below the touch panel 103 and the display unit 201, and detects the contact pressure when the opening and closing fingers contact the front side of the terminal 100, and can determine whether the opening and closing fingers have closed up to the terminal front side and can measure the contact strength when the opening and closing fingers close and abut against the terminal front side. On the other hand, the second pressure-sensitive sensor 202 is provided on the back side of the terminal 100, and detects the contact pressure from the thumb abutting against the back side of the terminal, and can determine whether the thumb is firmly holding the terminal 100 and can measure the strength of the thumb's holding. In addition, the pressures of the first pressure-sensitive sensor 202 and the second pressure-sensitive sensor 203 can be combined to obtain the strength of pressure when the fingers are closed.

[0044] The acceleration sensor 204 is a sensor that detects acceleration, which is a change in speed per unit time, and can capture movement, vibration, impact, and the like. The acceleration sensor 204 provided in the terminal 100 detects the magnitude and direction of vibration when the fingers are closed, and can measure the strength of the force when the fingers return and the strength of the force when the fingers touch. In addition, the gyro sensor 901 is a sensor that detects the angular velocity in the rotation direction, and can capture the vertical, horizontal, and diagonal posture states. Therefore, the movement of the terminal 100 can be detected using the acceleration sensor 204 and the gyro sensor 901 mounted on the terminal 100. The geomagnetic sensor 902 is a sensor that detects the magnetic force of the earth, and detects the direction in which the terminal 100 is facing. As the geomagnetic sensor, a three-axis type that detects geomagnetism in the up-down direction in addition to the forward-backward and left-right directions is used, and it is also possible to detect the movement of the terminal 100 by capturing the geomagnetic change in response to the movement of the terminal 100. With these, it is possible to detect and determine whether the display screen of the terminal 100 is in a position where it can be seen.

[0045] The finger tapping motion measurement and analysis processing unit 903 analyzes the position of the finger performing the tapping motion from the finger motion information captured by the first and second cameras 101, 801 and the finger motion information detected by the first and second distance measuring sensors 102, 802, and performs processing to measure the distance between the two fingers of the finger performing the tapping motion (e.g., the index finger) and the stationary finger (e.g., the thumb). The finger tapping motion measurement and analysis processing unit 903 also performs processing to calculate and analyze feature quantities that indicate the characteristics of the finger motion in each of the items of distance, speed, acceleration, jerk, tap interval, and phase difference based on the measured information of the distance between the two fingers that is displaced over time by the tapping motion. The calculated and analyzed feature quantities lead to an evaluation of the user's brain function, and can tell the state of the brain, enabling tests for early detection of dementia such as Alzheimer's, cerebrovascular, and Lewy body dementia, Parkinson's disease, and developmental coordination disorder (such as being unable to skip or jump rope). In addition, the calculated and analyzed feature values ​​can be used as a "ruler" to indicate the brain's health, making it possible to quantify and detect and recognize the dexterity of the fingers. In the feature distance item, the maximum amplitude of finger swing and total movement distance are calculated to evaluate how the magnitude of finger movement has changed. In the feature speed item, the maximum speed, opening speed, closing speed, etc. are calculated to evaluate how fast the fingers moved. In the feature acceleration item, the maximum amplitude of acceleration, the momentum at which the fingers start opening, the momentum at which the fingers finish opening, the momentum at which the fingers finish closing, the momentum at which the fingers start closing, and the finger contact time are calculated to evaluate the momentum of the finger movement. In the feature tap interval item, the number of taps, tap period, tap frequency, etc. are calculated to evaluate the timing of tapping and its variance. In the feature phase difference item, the misalignment between the hands and the similarity between the hands are calculated to evaluate the coordination between the hands.

[0046] The control unit 904 is composed of a CPU etc., and executes programs 911 such as an operating system (OS) 912 and various operation control applications 913, 914 stored in the memory unit 910, thereby performing operation control processing for the entire terminal 100 and controlling the startup operations of various applications.

[0047] The memory unit 910 is composed of a flash memory or the like, and stores programs 911 such as an operating system 912 and applications 913 and 914 for controlling various processes such as images, sounds, documents, display, and measurement. The memory unit 910 also stores information data 916 such as base data 917 required for basic operations by the operating system 912 and file data 918 and 919 used by the various applications 913 and 914. For example, an image processing application is started, an image is taken by a camera, and the captured file data is saved and stored. The processing in the finger tapping motion measurement and analysis processing unit 903 may be stored as one application A, and the measurement of the inter-finger distance and the calculation and analysis of the feature amount associated with the finger tapping motion may be processed by starting the application A. An external server device with high computing performance and large capacity may receive the inter-finger distance measured from the information processing terminal and calculate and analyze the feature amount.

[0048] The display unit 201 is configured with a liquid crystal panel or the like, and displays images and videos, as well as remaining battery capacity, various alarms, time and other notification information for the user, and icons of applications to be started on the display screen, etc. In particular, in this embodiment, the results of processing performed by the finger tapping motion measurement analysis processing unit 903 can be displayed.

[0049] The touch panel 103 is made of a highly transparent material, detects the position and area on the screen touched by a finger, and is provided on the surface of the information processing terminal 100. It can determine that an index finger, which is open and closed, has closed and touched the surface of the terminal 100. An input operation can be performed by touching the touch panel 103 while looking at icons displayed on the display unit 201. For example, the touch panel 103 is a touch pad type input means such as a capacitance type, and can detect an approach or contact operation (touch operation) by a finger or a touch pen as an operation input, and can set and input information that the user wants to input, or can easily select and specify icons of applications and files on the display screen by touch operation. In this embodiment, the finger detection unit 800 can detect the contact of the tapping finger as a closed position of the finger opening and closing of the finger tapping motion by the touch panel 103.

[0050] The microphone 931 collects sounds from the outside and the user's own voice, and the speaker 932 outputs sounds to the outside and informs the user of sounds such as notification information, music, etc. Also, instructions related to finger tap measurement may be given to the user by voice through the speaker.

[0051] The vibration generating unit 933 generates vibrations under the control of the control unit 904, and converts notification information for the user transmitted by the information processing terminal 100 into vibrations. The vibration generating unit 933 applies the generated vibrations to the fingers or the like holding the information processing terminal 100, thereby enabling the notification to be reliably conveyed to the user.

[0052] The communication unit 934 is a communication interface that performs wireless communication with a server device or the like located at another location by short-distance wireless communication, wireless LAN, or base station communication. During wireless communication, the communication unit 934 transmits and receives measurement data and analyzed and calculated features to and from the server device or the like via a transmission / reception antenna 935. The short-distance wireless communication is performed using, for example, an electronic tag, but is not limited to this. If the short-distance wireless communication is at least capable of wireless communication when the device is near another information terminal, the communication unit 934 may use a wireless LAN such as Bluetooth (registered trademark), IrDA (Infrared Data Association, registered trademark), Zigbee (registered trademark), HomeRF (Home Radio Frequency, registered trademark), or Wi-Fi (registered trademark). For base station communication, long-distance wireless communication such as W-CDMA (Wideband Code Division Multiple Access) or GSM (registered trademark) may be used. It is also possible to detect the positional relationship and orientation between terminals using an ultra-wideband wireless system (Ultra Wide Band: UWB). Although not shown, the communication unit 934 may use other methods such as communication by optical communication and sound waves as a means of wireless communication. In that case, a light emitting / receiving unit and a sound wave output / sound wave input unit are used in place of the transmitting / receiving antenna 935, respectively.

[0053] With such components shown in FIG. 9, it is possible to measure the two-finger distance between a finger (e.g., index finger) during a tapping motion and a finger (e.g., thumb) on the back side of the terminal 100, and further, it is possible to calculate and analyze a feature quantity representing the feature of the finger movement from the measured two-finger distance information. Then, it is possible to measure the two-finger tapping motion in a simple and easy-to-use manner using a widely used and inexpensive information processing terminal without wearing a special instrument. In addition, based on the measured information, it is possible to analyze and evaluate a feature quantity leading to the evaluation of the user's brain function (for example, by the finger tapping motion measurement and analysis processing unit 903 or by a device other than the terminal 100), and to realize a convenient test with little burden on the user for early detection of dementia such as Alzheimer's type, cerebrovascular type, and Lewy body type, Parkinson's disease, developmental coordination disorder (such as being unable to skip or jump rope), etc. Furthermore, in addition to being used to detect early signs of dementia and the like, the analyzed and evaluated features can be used to quantify and detect and recognize the fine motor function of the fingers as a "ruler" that indicates the health of the brain, making it possible to use it as a training or rehabilitation menu for improving brain function.

[0054] Next, the operation of finger tapping measurement processing terminal 100 according to this embodiment will be further described with reference to FIG. FIG. 10 is an example of a flowchart for explaining the operation of the finger tapping measurement processing terminal 100 according to this embodiment. Here, the configuration of FIG. 8 is assumed, and therefore, it is determined whether to capture the movement of the finger performing the tapping movement with the camera 101, 801 and measure the distance between the two fingers, or to detect the position of the finger with the distance measuring sensor 102, 802 and measure the distance between the two fingers (S1001). When the movement of the finger performing the tapping movement is captured with the camera 101, 801, the finger is set to a state in which the finger is opened to a predetermined target distance on the terminal 100 before capturing the image of the finger performing the tapping movement (S1002), and the finger is captured with the camera 101, 108 in this state, and the distance reference of the captured image during the finger tapping movement is obtained from the captured image. In addition, the user recognizes the set state of the finger opened to the predetermined target distance. Therefore, it is possible for the user to be conscious of opening the finger at the target distance when performing the tapping movement (S1003). After the preliminary photographing by the camera, the user starts a tapping motion for measurement (or is prompted to do so by a computer program) (S1004). On the other hand, when the finger position is detected by the distance measuring sensor 102, 802, it is not necessary to set the fingers to a predetermined target distance beforehand for measuring the inter-finger distance, and the user starts a tapping motion for measurement (S1004). Even in the case of detection by the distance measuring sensor 102, 802, by setting the fingers to a predetermined target distance, it is possible to make the user aware of the need to open the fingers at the target distance when performing a tapping motion. Of course, even when a distance measuring sensor is used, calibration may be performed in steps such as S1002 and 1003.

[0055] The finger tapping movement includes a two-handed simultaneous mode in which both hands perform finger tapping simultaneously, and a two-handed alternating mode in which both hands perform finger tapping alternately, and the user performs finger tapping in each mode (tapping step S1005). At this time, the first and second cameras 101, 801 and the first and second distance measuring sensors 102, 802 detect the movement of the tapping finger, and the finger tapping movement measurement analysis processing unit 903 analyzes the position of the finger performing the tapping movement from the finger movement information photographed by the first and second cameras 101, 801 and the finger movement information detected by the first and second distance measuring sensors 102, 802, and measures the two-finger interval distance between the finger performing the tapping movement and the fixed finger (finger detection step and processing step S1006). In this finger detection step and processing step S1006, the processing in steps S1002 and S1003 described above is reflected, and the operation / processing of the finger detection unit 800 and the finger tapping motion measurement / analysis processing unit 903 described in relation to Figs. 1 to 9 is performed. After that, if remeasurement is not necessary (S1007), the finger tapping motion measurement / analysis processing unit 903 calculates and analyzes feature quantities that indicate the characteristics of finger movement in each item, such as distance, speed, acceleration, tap interval, and phase difference, based on the measured inter-finger distance information that is displaced over time by the tapping motion (S1008). Furthermore, the calculated and analyzed feature quantities are displayed in an easy-to-understand table or graph on the display unit 201, or are outputted as voice from the speaker 932, to notify the user (S1009). If remeasurement and analysis are not to be performed, the process ends (S1010). If remeasurement is necessary, the process returns to step S1004, and if remeasurement and analysis are to be performed, the process returns to step S1001. Through the above operations, the distance between the two fingers during finger tapping movements can be accurately measured, and based on the measurement results, feature amounts that can be used to evaluate the user's brain function can be calculated and analyzed.

[0056] FIG. 11(a) shows the results of measurement in a two-handed simultaneous mode in which both hands perform finger tapping motion simultaneously, and FIG. 11(b) shows the results of measurement in a two-handed alternating mode in which both hands perform finger tapping motion alternately. In both figures, the vertical axis on the left side shows the distance between a finger (e.g., index finger) during the tapping motion and the terminal surface, and the vertical axis on the right side shows the inter-finger distance between a finger (e.g., index finger) during the tapping motion and a finger (e.g., thumb) fixed to the back of the terminal, and the movement of the left hand is shown on the top and the right hand index finger on the bottom. In the two-handed simultaneous mode, as shown in FIG. 11(a), the index finger of the left hand and the index finger of the right hand move in the same direction over time, but in the two-handed alternating mode, as shown in FIG. 11(b), the index finger of the left hand and the index finger of the right hand move in opposite directions over time, for example, at time 1101, the left index finger is open and the right index finger is closed. When the camera 101 (801) captures the movement of the index finger to measure the inter-finger distance, the measurement is performed in frame rate units (e.g., 60 fps) of the camera 101 (801), so the waveform shown in Fig. 11 is a sampling waveform that connects the values ​​of each observation point. Here, if the camera's angle of view does not include the surface of the display and the timing of touching the display cannot be measured accurately, the waveform can be obtained based on the timing of touching the touch panel.

[0057] Next, a finger tapping measurement processing system including the finger tapping measurement processing terminal 100 and the head-mounted information processing device 1201 will be described with reference to FIG. FIG. 12 shows an embodiment of a finger tapping measurement processing system that includes a finger tapping measurement processing terminal 100, a head-mounted information processing device 1201, and a server device 1203.

[0058] 12, a head-mounted information processing device 1201, which is worn on the head of a user who uses the finger tapping measurement processing terminal 100 and has a display processing unit (display unit) 1235 (see FIG. 13 described later) that displays real space information and virtual space information visibly to the user, and the finger tapping measurement processing terminal 100 transmit and receive various information via communication means such as wireless communication and an external network. Thus, the head-mounted information processing device 1201 receives, as various transmitted and received information, information displayed on a display screen (display) 1202 of the finger tapping measurement processing terminal 100 and finger movement information captured by a camera 101 (801) during a tapping motion from the finger tapping measurement processing terminal 100, and can display the received information on the display screen 1202, finger movement information, etc., as virtual space information on the display processing unit 1235. As a result, when measuring finger tapping motion, as shown in FIG. 12, due to the state in which the terminal 100 is held by the thumbs and index fingers of both hands, the display screen 1202 of the terminal 100 and the fingers performing the tapping motion are obstructed by the terminal 100 from the user's perspective, making it difficult to see them, but the display processing unit 1235 of the head-mounted information processing device 1201 makes it possible to clearly see them. In addition, as a transmission and reception of various information, instruction information such as operation instructions related to finger tapping measurement and start and end of operation can be transmitted and received between the two devices, and various instructions can be issued from the head-mounted information processing device 1201 to the terminal 100, and the operation instructions from the terminal 100 can be confirmed by display or voice on the head-mounted information processing device 1201. Note that when the finger tapping measurement processing terminal 100 has display screens on both sides, that is, when there is a display screen on the back side as well, the user can easily see the display screen on the back side, and operation guides and the like may be displayed on the display screen on the back side without using the head-mounted information processing device 1201. Furthermore, if the head-mounted information processing device 1201 is equipped with a distance sensor and a camera and can detect the positions of the left and right index fingers and thumbs, respectively, it is possible to measure the finger-tapping motion using only the head-mounted information processing device 1201.

[0059] Data may be transmitted and received via wireless communication between an externally installed server device 1203 and the finger tapping measurement processing terminal 100, and the data may be stored and managed by the large-capacity server device 1203. Alternatively, the server device 1203 with high computing performance and large capacity may receive the inter-finger distance, which is measurement data, from the finger tapping measurement processing terminal 100 and calculate and analyze features leading to the evaluation of the user's brain function. This allows various features to be calculated and analyzed at high speed and with high accuracy, and also makes it possible to store and manage a large amount of calculated and analyzed feature information.

[0060] Fig. 13 is a block diagram showing an example of the configuration of a head-mounted information processing device 1201. In Fig. 13, the head-mounted information processing device 1201 has a camera 1211, a right eye gaze detection unit 1212, a left eye gaze detection unit 1213, an acceleration sensor 1214, a gyro sensor 1215, a geomagnetic sensor 1216, a distance measurement sensor 1217, a control unit 1218, a memory unit 1220 consisting of a program 1225 and information data 1226, an operation input unit 1234, a display processing unit 1235, an external sound microphone 1236, a speech sound microphone 1237, headphones 1238, a vibration generation unit 1239, a communication unit 1240, and a transmission / reception antenna 1241, and these components are mutually connected via a bus 1250, except for the transmission / reception antenna 1241.

[0061] The right eye gaze detection unit 1212 and the left eye gaze detection unit 1213 detect the gazes of the right eye and the left eye, respectively. The process of detecting the gazes may utilize well-known technology that is generally used as eye tracking processing, and for example, in a method using corneal reflex, a technology is known in which an infrared LED (Light Emitting Diode) is irradiated onto the face, an image is taken with an infrared camera, and the position on the cornea of ​​the reflected light produced by the infrared LED irradiation (corneal reflex) is used as a reference point, and the gaze is detected based on the position of the pupil relative to the position of the corneal reflex.

[0062] The operation input unit 1234 is an input means such as a keyboard, key buttons, touch keys, etc., and is used for setting and inputting information that the user wants to input. The operation input unit 1234 may be provided in a position or form that allows the user to easily perform input operations in the head-mounted information processing device 1201, or may be separated from the main body of the head-mounted information processing device 1201 and connected by wire or wirelessly. In addition, an input operation screen may be displayed on the display screen of the display processing unit 1235, and input operation information may be captured based on the position on the input operation screen to which the gaze detected by the right eye gaze detection unit 1212 and the left eye gaze detection unit 1213 is directed, or a pointer may be displayed on the input operation screen and the pointer may be operated by the operation input unit 1234 to capture the input operation information. In addition, the user may utter a voice indicating an input operation, and the voice may be collected by the voice microphone 1237 to capture the input operation information.

[0063] In the case of an optically transparent head-mounted information processing device, the display processing unit 1235 is composed of, for example, a projection unit that projects various information such as playback information by a running application and notification information to a user, and a transparent half mirror that forms an image of the various projected information in front of the user's eyes. In the case of a video-transmitting head-mounted information processing device, the display processing unit 1235 is composed of a display such as a liquid crystal panel that displays various information together with a real space object in front of the user's eyes photographed by the camera 1211. This allows the user to view and listen to not only the image in the field of view in front of the user's eyes, but also other image information.

[0064] The peripheral sound microphone 1236 and the speech sound microphone 1237 collect sounds from the outside and the user's own speech, respectively. The headphones 1238 are worn on the user's ears to pick up sounds to be made to the user, and can inform the user of notification information by voice.

[0065] 13, by wirelessly transmitting and receiving information from the finger tapping measurement processing terminal 100 to the head-mounted information processing device 1201, the information on the display screen 1202 of the finger tapping measurement processing terminal 100 can be displayed as virtual space information by the display processing unit 1235 in the head-mounted information processing device 1201, and information such as an operation explanation for finger tapping measurement can be clearly visually recognized. Also, instruction information such as the start and end of an operation related to finger tapping measurement can be transmitted and received between the two, and various instructions can also be issued from the head-mounted information processing device 1201.

[0066] The present invention is not limited to the above-described embodiment, and may include various modified examples. For example, the above-described embodiment has been described in detail to clearly explain the present invention, and is not necessarily limited to those including all of the configurations described. In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

[0067] In addition, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. In addition, the above-mentioned configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as the program, table, file, etc. that realizes each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0068] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other.

[0069] In addition, in the above-mentioned embodiment, a form in which the terminal is held with two fingers of both hands has been described, but if the terminal is used by placing it on the user's lap, measurements can be easily performed using both hands without holding the terminal with the thumbs. [Explanation of symbols]

[0070] 100 Finger tapping measurement processing terminal 101,801,1211 Camera 102,802,1217 Distance sensor 103 Touch Panel 103A Display 111 left hand 112,122 Thumb 113,123 Index finger 201: Display unit 202,203 Pressure sensor 205 Fixture 401 Target length jig 800 Finger detection unit 903 Finger tapping motion measurement and analysis processing unit (processing unit) 934,1240 Communications Department 1201 Head-mounted information processing device 1203 Server device

Claims

1. A finger tapping measurement processing terminal that measures finger tapping motion and processes the measurement results, a touch panel provided on one surface of the terminal, capable of displaying information related to finger tapping motion, and tapped by one of two fingers of the same hand of a user holding the terminal; a finger detection unit that detects a movement of the one finger tapping on the touch panel; a processing unit that performs a process of measuring a distance between the other of the two fingers holding the other surface of the terminal located on the opposite side to the touch panel and the one finger tapping the touch panel, based on detection information obtained by the finger detection unit, by measuring a distance between the one finger and the one surface of the terminal that is hanging down and taking into account a thickness of the terminal; A finger tapping measurement processing terminal comprising:

2. 2. The finger tapping measurement processing terminal according to claim 1, wherein the finger detection unit detects the movement of the tapping finger of each of the user's hands holding the terminal on both sides, and the processing unit performs processing to measure the inter-finger distance for each of the user's hands.

3. 3. The finger tapping measurement and processing terminal according to claim 2, wherein the finger detection units are provided on both sides of the terminal.

4. 2. The finger tapping measurement processing terminal according to claim 1, wherein the finger detection unit has a camera that captures an image of the one finger that is tapping, detects a position of the one finger from an image captured by the camera, and the processing unit measures the distance between the two fingers based on the position detection data.

5. 5. The finger tapping measurement processing terminal according to claim 4, wherein the finger detection unit obtains a target distance between two fingers, which is a reference for an open position of the fingers in the finger tapping movement, as a reference value from the image captured by the camera, and calibrates the position detection data based on this reference value.

6. 6. The finger tapping measurement and processing terminal according to claim 5, wherein the finger detection unit defines a reference for an open position of fingers in the finger tapping movement by using a target length jig having a length corresponding to the target distance between two fingers.

7. 7. The finger tapping measurement and processing terminal according to claim 6, wherein the target length jig has transparency that does not impede detection by the finger detection unit.

8. 7. The finger tapping measurement processing terminal according to claim 6, wherein the finger detection unit detects an amount of deformation of the one finger that deforms while the target length jig is held between the surface of the touch panel and the one finger that is tapping, and calibrates the position detection data based on the deformation amount detection value.

9. The finger tapping measurement processing terminal according to claim 1, characterized in that the finger detection unit has a distance measurement sensor that detects the distance to the one finger being tapped and the angle thereof, detects the position of the one finger based on the distance and the angle detected by the distance measurement sensor, and the processing unit measures the distance between the two fingers based on the position detection information.

10. the touch panel is a touch panel display capable of displaying information on its surface, 2. The finger tapping measurement and processing terminal according to claim 1, wherein the finger detection unit detects a contact of the one finger that is tapping with the touch panel as a closed position of the fingers in the finger tapping motion.

11. 2. The finger tapping measurement and processing terminal according to claim 1, further comprising a fixing portion for fixing the other finger in a state of contact with the other surface of the terminal.

12. 2. The finger tapping measurement and processing terminal according to claim 1, wherein the touch panel has a pressure sensor that detects a contact pressure when the one finger tapping touches a surface of the touch panel.

13. 2. The finger tapping measurement processing terminal according to claim 1, wherein the processing unit performs a process of calculating and analyzing a feature amount leading to an evaluation of a user's brain function based on the measured inter-finger distance.

14. 2. The finger tapping measurement and processing terminal according to claim 1, wherein the touch panel displays a processing result processed by the processing unit.

15. A finger tapping measurement processing system comprising: a finger tapping measurement processing terminal that measures a finger tapping motion and processes the measurement results; and a head-mounted information processing device that is worn on the head of a user who uses the finger tapping measurement processing terminal and has a display unit that displays real space information and virtual space information visibly to the user, wherein the finger tapping measurement processing terminal and the head-mounted information processing device can communicate with each other via a communication means, The finger tapping measurement processing terminal has a touch panel that is provided on one surface of the terminal and is capable of displaying information related to a finger tapping motion, and is tapped with one of two fingers of the same hand of a user holding the terminal, a finger detection unit that detects the motion of the one finger tapping the touch panel, and a processing unit that performs a process of measuring a distance between the other of the two fingers holding the other surface of the terminal located on the opposite side to the touch panel and the one finger tapping the touch panel during a finger tapping motion, by measuring the distance between the one finger and the one surface of the hanging terminal and taking into account a thickness of the terminal, based on detection information obtained by the finger detection unit; the head-mounted information processing device receives information on the finger tapping measurement process via the communication means, and displays the received information on the display unit. A finger tapping measurement processing system.

16. 16. The finger tapping measurement processing system according to claim 15, wherein the head-mounted information processing device receives operation instructions regarding measurement of the finger tapping movement from the finger tapping measurement processing terminal via the communication means, and transmits instruction information including start and end of the operation to the finger tapping measurement processing terminal.

17. 16. The finger tapping measurement processing system according to claim 15, wherein the processing unit of the finger tapping measurement processing terminal performs a process of calculating and analyzing a feature amount leading to an evaluation of a user's brain function based on the measured inter-finger distance.

18. a server device capable of receiving the measurement result of the inter-finger distance from the processing unit of the finger tapping measurement processing terminal via the communication means; The finger tapping measurement processing system according to claim 15, characterized in that the server device has a calculation processing unit that performs processing to calculate and analyze features leading to an evaluation of the user's brain function based on the received measurement result of the inter-finger distance, and a memory unit that stores the processing result of the calculation processing unit, and data stored in the memory unit can be communicated to the finger tapping measurement processing terminal and the head-mounted information processing device via the communication means.

19. A finger tapping measurement and processing method for measuring finger tapping movements and processing the measurement results, comprising: a tapping step of tapping a touch panel provided on one surface of the terminal and capable of displaying information related to finger tapping motion, with one of two fingers of the same hand of a user holding the terminal; a finger detection step of detecting a movement of the one finger tapping on the touch panel; a processing step of performing a process of measuring a distance between the other of the two fingers holding the other surface of the terminal located on the opposite side to the touch panel and the one finger tapping the touch panel, based on the detection information obtained in the finger detection step, by measuring a distance between the one finger and the one surface of the terminal hanging down and taking into account a thickness of the terminal; 13. A finger tapping measurement processing method comprising:

20. 20. The finger tapping measurement processing method according to claim 19, wherein the finger detection step detects the movement of one of the fingers tapping on each of the user's hands holding the terminal on both sides, and the processing step performs processing to measure the inter-finger distance for each of the user's hands.

21. 20. The finger tapping measurement processing method according to claim 19, wherein the finger detection step detects a position of the one finger from an image of the one finger tapping, and the processing step measures the distance between the two fingers based on the position detection data.

22. 22. The finger tapping measurement processing method according to claim 21, wherein the finger detection step includes obtaining a reference value of a target distance between two fingers, which is a reference for an open position of the finger tapping movement, from the image captured by the camera, and calibrating the position detection data based on this reference value.

23. 20. The finger tapping measurement processing method according to claim 19, wherein the finger detection step detects the position of the one finger based on detection data from a distance measuring sensor that detects the distance to the one finger tapping and an angle thereof, and the processing step measures the distance between the two fingers based on the position detection information.

24. the touch panel is a touch panel display that displays information on its surface, 20. The finger tapping measurement processing method according to claim 19, wherein the finger detection step detects a contact of the one finger performing tapping by the touch panel, and detects the contact as a closed position of the finger opening and closing of the finger tapping motion.

25. 20. The finger tapping measurement processing method according to claim 19, wherein the processing step includes a process of calculating and analyzing a feature amount that leads to an evaluation of a user's brain function based on the measured inter-finger distance.

26. 20. The finger tapping measurement and processing method according to claim 19, wherein the touch panel displays a processing result obtained by the processing step.

27. A computer program that enables measurement of finger tapping motion using a terminal having a touch panel on one side and enables the measurement results to be displayed on a display, a tapping step in which a user of the terminal taps the touch panel with one of two fingers of the same hand holding the terminal; a finger detection step of detecting a movement of the one finger tapping on the touch panel; a processing step of performing a process of measuring a distance between the other of the two fingers holding the other surface of the terminal located on the opposite side to the touch panel and the one finger tapping the touch panel, based on the detection information obtained in the finger detection step, by measuring a distance between the one finger and the one surface of the terminal hanging down and taking into account a thickness of the terminal; A computer program characterized by causing a computer to execute the above.

28. 28. The computer program according to claim 27, wherein the finger detection step detects the movement of the tapping finger of each of the user's hands holding the terminal on both sides, and the processing step performs processing to measure the inter-finger distance for each of the user's hands.

29. 28. The computer program according to claim 27, wherein the finger detection step detects a position of the one finger from an image of the one finger tapping, and the processing step measures the distance between the two fingers based on the position detection data.

30. 30. The computer program according to claim 29, wherein the finger detection step includes obtaining a reference value for a target distance between two fingers, which is a reference for an open position of the fingers in the finger tapping movement, from the image captured by the camera, and calibrating the position detection data based on this reference value.

31. The computer program according to claim 27, characterized in that the finger detection step detects the position of the one finger based on detection data from a distance measuring sensor that detects the distance to the one finger tapping and the angle thereof, and the processing step measures the distance between the two fingers based on the position detection information.

32. the touch panel is a touch panel display that displays information on its surface, 28. The computer program according to claim 27, wherein the finger detection step detects contact of the one finger that is tapping with the touch panel as a closed position of the finger opening and closing of the finger tapping motion.

33. 28. The computer program according to claim 27, wherein the processing step includes a process of calculating and analyzing a feature amount that leads to an evaluation of a user's brain function based on the measured inter-finger distance.

34. 28. The computer program according to claim 27, wherein a processing result obtained by said processing step is displayed on said display.

Images