Finger tap measurement processing terminal, system, method and computer program

Abstract

The present application provides a finger tapping measurement processing terminal, system, method and computer program, without wearing special dedicated instruments, and without using expensive special devices for evaluating and analyzing measurement information, using widely popular terminals such as smart phones, tablet computers, etc. in a simple way, cheap and good usability for finger tapping movement measurement and processing. The finger tapping measurement processing terminal (100) measures finger tapping movement and processes the measurement results, comprising: a touch panel (103) arranged on one side of the terminal, capable of displaying information related to finger tapping movement and being tapped by one finger of two fingers of the same hand of the user holding the terminal; a finger detection unit (101, 102) detecting the movement of the finger tapping the touch panel (103); and a processing unit performing the following processing: based on the detection information obtained by the finger detection unit, measuring the distance between the two fingers when the finger tapping movement between the other finger of the two fingers and the finger tapping the touch panel, wherein the other finger holds the terminal on the other side opposite to the touch panel.

Description

TECHNICAL FIELD

[0001] The present application relates to a finger tapping measurement processing terminal, a finger tapping measurement processing system, a method, and a computer program that measure a finger tapping motion and process the measurement results thereof. BACKGROUND

[0002] As society is becoming older, the number of patients with Alzheimer-type cognitive impairment is increasing year by year, and if it can be detected early, it can be delayed by medication. However, it can be difficult to distinguish between symptoms caused by forgetfulness and the like due to aging and disease, and in many cases, the first diagnosis is made at a hospital after the symptoms have worsened.

[0003] Under such circumstances, as a screening for early detection of Alzheimer-type cognitive impairment, blood tests, olfactory tests, and tests that reproduce physician interviews on a tablet terminal have been performed in the past, but there are problems such as pain when blood is taken and a long examination time, which are burdensome for the examinee. On the other hand, as an examination that is less burdensome for the examinee, cognitive function evaluation can be performed by tapping a button and measuring finger motion of a single hand using a tablet terminal, but there is a disadvantage that sufficient high examination accuracy cannot be obtained. If simple screening can be performed with high accuracy and less burden on the examinee, early detection of Alzheimer-type cognitive impairment can be achieved, which contributes to improving the quality of life of patients, reducing medical and nursing costs.

[0004] In response to this, in recent years, it has been found that a motion pattern specific to Alzheimer-type cognitive impairment can be extracted from a two-finger opening and closing motion (finger tapping motion) performed by the thumbs and index fingers of both hands, and it has been confirmed that the motion measurement of the fingers has a high correlation with cognitive impairment examination by general interviews. It is believed that through finger tapping motion measurement, the decrease in the rhythmic motion function of both fingers caused by brain atrophy in Alzheimer-type cognitive impairment can be captured. In addition, the fingers are called the second brain, and most areas of the brain are related to the function of the fingers, and it is believed that the motion of the fingers is related not only to Alzheimer-type cognitive impairment but also to cognitive impairment such as cerebrovascular and Lewy body types, Parkinson's disease, developmental coordination disorder (inability to walk and jump rope, etc.), and the like. That is, the state of the brain can be known from the tapping motion of the fingers. Furthermore, by effectively using the tapping motion of the fingers as a "ruler" that indicates the state of the brain, the fine motor function of the fingers can be quantified, so it can also be used in various fields such as the health care field, the rehabilitation field, the life assistance field, and the like.

[0005] As a method of measuring and evaluating finger tapping motion with high precision, for example, Patent Literature 1 discloses a motion function evaluation system including a motion function measuring device that calculates motion data based on the relative distance between a pair of signal transmitting coils and a signal receiving coil installed on a movable portion of a living body, and an evaluation device that evaluates the motion function of the living body based on the motion data received from the motion function measuring device. That is, the technology given in this Patent Literature 1 is to convert the change in magnetic force that occurs due to the tapping motion of two fingers into an electric signal using a magnetic sensor installed on the fingertip, measure and quantify the motion to capture a feature quantity that represents the characteristics of the motion of the fingers, and thereby understand the state of brain function.

[0006] Prior Art Documents

[0007] Patent Literature

[0008] Patent Literature 1: Japanese Patent Application Publication No. 2016-49123 SUMMARY

[0009] PROBLEMS TO BE SOLVED BY THE INVENTION

[0010] The motion function evaluation system disclosed in the above Patent Literature 1 can measure and evaluate finger tapping motion using a magnetic sensor, but it is necessary to wear a magnetic sensor on the finger to measure finger tapping motion, and it is necessary to use an expensive dedicated device that evaluates and analyzes the measurement information detected by the magnetic sensor, and there is a problem that it is difficult to measure and evaluate finger tapping motion inexpensively.

[0011] The present application has been made in view of the above circumstances, and aims to provide a finger tapping measurement processing terminal, system, method, and computer program in which it is not necessary to wear a special dedicated tool and it is not necessary to use an expensive dedicated device that evaluates and analyzes measurement information, and in which it is possible to inexpensively and easily measure and process finger tapping motion in a simple manner using a terminal such as a smartphone or a tablet that is widely popular.

[0012] MEANS OF SOLVING THE PROBLEM

[0013] To solve the above problem, the present application provides a finger tapping measurement processing terminal that measures finger tapping motion and processes the measurement result thereof, characterized by comprising:

[0014] a touch panel that is provided on one face of the terminal, can display information related to finger tapping motion, and is tapped by one finger of two fingers of the same hand of a user of the terminal;

[0015] a finger detection section that detects the motion of the one finger that taps the touch panel; and

[0016] a processing section that measures an inter-digit distance at the time of a finger tapping motion between the other one of the two fingers and the one finger that taps the touch panel, based on the detection information obtained by the finger detection section, wherein the other one of the two fingers holds another surface of the terminal that is on the opposite side from the touch panel.

[0017] According to the above structure, since the finger detection section and the processing section are provided in the terminal with the touch panel, the finger tapping motion can be measured and processed only by tapping the touch panel of the terminal with the fingers, and thus the finger tapping motion can be easily measured and processed without wearing a special dedicated tool and without using an expensive dedicated evaluation analysis device. In particular, if a distance measuring sensor or the like, which is installed as a regular component in an existing terminal such as a smartphone or a tablet, is used as the finger detection section, and the processing section is implemented by using an arithmetic circuit of the existing terminal or newly introduced into the terminal as application software, the widely popular terminal such as a smartphone or a tablet can be effectively used to measure and process the finger tapping motion in a simple manner at a low cost and with good usability.

[0018] In addition, in the above structure, the finger detection section can detect the motion of the fingers using a camera and / or a distance measuring sensor or the like. The finger detection section can be provided on both sides of the terminal corresponding to the fingers of both hands of the user holding the terminal, respectively, or can be provided on one side of the terminal as a component shared by the fingers of both hands of the user. In addition, in a case where the position of the tapping finger is detected from an image captured by the camera as the finger detection section, there is a case where it is difficult to capture the correct position of the finger due to the angle of view and the distance of the camera, and thus it is preferable to obtain an inter-digit target distance, which is a reference of the open position of the fingers in the opening and closing action of the fingers in the finger tapping motion, as a reference value from the image captured by the camera in advance, and correct the position detection data based on the reference value. In this case, the finger detection section can use a target length jig having a length corresponding to the inter-digit target distance to define the reference of the open position of the fingers in the opening and closing action of the fingers in the finger tapping motion. In addition, such a target length jig preferably has transparency that does not interfere with the detection by the finger detection section. In addition, in a case where the target length jig is interposed between the surface of the touch panel and the tapping finger when the target length jig is used, the finger can be deformed due to the interposition, and thus it is preferable to effectively use the amount of deformation in correcting the position detection data.

[0019] In addition, in the above configuration, the processing section preferably calculates and analyzes a feature quantity related to the evaluation of the brain function of the user on the basis of the measurement result of the distance between the two fingers. Furthermore, the processing section can also evaluate the brain function and cognitive function of the user on the basis of the calculated feature quantity (for example, by comparison with data of a healthy person). Such evaluation is effective as a screening for the initial stage of determining cognitive impairment, and is useful for detecting cognitive impairment, for example, and can also be useful for determining the judgment ability at the time of driving a vehicle, and can be applied to a game for brain exercise and the like, and has a wide range of applications.

[0020] In addition, in the above configuration, the touch panel is preferably a touch panel display that has a touch panel that senses the position of a finger that comes into contact with the surface thereof and that is capable of displaying information. If such a touch panel display is used, the finger detection section can sense the contact of the finger that is tapping using the touch panel, and detect it as the closed position of the opening and closing action of the finger in the finger tapping motion.

[0021] In addition, in the above configuration, the other finger that holds the other face of the terminal is preferably fixed in a state of contact with the other face of the terminal by the fixing section. In this case, the fixing section can be provided on the terminal side, or can also be provided separately as a wearing tool that is separate from the terminal.

[0022] In addition, the finger tapping measurement processing terminal of the above configuration can constitute a finger tapping measurement processing system together with a head-mounted information processing device that is capable of being worn on the head of a user who uses the terminal and that has a display section that displays real space information and / or virtual space information in a manner that is visible to the user. In the case of this system, the finger tapping measurement processing terminal and the head-mounted information processing device preferably communicate with each other via the communication unit. By using the communication unit, the head-mounted information processing device can receive, for example, information displayed on the display of the finger tapping measurement processing terminal, and display the received information on the display section. In addition, in this case as well, the processing by the processing section of the finger tapping measurement processing terminal that calculates and analyzes a feature quantity related to the evaluation of the brain function of the user on the basis of the measured distance between the two fingers is preferably performed, but a server device that is provided remotely can also perform such calculation and analysis. Specifically, for example, the server device receives the measurement result of the distance between the two fingers from the processing section of the finger tapping measurement processing terminal via the communication unit, and calculates and analyzes a feature quantity related to the evaluation of the brain function of the user on the basis of the received measurement result of the distance between the two fingers using the arithmetic processing section. The processing result of the arithmetic processing section can be stored in the storage section of the server device, and the data stored in the storage section can be transmitted to the finger tapping measurement processing terminal and the head-mounted information processing device via the communication unit.

[0023] In addition to the aforementioned finger tapping measurement and processing terminal and finger tapping measurement and processing system, the present invention also provides a method and computer program for measuring finger tapping motion and processing the measurement results.

[0024] Invention Effects

[0025] According to the present invention, since a finger detection unit and a processing unit are provided in a terminal with a touch panel, the finger tapping motion can be measured and processed simply by tapping the touch panel of the terminal with a finger. Therefore, it is not necessary to wear special special equipment or use expensive special evaluation and analysis devices to easily measure and process the finger tapping motion. Attached Figure Description

[0026] Figure 1 This is a perspective view schematically showing the appearance of a finger tapping measurement and processing terminal according to an embodiment of the present invention, which has a finger detection unit on one side.

[0027] Figure 2 yes Figure 1 An enlarged cross-sectional view of a portion of the finger-tapping measurement processing terminal.

[0028] Figure 3 This is an explanation Figure 1 How to obtain a graph of the target distance between two fingers when using a camera to measure the distance between two fingers in a finger tapping measurement processing terminal.

[0029] Figure 4 This diagram illustrates how to use a target length fixture to obtain the target distance between two fingers.

[0030] Figure 5 This diagram provides supplementary explanation of how to use a target length fixture to obtain the target distance between two fingers.

[0031] Figure 6 This diagram illustrates the action of using a finger tapping measurement processing terminal with two displays to obtain the target distance between two fingers.

[0032] Figure 7 It means Figure 1 A diagram illustrating the measurement method using a distance sensor in a finger tapping measurement processing terminal.

[0033] Figure 8 This is a perspective view schematically showing the appearance of a finger tapping measurement and processing terminal with finger detection sections on both sides.

[0034] Figure 9 It means Figure 1 and Figure 8 The diagram shows a structural example of a finger tapping measurement and processing terminal.

[0035] Figure 10 is a flowchart showing one example of the operation of the finger tap measurement processing terminal. Figure 1 and Figure 8 is a flowchart showing one example of the operation of the finger tap measurement processing terminal.

[0036] Figure 11 In (a), (b) is a graph showing another example of the measurement data of the finger tap measurement processing terminal of the present embodiment.

[0037] Figure 12 is a perspective view schematically showing the external appearance of the finger tap measurement processing system of one embodiment of the present application.

[0038] Figure 13 is a block diagram showing an example of the structure of the head-mounted information processing device constituting the finger tap measurement processing system of Figure 12 DETAILED DESCRIPTION

[0039] Embodiments of the present application will be described below with reference to the drawings.

[0040] In addition, the finger tap measurement processing terminal of one embodiment of the present application described below is configured as a terminal capable of measuring and processing finger tap motions using a touch panel, as shown in Figure 12 may be any usage. In addition, the finger tap measurement processing terminal is configured as a small terminal such as a smartphone in the present embodiment, but can be in the form of a tablet-type thin computer or personal computer, or the like.

[0041] In addition, the following embodiments show a finger tap measurement processing terminal capable of measuring and processing finger tap motions independently because it has a touch panel, but the present application can also be embodied as a system or method capable of measuring and processing finger tap motions by cooperating with a separate display, or as a computer program that enables a computer to perform such measurement and processing of finger tap motions (for example, enables a terminal having a display on one face to measure finger tap motions and display the measurement results on the display), or a computer program product that stores such a computer program.

[0042] Figure 1 shows a specific embodiment of the present application, the finger tap measurement processing terminal (hereinafter sometimes simply referred to as a terminal) 100 shown in the drawing as an information processing terminal includes a camera 101, a distance measuring sensor 102, and a touch panel 103. The camera 101 and the distance measuring sensor 102 are provided on one side of the terminal 100​Figure 1 On the left side of the frame, camera 101 captures surrounding objects and scenery. Distance sensor 102 is a sensor capable of capturing the shape of objects such as people and objects in three dimensions, and can determine the distance and angle of objects. Additionally, touch panel 103 detects the position of a touch (sensing the position of a finger touching its surface), forming touch panel display 103A. Furthermore, this touch panel display 103A is disposed on one side (front) of the finger tapping measurement processing terminal 100, and can display information related to finger tapping movements. As described later, the user holds the terminal with two fingers of the same hand, and the touch panel 103 is tapped by one of the two fingers (index finger).

[0043] Tap one side of the measurement and processing terminal 100 with your finger ( Figure 1 The left side) is held by the user's left hand 111, and the opposite side ( Figure 1 The right side of the measurement processing terminal 100 is held by the user's right hand 121. The user's left thumb 112 supports the finger tapping the other side of the measurement processing terminal 100 (the side opposite to the touch panel 103), i.e., the back side. The user's left index finger 113 performs an opening and closing movement, i.e., a tapping movement, by opening or closing in contact with the touch panel 103. The same applies to the other side of the measurement processing terminal 100. The user's right thumb 122 supports the back side of the information processing terminal 100, and the user's right index finger 123 performs an opening and closing movement, i.e., a tapping movement 124, by opening or closing in contact with the touch panel 103. In addition, in Figure 1 In the middle, the left and right index fingers 113 and 123 are either open or closed. Index fingers 115 and 125 represent the open state of the index fingers, and index fingers 116 and 126 represent the closed state of the index fingers in the form of touching the touch panel 103.

[0044] In this state, the movement (opening and closing action) of the index fingers 113 and 123 tapping the touch panel 103 is detected. Specifically, the finger detection unit 800 (see reference) detects the position of the index fingers 113 and 123. Figure 9 The camera 101 can capture the positions of the index fingers 113 and 123 from the captured images. By correcting based on pre-captured images of predetermined positions (images of the index fingers spread at target intervals, described later), it can measure with high precision the distances between the index fingers 113 and 123 during the tapping motion and the surface of the finger tapping measurement processing terminal 100 (the surface of the touch panel 103) where the fingers are about to land, over time. Furthermore, the distance sensor 102, acting as a finger detection unit 800, can detect the distance and angle between itself and the index fingers 113 and 123, and can measure with high precision the distances between the index fingers 113 and 123 during the tapping motion and the surface of the finger tapping measurement processing terminal 100 where the fingers are about to land, over time.

[0045] Therefore, by taking into account the thickness of the finger tapping measurement processing terminal 100, the distance between the index finger 113 during tapping motion and the left thumb 112, which is stationary (keeping the back side) on the back of the finger tapping measurement processing terminal 100, can be measured. Similarly, for the right hand, by adding the thickness of the finger tapping measurement processing terminal 100, the distance between the index finger 123 during tapping motion and the right thumb 122, which is stationary on the back of the finger tapping measurement processing terminal 100, can be measured. Thus, without the need for special equipment, the tapping motion between two fingers can be measured easily and conveniently using a widely available and inexpensive information processing terminal. Furthermore, based on the measured information, the computational functions within the finger tapping measurement processing terminal 100 can be used to analyze and evaluate features related to the user's brain function assessment. This enables a user-friendly and convenient examination for the early detection of cognitive impairments such as Alzheimer's disease, cerebrovascular disease, Lewy body type, as well as Parkinson's disease and developmental coordination disorders (such as inability to jump or skip rope). Furthermore, it can not only detect and examine signs of cognitive impairment, but also quantitatively detect and identify the fine motor function of the fingers based on the characteristics obtained from analysis and evaluation, serving as a "yardstick" to represent the brain's health status. Therefore, it can be used in training and rehabilitation programs to improve brain function. For example, a training / rehabilitation program that uses sound prompts to open / close the fingertips to evaluate their fine motor skills can be cited.

[0046] Furthermore, in the example above, the tapping interval between the index finger and thumb was measured over time by measuring the distance between the index finger and the surface of the finger tapping measurement processing terminal 100. However, tapping movements between other two fingers, such as the middle finger and thumb, can also be used, and tapping movements can be measured even when multiple fingers are open / closed, achieving the same effect. Additionally, the following description primarily focuses on the case of tapping movements using the index finger, but the same applies to other fingers.

[0047] Figure 2 express Figure 1 The image shows a cross-section of a portion of the finger-tapping measurement processing terminal 100. Additionally, Figure 2 China for and Figure 1 The same parts are labeled with the same meaning. Figure 1 The same reference numerals are used in the accompanying drawings, but their descriptions are omitted.

[0048] Figure 2 This represents the cross-section of the periphery of the left hand holding the measurement processing terminal 100 and tapping it with fingers, but the cross-section of the periphery of the right hand holding the measurement processing terminal 100 and tapping it with fingers is the same. Figure 2In this device, the finger tapping measurement and processing terminal 100 has a touch panel 103, a display unit 201, and a pressure sensor 202 on the front side of the display, a pressure sensor 203 on the back side, and an accelerometer 204 and a fixture 205 on the bezel on the same side as the camera 101. The display unit 201 is composed of liquid crystal or the like and displays content through the highly transparent touch panel 103. The pressure sensors 202 and 203 sense the contact pressure (press) when the index finger 113 (123) touches the surface of the touch panel 103. The accelerometer 204 is a sensor that detects the change in velocity per unit time, i.e., acceleration, and can capture motion, vibration, impact, etc. In addition to acceleration, it also includes sensors that detect speed and direction of rotation. The fixing fixture 205 is a fixture (clamp, jig) that holds and supports the thumb 112 (142) of the finger tapping measurement processing terminal 100, fixing the thumb 112 (142) in a state of contact with the back of the terminal 100.

[0049] The touch panel 103 detects the position and contact area of ​​the index finger 113 (123) and can also detect positional deviations. The pressure sensor 202 detects the contact intensity, i.e., the contact pressure, when the index finger 113 (123) makes contact, and can reflect it as one of the characteristic quantities calculated based on the finger tapping motion. In addition, the pressure sensor 203 detects the contact pressure of the thumb 112 (142) supporting and holding the finger tapping measurement and processing terminal 100 in contact with the terminal 100, and can confirm whether the thumb 112 (142) is in a holding state with sufficient contact with the back of the terminal 100. In addition, the fixing fixture 205 can stably support and hold the terminal 100 without wobbling at a specified position. The accelerometer 204 can also measure the strength of the force when the finger returns by detecting the magnitude and direction of the vibration when the finger closes.

[0050] Next, use Figure 3 , Figure 4 , Figure 5 , Figure 6 The following describes in more detail the use of camera 101 to capture the position of index fingers 113 and 123 performing the tapping motion, and to measure the distances between index fingers 113 and 123 and the surface of the finger tapping measurement processing terminal 100 to which the fingers are about to land.

[0051] Figure 3 It is a cross-sectional view showing the state of the index fingers 113 and 123 when the camera 101 captures the tapping motion. Figure 4This describes how the index finger 113 is set to open at the target interval distance (target distance between two fingers) from the point where it taps the surface of the measurement and processing terminal 100 using the target length fixture 401, and how the camera 101 captures a picture of the index finger 113 in the open target interval distance state. Figure 5 This is to explain the cause. Figure 4 The diagram shows how to correct the depression (deformation) when the contact surface between the target length jig 401 and the index finger 113 (115) caused by the contact surface of the jig 401 and the index finger 113 (115) to cause the finger 113 (115) to be depressed (deformed). Figure 6 This describes how, when terminal 100 has two screens, the index fingers 113 and 123 are set to open at a target distance from the moment they tap the surface of the measuring terminal 100, and an image of the index fingers 113 and 123 in this open-distance state is captured by camera 101. Additionally, in... Figure 3 , Figure 4 , Figure 5 , Figure 6 In China, for the sake of Figure 1 and Figure 2 The same parts are labeled with the same meaning. Figure 1 and Figure 2 The same reference numerals are used in the accompanying drawings, but their descriptions are omitted.

[0052] Figure 3 In the middle, it is configured on one side of the finger tapping measurement and processing terminal 100 ( Figure 3 The camera 101 (located on the left side of the frame) captures the index finger 113 of the left hand 111 making a tapping motion near the camera 101. Based on the captured image of the index finger 113 (i.e., the detection information or position detection data obtained by the finger detection unit 800), the following will be used... Figure 9 The finger-tapping motion measurement and analysis processing unit 903, as described, analyzes and calculates the distance (distance from the surface of the terminal 100, i.e., the distance between the two fingers) 311 of the index finger 113 moving due to opening and closing. Additionally, the camera 101 captures an image of the index finger 123 of the right hand 121 performing the tapping motion, located at a distance from the camera 101. Based on the captured image of the index finger 123, the finger-tapping motion measurement and analysis processing unit 903 analyzes and calculates the distance 321 of the index finger 123 moving due to opening and closing. Here, the camera 101 and the distance sensor 102 need to be able to measure up to the surface of the display in order to detect finger contact. Therefore, by changing the lens position, the field of view of the camera 101 and the distance sensor 102 can be changed so that it can detect (capture) the finger even up to the vicinity of the surface.

[0053] However, it is difficult to correctly capture the distance 311, 321 of the index finger 113, 123 moved by the tapping motion only from the position and size of the index finger 113, 123 on the image captured by the camera 101. For example, even if the left and right index fingers 113, 123 perform the same distance of opening and closing motion, the index finger 113 of the left hand 111 close to the camera 101 is captured larger on the image captured by the camera 101 than the index finger 123 away from the camera 101, and the distance of the opening and closing motion looks longer, so it is difficult to correctly measure the distance of the opening and closing motion performed by the left and right index fingers 113, 123. Thus, in the present embodiment, before the index fingers 113, 123 are captured and the distance between the index fingers 113, 123 and the surface of the finger tapping measurement processing terminal 100 is measured, the index fingers 113, 123 are set in advance to a state of being opened by a predetermined target distance from the surface of the terminal 100, the set state of the index fingers 113, 123 is captured by the camera 101, the target distance between the index fingers 113, 123 and the surface of the terminal 100 is recognized on the captured image by the finger detection section 800 including the camera 101, and the recognized target distance on the image (two-finger target distance) is input as a length reference (a reference value as a reference of the opening position of the opening and closing motion of the fingers in the finger tapping motion). Then, when the index fingers 113, 123 performing the tapping motion are captured and the distance between the index fingers 113, 123 and the surface of the finger tapping measurement processing terminal 100 is measured, the image of the captured index fingers 113, 123 is corrected by the finger detection section 800 based on the length reference input in advance, so the finger tapping motion measurement analysis processing section 903 can calculate and measure the distance between the index fingers 113, 123 and the surface of the terminal 100 with high accuracy. In addition, when the brain function is evaluated based on the finger tapping motion, there is an optimal two-finger distance, which is input as a predetermined target distance in advance in the present embodiment.

[0054] As a specific example of the correction, as Figure 4As shown, the target length jig 401 that has a target length corresponding to the target distance between the two fingers is held between the extended index finger 115 and the surface of the terminal 100, and the extended index finger 115 is photographed by the camera 101. Thereby, the target distance between the index finger 115 and the surface of the terminal 100 in the photographed image is introduced as a length reference, and when the index finger 113 is photographed while performing the tapping motion, the distance between the index finger 113 and the surface of the terminal 100 in the photographed image is corrected based on the introduced length reference, so that the distance between the index finger 113 and the surface of the terminal 100 can be measured with high accuracy over time. Then, the distance between the index finger 113 and the surface of the terminal 100 is further considered in terms of the thickness of the terminal 100, so that the distance between the index finger 113 and the thumb 112 can be measured with high accuracy over time. In addition, Figure 4 The case of the extended index finger 115 of the left hand 111 is described above, but the case of the extended index finger 125 of the right hand 121 can also be corrected in the same manner.

[0055] In this way, the finger detection unit 800 can use the target length jig 401 to define the reference of the extended position of the opening and closing motion of the finger in the finger tapping motion, and the target length jig 401 is formed using a material having transparency, and the degree of transparency is such that it does not obstruct the photographing operation of the camera (does not interfere with the detection performed by the finger detection unit 800). Thereby, even if the target length jig 401 is present, the camera 101 can photograph the motion of the index finger through the target length jig 401, and in particular, the target length jig 401 set for the left hand does not interfere with the photographing of the motion of the index finger 123 of the right hand that is far from the camera 101.

[0056] In addition, when the target distance between the index finger 113 and the surface of the terminal 100 is measured using the target length jig 401, the index finger 115 that presses the target length jig 401 in the extended state is Figure 5As shown, at the contact surface 501 that contacts the target length fixture 401, the originally present raised portion becomes a flat state (deformed state) due to the pressure applied, and the distance between the index finger 115 and the surface of the terminal 100 will produce an error equivalent to the amount of indentation (deformation) 502. Therefore, in this embodiment, in order to achieve the specified target distance 503 when the index finger 113 is in the open state, the length of the target length fixture 401 is made the length after increasing the amount of indentation 502. As a result, the length reference for correction can be correctly introduced, and the distance between the index finger and the terminal surface can be measured with higher accuracy over time. That is, in this embodiment, the finger detection unit 800 detects the amount of deformation of the finger in the state where the target length fixture 401 is clamped between the surface of the display and the finger that is tapping, and corrects the position detection data based on the deformation detection value.

[0057] As a specific example of other corrections, such as Figure 6 As shown, when the terminal 100 has two screens, a fixed display screen 601 and a closable display screen 602, with the closable display screen 602 open, the index finger 113 of the left hand 111 is spread out on the fixed display screen 601 to become the open index finger 115. Then, by viewing and aligning with a reference such as the "target height position when the index finger is spread out" indicated by the display content 603 on the closable display screen 602, the index finger 115 is moved to the target height position. Thus, the index finger 115 can be set to align with a position 604 away from the surface of the terminal 100 from the target height. In this state, if the camera 101 captures a picture of the index finger 115, the target distance between the index finger 115 and the surface of the terminal 100 in the captured image can be used as a length reference. Similarly, the same applies to the index finger 123 of the right hand 121. A reference such as the "target height position when the index finger is open," displayed on the closable display screen 602 (showing content 605), can be used to move the index finger 125 to the target height position and align it with a position 606 away from the target height distance from the surface of the terminal 100. In this state, if the camera 101 captures a picture of the index finger 125, the target distance between the index finger 125 and the surface of the terminal 100 in the captured image can be imported as a length reference. Therefore, when capturing a picture of the index finger making a tapping motion, the distance between the captured index finger and the surface of the terminal 100 can be corrected based on the pre-imported length reference. This allows for high-precision measurement of the distance between the index finger and the surface of the terminal 100 over time. Furthermore, by taking into account the thickness of the terminal 100, the measured distance between the index finger and the thumb can be measured with high precision over time.

[0058] Next, useFigure 7 The case where the position of the index finger 113, 123 performing the tapping motion is captured by the distance measuring sensor 102, and the distance between the index finger 113, 123 and the surface of the finger tapping measurement processing terminal 100 onto which the finger is to be dropped is measured, is explained in more detail. Figure 7 is a cross-sectional view showing the state when the distance of the index finger 113, 123 performing the tapping motion and the angle thereof are detected by the distance measuring sensor 102. In addition, in Figure 7 , the same parts as Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 , Figure 6 are denoted by the same reference numerals as Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 , Figure 6 and the explanation thereof is omitted.

[0059] Figure 7In this case, the distance Llc of the index finger 113 of the left hand 111 that performs the tapping motion and the angle θl thereof are detected by the distance measuring sensor 102 arranged on one side (the right side frame portion) of the terminal 100. The distance Lla between the index finger 113 and the surface of the terminal 100 can be calculated as Llc x sin θl from the detected distance Llc and angle θl, and thus the distance Lla between the index finger 113 that performs the tapping motion and the surface of the terminal 100 can be measured over time. Also, the same applies to the index finger 123 of the right hand 121, and the distance Lrc of the index finger 123 of the right hand 121 that performs the tapping motion and the angle θr thereof are detected by the distance measuring sensor 102 arranged at a position away from the distance measuring sensor 102. The distance Lra between the index finger 123 and the surface of the terminal 100 can be calculated as Lrc x sin θr from the detected distance Lrc and angle θr, and thus the distance Lra between the index finger 123 that performs the tapping motion and the surface of the terminal 100 can be measured over time. The distance between the index finger and the surface of the terminal 100 is further considered in relation to the thickness of the terminal 100, and the distance between the index finger and the thumb can be measured with high accuracy over time. Also, by detecting the position at which the index fingers 113, 123 contact the touch panel 103, in combination with the detection of the distance of the fingers by the distance measuring sensor 102, the time period from when the index fingers 113, 123 are away from the touch panel 103 to when they return and contact again, and the movement trajectory of the index fingers 113, 123 during this period can be measured with high accuracy. In this way, in the present embodiment, the finger detection portion 800 can detect the position of the finger that performs the tapping based on the distance and angle detected by the distance measuring sensor 102, and the distance between the two fingers can be measured by the finger tapping motion measurement analysis processing portion 903 based on this position detection information.

[0060] Next, the configuration of the finger tapping measurement processing terminal 100 will be described with reference to FIG. 8. Figure 8 The case in which one camera 101 and one distance measuring sensor 102 are provided on each of the opposite sides of the terminal 100 (finger detection portions are provided on both sides of the terminal 100) will be described.

[0061] Figure 8 The finger tapping measurement processing terminal 100 in which the camera 101 and the distance measuring sensor 102 are provided on one side of the terminal 100, and the camera 801 and the distance measuring sensor 802 are also provided on the opposite side will be described with reference to FIG. 9. Figure 8 In the case in which the camera 101 and the distance measuring sensor 102 are provided on each of the opposite sides of the terminal 100, the same applies to the case in which the camera 801 and the distance measuring sensor 802 are provided on each of the opposite sides of the terminal 100. Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 1 Figure 2 Figure 3 ,​​​​​​​​​Figure 4 、 Figure 5 、 Figure 6 、 Figure 7 The same reference numerals are used and the explanation thereof is omitted.

[0062] As shown in the above Figure 1 , in the case where the camera 101 and the distance measuring sensor 102 are provided only on one side of the terminal 100, since the left hand 111 is located near the camera 101 and the distance measuring sensor 102, the distance between the index finger 113 of the left hand 111 and the surface of the terminal 100 can be easily measured with high precision, but the right hand 121 is located far from the camera 101 and the distance measuring sensor 102, so there is a case where the distance between the index finger 123 of the right hand 121 and the surface of the terminal 100 cannot be easily measured with high precision. In contrast, as shown in the above Figure 8 , by newly adding the camera 801 and the distance measuring sensor 802 on the other side near the right hand 121, the distance between the index finger 123 of the right hand 121 and the surface of the terminal 100 can be easily measured with high precision using the camera 801 and the distance measuring sensor 802 near the right hand 121. That is, for both the left hand and the right hand, the distance between the index finger and the surface of the terminal can be measured with high precision using the camera and the distance measuring sensor near the hand without being hindered by the index finger of the other hand, and the distance between the index finger and the surface of the terminal can be measured with higher precision. That is, according to the above Figure 8 , the finger detection section 800 can detect the motion of the fingers of the two hands of the user holding the terminal 100 on both sides, and the finger tapping motion measurement and analysis processing section 903 can perform the processing of measuring the distance between the two fingers for each of the two hands of the user.

[0063] In addition, Figure 1 and Figure 8 , the operation of each of the camera 101 and the distance measuring sensor 102 in the case where the camera 101 and the distance measuring sensor 102 are provided on one side or both sides of the terminal 100 (or up and down (or left and right)) is explained, but the camera 101 and the distance measuring sensor 102 can also simultaneously photograph and detect the motion of the fingers in the tapping motion. Thereby, for example, for a situation where two fingers are overlapped or one finger is not visible from the camera 101 or the distance measuring sensor 102, the motion of the fingers can be measured complementarily by the camera 101 and the distance measuring sensor 102 which are misaligned with each other, and the effect that the positions of the left and right fingers can be accurately measured without omission can be obtained. Of course, the camera 101 and the distance measuring sensor 102 can also be used together to improve the precision at the time of normal measurement.

[0064] Figure 9 is a block diagram showing the structure of the finger tapping measurement processing terminal 100 of the present embodiment. Figure 9In this case, the terminal 100 includes first and second cameras 101, 801, first and second distance measuring sensors 102, 802, first and second pressure sensitive sensors 202, 203, an acceleration sensor 204, a gyro sensor 901, a geomagnetic sensor 902, a finger tapping motion measurement analysis processing section 903, a control section 904, a memory section 910 constituted by a program 911 and information data 916, a touch panel 103, a display unit section 201, a microphone 931, a speaker 932, a vibration generation section 933, a communication section 934, a transceiving antenna 935, and these constituent elements are connected to each other via a bus 950 except for the transceiving antenna 935. In this case, the first and second cameras 101, 801, the first and second distance measuring sensors 102, 802, the first and second pressure sensitive sensors 202, 203, and the touch panel 103 constitute a finger detection section 800 capable of detecting the motion of a finger tapping a display.

[0065] The first camera 101 is disposed on one side of the terminal 100 along the up-down direction (or the left-right direction), and the second camera 801 is disposed on the other side of the terminal 100 along the up-down direction (or the left-right direction) to photograph the field of view in front of the user's finger as the main object. In Figure 1 Figure 3 In the case where the left and right index fingers 113, 123 performing the tapping motion are photographed by the first camera 101. In Figure 8

[0066] The first and second distance measuring sensors 102, 802 are sensors capable of stereoscopically capturing the shape of an object such as a person and an object. As such a sensor, there can be mentioned a LiDAR (Light Detection and Ranging) which analyzes and detects the distance of an object located at a long distance and the state of the object by irradiating laser light such as infrared light to the object and measuring the scattered light of the reflection, a TOF (Time Of Flight) sensor which measures the reflection time of pulsed light irradiated to a subject by each pixel to perform distance measurement, a millimeter wave radar which emits electromagnetic waves of millimeter waves and captures the reflection waves thereof to detect the distance of an object of reflection and the state of the object, and the like. In Figure 1 Figure 7 In the case where the left and right index fingers 113, 123 performing the tapping motion are captured by the first distance measuring sensor 102, and the distance and the angle of each index finger 113, 123 are detected. In Figure 8 ​​​In this case, the distance and angle of the index finger 113 of the left hand performing the tapping motion can be detected by the first distance measuring sensor 102, and the distance and angle of the index finger 123 of the right hand performing the tapping motion can be detected by the second distance measuring sensor 802, which are introduced into the terminal 100.

[0067] The first and second pressure sensitive sensors 202, 203 are sensors that detect the pressure (contact pressure) generated by contact. The first pressure sensitive sensor 202 is provided on the front side of the terminal 100 below the touch panel 103 and the display unit 201, detects the contact pressure when the open / close finger contacts the front surface of the terminal 100, can determine whether the open / close finger is closed to the terminal surface, and can measure the contact strength when the open / close finger is closed to abut against the terminal surface. On the other hand, the second pressure sensitive sensor 202 is provided on the back surface of the terminal 100, detects the contact pressure of the thumb abutting against the back surface of the terminal, can determine whether the thumb stably holds the terminal 100, and can measure the strength of the holding by the thumb. In addition, the pressures of the first and second pressure sensitive sensors 202, 203 can be added to obtain the strength of the pressure when the fingers are closed.

[0068] The acceleration sensor 204 is a sensor that detects the acceleration, i.e., the change in speed per unit time, and can capture motion, vibration, impact, and the like. By detecting the magnitude and direction of the vibration when the fingers are closed using the acceleration sensor 204 provided in the terminal 100, the strength of the force when the fingers are returned and the strength of the force when the fingers are contacted can also be measured. In addition, the gyro sensor 901 is a sensor that detects the angular velocity of the rotation direction, and can capture the state of the posture in the vertical, horizontal, and diagonal directions. Thus, the motion of the terminal 100 can be detected using the acceleration sensor 204 and the gyro sensor 901 mounted in the terminal 100. The geomagnetic sensor 902 is a sensor that detects the magnetic force of the earth, and can detect the direction in which the terminal 100 is oriented. As the geomagnetic sensor, a three-axis type sensor that detects the geomagnetic field in the up-down direction in addition to the front-back direction and the left-right direction is used, and by capturing the change in the geomagnetic field corresponding to the motion of the terminal 100, the motion of the terminal 100 can also be detected. Thus, it can be detected and determined whether the terminal 100 is in a configuration state in which the display screen is visible.

[0069] The finger tapping motion measurement analysis processing section 903 analyzes the position of the finger performing the tapping motion, and performs processing to measure the two-finger interval distance between the finger (e.g., the index finger) performing the tapping motion and the stationary finger (e.g., the thumb) based on the motion information of the finger captured by the first and second cameras 101, 801 and / or the motion information of the finger detected by the first and second distance sensors 102, 802. In addition, the finger tapping motion measurement analysis processing section 903 performs processing to calculate and analyze the characteristic quantity representing the characteristics of the motion of the finger in terms of distance, velocity, acceleration, jerk, tapping interval time, phase difference, and the like, based on the measured two-finger interval distance information that changes over time due to the tapping motion. The characteristic quantity calculated and analyzed is related to the evaluation of the brain function of the user, and thus the state of the brain can be known, and the examination for early detection of cognitive disorders such as Alzheimer's, cerebrovascular, Lewy body, and the like, and Parkinson's disease, developmental coordination disorder (inability to walk and jump rope, and the like), and the like can be performed. In addition, the characteristic quantity calculated and analyzed can quantitatively detect and recognize the fine motor function of the finger as a "ruler" representing the state of the brain. In the distance item of the characteristic quantity, the maximum amplitude, total movement distance, and the like of the finger are calculated, and how the size of the motion of the finger changes is evaluated. In the velocity item of the characteristic quantity, the maximum velocity, opening velocity, closing velocity, and the like are calculated, and how fast the finger can move is evaluated. In the acceleration item of the characteristic quantity, the maximum amplitude of the acceleration, the trend of the start of opening (momentum), the trend of the end of opening, the trend of the end of closing, the trend of the start of closing, the finger contact time, and the like are calculated, and the trend of the motion of the finger is evaluated. In the tapping interval time item of the characteristic quantity, the number of taps, tapping cycle, and the like are calculated, and the timing of the tapping and the error thereof are evaluated. In the phase difference item of the characteristic quantity, the two-hand deviation, two-hand similarity, and the like are calculated, and the coordination between the two hands is evaluated.

[0070] The control section 904 is configured of a CPU or the like, and performs processing to control the operation of the entire terminal 100 by executing programs 911 such as an operating system (OS) 912 and various action control applications 913, 914 stored and held in the memory section 910, and controls the start operation of various applications.

[0071] The memory section 910 is configured of a flash memory or the like, and stores an operating system 912 and programs 911 such as image, sound, document, display, measurement, and the like, and action control applications 913, 914 of various processes. In addition, the memory section 910 holds basic data 917 required for the operating system 912 and the like to perform basic actions, and information data 916 such as file data 918, 919 used by the various applications 913, 914 and the like. For example, the image processing application is started and shooting is performed using the camera, and the file data of the shooting is saved. In addition, the processing in the finger tapping motion measurement analysis processing section 903 can be stored as one application A, and the measurement of the distance between two fingers and the calculation analysis of the feature amount of the finger tapping motion can be performed by starting the application A. In addition, the distance between two fingers measured from the information processing terminal can be received by an external server device or the like having a high arithmetic performance and a large capacity, and the calculation analysis of the feature amount can be performed.

[0072] The display unit section 201 is configured of a liquid crystal panel or the like, and displays an image, a video, and displays the remaining amount of the battery capacity, various alarms, the time, and the like, notification information to the user, and icons of the applications started in the display screen, and the like. In particular, the present embodiment can display the processing result obtained by the finger tapping motion measurement analysis processing section 903.

[0073] The touch panel 103 is configured of a material having high transparency, and detects the position and the area on the screen contacted by a finger, and is provided on the surface of the information processing terminal 100, and can determine whether the index finger that is opened and closed touches the surface of the terminal 100. In addition, the input operation can be performed while the icon or the like displayed in the display unit section 201 is viewed and touched on the touch panel 103. For example, the touch panel 103 is an input unit of a touch panel method such as an electrostatic capacitance type, and can detect the approach or contact operation (touch operation) of a finger or a stylus or the like as an operation input, and the user can set to input the information intended to be input, or easily select and specify the icons of the applications and the files on the display screen with the touch operation. In the present embodiment, the finger detection section 800 can detect the closed position of the opening and closing action of the finger tapping motion by sensing the contact of the finger performing the tapping action using such a touch panel 103.

[0074] The microphone 931 collects sounds from the outside and sounds emitted by the user himself or herself, and the speaker 932 outputs sounds to the outside to deliver notification information and music and the like to the user. In addition, the user can be informed of the instruction on the finger tapping measurement by the sound from the speaker.

[0075] The vibration generation section 933 generates vibration by the control of the control section 904, and converts the notification information to the user emitted in the information processing terminal 100 into vibration. The vibration generation section 933 applies the generated vibration to the finger or the like holding the information processing terminal 100, and can reliably deliver the notification to the user.

[0076] The communication section 934 is a communication interface that performs wireless communication with a server device or the like located at another place through close-range wireless communication, wireless LAN, or base station communication, and transmits and receives measurement data and characteristic amounts and the like obtained by analysis and calculation to and from the server device or the like via the transceiving antenna 935 at the time of wireless communication. Note that, for example, close-range wireless communication can be performed using an electronic tag, but is not limited thereto, and can be performed using wireless LAN such as Bluetooth (registered trademark), IrDA (Infrared Data Association, registered trademark), Zigbee (registered trademark), HomeRF (Home Radio Frequency, registered trademark), or Wi-Fi (registered trademark) as long as wireless communication is possible at least when the information terminal is located in the vicinity. Further, as base station communication, long-range wireless communication such as W-CDMA (Wideband Code Division Multiple Access) and GSM (registered trademark) (Global System for Mobile communications) can be used. Further, it is also possible to detect the positional relationship and direction between terminals using an ultra-wideband wireless system (Ultra Wide Band: UWB). Although not illustrated, the communication section 934 can also use other methods such as optical communication and acoustic wave communication as means for wireless communication. In this case, a light emitting / receiving section and an acoustic wave outputting / acoustic wave inputting section are used instead of the transceiving antenna 935.

[0077] With the configuration elements as shown in Figure 9 , it is possible to measure the two-finger interval distance between the finger (for example, the index finger) in the tapping motion and the finger (for example, the thumb) on the back of the terminal 100, and further to calculate and analyze a characteristic amount representing a characteristic of the finger motion from the measured two-finger interval distance information. Moreover, without the need to install a special device, it is possible to measure the two-finger tapping motion in a simple manner with good usability using a widely popular inexpensive information processing terminal. Further, based on the measured information, it is possible to analyze and evaluate a characteristic amount related to the user's brain function evaluation (for example, using the finger tapping motion measurement analysis processing section 903 or with a device other than the terminal 100), and to realize a check for early detection of cognitive impairment such as Alzheimer's, cerebrovascular, Lewy body, and the like, and Parkinson's disease, developmental coordination disorder (inability to bounce walk and jump rope, and the like), and the like, with a small burden on the user and good convenience. Further, not only is it possible to perform a symptom detection and check for cognitive impairment and the like, but also it is possible to quantitatively detect and recognize the fine motor function of the finger as a "ruler" representing the state of brain health based on the characteristic amount obtained by analysis and evaluation, so it can be used for exercise and rehabilitation plans for brain function improvement.

[0078] Next, regarding the operation of the finger tapping measurement and processing terminal 100 in this embodiment, refer to... Figure 10 Further explanation is needed.

[0079] Figure 10 This is a flowchart illustrating the operation of the finger tapping measurement and processing terminal 100 in this embodiment. (Hereinafter referred to as...) Figure 8 Given the structure of the device, it is necessary to decide whether to measure the distance between two fingers by filming the movement of the fingers during the tapping motion using cameras 101 and 801, or by detecting the position of the fingers using distance sensors 102 and 802 (S1001). When filming the movement of the fingers during the tapping motion using cameras 101 and 801, the fingers are set to be spread out at a predetermined target distance on the terminal 100 before filming (S1002). In this state, the fingers are filmed using cameras 101 and 108, and a distance reference is obtained from the filmed images of the fingers during the tapping motion. Furthermore, the user can recognize the set state of the fingers being spread out at the predetermined target distance. Thus, the user is aware that the fingers should be spread out at the target distance when performing the tapping motion (S1003). After the cameras have completed the pre-filming, the user begins the tapping motion for measurement (or is prompted to perform the tapping motion by a computer program) (S1004). On the other hand, when the finger position is detected using distance sensors 102 and 802, it is not necessary to set the fingers to a predetermined target distance before detection in order to measure the distance between the two fingers; the user can directly begin the tapping motion for measurement (S1004). Alternatively, when using distance sensors 102 and 802, the user can be made aware that the fingers should be spread at the target distance when performing the tapping motion by setting the fingers to a predetermined target distance apart. Of course, when using distance sensors, calibration can also be performed using steps such as S1002 and 1003.

[0080] As the finger tapping motion, there are a simultaneous mode in which both hands perform the finger tapping motion simultaneously, and an alternate mode in which both hands perform the finger tapping motion alternately, and the user performs the finger tapping motion in each mode (tapping step S1005). At this time, the first and second cameras 101, 801 and / or the first and second distance measuring sensors 102, 802 detect the motion of the tapping finger, and the finger tapping motion measurement and analysis processing section 903 analyzes the position of the finger performing the tapping motion and measures the two-finger interval distance between the finger in the tapping motion and the fixed finger based on the motion information of the finger captured by the first and second cameras 101, 801 and / or the motion information of the finger detected by the first and second distance measuring sensors 102, 802 (finger detection step and processing step S1006). In the finger detection step and processing step S1006, the processing in the above steps S1002, 1003 is reflected therein, and the operations / processes associated with the finger detection section 800 and the finger tapping motion measurement and analysis processing section 903 are performed. Figures 1-9 After that, in the case where re-measurement is not required (S1007), the finger tapping motion measurement and analysis processing section 903 calculates and analyzes the characteristic quantity representing the characteristics of the motion of the finger based on the measured two-finger interval distance information which changes over time due to the tapping motion, in terms of distance, speed, acceleration, tapping interval time, phase difference, and the like (S1008). Further, the calculated and analyzed characteristic quantity is displayed on the display unit section 201 in a table or graph for easy understanding, or a sound is emitted from the speaker 932 to notify the user (S1009). The processing is ended without performing the measurement and analysis again (S1010). In addition, in the case where re-measurement is required, the processing returns to step S1004, and in the case where the measurement and analysis are to be performed again, the processing returns to step S1001. By the above operations, the two-finger interval distance in the finger tapping motion can be correctly measured, and based on the measurement result, the characteristic quantity related to the brain function evaluation of the user can be calculated and analyzed.

[0081] Figure 11 (a) is a graph representing the result measured in the simultaneous mode in which both hands perform the finger tapping motion simultaneously, Figure 11 (b) is a graph representing the result measured in the alternate mode in which both hands perform the finger tapping motion alternately. In both graphs, the vertical axis on the left side represents the distance between the finger (e.g., the index finger) in the tapping motion and the surface of the terminal, the vertical axis on the right side represents the two-finger interval distance between the finger (e.g., the index finger) in the tapping motion and the finger (e.g., the thumb) fixed to the back of the terminal, and the upper side represents the motion of the index finger of the left hand and the lower side represents the motion of the index finger of the right hand. As can be seen from the graphs, in the simultaneous mode, as shown in (a) of FIG. 10, the index finger of the left hand and the index finger of the right hand move in the same direction over time, but in the alternate mode, as shown in (b) of FIG. 10, the index finger of the left hand and the index finger of the right hand move in opposite directions over time. Figure 11 Figure 11 ​As shown in (b), the index fingers of the left and right hands move in opposite directions over time; for example, at time 1101, the left index finger is open and the right index finger is closed. Furthermore, when measuring the distance between the two fingers by filming the movement of the index fingers with camera 101 (801), since camera 101 (801) measures in frame rate units (60fps, etc.), Figure 11 The waveform shown is a sampled waveform obtained by connecting the values ​​of each observation point. Here, even when the camera's field of view does not extend to the surface of the display and the timing of finger contact with the display cannot be accurately measured, the waveform can be obtained based on the timing of contact with the touch panel.

[0082] Next, for the finger tapping measurement and processing system consisting of the finger tapping measurement and processing terminal 100 and the head-mounted information processing device 1201, refer to Figure 12 Please provide an explanation.

[0083] Figure 12 This illustrates an embodiment of a finger tapping measurement and processing system comprising a finger tapping measurement and processing terminal 100, a head-mounted information processing device 1201, and a server device 1203.

[0084] Figure 12 In this context, the head-mounted information processing device 1201 is worn on the head of a user who taps the measurement processing terminal 100 with their fingers, and has a display processing unit (display unit) 1235 (described later) capable of displaying real-world spatial information and virtual-world spatial information in a user-visible manner. Figure 13 The head-mounted information processing device 1201 and the finger tapping measurement processing terminal 100 transmit and receive various types of information via communication units such as wireless communication and external networks. Thus, the head-mounted information processing device 1201 can receive information displayed on the display screen (monitor) 1202 of the finger tapping measurement processing terminal 100 and / or finger movement information captured by the camera 101 (801) during tapping movements as various types of transmitted and received information, and display the received information on the display screen 1202 and the finger movement information as virtual spatial information on the display processing unit 1235. Therefore, when measuring finger tapping movements, due to the gripping state of the thumbs and index fingers of both hands on the terminal 100, such as... Figure 12indication information on the finger tapping measurement, operation instruction, start / end of operation, and the like are exchanged between the head-mounted information processing device 1201 and the terminal 100, various instructions are issued from the head-mounted information processing device 1201 to the terminal 100, or the operation instruction from the terminal 100 is confirmed by display and / or sound in the head-mounted information processing device 1201. In addition, when the terminal 100 having the finger tapping measurement processing function has a display screen on both the front and back surfaces, that is, when the terminal 1201 has a display screen on the back surface, the user can easily view the display screen on the back surface, and thus it is also possible to display the operation guidance and the like on the display screen on the back surface without using the head-mounted information processing device 1201. In addition, when the head-mounted information processing device 1201 is provided with a distance measuring sensor and a camera, and is capable of detecting the positions of the left index finger and the right thumb, respectively, it is also possible to measure the motion of the finger tapping motion using only the head-mounted information processing device 1201.

[0085] In addition, it is also possible to exchange data between the server device 1203 provided outside and the terminal 100 having the finger tapping measurement processing function via wireless communication, to perform data saving management using the server device 1203 having a large capacity, and to perform calculation and analysis of the characteristic quantity related to the brain function evaluation of the user using the server device 1203 having a high arithmetic performance and a large capacity, from the terminal 100 having the finger tapping measurement processing function, that is, the distance between the two fingers. Thus, it is possible to perform calculation and analysis of various characteristic quantities at high speed and high precision, and it is possible to save and manage a large amount of characteristic quantity information obtained by calculation and analysis.

[0086] Figure 13 is a block diagram showing a configuration example of the head-mounted information processing device 1201. Figure 13 In the head-mounted information processing device 1201, the camera 1211, the right eye line detection section 1212, the left eye line detection section 1213, the acceleration sensor 1214, the gyro sensor 1215, the geomagnetic sensor 1216, the distance measuring sensor 1217, the control section 1218, the memory section 1220 constituted by the program 1225 and the information data 1226, the operation input section 1234, the display processing section 1235, the external environment sound microphone 1236, the voice microphone 1237, the earphone 1238, the vibration generation section 1239, the communication section 1240, and the transceiving antenna 1241 are connected to each other via the bus 1250, except for the transceiving antenna 1241.

[0087] The right-eye line-of-sight detection section 1212 and the left-eye line-of-sight detection section 1213 respectively detect the line of sight of the right eye and the left eye. Among them, the processing of detecting the line of sight can use a known technique generally used as eye movement tracking processing, for example, in a method using a corneal reflection, a technique is known in which an infrared LED (Light Emitting Diode) is made to irradiate a face and an infrared camera is used to take a picture, and the position of the reflection light on the cornea (corneal reflection) due to the infrared LED irradiation is used as a reference point, and the line of sight is detected based on the position of the pupil with respect to the position of the corneal reflection.

[0088] The operation input section 1234 is, for example, an input unit such as a keyboard, a button, a touch key, or the like, and is used by the user to set and input information that the user wants to input. The operation input section 1234 can be provided in the head-mounted information processing device 1201 in a position and in a manner that the user can easily perform an input operation, or can be provided in a manner that is separate from the body of the head-mounted information processing device 1201 and is connected by wire or wirelessly. In addition, an input operation screen can be displayed in the display screen of the display processing section 1235, and input operation information can be acquired based on the position on the input operation screen to which the line of sight detected by the right-eye line-of-sight detection section 1212 and the left-eye line-of-sight detection section 1213 is directed, and a pointer can be displayed on the input operation screen and the input operation information can be acquired by operating the pointer with the operation input section 1234. In addition, the user can utter a sound indicating an input operation, and the input operation information can be acquired by picking up the sound with the voice microphone 1237.

[0089] The display processing section 1235 is constituted by, for example, a projection section that projects various information such as reproduction information generated by an activated application, notification information to the user, and a transparent half mirror that images and displays the projected various information in front of the eyes, in the case of an optical see-through type head-mounted information processing device. In addition, the display processing section 1235 is constituted by a display such as a liquid crystal panel that displays the real space object in front of the eyes photographed by the camera 1211 together with various information, in the case of a video see-through type head-mounted information processing device. Thus, the user can see other image information in addition to the image in the field of view in front of the eyes.

[0090] The external environment sound microphone 1236 and the voice microphone 1237 respectively pick up sound from the outside and sound uttered by the user himself or herself. The earphone 1238 is worn on the ear of the user to make the user hear sound output to the user, and can convey notification information to the user by sound.

[0091] The above-described Figure 13The finger-tap measurement processing terminal 100 measures the information by wireless communication with the head-mounted information processing device 1201, and the display processing section 1235 of the head-mounted information processing device 1201 displays the information on the display screen 1202 of the finger-tap measurement processing terminal 100 as virtual space information, so that the user can clearly view the operation instruction information and the like of the finger-tap measurement. In addition, the head-mounted information processing device 1201 can transmit and receive the instruction information on the start and end of the finger-tap measurement, and can issue various instructions.

[0092] In addition, the present application is not limited to the above-described embodiments, and various modifications are included. For example, the above-described embodiments are described in detail for easy understanding of the present application, but are not limited to necessarily include all the described structures. In addition, a part of the structure of one embodiment can be replaced with the structure of another embodiment, and the structure of one embodiment can be added with the structure of another embodiment. In addition, to a part of the structure of each embodiment, other structures can be added, deleted, or replaced.

[0093] In addition, for example, a part or all of each structure, function, processing section, processing unit, and the like described above can be realized by hardware through integrated circuit design. In addition, each structure, function, and the like described above can be realized by software through a program interpreted and executed by a processor to realize each function. The program, table, file, and the like information to realize each function can be stored in a memory, a hard disk, an SSD (Solid State Drive), or the like recording device, or an IC card, an SD card, a DVD, or the like recording medium.

[0094] In addition, the control lines and the information lines indicate the necessary parts for explanation, and do not necessarily indicate all the control lines and the information lines on the product. In fact, it can be considered that almost all the structures are connected to each other.

[0095] In addition, the above-described embodiments describe the method of holding the terminal with two fingers of both hands, but if the terminal is placed on the user's lap for use, the measurement can be easily performed with both hands without holding the terminal with the thumb.

[0096] Explanation of Reference Numerals

[0097] 100 finger-tap measurement processing terminal

[0098] 101, 801, 1211 camera

[0099] 102, 802, 1217 distance measuring sensor

[0100] 103 touch panel

[0101] 103A display

[0102] 111 left hand

[0103] 112, 122 thumb

[0104] 113, 123 index finger

[0105] 201: display unit section

[0106] 202, 203 pressure-sensitive sensor

[0107] 205 fixing jig

[0108] 401 target length jig

[0109] 800 finger detection section

[0110] 903 finger tapping motion measurement analysis processing section (processing section)

[0111] 934, 1240 communication section

[0112] 1201 head-mounted information processing device

[0113] 1203 server device

Claims

1. A finger tapping measurement and processing terminal for measuring finger tapping motion and processing the measurement results, characterized in that, include: A touch panel, which is disposed on one side of the terminal, is capable of displaying information related to finger tapping movements and can be tapped by one of two fingers of the same hand held by the user holding the terminal; A finger detection unit detects the movement of the finger tapping the touch panel; and The processing unit performs the following processing: based on the detection information obtained by the finger detection unit, it measures the distance between the two fingers during the finger-tapping movement between the other finger of the two fingers and the finger that taps the touch panel, wherein the other finger keeps the terminal on the opposite side of the touch panel.

2. The finger tapping measurement and processing terminal as described in claim 1, characterized in that: The finger detection unit detects the movement of one finger in each of the user's two hands that are holding the terminal on both sides, and the processing unit measures the distance between the two fingers in each of the user's two hands.

3. The finger tapping measurement and processing terminal as described in claim 2, characterized in that: The finger detection unit is provided on both sides of the terminal.

4. The finger tapping measurement and processing terminal as described in any one of claims 1 to 3, characterized in that: The finger detection unit includes a camera that captures images of the finger making the tapping motion, and detects the position of the finger from the images captured by the camera. Based on the position detection data, the processing unit measures the distance between the two fingers.

5. The finger tapping measurement and processing terminal as described in claim 4, characterized in that: The finger detection unit obtains the target distance between two fingers from the image captured by the camera as a reference value, which serves as the reference for the opening position of the fingers during the finger-tapping motion, and corrects the position detection data based on this reference value.

6. The finger tapping measurement and processing terminal as described in claim 5, characterized in that: The finger detection unit uses a target length fixture with a length corresponding to the target distance between the two fingers to define a reference for the opening position of the fingers during the finger-tapping motion.

7. The finger tapping measurement and processing terminal as described in claim 6, characterized in that: The target length fixture is transparent and does not obstruct the detection performed by the finger detection unit.

8. The finger tapping measurement and processing terminal as described in claim 6 or 7, characterized in that: The finger detection unit detects the amount of deformation of a finger that is deformed while the target length fixture is held between the surface of the touch panel and the finger that is tapping, and corrects the position detection data based on the deformation amount detection value.

9. The finger tapping measurement and processing terminal as described in any one of claims 1 to 3, characterized in that: The finger detection unit has a distance sensor that detects the distance and angle between itself and the finger that is tapping, and detects the position of the finger based on the distance and angle detected by the distance sensor. Based on the position detection information, the processing unit measures the distance between the two fingers.

10. The finger tapping measurement and processing terminal as described in any one of claims 1 to 3, characterized in that: The touch panel is a touch panel display capable of displaying information on its surface. The finger detection unit detects the finger as the closed position of the finger's opening and closing motion during the tapping movement by sensing the contact of the finger being tapped by the touch panel.

11. The finger tapping measurement and processing terminal as described in any one of claims 1 to 3, characterized in that: It also has a fixing part for fixing the other finger in contact with the other surface of the terminal.

12. The finger tapping measurement and processing terminal as described in any one of claims 1 to 3, characterized in that: The touch panel has a pressure-sensitive sensor that senses the contact pressure when a finger making a tap contacts the surface of the touch panel.

13. The finger tapping measurement and processing terminal as described in any one of claims 1 to 3, characterized in that: The processing unit performs calculations based on the measured distance between the two fingers and analyzes feature quantities related to the user's brain function evaluation.

14. The finger tapping measurement and processing terminal as described in any one of claims 1 to 3, characterized in that: The touch panel displays the processing results obtained by the processing unit.

15. A finger tapping measurement and processing system, comprising a finger tapping measurement and processing terminal and a head-mounted information processing device, wherein the finger tapping measurement and processing terminal measures finger tapping motion and processes the measurement results, and the head-mounted information processing device is worn on the head of a user using the finger tapping measurement and processing terminal and has a display section that displays real-world spatial information and / or virtual-world information in a user-visible manner, and the finger tapping measurement and processing terminal and the head-mounted information processing device are capable of communicating with each other via a communication unit. Its features are: The finger tapping measurement and processing terminal includes: a touch panel disposed on one surface of the terminal, capable of displaying information related to finger tapping movements, and capable of being tapped by one of two fingers of the same hand held by the user; a finger detection unit that detects the movement of the finger tapping the touch panel; and a processing unit that performs the following processing: based on the detection information obtained by the finger detection unit, measuring the distance between the other finger and the finger tapping the touch panel during the finger tapping movement, wherein the other finger holds the terminal on another surface located opposite the touch panel. The head-mounted information processing device receives the information from the finger tapping measurement and processing via the communication unit, and displays the received information on the display unit.

16. The finger tapping measurement and processing system as described in claim 15, characterized in that: The head-mounted information processing device receives operation instructions regarding the measurement of finger tapping movements from the finger tapping measurement and processing terminal via the communication unit, and sends instruction information including the start and end of the operation to the finger tapping measurement and processing terminal.

17. The finger tapping measurement and processing system as described in claim 15 or 16, characterized in that: The processing unit of the finger tapping measurement and processing terminal performs processing based on the measured distance between the two fingers to calculate and analyze feature quantities related to the user's brain function evaluation.

18. The finger tapping measurement and processing system as described in claim 15, characterized in that: It also includes a server device that can receive the measurement result of the distance between the two fingers from the processing unit of the finger tapping measurement processing terminal via the communication unit. The server device has a processing unit and a storage unit. The processing unit calculates and analyzes feature quantities related to the user's brain function evaluation based on the received measurement results of the distance between the two fingers. The storage unit stores the processing results of the processing unit. The server device can transmit the data stored in the storage unit to the finger tapping measurement processing terminal and the head-mounted information processing device via the communication unit.

19. A method for measuring and processing finger tapping motion, characterized in that, include: The tapping step involves tapping a touch panel on one side of the terminal that displays information related to finger tapping movements, using one of two fingers of the same hand held by the user holding the terminal. The finger detection step detects the movement of the finger tapping the touch panel; and The processing steps include the following processing: based on the detection information obtained in the finger detection step, measuring the distance between the other finger of the two fingers and the finger that taps the touch panel during the finger tapping movement, wherein the other finger keeps the terminal on the opposite side of the touch panel.

20. The finger tapping measurement and processing method as described in claim 19, characterized in that: The finger detection step detects the movement of one finger in each of the user's two hands that are holding the terminal on both sides, and the processing step measures the distance between the two fingers of the user's two hands respectively.

21. The finger tapping measurement and processing method as described in claim 19 or 20, characterized in that: The finger detection step detects the position of the finger from an image captured by a camera of the finger making the tapping motion, and the processing step measures the distance between the two fingers based on the position detection data.

22. The finger tapping measurement and processing method as described in claim 21, characterized in that: The finger detection step obtains the target distance between two fingers from the image captured by the camera as a reference value, which serves as the reference for the opening position of the fingers in the finger-tapping motion, and corrects the position detection data based on this reference value.

23. The finger tapping measurement and processing method as described in claim 19 or 20, characterized in that: The finger detection step detects the position of the finger based on detection data from the ranging sensor, and the processing step measures the distance between the two fingers based on the position detection information, wherein the ranging sensor detects the distance and angle to the finger that is tapping.

24. The finger tapping measurement and processing method as described in claim 19 or 20, characterized in that: The touch panel is a touch panel display capable of displaying information on its surface. The finger detection step detects the contact of the finger that is tapping by sensing the touch panel as the closed position of the finger's opening and closing motion during the tapping movement.

25. The finger tapping measurement and processing method as described in claim 19 or 20, characterized in that: The processing steps involve calculating and analyzing feature quantities related to the user's brain function evaluation based on the measured distance between the two fingers.

26. The finger tapping measurement and processing method as described in claim 19 or 20, characterized in that: The touch panel displays the processing results obtained from the processing steps.

27. A computer program product, comprising a computer program, characterized in that, The computer program enables the measurement of finger tapping movements using a terminal with a touch panel on one side and displays the measurement results on a display. The computer program enables the computer to perform the following steps: The tapping step, for the user of the terminal, involves tapping the touch panel with one of two fingers of the same hand holding the terminal; The finger detection step detects the movement of the finger tapping the touch panel; and The processing steps include the following processing: based on the detection information obtained in the finger detection step, measuring the distance between the other finger of the two fingers and the finger that taps the touch panel during the finger tapping movement, wherein the other finger keeps the terminal on the opposite side of the touch panel.

28. The computer program product as described in claim 27, characterized in that: The finger detection step detects the movement of one finger in each of the user's two hands that are holding the terminal on both sides, and the processing step measures the distance between the two fingers of the user's two hands respectively.

29. The computer program product as described in claim 27 or 28, characterized in that: The finger detection step detects the position of the finger from an image captured by a camera of the finger making the tapping motion, and the processing step measures the distance between the two fingers based on the position detection data.

30. The computer program product as described in claim 29, characterized in that: The finger detection step obtains the target distance between two fingers from the image captured by the camera as a reference value, which serves as the reference for the opening position of the fingers in the finger-tapping motion, and corrects the position detection data based on this reference value.

31. The computer program product as described in claim 27 or 28, characterized in that: The finger detection step detects the position of the finger based on detection data from the ranging sensor, and the processing step measures the distance between the two fingers based on the position detection information, wherein the ranging sensor detects the distance and angle to the finger that is tapping.

32. The computer program product as described in claim 27 or 28, characterized in that: The touch panel is a touch panel display capable of displaying information on its surface. The finger detection step detects the contact of the finger that is tapping by sensing the touch panel as the closed position of the finger's opening and closing motion during the tapping movement.

33. The computer program product as described in claim 27 or 28, characterized in that: The processing steps involve calculating and analyzing feature quantities related to the user's brain function evaluation based on the measured distance between the two fingers.

34. The computer program product as described in claim 27 or 28, characterized in that: The touch panel displays the processing result obtained from the processing steps.

Images