Operation estimation device

Abstract

To accurately estimate operation of finger operation.SOLUTION: An operation estimation device includes: a band 3 attached to a front arm so as to overlap a boundary at a second division in order from a wrist when a part from the wrist to an elbow of a front arm is divided into five; a plurality of sensor elements 4 that are provided inside the band 3, are arranged on a palm side of the front arm, and detects a pressure transmitted from the front arm; and a controller 12 that estimates finger operation on the basis of a plurality of sensor signals output from the plurality of sensor elements 4.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to an operation estimation device that estimates a finger operation. [Background technology]

[0002] Patent Document 1 discloses a mobile terminal device that uses piezoelectric sensors. The mobile terminal device of Patent Document 1 has multiple piezoelectric sensors arranged on the back of the wrist. The mobile terminal device of Patent Document 1 measures the movement of the user's (wearer's) fingers using detection signals from the multiple piezoelectric sensors.

[0003] Patent Document 2 discloses an estimation device that measures finger movements (finger operations). In Patent Document 2, finger operations are accurately learned using characteristic portions of a sensor signal corresponding to the displacement of the body surface at the wrist (movement of the wrist tendons), and the operations are estimated using the learning results. Furthermore, the displacement of the body surface at the wrist is closely linked to the finger movements. These features increase the estimation accuracy of finger operations.

[0004] In Patent Documents 1 and 2, a piezoelectric sensor is attached to the wrist and finger manipulation is estimated by detecting changes in the wrist surface. However, because the wrist moves significantly not only due to finger manipulation but also due to hand movements such as supination, pronation, palmar flexion, dorsiflexion, flexion, and ulnar flexion, measuring changes in the wrist surface requires close contact between the sensor and the skin. In other words, to prevent the device worn on the wrist from slipping due to these movements, a mechanism is required to tightly fix the device or to constantly follow the shape of the wrist. Furthermore, movements other than those of the fingers have a significant impact on changes in the wrist surface, so changes due to finger movement must be extracted from the combined output. This requires learning, which can easily lead to complex calculations.

[0005] Furthermore, as mentioned above, when measuring on the wrist, the device needs to be tightly fixed so that it does not move with wrist movement. However, with wearable devices, it is generally important to eliminate the feeling of being worn and to make them as unnoticeable as possible. Therefore, devices that require tight fixing or devices equipped with mechanisms for this purpose are likely to increase the discomfort when worn.

[0006] Furthermore, when measuring at the wrist, the movement of the tendons is mainly acquired. Unlike muscles, tendons do not expand or contract much with finger movement, but only move back and forth, so the waveform output by the sensor tends to be complex. Furthermore, when measuring at the wrist, the waveform output by the sensor tends to be complex because it is greatly influenced and combined by wrinkles on the skin and the movement of the wrist itself, in addition to the tendons and muscles of the wrist. Therefore, machine learning and other techniques are likely to be required, which will complicate calculations and increase the amount of judgment processing. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-352739 [Patent Document 2] International Publication No. 2022 / 130684 Summary of the Invention [Problem to be solved by the invention]

[0008] The present invention provides an operation estimation device that can accurately estimate a finger operation. [Means for solving the problem]

[0009] According to a first aspect of the present invention, there is provided an operation estimation device comprising: a band worn on the forearm so as to overlap the boundary line that divides the forearm into five equal parts from the wrist side when the forearm is divided into five equal parts; a plurality of sensor elements provided within the band and positioned on the palm side of the forearm, the sensor elements detecting pressure from the forearm; and a control unit that estimates finger operations based on a plurality of sensor signals output from the plurality of sensor elements.

[0010] According to a second aspect of the present invention, there is provided the operation estimation device according to the first aspect, wherein the plurality of sensor elements are configured to extend in an extension direction of the forearm and are arranged in a line in a circumferential direction of the forearm.

[0011] According to a third aspect of the present invention, there is provided the operation estimation device according to the first aspect, wherein the plurality of sensor elements are piezoelectric elements.

[0012] According to a fourth aspect of the present invention, there is provided the manipulation estimation device according to the first aspect, wherein the plurality of sensor elements include five sensor elements, and the five sensor elements are arranged so as to overlap muscles of the little finger, ring finger, middle finger, index finger, and thumb.

[0013] According to a fifth aspect of the present invention, there is provided the operation estimation device according to the first aspect, wherein the control unit receives a plurality of measurement signals corresponding to the plurality of sensor signals, determines a plurality of maximum values ​​and a plurality of minimum values ​​for each of the plurality of measurement signals, and estimates an operation of the finger based on the plurality of maximum values ​​and the plurality of minimum values.

[0014] According to a sixth aspect of the present invention, there is provided the operation estimation device according to the fifth aspect, wherein the plurality of sensor elements include a first sensor element arranged so as to overlap a muscle of the thumb, and the control unit determines that the operation is of the thumb when a first maximum value of a first measurement signal by the first sensor element is larger than maximum values ​​by other sensor elements.

[0015] According to a seventh aspect of the present invention, there is provided an operation estimation device according to the sixth aspect, wherein the plurality of sensor elements includes a second sensor element arranged so as to overlap a muscle of the little finger, and the control unit determines that the operation is of the little finger when a second maximum value of a second measurement signal by the second sensor element is larger than maximum values ​​by other sensor elements.

[0016] According to an eighth aspect of the present invention, there is provided the operation estimation device according to the seventh aspect, wherein the plurality of sensor elements includes a third sensor element arranged so as to overlap a muscle of a middle finger, and the control unit determines that the operation is of the middle finger when a third maximum value of a third measurement signal by the third sensor element is greater than maximum values ​​by other sensor elements.

[0017] According to a ninth aspect of the present invention, there is provided the operation estimation device according to the fifth aspect, wherein the plurality of sensor elements include a first sensor element arranged so as to overlap a muscle of the thumb, and the control unit determines that the operation is of the thumb when a first maximum value of a first measurement signal by the first sensor element is equal to or greater than a threshold value.

[0018] According to a tenth aspect of the present invention, there is provided the operation estimation device according to the first aspect, wherein the bisection boundary line corresponds to a position where the flexor pollicis longus and flexor digitorum superficialis of the forearm start to overlap. [Effects of the Invention]

[0019] According to the present invention, it is possible to provide an operation estimation device that can accurately estimate a finger operation. [Brief explanation of the drawings]

[0020] [Figure 1] FIG. 1 is an external view of an operation estimation device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a schematic diagram illustrating the configuration of the palm side of the operation estimation device. [Figure 3] FIG. 3 is a schematic diagram illustrating another configuration of the operation estimation device. [Figure 4]FIG. 4 is a schematic diagram illustrating the mounting position of the operation estimation device. [Figure 5] FIG. 5 is a block diagram illustrating the configuration of the operation estimation device. [Figure 6] FIG. 6 is a flowchart illustrating the overall operation of the operation estimation device. [Figure 7] FIG. 7 is a diagram illustrating an example of waveforms of a plurality of measurement signals. [Figure 8] FIG. 8 is a diagram illustrating an example of the operation of determining the maximum and minimum values ​​of the measurement signal. [Figure 9] FIG. 9 is a diagram illustrating an example of the maximum values ​​of the channels CH1 to CH12. [Figure 10] FIG. 10 is a diagram illustrating an example of the minimum values ​​of the channels CH1 to CH12. [Figure 11] FIG. 11 is a diagram illustrating an example of maximum values ​​of measurement signals when five fingers are extended. [Figure 12] FIG. 12 is a diagram illustrating an example of the minimum values ​​of the measurement signals when the five fingers are extended. [Figure 13] FIG. 13 is a diagram illustrating an example of maximum values ​​of measurement signals when five fingers are bent. [Figure 14] FIG. 14 is a diagram illustrating an example of the minimum values ​​of the measurement signals when five fingers are bent. [Figure 15] FIG. 15 shows normalized maximum values ​​of extension of the index finger, middle finger, and ring finger. [Figure 16] FIG. 16 is a diagram illustrating an example of maximum values ​​of measurement signals when five fingers are extended according to the first embodiment. [Figure 17] FIG. 17 is a diagram illustrating an example of the minimum values ​​of the measurement signals when the five fingers are extended according to the first embodiment. [Figure 18] FIG. 18 is a diagram illustrating an example of maximum values ​​of measurement signals when five fingers are bent according to the first embodiment. [Figure 19] FIG. 19 is a diagram illustrating an example of minimum values ​​of measurement signals when five fingers are bent according to the first embodiment. [Figure 20]FIG. 20 is a flowchart illustrating the operation estimation operation of the operation estimation device according to the first embodiment. [Figure 21] FIG. 21 is a flowchart illustrating the operation estimation operation of the operation estimation device according to the first embodiment. [Figure 22] FIG. 22 is a flowchart illustrating the operation estimation operation of the operation estimation device according to the second embodiment. [Figure 23] FIG. 23 is a flowchart illustrating the operation estimation operation of the operation estimation device according to the second embodiment. [Figure 24] FIG. 24 is a diagram illustrating multiple attachment positions of the band. [Figure 25] FIG. 25 is a diagram illustrating a measurement signal at the mounting position 1. In FIG. [Figure 26] FIG. 26 is a diagram illustrating a measurement signal at the mounting position 1. [Figure 27] FIG. 27 is a diagram illustrating a measurement signal at the mounting position 1. In FIG. [Figure 28] FIG. 28 is a diagram illustrating a measurement signal at the mounting position 1. In FIG. [Figure 29] FIG. 29 is a diagram illustrating the measurement results of the minimum value when the middle finger is bent five times in succession. [Figure 30] FIG. 30 is a diagram illustrating a measurement signal at the mounting position 2. [Figure 31] FIG. 31 is a diagram illustrating a measurement signal at the mounting position 2. In FIG. [Figure 32] FIG. 32 is a diagram illustrating the measurement signal at the mounting position 2. [Figure 33] FIG. 33 is a diagram illustrating a measurement signal at the mounting position 2. [Figure 34] FIG. 34 is a diagram illustrating the measurement signal at the mounting position 4. [Figure 35] FIG. 35 is a diagram illustrating a measurement signal at the mounting position 4. [Figure 36] FIG. 36 is a diagram illustrating a measurement signal at the mounting position 4. [Figure 37]FIG. 37 is a diagram illustrating a measurement signal at the mounting position 4. DETAILED DESCRIPTION OF THE INVENTION

[0021] Hereinafter, embodiments will be described with reference to the drawings. However, the drawings are schematic or conceptual, and the dimensions and proportions of each drawing are not necessarily the same as those of the actual drawing. Furthermore, even when the same parts are shown in different drawings, the dimensional relationships and proportions may be different. In particular, the following embodiments are illustrative of devices and methods for embodying the technical concept of the present invention, and the shape, structure, arrangement, etc. of the components do not specify the technical concept of the present invention. In the following description, elements having the same function and configuration are designated by the same reference numerals, and redundant description will be omitted.

[0022] [1] Configuration of operation estimation device 1 FIG. 1 is an external view of an operation estimation device 1 according to an embodiment of the present invention. The operation estimation device 1 is worn on the forearm of a user when in use. The operation estimation device 1 detects pressure from the body surface at the position where the device is worn, and estimates operations (movements) of the user's five fingers based on the detection results. The operation estimation device 1 includes a housing 2 and a band 3.

[0023] In this embodiment, an example will be described in which the operation estimation device 1 is worn on the left arm. The operation estimation device 1 may also be worn on the right arm. When the operation estimation device 1 is worn on the right arm, multiple sensor elements are arranged according to the positions of the five fingers of the right hand (thumb, index finger, middle finger, ring finger, and little finger), and channel numbers, which will be described later, are set according to the arrangement of the five fingers of the right hand.

[0024] The housing 2 is fixed to the band 3 and is placed on the back of the hand. The housing 2 houses electronic components for controlling the operation estimation device 1.

[0025] The band 3 has a circular shape that surrounds the forearm and is made of a stretchable material. The band 3 fits snugly around the forearm. The band 3 may have a structure that allows the length to be adjusted according to the thickness of the forearm.

[0026] FIG. 2 is a schematic diagram illustrating the configuration of the palm side of the manipulation estimation device 1. The manipulation estimation device 1 includes multiple sensor elements 4. For example, the manipulation estimation device 1 includes 12 sensor elements 4. The sensor elements 4 are configured, for example, by piezoelectric elements. Piezoelectric elements are sensor elements that utilize the piezoelectric effect and are capable of outputting an electrical signal in response to pressure applied to a piezoelectric body. The piezoelectric elements are configured by laminating a lower electrode, a piezoelectric layer, and an upper electrode in this order. The piezoelectric elements may have the lower electrode, piezoelectric layer, and upper electrode individually provided, the lower electrode and piezoelectric layer individually provided, and the upper electrode commonly provided for multiple piezoelectric elements, or only the lower electrode individually provided, and the piezoelectric layer and upper electrode commonly provided for multiple piezoelectric elements. The piezoelectric layer may be configured from a flexible piezoelectric material.

[0027] The multiple sensor elements 4 are provided inside the band 3 and arranged on the palm side. Each sensor element 4 has a rectangular shape extending in the extension direction of the forearm. That is, the rectangle of the sensor element 4 is configured so that its long sides are parallel to the extension direction of the forearm. The size of the sensor element 4 is, for example, 1.5 cm x 5 mm. The planar shape of the sensor element 4 is determined by the planar shape of the electrodes included in the sensor element 4. The multiple sensor elements 4 are arranged side by side in the circumferential direction of the forearm. The multiple sensor elements 4 have the function of detecting the movements of the muscles and tendons of five fingers (thumb, index finger, middle finger, ring finger, and little finger). Each of the multiple sensor elements 4 (specifically, the signals output by the multiple sensor elements 4) is called a channel (CH). The operation estimation device 1 has 12 channels (first channel CH1 to twelfth channel CH12) corresponding to the 12 sensor elements 4. When viewing the band 3 from the palm side, the rightmost sensor element 4 is channel CH1, and the leftmost sensor element 4 is channel CH12. In other words, multiple channels are defined such that the channel numbers increase by one from the little finger to the thumb.

[0028] FIG. 3 is a schematic diagram illustrating another configuration of the operation estimation device 1. The operation estimation device 1 includes five sensor elements 4. The five sensor elements 4 are provided corresponding to the five fingers (thumb, index finger, middle finger, ring finger, and little finger) and are arranged corresponding to the positions of the muscles and tendons of the thumb, index finger, middle finger, ring finger, and little finger. The operation estimation device 1 includes five channels (first channel CH1 to fifth channel CH5) corresponding to the five sensor elements 4. When the band 3 is viewed from the palm side, the rightmost sensor element 4 is channel CH1, and the leftmost sensor element 4 is channel CH5. In this way, the operation estimation device 1 is only required to include a minimum of five sensor elements 4.

[0029] FIG. 4 is a schematic diagram illustrating the wearing position of the operation estimation device 1. The forearm from the wrist to the elbow is divided into 5 equal parts. The boundary line at the wrist position is defined as the 0th boundary line. From the wrist to the elbow, the boundaries are defined as the 1st boundary line, the 2nd boundary line, the 3rd boundary line, the 4th boundary line, and the 5th boundary line, in that order. The 5th boundary line is the boundary line at the elbow position.

[0030] In this embodiment, the band 3 of the operation estimation device 1 is worn on the forearm so that the sensor element 4 overlaps the boundary line that divides the band into two equal parts. For example, the band 3 is worn on the forearm so that one end of the sensor element 4 contacts the boundary line that divides the band into two equal parts and the sensor element 4 extends toward the wrist. When the sensor element 4 is configured to extend to near the end of the band 3, the band 3 is worn on the forearm so that it overlaps the boundary line that divides the band into two equal parts.

[0031] The boundary line that bisects the forearm is the optimal position for determining the movement of the tendons and muscles of the thumb, index finger, middle finger, ring finger, and little finger. The boundary line that bisects the forearm corresponds to the position where the flexor pollicis longus and flexor digitorum superficialis begin to overlap. In this embodiment, by arranging the sensor element 4 so that it overlaps the boundary line that bisects the forearm, the movement of the five fingers can be determined more accurately.

[0032] 5 is a block diagram illustrating the configuration of the operation estimation device 1. The operation estimation device 1 includes a sensor unit 10, a signal processing unit 11, a control unit 12, and a storage unit 13.

[0033] The sensor unit 10 includes a plurality of sensor elements 4. For example, the sensor unit 10 includes, for example, 12 sensor elements 4. Each of the plurality of sensor elements 4 outputs a plurality of sensor signals.

[0034] The signal processing unit 11 receives a plurality of sensor signals from the sensor unit 10. The signal processing unit 11 has a function of converting the plurality of sensor signals (analog signals) into digital signals that can be processed by the control unit 12. For example, the signal processing unit 11 performs DC component removal processing, amplification processing, A / D conversion processing, filtering processing, etc. on the plurality of sensor signals. Note that the order of the processes performed by the signal processing unit 11 is not limited to the above and can be set as appropriate. The filtering processing is, for example, low-pass filtering. The signal processing unit 11 generates a plurality of measurement signals corresponding to the plurality of sensor signals and transmits the plurality of measurement signals to the control unit 12.

[0035] The control unit 12 is configured with one or more processors such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The control unit 12 realizes various functions by executing programs stored in the storage unit 13. The control unit 12 includes an operation period determination unit 12A, a voltage determination unit 12B, and an operation determination unit 12C.

[0036] The control unit 12 can output an output signal OUT to the outside. The output signal OUT includes information (estimated information) related to the estimation result of the finger operation. The operation estimation device 1 may transmit the estimated information to the outside using a communication unit (not shown). The operation estimation device 1 may display the estimated information using a display unit (not shown).

[0037] The operation period determination unit 12A detects voltage changes of a plurality of measurement signals, and determines an operation period during which a finger is operated based on the detection result.

[0038] The voltage determination unit 12B determines the maximum and minimum values ​​of the measurement signal for each channel during the operation period, and also performs normalization processing on the maximum and minimum values ​​of the measurement signal.

[0039] The operation determination unit 12C determines the finger operation based on the plurality of measurement signals. Furthermore, the operation determination unit 12C determines which of the five fingers (thumb, index finger, middle finger, ring finger, and little finger) is being operated based on the maximum and minimum values ​​of the measurement signals, and determines whether the finger is extended or flexed.

[0040] The storage unit 13 includes a non-volatile memory including a ROM (Read Only Memory) and a volatile memory including a RAM (Random Access Memory) and a register. The storage unit 13 stores programs executed by the control unit 12. The storage unit 13 stores various data necessary for the control of the control unit 12. The storage unit 13 also functions as a working area for temporarily storing data of the control unit 12.

[0041] [2] Operation of the operation estimation device 1 Next, the operation of the operation estimation device 1 will be described.

[0042] FIG. 6 is a flowchart illustrating the overall operation of the operation estimation device 1. The sensor unit 10 includes a plurality of sensor elements 4 that detect pressure applied from the forearm and output a sensor signal corresponding to the detected pressure. The sensor signal is a voltage signal.

[0043] The signal processing unit 11 acquires a plurality of sensor signals from a plurality of sensor elements 4 (step S100). The signal processing unit 11 performs predetermined signal processing on the plurality of sensor signals to generate a plurality of measurement signals corresponding to the plurality of sensor signals (step S101). The plurality of measurement signals are generated for each of the first channel CH1 to the twelfth channel CH12. The plurality of measurement signals are sent to the control unit 12. The control unit 12 stores the received plurality of measurement signals in the memory unit 13. The control unit 12 can perform a predetermined calculation using the plurality of measurement signals stored in the memory unit 13.

[0044] Fig. 7 is a diagram illustrating an example of waveforms of a plurality of measurement signals. Fig. 7 shows 12 measurement signals corresponding to the first channel CH1 to the twelfth channel CH12, respectively. The measurement signals are voltage waveforms (V). The horizontal axis of Fig. 7 represents time (s).

[0045] 7 shows, from the left, measurement signals obtained when the thumb is bent, the thumb is extended, the index finger is bent, the index finger is extended, the middle finger is bent, the middle finger is extended, the ring finger is bent, the ring finger is extended, the little finger is bent, and the little finger is extended. In this embodiment, as an example, the sensor element 4 is configured to output a negative voltage when compressed (when pressure is applied) and a positive voltage when the compression is released (when pressure is reduced). When the finger is extended, the muscle contracts, so the sensor element 4 outputs a positive voltage. The polarity of the output voltage of the sensor element 4 can be set arbitrarily. If the polarity of the output voltage of the sensor element 4 is opposite to that described above, the polarity in the determination operation will also be opposite.

[0046] As can be seen from Figure 7, the maximum and minimum values ​​of the measurement signal for each channel are different for the 10 types of manipulations. Therefore, by comparing the maximum and minimum values ​​of the measurement signals between multiple channels, it is possible to determine which finger is flexed or extended.

[0047] Next, the operation period determination unit 12A uses the multiple measurement signals to determine whether any of the five fingers has been operated (step S102). The voltage value of the measurement signal when there is no operation is called a reference voltage, and this reference voltage is set to 0 V, for example. The reference voltage is set appropriately according to the specifications of the sensor elements 4. The reference voltage may be set to the average value of the output voltages when no pressure is applied to the multiple sensor elements 4. The operation period determination unit 12A determines whether at least one of the multiple measurement signals (one or more selected measurement signals) has changed from the reference voltage. If the measurement signal has changed from the reference voltage, the operation period determination unit 12A determines that a finger operation has been performed. The time when the measurement signal changed from the reference voltage is called the operation start time.

[0048] If a finger operation has been performed (step S102=Yes), the operation period determination unit 12A determines the operation period from the operation start time to the operation end time (step S103). Specifically, the operation period determination unit 12A determines the time when the measurement signal returns to the reference voltage. The time when the measurement signal returns to the reference voltage is called the operation end time. The operation period determination unit 12A determines the time from the operation start time to the operation end time as the operation period.

[0049] Next, the voltage evaluation unit 12B determines the maximum and minimum values ​​of the measurement signal for each channel during the operation period (step S104). That is, the voltage evaluation unit 12B determines the largest voltage value of the measurement signal during the operation period as the maximum value, and determines the smallest voltage value of the measurement signal as the minimum value. FIG. 8 is a diagram illustrating an example of the operation of determining the maximum and minimum values ​​of the measurement signal. FIG. 8 shows an extracted waveform of channel CH12, thumb, and flexion in FIG. 7. During the operation period, the voltage evaluation unit 12B determines "+0.2 V" as the maximum value of the measurement signal and "0 V" as the minimum value of the measurement signal.

[0050] FIG. 9 is a diagram illustrating an example of the maximum values ​​of the first to twelfth channels CH1 to CH12. FIG. 9 shows the maximum values ​​of the first to twelfth channels CH1 to CH12 in waveform form. FIG. 10 is a diagram illustrating an example of the minimum values ​​of the first to twelfth channels CH1 to CH12. FIG. 10 shows the minimum values ​​of the first to twelfth channels CH1 to CH12 in waveform form. The horizontal axis of FIGS. 9 and 10 represents the channel number (CH number), and the vertical axis represents the voltage (V). In FIGS. 9 and 10, the maximum and minimum values ​​are calculated from the measurement signal when the thumb is bent as shown in FIG. 7.

[0051] The three sensor elements 4 corresponding to channels CH10 to CH12 are placed near the tendons and muscles of the thumb. As can be seen from Fig. 9, when the thumb is bent, the maximum values ​​of channels CH10 to CH12 become large. Therefore, by comparing the maximum values ​​of channels CH10 to CH12 with a threshold, it is possible to determine whether the thumb is bent.

[0052] Next, the operation determination unit 12C determines the operation of the fingers using the maximum and minimum values ​​of the operation signal (step S105). Specifically, the operation determination unit 12C compares the maximum and minimum values ​​of the operation signal with thresholds, and determines the flexion and extension of the five fingers based on the comparison result.

[0053] Fig. 11 is a diagram illustrating an example of the maximum value of the measurement signal when five fingers are extended. Fig. 12 is a diagram illustrating an example of the minimum value of the measurement signal when five fingers are extended. Fig. 13 is a diagram illustrating an example of the maximum value of the measurement signal when five fingers are bent. Fig. 14 is a diagram illustrating an example of the minimum value of the measurement signal when five fingers are bent. The horizontal axis of Figs. 11 to 14 is the channel number (CH number), and the vertical axis is the voltage ratio.

[0054] In this embodiment, the maximum values ​​of multiple channels are normalized. Normalization involves converting data (voltage) so that it falls within a certain range; specifically, it involves scaling the data from a minimum value of 0 to a maximum value of 1. The maximum value of the measurement signal is scaled from 0 to 1. The normalized voltage is called a voltage ratio. Similarly, the minimum values ​​of multiple channels are normalized. The minimum value of the measurement signal is scaled from -1 to 0. When the minimum value is normalized, the maximum absolute value is -1. In this embodiment, the minimum value of the measurement signal is scaled with a negative value to facilitate comparison with the maximum value. Normalizing the voltage makes it easier to compare operations with five fingers.

[0055] The tendons and muscles that extend and flex the fingers are arranged in the order of the thumb, index finger, middle finger, ring finger, and little finger. In this embodiment, the little finger side is designated as channel CH1, so the finger operation can be determined by assuming that the channels at which the measurement signal peaks appear in order starting from channel CH1 on the little finger side. In this specification, the "maximum peak" means the largest maximum value among the multiple maximum values ​​of all channels. The "minimum peak" means the smallest minimum value (the minimum value with the lowest voltage) among the minimum values ​​of all channels. The "peak of the measurement signal" has the same meaning as the "maximum peak" (or the minimum peak). When the voltage is normalized, the maximum and minimum values ​​are expressed in units of voltage ratio.

[0056] In this embodiment, a positive voltage is output when the pressure applied to the sensor element 4 decreases. Since muscles contract when fingers are extended, the output from the muscles linked to each finger can be determined based on the maximum value. As shown in FIG. 11, the measurement signals for the thumb and little finger are larger on the positive side, with the little finger's measurement signal being larger in the first half of the channel and the thumb's measurement signal being larger in the second half of the channel. Therefore, for example, if the maximum value of channel CH2 is equal to or greater than a threshold value (e.g., 0.7), it is determined that the little finger is being extended. Also, for example, if the maximum value of channel CH12 is equal to or greater than a threshold value (e.g., 0.7), it is determined that the thumb is being extended.

[0057] FIG. 15 is a diagram showing normalized maximum values ​​for the extension of the index finger, middle finger, and ring finger. From FIG. 15, it can be seen that peaks appear in the channels in the order of ring finger, middle finger, and index finger. Therefore, for example, if the maximum value of channel CH2 is equal to or greater than a threshold value (e.g., 0.7), it is determined that the ring finger is extended. For example, if the maximum value of channel CH4 is equal to or greater than a threshold value (e.g., 0.7), it is determined that the middle finger is extended. For example, if the maximum value of channel CH5 is equal to or greater than a threshold value (e.g., 0.7), it is determined that the index finger is extended. To determine whether the little finger is extended or the ring finger is extended in channel CH2, a threshold value that can be distinguished from each other is set, and if it is equal to or greater than the threshold, it is determined that the little finger is extended, and if it is less than the threshold, it is determined that the ring finger is extended.

[0058] Alternatively, the channel with the peak (the channel with the largest maximum value) among the maximum values ​​of channels CH1 to CH12 may be determined, and the peak channel may be associated with the finger operation for determination. Using FIG. 11 as an example, if the maximum value of channel CH12 is the peak, it is determined that the thumb is being extended. If the maximum value of channel CH2 is the peak, it is determined that the little finger is being extended. In this determination operation, it is sufficient to compare the maximum values ​​of channels CH1 to CH12.

[0059] Next, the operation of determining finger flexion will be explained. Finger flexion can be determined basically in the same way as the operation of determining extension. As can be seen from FIG. 14, in addition to the thumb and little finger, the measurement signal of the index finger also peaks (becomes smallest) in the channel near the center. Therefore, for example, if the minimum value of channel CH2 is less than a threshold value (for example, -0.7), it is determined that the little finger is flexed. For example, if the minimum value of channel CH5 is less than a threshold value (for example, -0.7), it is determined that the index finger is flexed. For example, if the minimum value of channel CH12 is less than a threshold value (for example, -0.7), it is determined that the thumb is flexed.

[0060] From FIG. 13, it may be determined that the maximum values ​​in all channels are low, that is, that the maximum values ​​in all channels are less than a threshold value (for example, 0.4), and the bending of the thumb and little finger may be determined.

[0061] Furthermore, the distinction between flexion and extension may be determined by, for example, comparing the maximum and minimum values ​​for a particular channel or a particular group of channels.

[0062] The determination operation of the operation determination unit 12C is performed with reference to data (user data) stored in advance in the storage unit 13. In an initial stage, with the user wearing the operation estimation device 1 on their forearm, the user extends and flexes each of their five fingers, and measures and determines the maximum and minimum values ​​of the measurement signal for each operation, information obtained by normalizing the maximum and minimum values, the channel numbers at which the multiple maximum values ​​peak for each operation, and the channel numbers at which the multiple minimum values ​​peak for each operation. This measured and determined user data is stored in advance in the storage unit 13. In addition, information on multiple thresholds used in the determination operation is stored in the storage unit 13 in advance.

[0063] [3] First Example Next, a more detailed example (first example) of the operation estimation operation will be described.

[0064] Fig. 16 is a diagram illustrating an example of the maximum value of the measurement signal when five fingers are extended. Fig. 17 is a diagram illustrating an example of the minimum value of the measurement signal when five fingers are extended. Fig. 18 is a diagram illustrating an example of the maximum value of the measurement signal when five fingers are bent. Fig. 19 is a diagram illustrating an example of the minimum value of the measurement signal when five fingers are bent.

[0065] In the first embodiment, five channels CH1 to CH5 are used to perform the operation estimation. Channels CH1 to CH5 in the first embodiment correspond to channels CH2, CH4, CH5, CH6, and CH12 in FIGS. 11 to 14, respectively. The positions of the five sensor elements 4 in channels CH2, CH4, CH5, CH6, and CH12 in FIGS. 11 to 14 are assumed to be the tendons and muscles of the little finger, ring finger, middle finger, index finger, and thumb. The voltage ratios in FIGS. 16 to 19 are extracted from the voltage ratios of the corresponding channels in FIGS. 11 to 14. The maximum threshold is set to 0.5, for example. The minimum threshold is set to -0.5, for example.

[0066] Fig. 20 is a flowchart illustrating the operation estimation operation of the operation estimation device 1 according to the first embodiment. Fig. 20 shows the process of estimating finger extension. The operation in Fig. 20 is based on the measurement signals in Figs. 16 and 17.

[0067] The operation determining unit 12C determines whether or not the channel CH5 is at the peak among the maximum values ​​of all the channels (step S200). If step S200 is "Yes", the operation determining unit 12C determines that the thumb is being extended (step S201).

[0068] Next, the operation determination unit 12C determines whether or not the channel CH1 is at the peak and the voltage ratio of the channel CH1 is 0.5 or more among the maximum values ​​of all the channels (step S202). If step S202 is "Yes", the operation determination unit 12C determines that the little finger is being extended (step S203).

[0069] Next, the operation determining unit 12C determines whether or not the channel CH3 is at the peak among the maximum values ​​of all the channels (step S204). If step S204 is "Yes", the operation determining unit 12C determines that the index finger is being extended (step S205).

[0070] Next, the operation determining unit 12C determines whether or not the channel CH2 is at the peak among the maximum values ​​of all the channels (step S206). If step S206 is "Yes", the operation determining unit 12C determines that the middle finger is extended (step S205).

[0071] Next, the operation determining unit 12C determines whether or not the channel CH4 is at the peak among the minimum values ​​of all the channels (step S208). If step S208 is "Yes", the operation determining unit 12C determines that the ring finger is being extended (step S209).

[0072] In this way, the operation estimation device 1 can estimate an operation performed by extending five fingers.

[0073] Fig. 21 is a flowchart illustrating the operation estimation operation of the operation estimation device 1 according to the first embodiment. Fig. 21 shows the process of estimating finger flexion. The operation in Fig. 21 is based on the measurement signals in Figs. 18 and 19.

[0074] The operation determination unit 12C determines whether or not the voltage ratio of channel CH4 is equal to or greater than −0.5 at the minimum value of all channels (step S300). If step S300 is “Yes,” the operation determination unit 12C determines that the thumb is being bent (step S301).

[0075] Next, the operation determination unit 12C determines whether or not the channel CH1 is at the peak and the voltage ratio of the channel CH5 is equal to or greater than -0.5 at the minimum value of all the channels (step S302). If step S302 is "Yes," the operation determination unit 12C determines that the operation is a flexion of the little finger (step S303).

[0076] Next, the operation determination unit 12C determines whether or not the channel CH3 is at the peak and the voltage ratio of the channel CH1 is equal to or greater than -0.5 at the minimum value of all the channels (step S304). If step S304 is "Yes," the operation determination unit 12C determines that the operation is a bending of the index finger (step S305).

[0077] Next, the operation determination unit 12C determines whether or not the channel CH3 is at the peak and the voltage ratio of the channel CH3 is 0.5 or more among the maximum values ​​of all the channels (step S306). If step S306 is "Yes", the operation determination unit 12C determines that the operation is a bending of the middle finger (step S307).

[0078] Next, the operation determining unit 12C determines whether or not the channel CH4 is at the peak among the maximum values ​​of all the channels (step S308). If step S308 is "Yes", the operation determining unit 12C determines that the operation is a bending of the ring finger (step S309).

[0079] In this way, the operation estimation device 1 can estimate an operation performed by bending five fingers.

[0080] It should be noted that instead of determining the extension and flexion of all five fingers, the extension and / or flexion of one or more fingers may be determined.

[0081] [4] Second Example Next, another embodiment (second embodiment) of the operation estimation operation will be described in more detail. In the second embodiment, as in the first embodiment, the operation estimation operation is performed using five channels CH1 to CH5 shown in Figures 16 to 19.

[0082] Fig. 22 is a flowchart illustrating the operation estimation operation of the operation estimation device 1 according to the second embodiment. Fig. 22 shows the process of estimating finger extension. The operation in Fig. 22 is based on the measurement signals in Figs. 16 and 17.

[0083] The operation determining unit 12C determines whether the minimum value of all channels is equal to or greater than -0.5 (step S400). If step S400 is "No", the process proceeds to step S403.

[0084] If step S400 is "Yes", the operation determining unit 12C determines whether the maximum value of channel CH5 is equal to or greater than 0.5 (step S401). If step S401 is "Yes", the operation determining unit 12C determines that the thumb is extended (step S402).

[0085] Next, the operation determining unit 12C determines whether the maximum value of channel CH1 is equal to or greater than 0.5 (step S403). If step S403 is "Yes", the operation determining unit 12C determines that the operation is an extension of the little finger (step S404).

[0086] Next, the operation determining unit 12C determines whether the minimum value of channel CH3 is equal to or greater than -0.5 (step S405). If step S405 is "Yes", the operation determining unit 12C determines that the operation is an extension of the index finger (step S406).

[0087] Next, the operation determination unit 12C determines whether or not the channel CH3 is at the peak among the minimum values ​​of all the channels (step S407). If step S407 is "Yes", the operation determination unit 12C determines that the middle finger is extended (step S408).

[0088] Next, the operation determining unit 12C determines whether or not the channel CH4 is at the peak among the minimum values ​​of all the channels (step S409). If step S409 is "Yes", the operation determining unit 12C determines that the ring finger is being extended (step S409).

[0089] In this way, the operation estimation device 1 can estimate an operation performed by extending five fingers.

[0090] Fig. 23 is a flowchart illustrating the operation estimation operation of the operation estimation device 1 according to the second embodiment. Fig. 23 shows the process of estimating finger flexion. The operation in Fig. 23 is based on the measurement signals in Figs. 18 and 19.

[0091] The operation determining unit 12C determines whether the minimum value of all channels is equal to or greater than -0.5 (step S500). If step S500 is "Yes", the process proceeds to step S507.

[0092] If step S500 is "No", the operation determining unit 12C determines whether the minimum value of channel CH5 is less than -0.5 (step S501). If step S501 is "Yes", the operation determining unit 12C determines that the thumb is bent (step S502).

[0093] Next, the operation determining unit 12C determines whether the minimum value of channel CH1 is less than −0.5 (step S503). If step S503 is “Yes”, the operation determining unit 12C determines that the operation is a bending of the little finger (step S504).

[0094] Next, the operation determining unit 12C determines whether the minimum value of channel CH3 is less than −0.5 (step S505). If step S505 is “Yes”, the operation determining unit 12C determines that the operation is a bending of the index finger (step S506).

[0095] Next, the operation determination unit 12C determines whether the maximum value of channel CH2 is equal to or greater than 0.5 (step S507). If step S507 is "Yes", the operation determination unit 12C determines that the operation is a bending of the middle finger (step S508). If step S507 is "No", the operation determination unit 12C determines that the operation is a bending of the ring finger (step S509).

[0096] In this way, the operation estimation device 1 can estimate an operation performed by bending five fingers.

[0097] It should be noted that instead of determining the extension and flexion of all five fingers, the extension and / or flexion of one or more fingers may be determined.

[0098] [5] Band position Next, the wearing position of the band including the plurality of sensor elements will be described.

[0099] Fig. 24 is a diagram illustrating multiple wearing positions of the band. In this embodiment, the band 3 is worn on the forearm so as to overlap the boundary line that divides the forearm into five equal parts from the wrist to the elbow. That is, in this embodiment, the band 3 is worn at wearing position 3 in Fig. 24. Other positions on the wrist, i.e., positions that overlap the boundary line that divides the forearm into five equal parts, are referred to as wearing position 1. A position that overlaps the boundary line that divides the forearm into five equal parts is referred to as wearing position 2. A position that overlaps the boundary line that divides the forearm into three equal parts is referred to as wearing position 4.

[0100] 25 to 28 are diagrams illustrating measurement signals at wearing position 1. FIG. 25 is an example of the maximum value of the measurement signal when five fingers are extended. FIG. 26 is an example of the minimum value of the measurement signal when five fingers are extended. FIG. 27 is an example of the maximum value of the measurement signal when five fingers are bent. FIG. 28 is an example of the minimum value of the measurement signal when five fingers are bent. Wearing position 1 is assumed to be worn on the wrist.

[0101] 25 to 28, the outputs of the middle finger and ring finger are similar when bending, and it is difficult to distinguish between the operations of the middle finger and ring finger by comparing the channel peaks. In addition, the channel with the peak, especially when bending, changed significantly for each measurement, and the output was unstable.

[0102] Figure 29 illustrates the measurement results of the minimum value when the middle finger is bent five times consecutively. Figure 29 shows that the output is not stable when the middle finger is bent. This is because a simple fixation method such as a band causes the sensor element to float significantly during flexion. In other words, measurement at the wrist requires high adhesion between the sensor element and the skin. Furthermore, the index finger, middle finger, and little finger have similar characteristics in extension, making it difficult to distinguish them based on the channel peaks using a sensor element group on the palm side only. Therefore, discrimination at the wrist requires a sensor element group on the back of the hand in addition to the palm side, and is likely to require complex calculations such as machine learning.

[0103] 30 to 33 are diagrams illustrating the measurement signal at wearing position 2. FIG. 30 is an example of the maximum value of the measurement signal when five fingers are extended. FIG. 31 is an example of the minimum value of the measurement signal when five fingers are extended. FIG. 32 is an example of the maximum value of the measurement signal when five fingers are flexed. FIG. 33 is an example of the minimum value of the measurement signal when five fingers are flexed. Wearing position 2 is closer to the wrist than wearing position 3.

[0104] 30 to 33, the output tends to be stable when the device is moved slightly away from the wrist. However, because the tendons are densely packed at this wearing position 2, the output of each finger is similar overall, and it is difficult to distinguish the operation of each finger by comparing the channel peaks.

[0105] 34 to 37 are diagrams illustrating the measurement signal at wearing position 4. FIG. 34 is an example of the maximum value of the measurement signal when five fingers are extended. FIG. 35 is an example of the minimum value of the measurement signal when five fingers are extended. FIG. 36 is an example of the maximum value of the measurement signal when five fingers are flexed. FIG. 37 is an example of the minimum value of the measurement signal when five fingers are flexed. Wearing position 4 is closer to the elbow than wearing position 3.

[0106] 34 to 37, the output of the index finger and little finger during flexion is similar. Also, because the order of the peaks does not match the muscle arrangement, factors other than muscle movement affect the output. In this case, the output pattern may change depending on the wearing conditions, so it is not suitable for stable output.

[0107] Therefore, it can be said that wearing position 3 is appropriate because it is easy to see large differences in the characteristics caused by the movements of the five fingers and allows for finger operation to be determined with simple calculations. On the other hand, wearing position 4 closer to the elbow is inappropriate because it is likely to be affected by the movement of the elbow joint and make it difficult to easily wear the band.

[0108] [6] Effects of the embodiment According to this embodiment, the band 3 including the multiple sensor elements 4 is worn on the forearm so as to overlap the boundary line between the wrist and elbow, which divides the forearm into five equal parts. The multiple sensor elements 4 are also arranged on the palm side. The multiple sensor elements 4 detect the movements of the tendons and muscles of the five fingers as pressure. This allows for accurate estimation of finger operations.

[0109] Furthermore, because the band 3 is worn at a position away from the wrist, it is less affected by wrist movement. This makes it easier to fix the band 3 to the forearm and prevents the band 3 from slipping. This allows for more accurate estimation of finger operations.

[0110] Furthermore, since the correlation between the output waveforms of the multiple sensor elements 4 and the finger movement becomes clear, the finger operation determination can be performed with simpler calculation processing. Furthermore, there is no need to use machine learning or the like to determine the finger operation.

[0111] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected elements from the disclosed elements. For example, if the problem can be solved and the desired effect can be obtained even if some elements are deleted from all elements shown in the embodiments, the configuration from which these elements are deleted can be extracted as an invention. [Explanation of symbols]

[0112] 1...operation estimation device, 2...casing, 3...band, 4...sensor element, 10...sensor unit, 11...signal processing unit, 12...control unit, 12A...operation period determination unit, 12B...voltage determination unit, 12C...operation determination unit, 13...memory unit.

Claims

1. When the forearm is divided into five equal parts from the wrist to the elbow, the band is attached to the forearm so as to overlap the boundary line that divides the forearm into two equal parts from the wrist side. a plurality of sensor elements provided within the band and positioned on the palm side of the forearm to detect pressure from the forearm; a control unit that estimates a finger operation based on a plurality of sensor signals output from the plurality of sensor elements; An operation estimation device comprising:

2. The plurality of sensor elements are configured to extend in the extension direction of the forearm and are arranged to be aligned in the circumferential direction of the forearm. The operation estimation device according to claim 1 .

3. The plurality of sensor elements are piezoelectric elements. The operation estimation device according to claim 1 .

4. the plurality of sensor elements includes five sensor elements; The five sensor elements are positioned to overlap the muscles of the little finger, ring finger, middle finger, index finger, and thumb. The operation estimation device according to claim 1 .

5. The control unit receives a plurality of measurement signals corresponding to the plurality of sensor signals, determines a plurality of maximum values ​​and a plurality of minimum values ​​for each of the plurality of measurement signals, and estimates the operation of the finger based on the plurality of maximum values ​​and the plurality of minimum values. The operation estimation device according to claim 1 .

6. the plurality of sensor elements includes a first sensor element arranged to overlap a thumb muscle; The control unit determines that the operation is performed by the thumb when a first maximum value of the first measurement signal by the first sensor element is greater than maximum values ​​by the other sensor elements. The operation estimation device according to claim 5 .

7. the plurality of sensor elements includes a second sensor element arranged to overlap a muscle of the little finger; The control unit determines that the operation is performed by the little finger when a second maximum value of the second measurement signal by the second sensor element is greater than maximum values ​​by other sensor elements. The operation estimation device according to claim 6 .

8. the plurality of sensor elements includes a third sensor element arranged to overlap a muscle of a middle finger; The control unit determines that the operation is performed by the middle finger when a third maximum value of the third measurement signal by the third sensor element is greater than maximum values ​​by the other sensor elements. The operation estimation device according to claim 7 .

9. the plurality of sensor elements includes a first sensor element arranged to overlap a thumb muscle; The control unit determines that the operation is performed by the thumb when a first maximum value of the first measurement signal by the first sensor element is equal to or greater than a threshold value. The operation estimation device according to claim 5 .

10. The bisection boundary line corresponds to the position where the flexor pollicis longus and flexor digitorum superficialis muscles of the forearm begin to overlap. The operation estimation device according to claim 1 .

Images