Operation device and operation estimation method

The operation estimation technique uses wrist-mounted sensors and a calculation section to enhance the accuracy of finger movement detection by learning from wrist tendon displacements, addressing the inaccuracy of conventional piezoelectric sensor configurations.

JP7586193B2Active Publication Date: 2024-11-19MURATA MFG CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2022569704
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2020-12-14
Filing Date
2021-08-10
Publication Date
2024-11-19
Estimated Expiration
2041-08-10

AI Technical Summary

Technical Problem

Conventional mobile terminal devices using piezoelectric sensors struggle to accurately measure finger movements.

Method used

An operation estimation technique that includes a plurality of sensors attached to the wrist, a range setting section, and a calculation section to learn and estimate finger operations with high precision by analyzing sensor signals from the displacement of the wrist tendons.

Benefits of technology

Accurately detects finger operations with increased estimation accuracy by linking wrist tendon movements to finger movements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007586193000001
    Figure 0007586193000001
  • Figure 0007586193000002
    Figure 0007586193000002
  • Figure 0007586193000003
    Figure 0007586193000003
Patent Text Reader

Abstract

An operation device (10) is provided with a plurality of sensors (201-216), a range setting unit (42), and a calculation unit (43). The plurality of sensors (201-216) are attached to wrists, and output sensor signals corresponding to movements of tendons of the wrists. The range setting unit 42 sets an operation-learning time range that includes time points corresponding to feature points in measurement signals based on the sensor signals from the plurality of sensors (201-216). The calculation unit (43) performs operation learning by using the measurement signals based on the sensor signals from the plurality of sensors in the operation-learning time range. The calculation unit (43) performs operation inference by using a reference corresponding to details of the learning.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a technique for detecting an operation from a hand movement. [Background technology]

[0002] A portable terminal device using a piezoelectric sensor is described in Patent Document 1. The portable terminal device of Patent Document 1 has a plurality of piezoelectric sensors arranged on the back side of the wrist.

[0003] The mobile terminal device of Patent Document 1 measures the movement of the user's (wearer's) finger using detection signals from a plurality of piezoelectric elements. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2005-352739 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, with a conventional configuration such as the mobile terminal device described in Patent Document 1, it has been difficult to measure finger movements with high accuracy.

[0006] Therefore, an object of the present invention is to provide an operation estimation technique that can measure finger movements (operations by fingers) with high accuracy. [Means for solving the problem]

[0007] The operation device of the present invention includes a plurality of sensors, a range setting unit, and a calculation unit. The plurality of sensors are attached to the wrist and output sensor signals corresponding to the displacement of the body surface at the wrist. The range setting unit sets a time range for operation learning including the times of feature points of the sensor signals of the plurality of sensors. The calculation unit estimates an operation using the sensor signals of the plurality of sensors in the time range for operation learning.

[0008] In this configuration, operations are learned with high accuracy using characteristic parts of the sensor signal corresponding to the displacement of the body surface at the wrist (movement of the wrist tendons), and operations are estimated using the learning results. In addition, the displacement of the body surface at the wrist (movement of the wrist tendons) is closely linked to the movement of the fingers. This increases the accuracy of estimation of finger operations. Effect of the Invention

[0009] According to this invention, an operation by a finger can be detected with high accuracy. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a functional block diagram illustrating an example of the configuration of the operation device according to the first embodiment. [Diagram 2] 2(A) and 2(B) are diagrams showing a specific configuration and an installation example of the strain sensor. [Diagram 3] FIG. 3 is a diagram illustrating an example of a waveform of a measurement signal. [Figure 4] FIG. 4 is a functional block diagram illustrating an example of the configuration of the estimation unit according to the first embodiment. [Diagram 5] FIG. 5 is a functional block diagram illustrating an example of the configuration of the index value calculation unit. [Figure 6] 6(A) and 6(B) are charts showing the concept of total activity. [Figure 7] FIG. 7 is a functional block diagram illustrating an example of the configuration of the range setting unit according to the first embodiment. [Figure 8] FIG. 8 is a waveform diagram of the total activity used for range setting. [Figure 9] FIG. 9 is a functional block diagram illustrating an example of the configuration of the calculation unit according to the first embodiment. [Figure 10] FIG. 10 is a flowchart illustrating an example of the operation estimation method according to the first embodiment. [Figure 11] FIG. 11 is a diagram for explaining the concept of estimation. [Figure 12]FIG. 12 is a diagram showing a concept of determining whether a composite operation has been performed. [Figure 13] FIG. 13 is a diagram showing a concept of determining whether a composite operation has been performed. [Figure 14] FIG. 14 is a diagram showing a concept of determining whether a composite operation has been performed. [Figure 15] FIG. 15 is a flowchart illustrating an example of a manipulation estimation method according to the first embodiment. [Figure 16] FIG. 16 is a diagram showing an example of an application target of the operation device of this embodiment. [Figure 17] FIG. 17 is a functional block diagram illustrating an example of the configuration of the operation device according to the second embodiment. [Figure 18] FIG. 18 is a functional block diagram illustrating an example of the configuration of an operation device according to the third embodiment. [Figure 19] FIG. 19 is a diagram showing an example of how the operating device according to the third embodiment is worn. [Figure 20] FIG. 20 is a functional block diagram illustrating an example of the configuration of an operation device according to the fourth embodiment. [Figure 21] FIG. 21 is a functional block diagram illustrating an example of the configuration of an operation device according to the fifth embodiment. [Figure 22] FIG. 22 is a functional block diagram showing an example of the configuration of a calculation unit that only estimates an operation. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] [First embodiment] An operation estimation technique according to a first embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a functional block diagram showing an example of the configuration of an operation device according to the first embodiment.

[0012] 1, the operating device 10 includes a strain sensor 20, a pre-processing unit 30, an estimation unit 40, and a storage unit 50. The pre-processing unit 30, the estimation unit 40, and the storage unit 50 are formed of electronic components, electronic circuits, and the like, and are built into, for example, a predetermined housing.

[0013] (Structure and Processing of Strain Sensor 20) Figures 2(A) and 2(B) are diagrams showing a specific configuration of the strain sensor and an example of how to wear it. Figure 2(A) shows the front side of the hand and wrist, and Figure 2(B) shows the back side of the hand and wrist.

[0014] As shown in Figures 2(A) and 2(B), the strain sensor 20 is worn on the wrist. The strain sensor 20 includes a plurality of sensors 201-216. The plurality of sensors 201-216 includes a configuration in which detection electrodes are arranged on a flexible piezoelectric film. The piezoelectric film is, for example, made mainly of polylactic acid and stretched in a predetermined direction.

[0015] More specifically, the multiple sensors 201-208 are attached to a wrist surface 911. The wrist surface 911 is the surface of the wrist facing the back of the hand 91. The multiple sensors 201-208 are arranged at intervals along the circumferential direction of the wrist. The multiple sensors 201-208 are attached to the wrist surface 911 such that the longitudinal direction of the piezoelectric film and the electrodes is parallel to the extension direction of the wrist tendons.

[0016] The multiple sensors 209-216 are attached to the underside 912 of the wrist. The underside 912 of the wrist is the surface of the wrist facing the palm 92. The multiple sensors 209-216 are arranged at intervals along the circumferential direction of the wrist. The multiple sensors 209-216 are attached to the underside 912 of the wrist such that the longitudinal direction of the piezoelectric film and electrodes is parallel to the direction in which the tendons of the wrist extend. The strain sensor 20 may include lead wiring and the like for outputting the acquired sensor signal to the outside, but this is omitted in FIGS. 2(A) and 2(B).

[0017] If the piezoelectric film of the sensors 201-216 is made of polylactic acid (PLLA), the stretching direction may be approximately 45° with respect to the direction in which the tendons of the wrist extend. In this embodiment, the electrode shape is not limited to a rectangle, and may be other shapes such as a square or a circle. The piezoelectric film is not limited to polylactic acid. In addition, a film-shaped piezoelectric element is preferable in terms of conformity to the body surface, but is not essential.

[0018] When a person wearing the strain sensor 20 moves his / her fingers, the wrist tendons move and the body surface is displaced in response to the finger movement. For example, when operating a virtual keyboard described below, the wrist tendons move and the body surface is displaced in response to the finger movement. The multiple sensors 201-216 of the strain sensor 20 generate and output sensor signals in response to the wrist tendon movement (more specifically, the displacement of the skin surface (displacement of the body surface) due to the tendon movement). The sensor signals are generated with amplitudes corresponding to the magnitude of the wrist tendon movement and with waveforms corresponding to the time of the wrist tendon movement. The strain sensor 20 outputs sensor signals of the multiple sensors 201-216 (sensor signals of multiple detection channels) to the front-stage signal processing unit 30.

[0019] With this configuration, the strain sensor 20 can output sensor signals of the multiple sensors 201-216 that are detected with high accuracy in response to finger movements. Furthermore, with this configuration, the strain sensor 20 has flexibility, which reduces discomfort felt by the wearer and suppresses deterioration of operability by the wearer.

[0020] (Configuration and Processing of Pre-Signal Processing Unit 30) The pre-signal processing unit 30 performs DC component removal, amplification, A / D conversion, and filtering on the sensor signals of the multiple sensors 201-216. More specifically, the pre-signal processing unit 30 performs DC component removal on the sensor signals of the multiple sensors 201-216. The pre-signal processing unit 30 amplifies the sensor signals of the multiple sensors 201-216 after the DC component removal. The pre-signal processing unit 30 performs A / D conversion (analog-to-digital conversion) on the sensor signals of the multiple sensors 201-216 after the amplification. Note that the order of the processes performed by the pre-signal processing unit 30 is not limited to the above and can be set as appropriate.

[0021] The front-stage signal processing unit 30 performs filtering on the digitalized sensor signals of the multiple sensors 201-216. The filtering is, for example, N-th order digital Butterworth low-pass filtering. The front-stage signal processing unit 30 normalizes the filtered signal. Note that the normalization is, for example, a process of unifying the reference potentials of the sensor signals of the multiple sensors 201-206. The front-stage signal processing unit 30 outputs the normalized signal to the estimation unit 40 as a measurement signal yCH(t) corresponding to the sensor signals of the multiple sensors 201-216. Note that the normalization can be omitted, but using it can suppress variations in the measurement signal yCH(t).

[0022] The measurement signal is composed of low-frequency components excluding DC components by the above-mentioned processing of the front-stage signal processor 30. Therefore, noise contained in the sensor signal can be effectively removed, and the measurement signal reflects the tendon movement with high accuracy.

[0023] Fig. 3 is a diagram showing an example of a waveform of a measurement signal. In Fig. 3, the vertical axis indicates the amplitude of the measurement signal yCH(t) for each channel, and the horizontal axis indicates the measurement time. The channels CH1-CH16 shown on the vertical axis, i.e., the measurement signals yCH1(t)-yCH16(t), correspond to the sensor signals of the multiple sensors 201-216, respectively. Moreover, the operations A, B, C, D, and E shown in Fig. 3 each indicate a case where a different finger motion is performed.

[0024] 3, the combination of waveforms of the measurement signals yCH1(t)-yCH16(t) differs depending on the operation A, B, C, D, or E. In other words, depending on the operation, the combination of waveforms of the measurement signals yCH1(t)-yCH16(t) differs. Therefore, the operation can be estimated by using the measurement signals yCH1(t)-yCH16(t).

[0025] (Configuration and Processing of Estimation Unit 40) In summary, the estimation unit 40 detects feature points of the measurement signals (sensor signals) of the multiple sensors 201-216, and estimates the operation using the measurement signals (sensor signals) in a time range for operation estimation that includes the time of the feature points. At this time, the estimation unit 40 estimates the operation using a database for estimation stored in the storage unit 50.

[0026] Furthermore, the estimation unit 40 performs learning to estimate the above-mentioned operation using measurement signals (sensor signals) of the multiple sensors 201-216.

[0027] 4 is a functional block diagram showing an example of the configuration of an estimation unit according to the first embodiment. As shown in FIG. 4, the estimation unit 40 includes an index value calculation unit 41, a range setting unit 42, and a calculation unit 43.

[0028] The index value calculation unit 41 calculates the total activity S(t), which is a range setting index, using the measurement signals yCH1(t)-yCH16(t) of the multiple sensors 201-216.

[0029] The range setting unit 42 sets a time range for learning by using the feature points of the total activity level S(t).

[0030] When estimating a manipulation, the calculation unit 43 estimates the manipulation using a database for manipulation estimation stored in the storage unit 50 and the measurement signals yCH1(t)-yCH16(t) within the time window for manipulation estimation. When learning a manipulation, the calculation unit 43 performs learning for manipulation estimation using the measurement signals yCH1(t)-yCH16(t) in the time range for learning.

[0031] More specifically, each part of the estimation unit 40 executes the following process.

[0032] Fig. 5 is a functional block diagram showing an example of the configuration of the index value calculation unit 41. Fig. 6(A) and Fig. 6(B) are charts showing the concept of total activity. Fig. 6(A) shows a state where no operation is being performed (Low state), and Fig. 6(B) shows a state where an operation is being performed (Hi state).

[0033] As shown in FIG. 5, the index value calculation unit 41 includes a chart generation unit 411 and a total activity calculation unit 412. The chart generation unit 411 generates a chart using the measurement signals yCH1(t)-yCH16(t) of the multiple sensors 201-216. The chart is a diagram in which multiple channels CH1-CH16 corresponding to the measurement signals yCH1(t)-yCH16(t) are arranged on a circumference, the absolute value of the amplitude is set to 0 (zero) at the center, and the amplitude increases as the distance from the center increases, and the amplitude (absolute value) of the measurement signals yCH1(t)-yCH16(t) is plotted for each of the channels CH1-CH16. In other words, the distance from the center means the magnitude of the measurement signals yCH1(t)-yCH16(t) in each channel.

[0034] The chart generating unit 411 generates a chart diagram for the measurement signals yCH1(t)-yCH16(t) at a predetermined time interval (sampling interval). The chart generating unit 411 outputs the generated chart diagram for each time to the total activity calculating unit 412.

[0035] The total activity calculation unit 412 calculates the inner area of ​​the chart as the total activity S(t). The inner area of ​​the chart is the area of ​​the inner region (the region toward the center) of the region obtained by sequentially connecting the plot positions (positions representing the amplitudes of the measurement signals yCH1(t)-yCH16(t)) of the channels CH1-CH16 in the chart in a circumferential manner.

[0036] As shown in Fig. 6(A), if no operation is performed, the amplitude of the measurement signals yCH1(t)-yCH16(t) is small, so the total activity S(t), which is the internal area of ​​the chart, is small. On the other hand, as shown in Fig. 6(B), if an operation is performed, the amplitude of the measurement signals yCH1(t)-yCH16(t) is large, so the total activity S(t), which is the internal area of ​​the chart, is large. Therefore, the presence or absence of an operation can be detected by using the magnitude of the total activity S(t).

[0037] The total activity calculation unit 412 calculates the total activity S(t) for each time interval (the sampling interval for creating the chart) at which the chart generation unit 411 generates the chart. The total activity calculation unit 412 outputs the calculated total activity S(t) to the range setting unit 42.

[0038] (Configuration and processing of range setting unit 42) The range setting unit 42 is primarily used during learning.

[0039] Fig. 7 is a functional block diagram showing an example of the configuration of a range setting unit according to the first embodiment Fig. 8 is a waveform diagram obtained by fitting the total activity used for range setting with a Gaussian function.

[0040] As shown in FIG. 7, the range setting section 42 includes a Gaussian function fitting section 421, a peak detection section 422, and a start / end time determination section 423.

[0041] The Gaussian function fitting unit 421 fits the total activity S(t), which is a time function, with a Gaussian function that indicates a normal distribution. This suppresses noise contained in the total activity S(t), and the total activity S(t) has a waveform as shown in Fig. 8, with the peaks of the waveform becoming clearer.

[0042] Also, it is possible to extract only an arbitrary section centered on the peak of the waveform and use it for identification. A calculation unit 43 described later uses the signal obtained by the finger movement and the learning result to determine the identification operation. In order to accurately identify each movement, an appropriate section for extracting the measurement signal yCH(t) must be determined. For this reason, by using the time waveform (time function) of the total activity S(t) fitted with a Gaussian function, an appropriate section can be determined, and the identification operation described later can be accurately determined.

[0043] The Gaussian function fitting unit 421 outputs the total activity S(t) after the Gaussian function fitting to the peak detection unit 422.

[0044] Peak detection unit 422 detects the peak (maximum point) and its time of the total activity S(t) after Gaussian function fitting. For example, in the example of Fig. 8, peak detection unit 422 detects peak value a1 and peak value a2. These peak values ​​a1 and a2 correspond to the "characteristic point" of the present invention.

[0045] The peak detection unit 422 detects the peak time tp1 of the peak value a1 and the peak time tp2 of the peak value a2. The peak detection unit 422 outputs the peak time tp1 and the peak time tp2 of the total activity S(t) to the start / end time determination unit 423.

[0046] The start / end time determination unit 423 uses the peak time tp1 and the peak time tp2 to determine the start time and end time for determining the time range for operation estimation.

[0047] More specifically, the start / end time determination unit 423 sets a range setting time d1 for the peak time tp1. The range setting time d1 is set based on, for example, the spread (variance, etc.) of the waveform of the total activity S(t) at the point where the peak value a1 occurs. The start / end time determination unit 423 subtracts the range setting time d1 from the peak time tp1 to set a learning range start time t1s for the peak value a1. The start / end time determination unit 423 adds the range setting time d1 to the peak time tp1 to set a learning range end time t1e for the peak value a1. Then, the start / end time determination unit 423 sets the time from the learning range start time t1s to the learning range end time t1e as a time range PD1 for learning estimation.

[0048] Similarly, the start / end time determination unit 423 sets a range setting time d2 for the peak time tp2. The range setting time d2 is set based on, for example, the spread (variance, etc.) of the waveform of the total activity S(t) at the point where the peak value a2 occurs. The start / end time determination unit 423 subtracts the range setting time d2 from the peak time tp2 to set a learning range start time t2s for the peak value a2. The start / end time determination unit 423 adds the range setting time d2 to the peak time tp2 to set a learning range end time t2e for the peak value a2. Then, the start / end time determination unit 423 sets the time from the learning range start time t2s to the learning range end time t2e as a time range PD2 for learning estimation.

[0049] When identifying using multiple feature points from multiple actions, fitting is performed with a function consisting of the sum of multiple Gaussian functions to determine the range for extracting the measurement signal yCH(t).As an example, when identifying one action using two feature points from two actions shown in Figure 8, fitting is performed with a function consisting of the sum of Gaussian functions of two waveforms to determine the range setting times d1 and d2.

[0050] The start / end time determination unit 423 outputs to the calculation unit 43 the time range PD1 for learning estimation and the time range PD2 for learning estimation.

[0051] (Configuration and processing of calculation unit 43) 9 is a functional block diagram showing an example of the configuration of the calculation unit according to the first embodiment. As shown in FIG. 9, the calculation unit 43 includes a plurality of classifiers 4311 and 4312, a determination unit 432, and a learning unit 433.

[0052] (Study time) Measurement signals yCH1(t)-yCH16(t) of the multiple sensors 201-216 and the learning time ranges PD1 and PD2 are input to the discriminator 4311 and the discriminator 4312. The discriminator 4311 and the discriminator 4312 use the measurement signals yCH1(t)-yCH16(t) within the learning time ranges PD1 and PD2 to acquire a reference signal for discriminating between operations.

[0053] The discriminator 4311 and the discriminator 4312 acquire reference signals under different conditions. That is, the discriminator 4311 and the discriminator 4312 acquire reference signals used for operation estimation in different categories.

[0054] For example, the classifier 4311 obtains a reference signal for individually identifying the five fingers, and the classifier 4312 obtains a reference signal for identifying whether the fingers are raised or lowered.

[0055] The discriminator 4311 and the discriminator 4312 output the acquired reference signals to the learning unit 433 .

[0056] The learning unit 433 associates the acquired reference signal with the type of fingers and the finger motion corresponding to this reference signal, and stores them in the storage unit 50.

[0057] This allows the calculation unit 43 to learn a reference signal according to the type of fingers and the finger movements. During such learning, as described above, by using the measurement signals yCH1(t)-yCH16(t) in the learning time range PD1 and the learning time range PD2, learning can be performed using the measurement signals yCH1(t)-yCH16(t) suitable for learning. This improves the learning accuracy.

[0058] Furthermore, the learning unit 433 can adapt the threshold value Th(t) for detecting a motion during estimation based on the learned reference signal, etc. This allows motion to be detected more accurately during estimation, and ultimately improves the estimation accuracy.

[0059] (How to learn how to operate) FIG. 10 is a flowchart showing an example of the operation learning method according to the first embodiment.

[0060] The operation device 10 generates sensor signals corresponding to the movement of the tendons of the wrist (displacement of the skin surface) caused by the operation of the fingers using the multiple sensors 201-216 (S11). The operation device 10 generates measurement signals yCH1(t)-yCH16(t) using the sensor signals of the multiple sensors (S12).

[0061] The operation device 10 calculates a total activity S(t) which is a range setting index (index value) using measurement signals of a plurality of sensors (S13). The operation device 10 detects feature points of the range setting index from the time characteristics of the range setting index, and sets a time range for learning (S14). The operation device 10 learns the operation using the measurement signals yCH1(t)-yCH16(t) in the time range for learning (S15).

[0062] (at the time of estimation) (1) When performing operation estimation (identification, judgment) by setting a time range for operation estimation using Gaussian function fitting The Gaussian function fitting unit 421 fits the total activity S(t), which is a time function, with a Gaussian function that indicates a normal distribution. This suppresses noise contained in the total activity S(t), and the total activity S(t) has a waveform as shown in Fig. 8, with clear peaks in the waveform.

[0063] The Gaussian function fitting section 421 outputs the total activity S(t) after the Gaussian function fitting to the peak detection section 422.

[0064] Peak detection unit 422 detects the peak (maximum point) and its time of the total activity S(t) after Gaussian function fitting. For example, in the example of Fig. 8, peak detection unit 422 detects peak value a1 and peak value a2. These peak values ​​a1 and a2 correspond to the "characteristic point" of the present invention.

[0065] The peak detection unit 422 detects the peak time tp1 of the peak value a1 and the peak time tp2 of the peak value a2. The peak detection unit 422 outputs the peak time tp1 and the peak time tp2 of the total activity S(t) to the start / end time determination unit 423.

[0066] The start / end time determination unit 423 uses the peak time tp1 and the peak time tp2 to determine the start time and end time for determining the time range for operation estimation. More specifically, the start / end time determination unit 423 sets the range setting time d1 at the peak time tp1. The range setting time d1 is set based on, for example, the spread (variance, etc.) of the waveform of the total activity S(t) at the point where the peak value a1 occurs. The start / end time determination unit 423 subtracts the range setting time d1 from the peak time tp1 to set the estimated range start time t1s for the peak value a1. The start / end time determination unit 423 adds the range setting time d1 to the peak time tp1 to set the estimated range end time t1e for the peak value a1. Then, the start / end time determination unit 423 sets the time from the estimated range start time t1s to the estimated range end time t1e as the time range PD1 for operation estimation.

[0067] Similarly, the start / end time determination unit 423 sets a range setting time d2 at the peak time tp2. The range setting time d2 is set based on, for example, the spread (variance, etc.) of the waveform of the total activity S(t) at the point where the peak value a2 occurs. The start / end time determination unit 423 subtracts the range setting time d2 from the peak time tp2 to set an estimated range start time t2s for the peak value a2. The start / end time determination unit 423 adds the range setting time d2 to the peak time tp2 to set an estimated range end time t2e for the peak value a2. Then, the start / end time determination unit 423 sets the time from the estimated range start time t2s to the estimated range end time t2e as a time range PD2 for operation estimation.

[0068] When identifying using multiple feature points from multiple actions, fitting is performed with a function consisting of the sum of multiple Gaussian functions to determine the range for extracting the measurement signal yCH(t).As an example, when identifying one action using two feature points from two actions shown in Figure 8, fitting is performed with a function consisting of the sum of Gaussian functions of two waveforms to determine the range setting times d1 and d2.

[0069] The start / end time determination unit 423 outputs to the calculation unit 43 the time range PD1 for operation estimation and the time range PD2 for operation estimation.

[0070] The discriminator 4311 and the discriminator 4312 discriminate the operations using the measurement signals yCH1(t)-yCH16(t) within the time range PD1 for operation estimation and the time range PD2 for operation estimation.

[0071] The classifier 4311 and the classifier 4312 classify operations under different conditions. That is, the classifier 4311 and the classifier 4312 perform classification for use in operation estimation in different categories. The classification conditions and classification criteria for the classification conditions are stored in the storage unit 50 and are information that has been learned in advance. Note that, even during the advance learning, the same method as that used during the above-mentioned classification is used to set the time range of the learning data.

[0072] For example, the classifier 4311 performs classification of the five fingers. Specifically, reference signals (learning information) of the measurement signals yCH1(t)-yCH16(t) corresponding to the movements of the five fingers obtained by the above-mentioned learning are stored in the storage unit 50. The classifier 4311 compares the measurement signals yCH1(t)-yCH16(t) with the reference signals, and classifies the fingers that are likely to have been moved based on the comparison results.

[0073] On the other hand, the classifier 4312 executes classification of finger lifting and lowering. Specifically, reference signals (learning information) of the measurement signals yCH1(t)-yCH16(t) corresponding to the finger lifting and lowering movements obtained by the above-mentioned learning are stored in the storage unit 50. The classifier 4312 compares the measurement signals yCH1(t)-yCH16(t) with the reference signals, and classifies the movement that is likely to have been made from the comparison result.

[0074] The discriminators 4311 and 4312 output the discrimination results to the decision section 432 .

[0075] The determination unit 432 determines the operation using the classification results of the classifier 4311 and the classifier 4312. For example, the determination unit 432 determines which finger has moved in which direction using the classification result of the classifier 4311 on the five fingers and the classification result of the classifier 4312 on the up-down movement.

[0076] In this way, by using the configuration and processing of this embodiment, the operation device 10 can estimate the operation by the finger. At this time, as described above, the estimation is performed using the measurement signals yCH1(t)-yCH16(t), that is, the sensor signals, including the feature points indicating that an operation has been performed, and the portions where the amplitude corresponding to the operation is obtained (the time range PD1 for operation estimation and the time range PD2 for operation estimation). As a result, the operation device 10 uses the measurement signals (sensor signals) in a range that has a large effect on improving the accuracy of the estimation, and does not use the measurement signals (sensor signals) in a range that has almost no effect on improving the accuracy of the estimation or that may cause an error. Therefore, the operation device 10 can estimate the operation by the finger with high accuracy.

[0077] In addition, in this configuration and processing, the operation device 10 uses multiple classifiers to classify operations for each category, and then estimates the operation in an integrated manner. This allows the operation device 10 to reduce the load on each classifier for classification, and to perform classification more reliably and quickly. Therefore, the operation device 10 can estimate the operation more reliably and quickly.

[0078] In addition, in this configuration and processing, multiple sensors are attached to both the front surface 911 and the back surface 912 of the wrist. This makes it possible to more accurately detect the movement of the wrist tendons (displacement of the skin surface) caused by finger manipulation than if multiple sensors were attached only to the front surface 911 or only to the back surface 912 of the wrist. Therefore, the operating device 10 can estimate the finger manipulation with higher accuracy.

[0079] (2) When performing operation estimation (classification, judgment) without using operation estimation time by Gaussian function fitting Fig. 11 is a diagram for explaining the concept of estimation. In Fig. 11, the horizontal axis is time, the vertical axis is the value of total activity S(t), the solid line is the time characteristic of total activity S(t), the dotted line is the time characteristic of threshold Th(t), and each section set by the dashed lines is a plurality of time windows PWA, PWB, PWC, PED, PWG, PWH, PWI, and PWJ.

[0080] The calculation unit 43 sets a plurality of time windows for estimation (identification). The plurality of time windows are set to have a predetermined time length. The time length of the time window is longer than the sampling period in which identification is performed over time. In other words, the time length of the time window is set so that identification is performed multiple times during the time of one time window.

[0081] Furthermore, the multiple time windows are set in a predetermined arrangement on the time axis. For example, in the case of FIG. 11, adjacent time windows on the time axis partially overlap. Specifically, the time window PWA and the time window PWB are set so that the latter half of the time window PWA overlaps with the first half of the time window PWB. The time windows PWC and after are set in the same manner. For example, if the time length of the multiple time windows is 50 msec., adjacent time windows are set with a time shift of 25 msec.

[0082] Note that the time lengths and arrangement (degree of overlap) of the multiple time windows are not limited to this, and adjacent time windows do not have to overlap.

[0083] The classifiers 4311 and 4312 compare the total activity level S(t) with a threshold Th(t) for detecting a movement at each classification timing. If the total activity level S(t) is equal to or greater than the threshold Th(t), the classifiers 4311 and 4312 set a flag indicating that a movement has been made. If the total activity level S(t) is less than the threshold Th(t), the classifiers 4311 and 4312 set a flag indicating that a movement has not been made.

[0084] Furthermore, the discriminators 4311 and 4312 compare the signal with the above-mentioned reference signal at the timing when a flag indicating the presence of an action is set, and discriminate the operation.

[0085] The discriminator 4311 and the discriminator 4312 output a flag indicating the presence or absence of a motion and the discriminated operation to the determination unit 432.

[0086] The determination unit 432 determines the classified operation individually for each output of the discriminator 4311 and the discriminator 4312. In the following, the case of the discriminator 4311 is shown as an example, but the same applies to the case of the discriminator 4312.

[0087] The determination unit 432 separates, for each of a plurality of time windows, the flags indicating the presence or absence of a motion and the discrimination results of an operation sequentially obtained from the discriminator 4311. The determination unit 432 classifies the flags indicating the presence or absence of a motion and the discrimination results of an operation for each of the plurality of time windows.

[0088] If the flag indicates the presence of an action and the operation identification results match at all identification timings within the time window, the determination unit 432 determines the operation identification result for this time window.

[0089] 11, for example, the time window PWB and the time window PWC have an action flag at all identification timings. In this case, if all identification results in the time window PWB are operation A, the estimated result of the operation for the time window PWB is operation A. Similarly, if all identification results in the time window PWC are operation A, the estimated result of the operation for the time window PWC is operation A.

[0090] 11, the time windows PWH and PWI have the action flag at all identification timings. In this case, if all identification results in the time window PWH are operation B, the estimated operation result for the time window PWH is operation B. Similarly, if all identification results in the time window PWI are operation B, the estimated operation result for the time window PWI is operation B.

[0091] On the other hand, when a time window contains both an action flag and an action no flag, the determination unit 432 discards the operation identification result for this time window even if there is an action flag. In other words, the determination unit 432 determines that there is no identification result for this time window.

[0092] 11, the time window PWJ includes both action flags and no action flags. In this case, even if the action flag timing identification result within the time window PWJ is operation B, there will be no identification result for the time window PWJ.

[0093] Furthermore, if the operation classification results do not match at all classification timings within the time window, even if the flag indicates the presence of an action, the determination unit 432 discards these classification results. In other words, the determination unit 432 determines that there is no classification result for this time window.

[0094] For example, in the case of Fig. 11, the time window PWI has a flag indicating that an action has been performed at all identification timings. In this case, if the identification results in the time window PWI include a mixture of operation B and other operations, there will be no identification result for the time window PHI.

[0095] Furthermore, if the no-motion flag is present at all identification timings within the time window, the determination unit 432 determines that there is no identification result for this time window.

[0096] For example, in the case of Fig. 11, the time windows PWA, PWD, and PWG have no-motion flags at all identification timings, so there is no identification result for the time windows PWA, PHD, and PWG.

[0097] By performing such processing, the calculation unit 43 can estimate the operation. The calculation unit 43 can estimate the operation without setting the operation estimation time by Gaussian function fitting. This allows the calculation unit 43 to estimate the operation at a higher speed.

[0098] At this time, the reference signal and the threshold value Th(t) for estimation are set using the time range for learning estimation set by Gaussian function fitting as described above. Therefore, the comparison target used for estimation has high accuracy, and the calculation unit 43 can realize accurate estimation.

[0099] Furthermore, this method can be used to estimate composite operations. A composite operation is a combination of multiple operations that can be identified as a specific operation. For example, (lowering a finger) + (lowering a finger and then lifting the same finger) = (clicking operation). In this case, the time from (lowering a finger) to (lifting a finger) is also a determining factor for identifying a specific operation.

[0100] Figures 12, 13, and 14 are diagrams showing the concept of determining a composite operation. In Figures 12, 13, and 14, each frame represents a time window. Also, the hatched time windows indicate the operation identification results obtained as the time windows, and the operation contents differ depending on the type of hatching.

[0101] 12, the same operation (e.g., operation A) is identified in the time window PWB and the time window PWC. In this case, the determination unit 432 adopts the identification result of the time window PWB that first identified this operation (e.g., operation A). Then, the determination unit 432 discards the identification result of the time window PWC that follows the time window PWB.

[0102] Also, the same operation (e.g., operation B) is identified in the time window PWH and the time window PWI. In this case, the determination unit 432 adopts the identification result of the time window PWH that first identified this operation (e.g., operation B). Then, the determination unit 432 discards the identification result of the time window PWI that follows the time window PWH.

[0103] Then, the determination unit 432 determines a specific operation by combining the identification result of the time window PWB (operation A) and the identification result of the time window PWH (operation B). For example, if operation A is (lowering the right index finger) and operation B is (raising the right index finger), the determination unit 432 determines that it is (a click operation by the right index finger) based on these identification results.

[0104] At this time, the determination unit 432 counts time from the time window PWB as a starting point, and if the identification result of the next operation is not obtained within the determination hold time according to the identification of the characteristic operation, the determination unit 432 determines the operation identified in the time window PWB as a single operation. That is, if the identification result of the next operation is not obtained within the time according to the identification of the specific operation for the time window that is the starting point of the specific operation, the determination unit 432 discards the identification result of the operation identified in the time window set as the starting point. Note that if the identification result of the next operation is not obtained within the time according to the identification of the specific operation, it is also possible to detect the operation identified in the time window set as the starting point as a single operation.

[0105] The criteria for determining such a specific operation can be learned in the same manner as the learning of the individual operations described above, and are stored in the storage unit 50. The determination unit 432 refers to the stored contents to determine whether or not the specific operation is performed.

[0106] 13, the same operation (e.g., operation A) is identified in the time window PWB and the time window PWE. In this case, the determination unit 432 adopts the identification result of the time window PWE that last identified this operation (e.g., operation A). Then, the determination unit 432 discards the identification result of the time window PWB. In other words, when the same operation is identified in multiple time windows that are not adjacent to each other on the time axis, the determination unit 432 adopts the identification result of the time window in which the operation was last identified.

[0107] 13, an operation (e.g., operation B) different from that in the time window PWE is identified in the time window PWH. Since there is no identification result of the same operation as that in the time window PWH within a predetermined time before and after the time window PWH, the determination unit 432 adopts the identification result of the time window PWH.

[0108] Then, the determination unit 432 combines the identification result of the time window PWE (operation A) with the identification result of the time window PWH (operation B) to determine a specific operation.

[0109] In the case of Fig. 14, the same operation (e.g., operation A) is identified in the time window PWB and the time window PWH. In the time window PWE, since a part of the time window is a no-action flag, even if an operation different from the time window PWB and the time window PWH (e.g., operation B) is performed, no identification result is obtained.

[0110] In this case, the determination unit 432 performs composite operation determination based on the identification results of the time window PWB and the time window PWH. The time window PWB and the time window PWH are the same identification results, and are spaced apart on the time axis. Therefore, the determination unit 432 adopts the identification result of the time window PWH (operation A) and discards the identification result of the time window PWB. Then, the determination unit 432 holds the identification result of the time window PWH for a determination reservation time, and reserves the determination of the specific operation.

[0111] By performing such processing, the calculation unit 43 can identify (estimate) a composite operation. In this case, the operation can be estimated without setting an operation estimation time by Gaussian function fitting. This allows the calculation unit 43 to estimate the operation at a higher speed. Also, as described above, the comparison target used for estimation has high accuracy, and the calculation unit 43 can achieve accurate estimation.

[0112] In the above embodiment, the number of sensors is 16. However, the number of sensors is not limited to 16, and may be any number. For example, the number of sensors may be set to a predetermined number based on the number of fingers to detect the operation, the type of finger movement to be estimated, etc.

[0113] In the above embodiment, a chart is used to calculate the total activity S(t). However, the total activity S(t) can be calculated by using the sum of the amplitudes of the measurement signals yCH1(t)-yCH16(t).

[0114] In the above embodiment, two classifiers are used. However, the number of classifiers is not limited to this, and may be set appropriately according to the classification conditions. For example, in order to classify horizontal movement as a finger movement in addition to vertical movement, the operation device may further add a classifier for classifying horizontal movement. It is also possible to perform all classifications using one classifier.

[0115] (Operation estimation method) Fig. 15 is a flowchart showing an example of a manipulation estimation method according to the first embodiment. The process shown in Fig. 15 shows a case where the above-mentioned time window is used. The estimation method using the time range for manipulation estimation using Gaussian fitting can be realized by replacing the learning part in the learning method shown in Fig. 10 with estimation.

[0116] The operation device 10 generates sensor signals corresponding to the movement of the tendons of the wrist (displacement of the skin surface) caused by the operation of the fingers using the multiple sensors 201-216 (S21). The operation device 10 generates measurement signals yCH1(t)-yCH16(t) using the sensor signals of the multiple sensors (S22).

[0117] The operation device 10 calculates a total activity level S(t), which is a range setting index (index value), using measurement signals from a plurality of sensors (S23). The operation device 10 sets a time window for estimation (S24). The operation device 10 estimates an operation using measurement signals yCH1(t)-yCH16(t) in the time window for estimation (S25).

[0118] In the operation estimation (S25), as described above, it is also possible to estimate a composite operation from the identification results at multiple times, and further from the temporal connection of the multiple identification results. In other words, when multiple identification results satisfy the condition indicating one (one type) operation, this one operation is estimated using these multiple identification results. For example, when a down movement of a certain finger is identified and then a lifting of the same finger is identified, a tap operation is estimated.

[0119] On the other hand, if multiple identification results do not satisfy the condition that indicates one (one type) operation, these multiple identification results are each treated as an individual identification result and each individual operation is inferred. For example, if a down movement of a finger is identified followed by a lifting of another finger, these are inferred as separate operations.

[0120] (An example of what manipulation presumption applies to) Fig. 16 is a diagram showing an example of an application target of the operation device of this embodiment. In Fig. 16, each hatched circle indicates a default position PD of a finger. As shown in Fig. 16, the operation of a finger estimated by the operation device 10 can be used for inputting to a virtual keyboard 29, for example.

[0121] Specifically, a plurality of virtual keys 290 are arranged on the virtual keyboard 29. Coordinates are set for each of the plurality of virtual keys 290. A default position PD for each finger is set on the virtual keyboard 29. A default position is set for each finger, that is, for each of the five fingers of the right hand 90R and the five fingers of the left hand 90L. These default positions PD are set, for example, by prior learning or the like. The moved fingers and their movements are estimated by the operation device 10. This movement is assigned to the movement of the fingers operating the virtual keyboard 29, the pressing of the keys, and the like. This makes it possible to estimate and detect which virtual key 290 has been pressed on the virtual keyboard 29.

[0122] As a result, even if there is no physical character keyboard, the operation device 10 detects finger movements and operations in the air or on a desk, etc., and allows characters to be input to an electronic device (such as a smartphone or a PC) paired with the operation device 10. In other words, the operation device 10 functions as an input device.

[0123] [Second embodiment] The operation estimation technique according to the second embodiment of the present invention will be described with reference to Fig. 17. Fig. 17 is a functional block diagram showing an example of the configuration of an operation device according to the second embodiment.

[0124] 17, the manipulation device 10A according to the second embodiment differs from the manipulation device 10 according to the first embodiment in that an IMU sensor 60 is added and in the processing of an estimation unit 40A. Other configurations of the manipulation device 10A are similar to those of the manipulation device 10, and a description of similar parts will be omitted.

[0125] The operation device 10A includes an estimation unit 40A, a storage unit 50A, and an IMU sensor 60. The IMU sensor 60 includes a three-axis acceleration sensor, a three-axis angular velocity sensor, and the like. The IMU sensor 60 is attached to the wrist and measures the movement of the wrist. The IMU sensor 60 outputs an IMU measurement signal to the estimation unit 40A.

[0126] The estimation unit 40A estimates the operation by the finger using the IMU measurement signal together with the measurement signals yCH1(t)-yCH16(t) of the multiple sensors 201-216. At this time, the storage unit 50A stores a reference signal for the IMU measurement signal and a criterion for operation estimation for the IMU measurement signal. The estimation unit 40A refers to the reference signal and the criterion for operation estimation stored in the storage unit 50A and estimates the operation by the finger using the IMU measurement signal.

[0127] In this case, the estimation unit 40A can, for example, provide a classifier for the IMU measurement signal separate from the classifiers for the measurement signals yCH1(t)-yCH16(t) of the multiple sensors 201-216. By providing separate classifiers for these, it is possible to reduce the load on each classifier and improve the accuracy of operation estimation.

[0128] [Third embodiment] An operation estimation technique according to a third embodiment of the present invention will be described with reference to the drawings. Fig. 18 is a functional block diagram showing an example of the configuration of an operation device according to the third embodiment. Fig. 19 is a diagram showing an example of how the operation device according to the third embodiment is worn.

[0129] 18, the operation device 10B according to the third embodiment differs from the operation device 10A according to the second embodiment in that it includes an application execution unit 71 and a display unit 72. Other configurations of the operation device 10B are similar to those of the operation device 10, and a description of similar parts will be omitted. Note that the estimation unit 40B and the storage unit 50B of the operation device 10B are similar to the estimation unit 40A and the storage unit 50A of the operation device 10A, and a description thereof will be omitted.

[0130] The operation device 10B includes an application execution unit 71 and a display unit 72. The application execution unit 71 is configured with, for example, a CPU, a memory in which an application executed by the CPU is stored, etc. The application execution unit 71 receives an operation estimation result.

[0131] The application execution unit 71 executes, for example, a document creation application, an email application, an SNS application, etc. At this time, the application execution unit 71 estimates character input from the key operation status detected by the operation estimation result, and reflects it in various applications. The application execution unit 71 outputs the execution result of the application to the display unit 72. The display unit 72 displays the execution result of the application.

[0132] In such an embodiment, for example, as shown in Fig. 19, the operating device 10B has a structure like a smart watch. That is, as shown in Fig. 19, the operating device 10B has a housing 700. The housing 700 is sized so that it can be worn on the wrist. The housing 700 is mounted on the strain sensor 20 and connected to the sensor 20.

[0133] A display unit 72 is disposed on the surface of the housing 700. The functional units of the operation device 10B other than the strain sensor 20 and the display unit 72 are housed within the housing 700.

[0134] [Fourth embodiment] An operation estimation technique according to a fourth embodiment of the present invention will be described with reference to Fig. 20. Fig. 20 is a functional block diagram showing an example of the configuration of an operation device according to the fourth embodiment.

[0135] 20, the operation device 10C according to the fourth embodiment differs from the operation device 10 according to the first embodiment in that it includes a wireless communication unit 81 and a wireless communication unit 82. Other configurations of the operation device 10C are similar to those of the operation device 10, and a description of similar parts will be omitted.

[0136] The operation device 10C includes a wireless communication unit 81 and a wireless communication unit 82. The wireless communication unit 81 is connected to the output side of the signal pre-processing unit 30. The wireless communication unit 82 is connected to the input side of the estimation unit 40.

[0137] The wireless communication unit 81 transmits measurement signals yCH1(t)-yCH16(t) of the multiple sensors 201-216 to the wireless communication unit 82. The wireless communication unit 82 outputs the received measurement signals yCH1(t)-yCH16(t) to the estimation unit 40.

[0138] With this configuration, the controller 10C can separate the configuration for generating the measurement signals yCH1(t)-yCH16(t) from the configuration for estimating the operation. This allows the portion worn on the wrist to be smaller, and the controller 10C can further reduce discomfort to the wearer and further improve operability.

[0139] Although the position of the portion separated wirelessly is not limited to that in this embodiment, for example, in the configuration of this embodiment, measurement signals yCH1(t)-yCH16(t) that have relatively clear waveforms are transmitted and received as digital signals. Therefore, the occurrence of erroneous estimation due to noise can be suppressed more than when sensor signals are transmitted and received.

[0140] [Fifth embodiment] An operation estimation technique according to a fifth embodiment of the present invention will be described with reference to Fig. 21. Fig. 21 is a functional block diagram showing an example of the configuration of an operation device according to the fifth embodiment.

[0141] 21, the operation estimation system 1 includes a manipulation device 10D and an operation target device 2. The manipulation device 10D differs from the manipulation device 10 according to the first embodiment in that the manipulation device 10D includes a communication unit 70. Other configurations of the manipulation device 10D are similar to those of the manipulation device 10, and descriptions of similar parts will be omitted.

[0142] The communication unit 70 is connected to the output side of the estimation unit 40, and receives an estimation result of the operation from the estimation unit 40. The communication unit 70 has, for example, a wireless communication function and is capable of communicating with the operation target device 2. The communication unit 70 transmits the estimation result of the operation to the operation target device 2.

[0143] The operation target device 2 executes a predetermined application (for example, an application executed by the application execution unit 71 shown in the above-described embodiment) using the operation estimation result.

[0144] In this way, the finger operation estimation described above is not limited to being used by a single device, but can also be used as a system.

[0145] In the above description, the operation device has both the "learning" function and the "estimation" function. However, the operation device may have only the "estimation" function. FIG. 22 is a functional block diagram showing an example of the configuration of a calculation unit that only estimates the operation.

[0146] 22, a calculation unit 43ES of an operation device that does not have a learning function and only performs estimation includes a discriminator 4311, a discriminator 4312, and a determination unit 432. That is, the calculation unit 43ES is a unit in which the learning unit 433 in the above-mentioned calculation unit 43 is omitted.

[0147] In this case, the learning is performed by another operation device having a similar configuration to this operation device and including at least a learning unit 433. Then, the operation device having only an estimation function stores the learning result in advance in the storage unit 50, and the operation device having only an estimation function uses the stored learning result to estimate the operation.

[0148] Furthermore, if an operating device having only an estimation function has a communication function with the outside, the operating device having only an estimation function can appropriately acquire learning results stored in an external server, etc., and estimate operations.

[0149] The above-mentioned embodiments have been described with a focus on inputting operation of keys or the like using fingers. However, the configuration and processing of each embodiment of the present invention are not limited to key input. For example, the present invention can be applied to devices in other fields, such as game machines operated by moving fingers.

[0150] Furthermore, the configurations and processes of the above-described embodiments can be combined as appropriate, and effects corresponding to each combination can be achieved. [Explanation of symbols]

[0151] 1: Operation estimation system 2: Device to be operated 10, 10A, 10B, 10C, 10D: Operating device 20: Strain sensor 29: Virtual keyboard 30: Front-end signal processing section 40:Estimation part 40A: Estimation section 40B: Estimation section 41: Index value calculation unit 42: Range setting section 43, 43ES: Arithmetic section 50, 50A, 50B: Storage section 60: IMU sensor 70: Communications Department 71: Application execution unit 72: Display section 81, 82: Wireless communication section 90L: Left hand 90R:Right hand 91: Instep 201-216: Sensors 290: Virtual key 411: Chart generation unit 412: Total activity calculation unit 421: Gaussian function fitting part 422: Peak detector 423: Start and end time determination unit 432: Judgment section 700: Chassis 911: Surface 912: Back side 4311, 4312: Classifier

Claims

1. A plurality of sensors are attached to the wrist and output sensor signals corresponding to the displacement of the body surface at the wrist; a range setting unit which sets a time range for operation learning based on the spread of the waveform of the total activity, including a time of the feature point, using a peak value of the waveform of a time function of the total activity, which is the internal area of ​​a chart generated using the amplitude values ​​of a plurality of sensor signals from the plurality of sensors, as a feature point; a calculation unit that learns an operation by using sensor signals of the plurality of sensors within a time range for the operation learning; An operating device comprising:

2. an index value calculation unit that sets the total activity as a range setting index, The range setting unit is a time range for the operation learning is set by setting the feature points of the range setting indicators as feature points of the plurality of sensor signals; The operating device according to claim 1 .

3. The index value calculation unit calculates a sum of amplitude values ​​of the plurality of sensor signals of the plurality of sensors as the range setting index. The operating device according to claim 2 .

4. The range setting unit is detecting the feature points from the time characteristics of the range setting indicators; The operating device according to claim 2 or 3.

5. The range setting unit is Detecting a peak value of the range setting index as the feature point; The operating device according to claim 4.

6. The range setting unit is a time range for operation learning is set to a predetermined time range including the time of the characteristic point based on the time of the characteristic point; The operating device according to any one of claims 2 to 5.

7. The range setting unit is A predetermined time range including the time of the feature point is set by the spread of the time characteristic of the range setting indicator. The operating device according to claim 6.

8. the range setting unit performs fitting of the time characteristics of the range setting indicator based on a normal distribution, and then detects the feature points and sets the time range for the operation learning. The operating device according to any one of claims 2 to 7.

9. A plurality of sensors are attached to the wrist and output sensor signals corresponding to the displacement of the body surface at the wrist; a range setting unit that sets a time range for estimating an operation based on a spread of the waveform of the total activity, the time range including a time of the feature point, and a peak value of the waveform of the total activity, the peak value being an internal area of ​​a chart generated using amplitude values ​​of a plurality of sensor signals from the plurality of sensors, as a feature point; a calculation unit that estimates an operation by using the plurality of sensor signals of the plurality of sensors in the time range for estimating the operation; An operating device comprising:

10. A plurality of sensors are attached to the wrist and output sensor signals corresponding to the displacement of the body surface at the wrist; a total activity calculation unit that calculates a total activity, which is an internal area of ​​a chart that is generated using amplitude values ​​of the plurality of sensor signals of the plurality of sensors; a range setting unit that sets a time window for estimating an operation based on a spread of the waveform of the total activity, the time including a feature point represented by a peak value of the waveform of the time function of the total activity; a calculation unit that estimates an operation by using the total activity within the time window and the plurality of sensor signals; An operating device comprising:

11. The calculation unit is Inferring the operation using a magnitude of a total activity degree at a plurality of times within the time window and a result of identifying the operation based on the plurality of sensor signals. The operating device according to claim 10.

12. The calculation unit is If the total activity level of all the time periods within the time window is equal to or greater than the threshold for detecting the motion and the identification results of all the time periods are the same, the identification results are determined to be the operation in the time window. The operating device according to claim 11.

13. The calculation unit is If the identification results of the operation are the same in a plurality of consecutive time windows, the identification result of the first time window of the plurality of consecutive time windows is kept, and the identification results of the other time windows are discarded. The operating device according to any one of claims 10 to 12.

14. The calculation unit is If the identification results of the operation are the same in a plurality of non-consecutive time windows during a retention time of the identification results in the plurality of time windows, the identification result of the last time window is retained and the identification results of the other time windows are discarded. The operating device according to any one of claims 10 to 13.

15. The calculation unit is a plurality of classifiers each performing an operation classification under a different condition for the plurality of sensor signals of the plurality of sensors; a determination unit that determines an operation using results of classification by the plurality of classifiers; Equipped with The operating device according to any one of claims 1 to 14.

16. The plurality of classifiers include identifying the operation based on a relationship between a previously learned operation content and the plurality of sensor signals of the plurality of sensors; The operating device according to claim 15.

17. The plurality of sensors include A front sensor group attached to the front side of the wrist; A rear sensor group attached to the rear side of the wrist; having 17. The operating device according to claim 1.

18. the plurality of sensors output sensor signals corresponding to a displacement of a body surface at the wrist caused by a movement of at least one of a hand and a finger; 18. The operating device according to claim 1.

19. The plurality of sensors include A piezoelectric sensor in which electrodes are formed on a flexible piezoelectric film.

19. The operating device according to claim 1.

20. A display unit that displays the estimation result of the operation.

20. The operating device according to claim 1.

21. an application execution unit that executes an application using a result of the estimation of the operation; 21. The operating device according to claim 1.

22. A communication unit that transmits the estimated result of the operation to an external device to be operated.

22. The operating device according to claim 1.

23. generating sensor signals corresponding to a displacement of a body surface at the wrist from a plurality of sensors attached to the wrist; a peak value of a waveform of a time function of a total activity, which is an internal area of ​​a chart generated using amplitude values ​​of the sensor signals of the plurality of sensors, is set as a feature point, and a time range for operation estimation is set that includes the time of the feature point and is based on the spread of the waveform of the total activity; Inferring a manipulation using the plurality of sensor signals of the plurality of sensors in the time range for the manipulation estimation; Manipulation estimation method.

24. The estimation of the operation is The estimation is performed by using a combination of estimation results of a plurality of operations within the time range for operation estimation. The operation estimation method according to claim 23.

25. When the multiple estimation results satisfy a condition indicating one operation, the estimation is performed as the one operation. The operation estimation method according to claim 24.

26. If the multiple estimation results do not satisfy the condition indicating a single operation, each of the multiple estimation results is estimated as an individual operation. The operation estimation method according to claim 24.

27. The total activity is used as a range setting index; inferring the operation only when the range setting indicator satisfies an estimation possible condition; The operation estimation method according to any one of claims 23 to 26.

Citation Information

Patent Citations

  • Portable terminal device, input system and information input method

    JP2005352739A

  • Method and apparatus for combining myoelectric sensor signals and inertial sensor signals for gesture-based control

    JP2016507851A

  • Armband for tracking hand movements using electrical impedance measurements

    JP2020529067A

  • Motion and gesture input from a wearable device

    US20160091980A1

  • Method for recognizing a gesture and an electronic device thereof

    WO2018159949A1