Force estimation device, force estimation method, and program

Abstract

This force estimation device for estimating a force, relating to a muscle, to be estimated comprises: an acoustic wave transmission unit that transmits an acoustic wave; an acoustic wave reception unit that receives the acoustic wave; a force estimation unit that calculates an estimated value of the force to be estimated by analyzing the received acoustic wave; and an offset correction unit that calculates, as a corrected estimated value, a value obtained by subtracting a set offset correction value from the estimated value.

Description

[Technical field]

[0001] The present invention relates to a force estimation device, a force estimation method, and a program. [Background technology]

[0002] There is a known technology for estimating muscle strength using sound waves. For example, Non-Patent Document 1 discloses a device that estimates the strength of a user by attaching two piezoelectric elements to the back of the user's hand, one of which is a speaker and the other a microphone, transmitting and receiving ultrasonic waves, extracting features by frequency analysis, and applying a machine learning algorithm based on the extracted features. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Kubo, Yuki, et al. "AudioTouch: Minimally Invasive Sensing of Micro-Gestures via Active Bio-Acoustic Sensing." Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services. 2019. Summary of the Invention [Problem to be solved by the invention]

[0004] Conventionally, when a device is used for a long period of time, the temperature of the piezoelectric element changes due to heat generation, body heat, etc., which causes changes in the acoustic characteristics and leads to drift (baseline fluctuation) errors in the force estimate.

[0005] The disclosed technique aims to improve the accuracy of force estimates. [Means for solving the problem]

[0006] The disclosed technology is a force estimation device for estimating a force related to a muscle of an estimation target, comprising: an ultrasonic wave transmitting unit that transmits ultrasonic waves; an ultrasonic wave receiving unit that receives the ultrasonic waves; a force estimation unit that analyzes the received ultrasonic waves to calculate an estimate of the force of the muscle of the estimation target; and an offset correction unit that calculates a value obtained by subtracting a set offset correction value from the estimate as a corrected estimate value. A force estimation device further comprising: a force weakness determination unit that determines whether the estimation target is in a force weakness state; and an offset correction value setting unit that sets the offset correction value based on an estimated value of force in a time period determined to be in the force weakness state, wherein the sound wave receiving unit extracts sound waves in an audible range from received sound waves, and the force weakness determination unit calculates an absolute value of the amplitude of the extracted sound waves in the audible range, and when a value obtained by integrating the calculated absolute value over a certain window width is equal to or less than a predetermined threshold value, determines that the certain window width is a force weakness state. A force estimation device. Effect of the Invention

[0007] The accuracy of the force estimate can be improved. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a functional configuration diagram of the force estimation device. [Diagram 2] 13 is a flowchart showing an example of the flow of a force estimation process. [Diagram 3] 11 is a flowchart showing an example of the flow of a correction value setting process. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of a force estimation device. [Diagram 5] FIG. 11 is a diagram showing an example of a measurement result of a sound wave. [Figure 6] FIG. 13 is a diagram showing an example of a result of an experiment on force estimation. [Figure 7] FIG. 2 illustrates an example of a hardware configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention (the present embodiment) will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment.

[0010] The force estimation device according to the present embodiment is a device that uses sound waves to estimate force related to the muscle strength of an estimation target, such as grip strength, leg strength, back strength, explosive power, etc. The estimation target is assumed to be a user or a human other than a user, but it does not have to be a human as long as it has muscles, and may be an animal other than the human family, such as a dog or a monkey. The following description is based on the assumption that the estimation target is a human.

[0011] (Functional configuration of the force estimation device) 1 is a functional configuration diagram of a force estimation device 10. The force estimation device 10 includes an ultrasonic wave transmitter 11, an ultrasonic wave receiver 12, a force estimation unit 13, a force relaxation determination unit 14, an offset correction value setting unit 15, and an offset correction unit 16.

[0012] The sound wave transmitting unit 11 transmits sound waves. The transmitted sound waves include ultrasonic waves and sound waves in the audible range, and are transmitted from a position close to the surface of the skin of the estimation target.

[0013] The sound wave receiving unit 12 receives sound waves. Specifically, the sound wave receiving unit 12 receives ultrasonic waves and sound waves in the audible range, and extracts the ultrasonic waves or sound waves in the audible range from the received sound waves.

[0014] The force estimation unit 13 estimates the force of an estimation target. Specifically, the force estimation unit 13 calculates a frequency feature from the received sound wave, and inputs the calculated frequency feature into a regression model to obtain an estimated value of the force.

[0015] The weakness determination unit 14 determines whether the estimation target is in a relaxed state based on the sound waves received by the sound wave receiving unit 12. Specifically, the weakness determination unit 14 calculates the absolute value of the amplitude of the sound waves in the audible range extracted by the sound wave receiving unit 12, and calculates a value (integrated muscle sound) by integrating the calculated absolute value over a certain window width. Here, in order to increase the S / N ratio of the integrated muscle sound, the weakness determination unit 14 may calculate the integrated muscle sound after applying a low-pass filter with a cutoff frequency of about 100 Hz. If the integrated muscle sound is equal to or less than a predetermined threshold, the weakness determination unit 14 determines that the section corresponding to the calculated certain window width is a section in which the estimation target is in a relaxed state (weakness section).

[0016] The offset correction value setting unit 15 sets the offset correction value based on the estimated value of the force in the relaxation section. Specifically, the offset correction value setting unit 15 calculates an average value of the estimated value of the force in the relaxation section, and sets the calculated average value as the offset correction value.

[0017] The offset correction unit 16 executes offset correction based on the force estimate calculated by the force estimation unit 13 and the offset setting value. Specifically, the offset correction unit 16 sets a value obtained by subtracting the offset setting value from the force estimate value as the corrected estimate value.

[0018] (Operation of the force estimation device) Next, the operation of the force estimation device 10 will be described with reference to the drawings. The force estimation device 10 starts a force estimation process in response to a user's operation or the like. Note that it is assumed that an offset correction value has been set by an offset correction value setting process described later or based on the results of an experiment or the like when the force estimation process starts.

[0019] 2 is a flowchart showing an example of the flow of the force estimation process. The sound wave transmitting unit 11 transmits sound waves (step S11). The sound waves transmitted here include ultrasonic waves. For example, the sound wave transmitting unit 11 transmits sweep waves in the ultrasonic band (e.g., 20-40 kHz) at regular intervals (e.g., 20 ms intervals) using a piezoelectric element.

[0020] Next, the ultrasonic wave receiving unit 12 receives the ultrasonic waves (step S12). For example, the ultrasonic wave receiving unit 12 receives the ultrasonic waves using a piezoelectric element different from the piezoelectric element used for transmission.

[0021] Next, the force estimation unit 13 estimates the force using a regression model based on the frequency feature of the sound wave (step S13). Specifically, the force estimation unit 13 performs a fast Fourier transform (FFT) on the sound wave received in step S12 for every certain number of samples (e.g., 4,096 samples) to calculate a power spectrum, and treats the calculated power spectrum as a vector indicating the frequency feature (feature vector).

[0022] Then, the force estimation unit 13 inputs the feature vector into a regression model to obtain an estimated value of the force. The regression model is a trained model by machine learning, such as SupportVectorRegression. This utilizes the fact that when ultrasonic waves propagate through the skin, if there is deformation of the skin according to the amount of force applied by the measurement target, the acoustic characteristics change.

[0023] Next, the offset correction unit 16 subtracts the offset correction value from the force estimate value to obtain The corrected estimated value is output (step S14).

[0024] Next, a correction value setting process for setting an offset correction value will be described. The correction value setting process is executed before the start of the force estimation process.

[0025] 3 is a flowchart showing an example of the flow of the correction value setting process. The sound wave transmitting unit 11 transmits sound waves (step S21). The transmitted sound waves include ultrasonic waves and sound waves in the audible range. Next, the sound wave receiving unit 12 receives the sound waves and extracts sound waves in the audible range from the received sound waves (step S22).

[0026] Next, the force estimation unit 13 calculates the absolute value of the amplitude of the extracted sound wave in the audible range, and calculates a value (integrated muscle sound) by integrating the calculated absolute value over a certain window width (step S23). The certain window width is a time interval of a preset time (for example, one second).

[0027] Next, the relaxation determination unit 14 determines whether the integrated muscle sound is equal to or less than a threshold value (step S24). The threshold value is set in advance based on the results of experiments, etc. If the relaxation determination unit 14 determines that the integrated muscle sound is not equal to or less than the threshold value (step S24: No), the force estimation device 10 returns to the process of step S21 and executes the process for the next section.

[0028] When the relaxation determination unit 14 determines that the integrated muscle sound is equal to or less than the threshold (step S24: Yes), the force estimation unit 13 estimates the force using a regression model based on the frequency feature of the sound wave in that section (relaxation section) (step S25). The process of step S25 is the same as step S13 of the force estimation process shown in FIG. 2.

[0029] Note that the force estimation unit 13 may execute the process of step S24 in advance before the determination by the force loss determination unit 14, and store the execution result.

[0030] Next, the offset correction value setting unit 15 sets the average value of the force estimate value in that section (relaxation section) as the offset correction value (step S26).

[0031] (Hardware configuration of the force estimation device) 4 is a diagram showing an example of the hardware configuration of the force estimation device 10. The force estimation device 10 includes a computer 101, a speaker 102, and a microphone 103.

[0032] The computer 101 is a PC (Personal Computer), a smartphone, a server device, etc. The speaker 102 is a device that transmits sound waves. The sound wave transmitting unit 11 is realized by the computer 101 controlling the speaker 102.

[0033] The microphone 103 is a device for receiving sound waves. The sound wave receiving unit 12 is realized by the computer 101 controlling the microphone 103.

[0034] For example, the force estimation device 10 estimates grip strength by attaching a piezoelectric element that plays the role of speaker 102 and microphone 103 to the back of the hand, as in Non-Patent Document 1. For example, if the frequency characteristic of the piezoelectric element corresponding to microphone 103 has sufficient gain in both a low frequency band of 100 Hz or less and a high frequency band of 20-40 kHz, one piezoelectric element can be used for both. Also, if the piezoelectric element has frequency characteristics that are significantly attenuated in one of the frequency bands, an acoustic receiving device with different frequency characteristics that complements it can be separately attached, and the piezoelectric element and the acoustic receiving device can be used together as microphone 103.

[0035] (Sound wave measurement results) Fig. 5 is a diagram showing an example of the measurement results of sound waves. Fig. 5 shows an example of measuring muscle sounds on the back of the hand with a microphone attached to the back of the hand. The "relaxed state" in Fig. 5 is a state in which no force is exerted, and the "grip strength" section is a state in which force is exerted. When no force is exerted, the amplitude of the sound waves is small, but when force is exerted, the amplitude of the sound waves becomes large due to the influence of the muscle sounds.

[0036] From these measurement results, it can be said that the method of determining whether or not a person is in a relaxed state based on the amplitude of the sound waves is effective.

[0037] (Experimental results of force estimation) 6 is a diagram showing an example of the experimental results of force estimation. In the experiment, the estimated values ​​of grip force were calculated before and after offset correction processing. The dashed line 901 shows the estimated value before correction. The solid line 902 shows the corrected value. The dotted line 903 shows the estimated value after correction.

[0038] The correct value is the grip force value measured by the force sensor. The estimated value before correction was approximately 5N for the section where the correct value was 0N, and approximately 10N for the section where the correct value was 5N.

[0039] In contrast, the estimated value after offset correction was estimated to be close to the correct value by subtracting the offset correction value of 5N from the estimated force value. Note that, as a result of the offset correction value setting process, the force estimation device 10 determined that the section where the correct value was 0N was a force relaxation section, and set the offset correction value to 5N.

[0040] As a result of this experiment, the corrected estimated value was closer to the correct value than the uncorrected estimated value.

[0041] (Example of computer hardware configuration) Each of the functional units of the force estimation device 10 described above can be realized by causing a computer 101 to execute a program describing the processing contents described in this embodiment. Note that this "computer" may be a physical machine or a virtual machine on the cloud. When a virtual machine is used, the "hardware" described here is virtual hardware.

[0042] The above program can be recorded on a computer-readable recording medium (such as a portable memory) and can be stored or distributed. The above program can also be provided via a network such as the Internet or e-mail.

[0043] Fig. 7 is a diagram showing an example of the hardware configuration of the computer. The computer in Fig. 7 has a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, etc., which are all connected to each other via a bus B.

[0044] The program for realizing the processing on the computer is provided by a recording medium 1001 such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.

[0045] When a program start instruction is received, the memory device 1003 reads out and stores the program from the auxiliary storage device 1002. The CPU 1004 realizes the functions related to the device according to the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network. The display device 1006 displays a GUI (Graphical User Interface) or the like according to a program. The input device 1007 is composed of a keyboard, a mouse, a button, a touch panel, or the like, and is used to input various operation instructions. The output device 1008 outputs the calculation result. Note that the above computer may be equipped with a GPU (Graphics Processing Unit) or a TPU (Tensor processing unit) instead of the CPU 1004, or may be equipped with a GPU or a TPU in addition to the CPU 1004. In that case, the processes may be shared and executed, for example, the GPU or the TPU executes processes that require special calculations, and the CPU 1004 executes other processes.

[0046] (Effects of this embodiment) According to the force estimation device 10 of the present embodiment, the offset correction value is subtracted from the force estimate to obtain a corrected estimate, thereby improving the accuracy of the force estimate.

[0047] Furthermore, the force estimation device 10 determines whether the estimation target was in a relaxed state, and sets an offset correction value based on the force estimate in the time period in which the force estimate was determined to be in the relaxed state. This allows an error occurring in the force estimate to be set as an offset correction value, thereby further improving the force estimate.

[0048] (Summary of the embodiment) This specification describes at least the force estimation device, the force estimation method, and the program described in the following sections. (Section 1) A force estimation device for estimating a force related to a muscle to be estimated, comprising: A sound wave transmitting unit that transmits sound waves; A sonic wave receiving unit that receives the sonic waves; a force estimation unit that analyzes the received sound wave to calculate an estimate of the force of the estimation target; and an offset correction unit that calculates a corrected estimated value by subtracting a set offset correction value from the estimated value. Force estimation device. (Section 2) a weakness determination unit that determines whether the estimation target is in a weakness state; and an offset correction value setting unit that sets the offset correction value based on an estimated value of force in a time period determined to be the weak force state. 2. A force estimation device as described in claim 1. (Section 3) The sound wave receiving unit extracts sound waves in an audible range from the received sound waves, The weakness determination unit calculates an absolute value of the amplitude of the extracted sound wave in the audible range, and when a value obtained by integrating the calculated absolute value over a certain window width is equal to or smaller than a predetermined threshold value, determines that the section of the certain window width is a section in a weakness state. 3. A force estimation device according to claim 2. (Section 4) the force estimator calculates an estimate of a force of the estimation target in a section determined to be in the relaxed state; the offset correction value setting unit calculates an average value of the force estimate value in the section, and sets the calculated average value as the offset correction value; 4. A force estimation device according to claim 2 or 3. (Section 5) The force estimation unit calculates a power spectrum by performing a fast Fourier transform on the received sound wave for every fixed number of samples, and inputs the calculated power spectrum into a regression model as a vector indicating a frequency feature amount to obtain an estimate of the force. 5. A force estimation device according to any one of claims 1 to 4. (Section 6) 1. A computer implemented force estimation method comprising: transmitting sound waves; receiving the acoustic waves; Analysing the received sound waves to calculate an estimate of the force to be estimated; and calculating a value obtained by subtracting the set offset correction value from the estimated value as a corrected estimated value. Force estimation method. (Section 7) A program for causing a computer to function as each unit in the force estimation device according to any one of claims 1 to 5.

[0049] Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and variations are possible within the scope of the gist of the present invention described in the claims. [Explanation of symbols]

[0050] 10 Force estimation device 11 Sonic transmitter 12. Sonic receiver 13 Force estimation part 14 Weakness judgment section 15 Offset correction value setting section 16 Offset correction section 101 Computer 102 Speaker 103 Mike 1000 Drive Device 1001 Recording media 1002 Auxiliary storage 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input Device 1008 Output device

Claims

1. A force estimation device for estimating a force related to a muscle to be estimated, comprising: A sound wave transmitting unit that transmits sound waves; A sonic wave receiving unit that receives the sonic waves; a force estimation unit that analyzes the received sound wave to calculate an estimate of the force of the estimation target; an offset correction unit that calculates a corrected estimated value by subtracting a set offset correction value from the estimated value, a weakness determination unit that determines whether the estimation target is in a weakness state; and an offset correction value setting unit that sets the offset correction value based on an estimated value of force in a time period determined to be the weak force state, The sound wave receiving unit extracts sound waves in an audible range from the received sound waves, The weakness determination unit calculates an absolute value of the amplitude of the extracted sound wave in the audible range, and when a value obtained by integrating the calculated absolute value over a certain window width is equal to or smaller than a predetermined threshold value, determines that the section of the certain window width is a section in a weakness state. Force estimation device.

2. the force estimator calculates an estimate of a force of the estimation target in a section determined to be in the relaxed state; the offset correction value setting unit calculates an average value of the force estimate value in the section, and sets the calculated average value as the offset correction value; The force estimation device according to claim 1 .

3. The force estimation unit calculates a power spectrum by performing a fast Fourier transform on the received sound wave for every fixed number of samples, and inputs the calculated power spectrum into a regression model as a vector indicating a frequency feature amount to obtain an estimate of the force. The force estimation device according to claim 1 or 2.

4. 1. A computer implemented force estimation method comprising: a sonic wave transmitting step of transmitting a sonic wave; a sonic wave receiving step of receiving the sonic wave; a force estimating step of analyzing the received sound wave to calculate an estimate of a force to be estimated; an offset correction step of subtracting a set offset correction value from the estimated value to obtain a corrected estimated value, a weakness determination step of determining whether the estimation target is in a weakness state; and setting the offset correction value based on an estimated value of the force in the time period determined to be the weak force state. In the sound wave receiving step, audible sound waves are extracted from the received sound waves; In the step of determining whether or not the sound waves are weak, an absolute value of the amplitude of the extracted sound waves in the audible range is calculated, and if a value obtained by integrating the calculated absolute value over a certain window width is equal to or smaller than a predetermined threshold value, the section of the certain window width is determined to be a section in which the sound waves are weak. Force estimation method.

5. A program for causing a computer to function as each unit in the force estimation device according to any one of claims 1 to 3.

Images