Wearable device and gesture recognition method

Abstract

A wearable device, a gesture recognition method and a non-transitory computer readable storage medium thereof are provided. The device judges whether a single-channel electromyogram signal in a first time interval is in a non-neutral state or a neutral state at a first time point. The device recognizes a gesture of a user at the first point in time based on a plurality of inertial measurement parameters in response to the single-channel electromyogram signal for the first time interval being in the non-neutral state. The gesture recognition technology provided by the invention can reduce the cost of computing resources and reduce the opportunity of misjudgment.

Description

Technical Field

[0001] The present invention relates to a wearable device, a gesture recognition method and a non-transient computer-readable storage medium thereof. Specifically, the present invention relates to a wearable device capable of correctly recognizing gestures, a gesture recognition method and a non-transient computer-readable storage medium thereof. Background Art

[0002] In recent years, various technologies related to virtual reality have developed rapidly, and various head-mounted display technologies and applications have been proposed one after another.

[0003] In the prior art, user gestures can be identified mainly through three methods: image recognition, inertial measurement parameters of wearable devices, or electromyography signals. However, each method has its disadvantages when implemented alone.

[0004] Specifically, when performing image recognition through images captured by an image capturer, when the user's hand exceeds the field of view (FOV) of the image capturer or when the hand is blocked, it will be difficult to correctly recognize the user's gesture.

[0005] In addition, inertial measurement parameters can only classify instantaneous gestures, but cannot detect the continuity of gestures (for example, if a user continues to maintain a gesture, it will not be detected).

[0006] In addition, although the electromyography signal can detect the continuity of the gesture, the classification accuracy of the electromyography signal is lower than that of the inertial measurement parameter, so it needs to be detected by a multi-channel electromyography sensor. However, since the multi-channel electromyography sensor needs to be calibrated before use (for example, calibrating the multi-channel electromyography sensor at the user's hand position), the user experience is not good.

[0007] In view of this, how to provide a technology that can correctly recognize gestures is a goal that the industry urgently needs to work hard on. Summary of the invention

[0008] One object of the present invention is to provide a wearable device. The wearable device includes an inertial sensor, a single-channel electromyography sensor and a processor, and the processor is coupled to the inertial sensor and the single-channel electromyography sensor. The inertial sensor is used to generate a plurality of inertial measurement parameters corresponding to a hand of a user. The single-channel electromyography sensor is used to generate a single-channel electromyography signal corresponding to the hand. At a first time point, the processor determines whether the single-channel electromyography signal of a first time interval is in a non-neutral state or a neutral state. In response to the single-channel electromyography signal of the first time interval being in the non-neutral state, the processor identifies a gesture of the user at the first time point based on the plurality of inertial measurement parameters.

[0009] Another object of the present invention is to provide a gesture recognition method, which is used in an electronic device. The gesture recognition method comprises the following steps: receiving a plurality of inertial measurement parameters corresponding to a hand of a user and a single-channel electromyography signal corresponding to the hand; at a first time point, determining whether the single-channel electromyography signal in a first time interval is in a non-neutral state or a neutral state; and in response to the single-channel electromyography signal in the first time interval being in the non-neutral state, recognizing a gesture of the user at the first time point based on the plurality of inertial measurement parameters.

[0010] Another object of the present invention is to provide a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores a computer program, wherein the computer program includes a plurality of program instructions, and wherein the computer program executes a gesture recognition method after being loaded into an electronic device, wherein the gesture recognition method includes the following steps: receiving a plurality of inertial measurement parameters corresponding to a hand of a user and a single-channel electromyography signal corresponding to the hand; at a first time point, determining whether the single-channel electromyography signal of a first time interval is in a non-neutral state or a neutral state; and in response to the single-channel electromyography signal of the first time interval being in the non-neutral state, recognizing a gesture of the user at the first time point based on the plurality of inertial measurement parameters.

[0011] In one embodiment of the present invention, the processor further performs the following operations: in response to the single-channel electromyography signal in the first time interval being in the neutral state, not recognizing the gesture of the user at the first time point.

[0012] In one embodiment of the present invention, determining whether the single-channel electromyography signal is in the non-neutral state or the neutral state includes the following operations: comparing whether an amplitude value of the single-channel electromyography signal in the first time interval exceeds a preset threshold; in response to the amplitude value of the single-channel electromyography signal in the first time interval exceeding the preset threshold, determining that the single-channel electromyography signal is in the non-neutral state; and in response to the amplitude value of the single-channel electromyography signal in the first time interval being lower than the preset threshold, determining that the single-channel electromyography signal is in the neutral state.

[0013] In one embodiment of the present invention, identifying the gesture of the user at the first time point includes the following operations: identifying the gesture of the user at the first time point based on the multiple inertial measurement parameters of a second time interval, wherein the second time interval is a part of the first time interval.

[0014] In one embodiment of the present invention, the processor further performs the following operations: at a second time point, determining whether the single-channel electromyography signal of a second time interval is in the non-neutral state or the neutral state; in response to the single-channel electromyography signal being in the non-neutral state, calculating a duration of the gesture; and based on the gesture and the duration, determining a first control signal corresponding to a first output event among multiple output events.

[0015] In one embodiment of the present invention, the processor further performs the following operations: at a third time point, determining whether the single-channel electromyography signal of a third time interval is in the non-neutral state or the neutral state; in response to the single-channel electromyography signal being in the non-neutral state, generating a first movement value corresponding to the gesture based on the multiple inertial measurement parameters between the first time point and the third time point; and generating a second control signal corresponding to a second output event among multiple output events based on the gesture and the first movement value.

[0016] In one embodiment of the present invention, the processor further performs the following operations: at a fourth time point, determining whether the single-channel electromyography signal of a fourth time interval is in the non-neutral state or the neutral state; and in response to the single-channel electromyography signal being in the neutral state, generating a third control signal corresponding to a third output event among the multiple output events.

[0017] In one embodiment of the present invention, the wearable device further includes: a result data buffer, coupled to the processor, and used to store a judgment result of the single-channel electromyography signal; wherein the processor further performs the following operations: in response to the single-channel electromyography signal in the first time interval being in the non-neutral state, storing the judgment result of the single-channel electromyography signal corresponding to the first time interval to the result data buffer; and in response to the single-channel electromyography signal in the first time interval being in the neutral state, deleting the judgment result of the single-channel electromyography signal corresponding to the first time interval in the result data buffer.

[0018] In one embodiment of the present invention, the processor further performs the following operations: in response to the single-channel electromyography signal of the first time interval being in the non-neutral state, identifying the gesture of the user at the first time point based on the multiple inertial measurement parameters and the single-channel electromyography signal.

[0019] The gesture recognition technology provided by the present disclosure (at least including a device, a method and a non-transitory computer-readable storage medium thereof) first determines whether a single-channel electromyogram signal in a time interval is in a non-neutral state or a neutral state. Then, when it is determined that the single-channel electromyogram signal in the first time interval is in the non-neutral state, the gesture of the user at the first time point is identified based on the multiple inertial measurement parameters. In addition, when it is determined that the single-channel electromyogram signal in the first time interval is in the neutral state, the gesture of the user at the first time point is not identified. The gesture recognition technology provided by the present disclosure first refers to the value of the single-channel electromyogram signal before deciding whether to perform a gesture recognition operation, thereby reducing the cost of computing resources and reducing the chance of misjudgment. In addition, since the present disclosure refers to both the inertial measurement parameters and the electromyogram signal, the device of the present disclosure does not need to be equipped with a multi-channel electromyogram sensor, and can achieve correct gesture recognition operations.

[0020] The detailed technology and implementation methods of the present invention are described below in conjunction with the accompanying drawings so that a person having ordinary knowledge in the technical field to which the present invention belongs can understand the technical features of the invention for which protection is sought. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram showing an applicable scenario of the wearable device of the first embodiment;

[0022] Figure 2 A schematic diagram showing the architecture of a wearable device according to certain embodiments;

[0023] Figure 3A A schematic diagram showing the architecture of a wearable device and an expansion device according to certain embodiments;

[0024] Figure 3B A schematic diagram showing the architecture of a wearable device and an expansion device according to certain embodiments;

[0025] Figure 4 A timing diagram illustrating certain embodiments;

[0026] Figure 5A A schematic diagram illustrating the operation of certain embodiments;

[0027] Figure 5B A schematic diagram illustrating the operation of certain embodiments;

[0028] Fig. 6A A schematic diagram illustrating the operation of certain embodiments;

[0029] Figure 6B schematic diagrams illustrating the operation of certain embodiments; and

[0030] Figure 7 A partial flow chart of a gesture recognition method according to the second embodiment is shown.

[0031] Explanation of symbols:

[0032] 1: Wearable devices

[0033] HMD: Head-mounted display

[0034] C: User

[0035] IMU: Inertial Sensor

[0036] SC-EMGS: Single Channel Electromyography Sensor

[0037] PS: Processor

[0038] WD: Wearable devices

[0039] ED: Extension Device

[0040] TI: Transceiver Interface

[0041] 400: Timing diagram

[0042] GS: Gestures

[0043] EMG: electromyography

[0044] SW: Sliding Window

[0045] ACC: Acceleration

[0046] Gyro: angular acceleration

[0047] STA: Status

[0048] TIM: Timeline

[0049] P1, P2, P3, Pn: time point

[0050] T1: Time interval

[0051] NN: non-neutral state

[0052] N: Neutral state

[0053] G1, G2, G3, G4: Gestures

[0054] SL: Show Menu

[0055] TO: target object

[0056] 700: Gesture recognition method

[0057] S701, S703, S705: Steps DETAILED DESCRIPTION

[0058] The following will explain a wearable device, a gesture recognition method and a non-transitory computer-readable storage medium provided by the present invention through implementation examples. However, these implementation examples are not intended to limit the present invention to any environment, application or method as described in these implementation examples. Therefore, the description of the implementation examples is only for the purpose of illustrating the present invention, and is not intended to limit the scope of the present invention. It should be understood that in the following implementation examples and drawings, elements that are not directly related to the present invention have been omitted and are not shown, and the sizes of each element and the size ratios between the elements are only examples, and are not intended to limit the scope of the present invention.

[0059] First, the applicable scenario of this embodiment is described, and its schematic diagram is shown in Figure 1 .like Figure 1 As shown, in the application environment of the present invention, a head-mounted display HMD can be used by a user C, and the user C wears one or more wearable devices 1 (e.g., smart bracelets, smart watches, etc.) on the hands (e.g., fingers, wrists, etc.) to perform input operations corresponding to the head-mounted display HMD (e.g., gesture input operations).

[0060] It should be noted that Figure 1 This is just an example, and the present invention does not limit the number of wearable devices 1 connected to the head-mounted display HMD. The head-mounted display HMD can be connected to one or more wearable devices via a network at the same time, depending on the scale of the device and the actual needs of the user C.

[0061] In the first embodiment of the present invention, Figure 2As shown, the wearable device 1 includes an inertial sensor IMU, a single-channel electromyography sensor SC-EMGS and a processor PS, and the processor PS is coupled to the inertial sensor IMU and the single-channel electromyography sensor SC-EMGS.

[0062] In this embodiment, the inertial sensor IMU is used to generate a plurality of inertial measurement parameters corresponding to the hand of the user C. Specifically, the inertial sensor can continuously generate a sequence of a plurality of inertial measurement parameters (e.g., a stream of inertial measurement parameters generated at a frequency of 10 times per second). It should be noted that in the present disclosure, the inertial sensor IMU may only include a gyroscope and an accelerometer, without the configuration of a magnetometer or a barometer, etc.

[0063] In this embodiment, the single-channel electromyography sensor SC-EMGS is used to generate a single-channel electromyography (EMG) signal corresponding to the hand. It should be noted that since the present disclosure only requires a single-channel electromyography sensor to operate, it is not necessary to frequently calibrate the position of the electromyography sensor.

[0064] It should be noted that the processor PS can be various processing units, a central processing unit (CPU), a microprocessor, or other computing devices known to those having ordinary knowledge in the technical field to which the present invention belongs.

[0065] For easier understanding, please refer to Figure 4 The timing diagram 400 illustrates the changes in the values ​​of each parameter signal collected at each time point (for example, from time point P1 to time point Pn on the time axis TIM), the state STA corresponding to each time interval after judgment (for example, non-neutral state NN (non-neutral state) or neutral state N (neutral state)), and the corresponding gesture GS after recognition.

[0066] like Figure 4 As shown, the plurality of inertial measurement parameters may include acceleration ACC and angular acceleration Gyro corresponding to different time points. In addition, the single-channel electromyography signal EMGS includes single-channel electromyography EMG corresponding to different time points.

[0067] From the data changes in the timing diagram 400, it can be seen that the inertial measurement parameters will only collect short-term and drastic changes when the user C performs an action. However, if the user C continues to move (for example, continue to maintain the gesture), the inertial measurement parameters will not be able to provide complete activity information of the user C. On the other hand, if the user C continues to move, the single-channel electromyography EMG can provide complete activity information of the user C over a continuous period of time (i.e., with obvious changes).

[0068] First, in this embodiment, if Figure 4 As shown, the processor PS determines whether the single-channel electromyography signal EMGS of the time interval T1 (for example, the first time interval in some embodiments) is in the non-neutral state NN or the neutral state N at the time point P1 (for example, the first time point in some embodiments).

[0069] It should be noted that the stronger the muscle force applied by the user C wearing the single-channel electromyography sensor SC-EMGS, the higher the amplitude of the single-channel electromyography signal generated. Therefore, if the single-channel electromyography signal EMGS is in a non-neutral state NN, it can represent that the user C is in a state of applying force. In addition, if the single-channel electromyography signal EMGS is in a neutral state N, it can represent that the user C is in a relatively static state.

[0070] In some embodiments, the processor PS can determine the non-neutral state NN or the neutral state N by comparing whether the single-channel electromyogram signal EMGS in the time interval exceeds the amplitude value. For example, the processor PS compares whether an amplitude value of the single-channel electromyogram signal EMGS in the first time interval exceeds a preset threshold. Then, the processor PS judges that the single-channel electromyogram signal EMGS is in the non-neutral state NN in response to the amplitude value of the single-channel electromyogram signal EMGS in the first time interval exceeding the preset threshold. In addition, the processor PS judges that the single-channel electromyogram signal EMGS is in the neutral state N in response to the amplitude value of the single-channel electromyogram signal EMGS in the first time interval being lower than the preset threshold.

[0071] In some embodiments, the processor PS may be configured to perform the following operations at a dynamic time interval (e.g.: Figure 4 The sliding window SW (sliding window) in the single-channel electromyography signal is compared to see whether the state of the single-channel electromyography signal exceeds the amplitude value to determine whether the single-channel electromyography signal EMGS is in the non-neutral state NN or the neutral state N.

[0072] It should be noted that Figure 4 The relative relationship and position of the time point P1 and the time interval T1 in the example are only for illustration and are not intended to limit the scope of the present disclosure.

[0073] Next, in this embodiment, in response to the single-channel electromyography signal EMGS in the first time interval being in the non-neutral state NN, the processor PS identifies the gesture of the user at the first time point based on the multiple inertial measurement parameters.

[0074] In some embodiments, the processor PS may input the plurality of inertial measurement parameters into a convolutional neural network (e.g., a trained convolutional neural network) to generate a gesture corresponding to the user C.

[0075] It should be noted that, if the body part is a finger, for example, the trained convolutional neural network can be a gesture classifier. Specifically, the convolutional neural network can be used to analyze the multiple inertial measurement parameters and generate gestures corresponding to the multiple inertial measurement parameters. In some embodiments, the convolutional neural network can be trained by annotated inertial measurement parameter data (e.g., inertial measurement parameters corresponding to fingers).

[0076] In some embodiments, in order to save computational cost, the processor PS performs subsequent gesture recognition only when the single-channel electromyography signal EMGS is in the non-neutral state NN. Conversely, when the single-channel electromyography signal EMGS is in the neutral state N, no gesture recognition operation is performed.

[0077] Specifically, the processor PS responds to the single-channel electromyography signal EMGS in the first time interval being in a neutral state N and does not recognize the gesture of the user C at the first time point (for example: Figure 4 The time interval from time point P2 to time point P3).

[0078] In some embodiments, in order to more accurately identify the gesture, the processor PS may perform gesture recognition operation through the multiple inertial measurement parameters of a time interval, and the time interval corresponds to a part of the time interval used to determine the corresponding single-channel electromyography signal EMGS. Specifically, the processor PS identifies the gesture of the user C at the first time point based on the multiple inertial measurement parameters of a second time interval, wherein the second time interval is a part of the first time interval (for example: Figure 4 , the second half of time interval T1).

[0079] In some embodiments, in order to more accurately identify gestures, the processor PS may also perform gesture recognition operations through the multiple inertial measurement parameters and the single-channel electromyography signal EMGS at the same time. Specifically, in response to the single-channel electromyography signal EMGS in the first time interval being in a non-neutral state NN, the processor PS identifies the gesture of the user at the first time point based on the multiple inertial measurement parameters and the single-channel electromyography signal EMGS.

[0080] In some embodiments, the wearable device 1 may further include a result data buffer (not shown), an inertial measurement parameter data buffer (not shown), or a single-channel electromyography signal data buffer (not shown) coupled to the processor PS.

[0081] It should be noted that the inertial measurement parameter data buffer and the single-channel electromyography signal data buffer can be used to temporarily store the data generated by the inertial sensor IMU and the single-channel electromyography sensor SC-EMGS.

[0082] In addition, the result data buffer may be used to store the determination result of the single-channel electromyography signal EMGS, and the processor PS may know whether there is a determined gesture (ie, a gesture that may be ongoing) through the data in the result data buffer.

[0083] Specifically, in response to the single-channel electromyography signal EMGS in the first time interval being in a non-neutral state NN, the processor PS stores the judgment result corresponding to the single-channel electromyography signal EMGS in the first time interval to the result data buffer. In addition, in response to the single-channel electromyography signal EMGS in the first time interval being in a neutral state N, the processor PS deletes the judgment result corresponding to the single-channel electromyography signal EMGS in the first time interval in the result data buffer.

[0084] In some embodiments, the processor PS can determine whether the gesture is a continuous action by determining whether the result data buffer is empty, without performing the gesture recognition operation again. Specifically, the processor PS determines at a time point (e.g., the fifth time point referred to in some embodiments) that the single-channel electromyography signal EMGS of a time interval (e.g., the fifth time interval referred to in some embodiments) is in a non-neutral state NN or a neutral state N. Then, in response to the single-channel electromyography signal EMGS of the time interval being in a non-neutral state NN, the processor PS determines whether the result data buffer is empty. In response to the result data buffer not being empty, the processor PS does not recognize the gesture of the user at the time point.

[0085] In addition, in some embodiments, the corresponding gesture application can be determined by calculating the duration of the gesture. Specifically, the processor PS determines at a time point (e.g., the second time point referred to in some embodiments) whether the single-channel electromyography signal EMGS of a time interval (e.g., the second time interval referred to in some embodiments) is in a non-neutral state NN or a neutral state N. Then, the processor PS calculates a duration of the gesture in response to the single-channel electromyography signal EMGS being in the non-neutral state NN. Finally, the processor PS determines a first control signal corresponding to a first output event among multiple output events based on the gesture and the duration.

[0086] For example, Figure 5AAs shown, the processor PS determines that the gesture is G1 and the gesture of the user C lasts for more than 3 seconds. The processor PS compares the rules in the plurality of output events and generates an output event corresponding to the display menu SL.

[0087] In addition, in some embodiments, the processor PS can determine the corresponding gesture application by calculating the movement value of the gesture. Specifically, the processor PS determines whether the single-channel electromyography signal EMGS of a time interval (for example, the third time interval referred to in some embodiments) is in a non-neutral state NN or a neutral state N at a time point (for example, the third time point referred to in some embodiments). Then, the processor PS generates a first movement value corresponding to the gesture based on the multiple inertial measurement parameters between the first time point and the third time point in response to the single-channel electromyography signal EMGS being in the non-neutral state NN. Finally, the processor PS generates a second control signal corresponding to a second output event among multiple output events based on the gesture and the first movement value.

[0088] For example, Figure 5B As shown, the processor PS determines that the gesture is G2 and the gesture of the user C has a movement value in a direction. The processor PS compares the rules in the plurality of output events and generates an output event for moving the target object TO (eg, curtain) based on the movement value.

[0089] For example, Fig. 6A As shown, the processor PS determines that the gesture is G1 and the gesture of the user C has a movement value in a direction. The processor PS compares the rules in the plurality of output events and generates an output event for adjusting the volume value based on the movement value.

[0090] In addition, in some embodiments, the corresponding gesture application can be determined by calculating the movement value of the gesture and the subsequent continuous actions. Specifically, the processor PS determines at a time point (e.g., the fourth time point referred to in some embodiments) whether the single-channel electromyography signal EMGS of a time interval (e.g., the fourth time interval referred to in some embodiments) is in a non-neutral state NN or a neutral state N. Then, in response to the single-channel electromyography signal being in the neutral state N, the processor PS generates a third control signal corresponding to a third output event among the multiple output events.

[0091] For example, Figure 6BAs shown, the processor PS determines that the gesture is G3 at the first time point and the gesture of the user C has a movement value in a direction, and at the second time point, the single-channel electromyography signal is in a neutral state N (i.e., the hand-released gesture G4). The processor PS compares the rules in the multiple output events and generates an output event of moving the target object TO (e.g., a box) based on the movement value and moving the target object TO to a position and putting it down.

[0092] In some embodiments, the present disclosure can also be used in conjunction with a wearable device WD and an expansion device ED, where the expansion device ED transmits data to the wearable device WD for data processing. Figure 3A As shown, the wearable device WD includes an inertial sensor IMU, a processor PS and a transceiver interface TI. The extended device ED includes a single-channel electromyography sensor SC-EMGS and a transceiver interface TI.

[0093] For example, Figure 3B As shown, the wearable device WD includes a single-channel electromyography sensor SC-EMGS, a processor PS and a transceiver interface TI. The extended device ED includes an inertial sensor IMU and a transceiver interface TI.

[0094] As can be seen from the above description, the wearable device 1 provided by the present invention first determines whether the single-channel electromyography signal in a time interval is in a non-neutral state or a neutral state. Then, when it is determined that the single-channel electromyography signal in the first time interval is in the non-neutral state, the gesture of the user at the first time point is identified based on the multiple inertial measurement parameters. In addition, when it is determined that the single-channel electromyography signal in the first time interval is in the neutral state, the gesture of the user at the first time point is not identified. The wearable device 1 provided by the present disclosure first refers to the value of the single-channel electromyography signal before deciding whether to perform a gesture recognition operation, thereby reducing the cost of computing resources and reducing the chance of misjudgment. In addition, since the present disclosure refers to both the inertial measurement parameters and the electromyography signal, the device of the present disclosure does not need to be equipped with a multi-channel electromyography sensor, and can achieve correct gesture recognition operations.

[0095] The second embodiment of the present invention is a gesture recognition method, the flow chart of which is shown in Figure 7 The gesture recognition method 700 is applicable to an electronic device, such as the wearable device 1 described in the first embodiment, the wearable device WD with an expansion device ED, or a head mounted display HMD. The gesture recognition method 700 recognizes the user's gesture through steps S701 to S705.

[0096] In step S701, a plurality of inertial measurement parameters corresponding to a hand of a user and a single-channel electromyography signal corresponding to the hand are received by an electronic device. Then, in step S703, the electronic device determines whether the single-channel electromyography signal of a first time interval is in a non-neutral state or a neutral state at a first time point.

[0097] Finally, in step S705 , in response to the single-channel electromyography signal in the first time interval being in the non-neutral state, the electronic device identifies a gesture of the user at the first time point based on the multiple inertial measurement parameters.

[0098] In some embodiments, the gesture recognition method 700 further includes the following steps: in response to the single-channel electromyography signal in the first time interval being in the neutral state, not recognizing the gesture of the user at the first time point.

[0099] In some embodiments, the step of determining whether the single-channel electromyography signal is in the non-neutral state or the neutral state includes the following steps: comparing whether an amplitude value of the single-channel electromyography signal in the first time interval exceeds a preset threshold; in response to the amplitude value of the single-channel electromyography signal in the first time interval exceeding the preset threshold, determining that the single-channel electromyography signal is in the non-neutral state; and in response to the amplitude value of the single-channel electromyography signal in the first time interval being lower than the preset threshold, determining that the single-channel electromyography signal is in the neutral state.

[0100] In some embodiments, the step of identifying the gesture of the user at the first time point includes the following operations: identifying the gesture of the user at the first time point based on the multiple inertial measurement parameters in a second time interval, wherein the second time interval is a part of the first time interval.

[0101] In some embodiments, the gesture recognition method 700 further includes the following steps: at a second time point, determining whether the single-channel electromyography signal of a second time interval is in the non-neutral state or the neutral state; in response to the single-channel electromyography signal being in the non-neutral state, calculating a duration of the gesture; and based on the gesture and the duration, determining a first control signal corresponding to a first output event among multiple output events.

[0102] In some embodiments, the gesture recognition method 700 further includes the following steps: at a third time point, determining whether the single-channel electromyography signal of a third time interval is in the non-neutral state or the neutral state; in response to the single-channel electromyography signal being in the non-neutral state, generating a first movement value corresponding to the gesture based on the multiple inertial measurement parameters between the first time point and the third time point; and generating a second control signal corresponding to a second output event among multiple output events based on the gesture and the first movement value.

[0103] In some embodiments, the gesture recognition method 700 further includes the following steps: at a fourth time point, determining whether the single-channel electromyography signal of a fourth time interval is in the non-neutral state or the neutral state; and in response to the single-channel electromyography signal being in the neutral state, generating a third control signal corresponding to a third output event among the multiple output events.

[0104] In some embodiments, the gesture recognition method 700 further includes the following steps: in response to the single-channel electromyography signal in the first time interval being in the non-neutral state, storing a judgment result corresponding to the single-channel electromyography signal in the first time interval to a result data buffer; and in response to the single-channel electromyography signal in the first time interval being in the neutral state, deleting the judgment result of the single-channel electromyography signal in the first time interval in the result data buffer.

[0105] In some embodiments, the gesture recognition method 700 further includes the following steps: at a fifth time point, determining whether the single-channel electromyography signal of a fifth time interval is in the non-neutral state or the neutral state; in response to the single-channel electromyography signal of the fifth time interval being in the non-neutral state, determining whether the result data buffer is empty; and in response to the result data buffer being not empty, not recognizing the gesture of the user at the fifth time point.

[0106] In addition to the above steps, the second embodiment can also perform all operations and steps of the wearable device 1 described in the first embodiment, has the same functions, and achieves the same technical effects. A person with ordinary knowledge in the technical field to which the present invention belongs can directly understand how the second embodiment performs these operations and steps based on the above-mentioned first embodiment, has the same functions, and achieves the same technical effects, so it is not repeated.

[0107] The control method described in the second embodiment can be implemented by a computer program having multiple instructions. Each computer program can be a file that can be transmitted on a network, or can be stored in a non-transitory computer-readable storage medium. For each computer program, after the multiple instructions contained therein are loaded into an electronic device (e.g., a wearable device 1, a wearable device WD with an expansion device ED, or a head-mounted display HMD), the computer program executes the gesture recognition method described in the second embodiment. The non-transitory computer-readable storage medium can be an electronic product, such as a read only memory (ROM), a flash memory, a floppy disk, a hard disk, a compact disk (CD), a portable disk, a database accessible by a network, or any other storage medium known to a person of ordinary skill in the technical field to which the present invention belongs and having the same function.

[0108] It should be noted that in the patent specification and claims of the present invention, certain terms (including: time points, time intervals, etc.) are preceded by "first", "second", "third", "fourth" or "fifth". These "first", "second", "third", "fourth" or "fifth" are only used to distinguish different terms. For example, the "first" and "second" in the first time point and the second time point are only used to indicate different time points used in different operations.

[0109] In summary, the gesture recognition technology provided by the present disclosure (at least including the device, method and its non-transitory computer-readable storage medium) first determines whether the single-channel electromyography signal in a time interval is in a non-neutral state or a neutral state. Then, when it is determined that the single-channel electromyography signal in the first time interval is in the non-neutral state, the gesture of the user at the first time point is recognized based on the multiple inertial measurement parameters. In addition, when it is determined that the single-channel electromyography signal in the first time interval is in the neutral state, the gesture of the user at the first time point is not recognized. The gesture recognition technology provided by the present disclosure first refers to the value of the single-channel electromyography signal before deciding whether to perform a gesture recognition operation, thereby reducing the cost of computing resources and reducing the chance of misjudgment. In addition, since the present disclosure refers to both the inertial measurement parameters and the electromyography signal, the device of the present disclosure does not need to be equipped with a multi-channel electromyography sensor, and can achieve correct gesture recognition operations.

[0110] The above embodiments are only used to illustrate some embodiments of the present invention and to explain the technical features of the present invention, and are not used to limit the protection category and scope of the present invention. Any changes or equivalent arrangements that can be easily completed by a person with ordinary knowledge in the technical field to which the present invention belongs are within the scope claimed by the present invention, and the scope of protection of the present invention is subject to the claims.

Claims

1. A wearable device, characterized in that: Include: an inertial sensor for generating a plurality of inertial measurement parameters corresponding to a hand of a user; a single-channel electromyography sensor for generating a single-channel electromyography signal corresponding to the hand; and A processor is coupled to the inertial sensor and the single-channel electromyography sensor, and performs the following operations: At a first time point, determining whether the single-channel electromyography signal in a first time interval is in a non-neutral state or a neutral state; and In response to the single-channel electromyography signal in the first time interval being in the non-neutral state, a gesture of the user at the first time point is identified based on the plurality of inertial measurement parameters.

2. The wearable device according to claim 1, wherein: The processor further performs the following operations: In response to the single-channel electromyography signal in the first time interval being in the neutral state, the gesture of the user at the first time point is not recognized.

3. The wearable device according to claim 1, wherein: Wherein, judging whether the single-channel electromyography signal is in the non-neutral state or the neutral state comprises the following operations: Comparing whether an amplitude value of the single-channel electromyography signal in the first time interval exceeds a preset threshold; In response to the amplitude value of the single-channel electromyography signal in the first time interval exceeding the preset threshold, determining that the single-channel electromyography signal is in the non-neutral state; and In response to the amplitude value of the single-channel electromyography signal in the first time interval being lower than the preset threshold, it is determined that the single-channel electromyography signal is in the neutral state.

4. The wearable device according to claim 1, wherein: Wherein identifying the gesture of the user at the first time point comprises the following operations: The gesture of the user at the first time point is identified based on the plurality of inertial measurement parameters in a second time interval, wherein the second time interval is a portion of the first time interval.

5. The wearable device according to claim 1, wherein: The processor further performs the following operations: At a second time point, determining whether the single-channel electromyography signal in a second time interval is in the non-neutral state or the neutral state; In response to the single-channel electromyography signal being in the non-neutral state, calculating a duration of the gesture; as well as Based on the gesture and the duration, a first control signal corresponding to a first output event among a plurality of output events is determined.

6. The wearable device according to claim 1, wherein: The processor further performs the following operations: At a third time point, determining whether the single-channel electromyography signal in a third time interval is in the non-neutral state or the neutral state; In response to the single-channel electromyography signal being in the non-neutral state, generating a first movement value corresponding to the gesture based on the plurality of inertial measurement parameters between the first time point and the third time point; as well as Based on the gesture and the first movement value, a second control signal corresponding to a second output event among a plurality of output events is generated.

7. The wearable device according to claim 6, wherein: The processor further performs the following operations: At a fourth time point, determining whether the single-channel electromyography signal in a fourth time interval is in the non-neutral state or the neutral state; and In response to the single-channel electromyography signal being in the neutral state, a third control signal corresponding to a third output event among the plurality of output events is generated.

8. The wearable device according to claim 1, wherein: The wearable device further comprises: a result data buffer, coupled to the processor and used to store a determination result of the single-channel electromyography signal; The processor further performs the following operations: In response to the single-channel electromyography signal in the first time interval being in the non-neutral state, storing the determination result corresponding to the single-channel electromyography signal in the first time interval in the result data buffer; and In response to the single-channel electromyography signal in the first time interval being in the neutral state, the judgment result of the single-channel electromyography signal in the first time interval in the result data buffer is deleted.

9. The wearable device according to claim 1, wherein: The processor further performs the following operations: In response to the single-channel electromyography signal in the first time interval being in the non-neutral state, the gesture of the user at the first time point is identified based on the multiple inertial measurement parameters and the single-channel electromyography signal.

10. A gesture recognition method, characterized in that: For an electronic device, the gesture recognition method comprises the following steps: receiving a plurality of inertial measurement parameters corresponding to a hand of a user and a single-channel electromyography signal corresponding to the hand; At a first time point, determining whether the single-channel electromyography signal in a first time interval is in a non-neutral state or a neutral state; and In response to the single-channel electromyography signal in the first time interval being in the non-neutral state, a gesture of the user at the first time point is identified based on the plurality of inertial measurement parameters.

11. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores a computer program, wherein the computer program includes a plurality of program instructions. After the computer program is loaded into an electronic device, a gesture recognition method is executed. The gesture recognition method includes the following steps: receiving a plurality of inertial measurement parameters corresponding to a hand of a user and a single-channel electromyography signal corresponding to the hand; At a first time point, determining whether the single-channel electromyography signal in a first time interval is in a non-neutral state or a neutral state; and In response to the single-channel electromyography signal in the first time interval being in the non-neutral state, a gesture of the user at the first time point is identified based on the plurality of inertial measurement parameters.

Images