Control Method, Device and Wearable Device Based on Surface Electromyogram Signal

Abstract

The present application discloses a control method, device and wearable device based on surface electromyogram signals. The control method of the present application includes: when the transmitting electrode attached to the second limb of the human body releases an excitation signal and the second limb makes a gesture on the human skin of the first limb, two sets of electrode response signals of the excitation signal are obtained through two sets of receiving electrodes attached to the first limb; two sets of surface electromyogram signals to be recognized are obtained according to the two sets of electrode response signals, and the gesture is recognized according to the time-domain waveform change characteristics of the two sets of surface electromyogram signals; the recognized gesture is sent to the head-mounted device for controlling the head-mounted device. The technical solution of the present application performs gesture recognition based on the time-domain waveform change characteristics of two sets of surface electromyogram signals, which can improve the recognition accuracy and efficiency.

Description

Technical Field

[0001] The present application relates to the technical field of surface electromyogram signal processing, and particularly relates to a control method, device and wearable device based on surface electromyogram signals. Background Art

[0002] With the rapid development of the fields of Virtual Reality (VR) and Augmented Reality (AR), more and more research and development of their supporting devices have been carried out to improve the convenience and comfort of VR / AR devices based on the supporting devices. Currently, when users interact with VR / AR devices, most of them adopt visual solutions, such as using a camera to track the user's hand and capture hand images, and recognizing gesture actions according to the hand graphics. However, visual solutions are easily affected by factors such as light and shooting angle, with low gesture recognition accuracy, and the camera needs to be in a working state all the time, which not only occupies computing resources but also consumes a lot of power. Summary of the Invention

[0003] Embodiments of the present application provide a control method, device and wearable device based on surface electromyogram signals, which can avoid the influence of light and shooting angle on the control process while ensuring the convenience and comfort of controlling a head-mounted device.

[0004] Embodiments of the present application adopt the following technical solutions:

[0005] In a first aspect, an embodiment of the present application provides a control method based on surface electromyogram signals, including:

[0006] When a transmitting electrode attached to a second limb of a human body releases an excitation signal and the second limb makes a gesture action on the human skin of a first limb, two groups of electrode response signals of the excitation signal are acquired through two groups of receiving electrodes attached to the first limb;

[0007] Two groups of surface electromyogram signals to be recognized are acquired according to the two groups of electrode response signals, and gesture actions are recognized according to the time-domain waveform change characteristics of the two groups of surface electromyogram signals;

[0008] The recognized gesture actions are sent to a head-mounted device for controlling the head-mounted device.

[0009] In a second aspect, an embodiment of the present application provides a control device based on surface electromyogram signals, including:

[0010] A signal acquisition unit, configured to receive two groups of electrode response signals of an excitation signal through two groups of receiving electrodes attached to a first limb when a transmitting electrode attached to a second limb of a human body releases an excitation signal and the second limb makes a gesture action on the human skin of the first limb;

[0011] A signal processing unit, configured to obtain two sets of surface electromyography signals to be recognized according to two sets of electrode response signals, and recognize gesture actions according to the time-domain waveform change characteristics of the two sets of surface electromyography signals;

[0012] A gesture sending unit, configured to send the recognized gesture actions to a head-mounted device for controlling the head-mounted device.

[0013] In a third aspect, an embodiment of the present application provides a wearable device, including a processor; and a memory arranged to store computer-executable instructions, the executable instructions, when executed, cause the processor to execute a control method based on surface electromyography signals, and the control method includes:

[0014] When a transmitting electrode attached to a second limb of a human body releases an excitation signal and the second limb makes a gesture action on the human skin of the first limb, two sets of electrode response signals of the excitation signal are obtained through two sets of receiving electrodes attached to the first limb;

[0015] Obtain two sets of surface electromyography signals to be recognized according to the two sets of electrode response signals, and recognize gesture actions according to the time-domain waveform change characteristics of the two sets of surface electromyography signals;

[0016] Send the recognized gesture actions to a head-mounted device for controlling the head-mounted device.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, and the computer-readable storage medium stores one or more programs, and the one or more programs, when executed by a processor, implement a control method based on surface electromyography signals, and the control method includes:

[0018] When a transmitting electrode attached to a second limb of a human body releases an excitation signal and the second limb makes a gesture action on the human skin of the first limb, two sets of electrode response signals of the excitation signal are obtained through two sets of receiving electrodes attached to the first limb;

[0019] Obtain two sets of surface electromyography signals to be recognized according to the two sets of electrode response signals, and recognize gesture actions according to the time-domain waveform change characteristics of the two sets of surface electromyography signals;

[0020] Send the recognized gesture actions to a head-mounted device for controlling the head-mounted device.

[0021] The above at least one technical solution adopted in the embodiments of the present application can achieve the following beneficial effects: When the transmitting electrode attached to the second limb of the human body releases an excitation signal and the second limb makes a gesture on the human skin of the first limb, first, two groups of electrode response signals of the excitation signal are obtained through two groups of receiving electrodes attached to the first limb, then two groups of surface electromyography signals to be recognized are obtained according to the two groups of electrode response signals, the gesture is recognized according to the time-domain waveform change characteristics of the two groups of surface electromyography signals, and finally the recognized gesture is sent to the head-mounted device for controlling the head-mounted device.

[0022] In the embodiments of the present application, the signal acquisition process and the signal recognition process of the gesture are separated from the head-mounted device and executed by the wearable device, which saves the power consumption and system resources of the head-mounted device, avoids the influence of factors such as light and shooting angle existing in the visual solution on the interactive control, and can improve the recognition accuracy and efficiency while ensuring the convenience and comfort of controlling the head-mounted device. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0024] Figure 1 It is a flowchart of a control method based on surface electromyography signals in the embodiments of the present application;

[0025] Figure 2 It is a schematic structural diagram of a gesture recognition system in the embodiments of the present application;

[0026] Figure 3 It is a schematic structural diagram of a first wearable device in the embodiments of the present application;

[0027] Figure 4 It is a schematic structural diagram of a second wearable device in the embodiments of the present application;

[0028] Figure 5 It is a schematic diagram of the right finger wearing the first wearable device making a tapping action on the surface of the left palm wearing the second wearable device in the embodiments of the present application;

[0029] Figure 6-1 It is a schematic diagram of the electrode response signal corresponding to a single click action collected by any group of receiving electrodes in the embodiments of the present application;

[0030] Figure 6-2 It is a waveform change schematic diagram of any group of surface electromyography signals corresponding to a single click action in the embodiments of the present application;

[0031] Figure 7-1Schematic diagram of electrode response signals corresponding to double - click actions collected by any group of receiving electrodes in the embodiments of the present application;

[0032] Figure 7-2 Schematic diagram of waveform changes of any group of surface electromyogram signals corresponding to double - click actions in the embodiments of the present application;

[0033] Figure 8-1 Schematic diagram of electrode response signals corresponding to long - press actions collected by any group of receiving electrodes in the embodiments of the present application;

[0034] Figure 8-2 Schematic diagram of waveform changes of any group of surface electromyogram signals corresponding to long - press actions in the embodiments of the present application;

[0035] Figure 9-1 Schematic diagram of the right finger wearing the first wearable device sliding left on the surface of the left palm wearing the second wearable device in the embodiments of the present application;

[0036] Figure 9-2 Schematic diagram of electrode response signals corresponding to left - sliding actions collected by any group of receiving electrodes in the embodiments of the present application;

[0037] Figure 9-3 Schematic diagram of waveform changes of any group of surface electromyogram signals corresponding to left - sliding actions in the embodiments of the present application;

[0038] Figure 10-1 Schematic diagram of the right finger wearing the first wearable device sliding right on the surface of the left palm wearing the second wearable device in the embodiments of the present application;

[0039] Figure 10-2 Schematic diagram of electrode response signals corresponding to right - sliding actions collected by any group of receiving electrodes in the embodiments of the present application;

[0040] Figure 10-3 Schematic diagram of waveform changes of any group of surface electromyogram signals corresponding to right - sliding actions in the embodiments of the present application;

[0041] Figure 11-1 Schematic diagram of the right finger wearing the first wearable device sliding up on the surface of the left palm wearing the second wearable device in the embodiments of the present application;

[0042] Figure 11-2 Schematic diagram of electrode response signals corresponding to up - sliding actions collected by the first receiving electrode RS1 in the embodiments of the present application;

[0043] Figure 11-3 Schematic diagram of waveform changes of the first group of surface electromyogram signals corresponding to up - sliding actions in the embodiments of the present application;

[0044] Figure 11-4 It is a schematic diagram of the electrode response signal corresponding to the upward sliding action collected by the second receiving electrode RS2 in the embodiment of the present application;

[0045] Figure 11-5 It is a schematic diagram of the waveform change of the second group of surface electromyography signals for the upward sliding action in the embodiment of the present application;

[0046] Figure 12-1 It is a schematic diagram of the right finger wearing the first wearable device sliding downward on the surface of the left palm wearing the second wearable device in the embodiment of the present application;

[0047] Figure 12-2 It is a schematic diagram of the electrode response signal corresponding to the downward sliding action collected by the first receiving electrode RS1 in the embodiment of the present application;

[0048] Figure 12-3 It is a schematic diagram of the waveform change of the first group of surface electromyography signals for the downward sliding action in the embodiment of the present application;

[0049] Figure 12-4 It is a schematic diagram of the electrode response signal corresponding to the downward sliding action collected by the second receiving electrode in the embodiment of the present application;

[0050] Figure 12-5 It is a schematic diagram of the waveform change of the second group of surface electromyography signals for the downward sliding action in the embodiment of the present application;

[0051] Figure 13 It is a schematic diagram of the recognition process of a gesture action in the embodiment of the present application;

[0052] Figure 14 It is a schematic diagram of the structure of a control device based on surface electromyography signals in the embodiment of the present application;

[0053] Figure 15 It is a schematic diagram of the structure of a wearable device in the embodiment of the present application. Detailed implementation manners

[0054] To make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0055] As mentioned above, visual solutions are easily affected by light, shooting angle, etc., and the gesture recognition accuracy is low. In addition, the camera needs to be in working state all the time, which takes up computing resources and consumes a lot of power. To address this problem, the embodiment of the present application separates the collection and recognition process of gesture action signals from the head-mounted device and performs it on wearable devices such as smart bracelets and smart watches. After the wearable device recognizes the gesture action, it can send the gesture action to the head-mounted device via wired or wireless means to save the power consumption and system resources of the head-mounted device.

[0056] Based on this, the embodiment of the present application provides a control gesture recognition method based on surface electromyography signals, such as Figure 1 As shown, a flow chart of a control method based on surface electromyography signals in an embodiment of the present application is provided, and the method at least includes the following steps S110 to S130:

[0057] Step S110, when the transmitting electrode attached to the second limb of the human body releases an excitation signal and the second limb performs a gesture on the human skin of the first limb, two groups of response signals of the excitation signal are obtained through two groups of receiving electrodes attached to the first limb.

[0058] The embodiments of the present application realize control of a head-mounted device based on bare-hand interaction, specifically, realize bare-hand interaction based on surface electromyography (sEMG). Surface electromyography is a bioelectric current generated by the contraction of muscles on the surface of the human body. In layman's terms, the human body is likened to a huge battery. When the muscles are active, some electricity is released, and the time series of these changes in electricity is the surface electromyography.

[0059] In the embodiment of the present application, when the transmitting electrode attached to the second limb of the human body releases an excitation signal, and when the second limb performs a gesture on the human skin of the first limb, the response signal is collected by two groups of receiving electrodes attached to the first limb. In the embodiment of the present application, the first limb includes, for example, the right hand, and the second limb includes, for example, the left hand. The first wearable device is worn on the first limb, and the second wearable device is worn on the second limb. The transmitting electrode refers to an electromyographic electrode disposed on the first wearable device and attached to the skin of the first limb. The two groups of receiving electrodes refer to two groups of electromyographic electrodes disposed on the second wearable device and attached to the skin of the second limb. The two groups of receiving electrodes correspond to different conductive circuits.

[0060] exist Figure 2 In the gesture recognition system shown in FIG. 1 , the transmitting electrode TS of the first wearable device is controlled to release the excitation signal to the surface of the human skin, such as Figure 3As shown in the figure, the first wearable device includes an excitation source, a filter circuit, and an operational amplifier connected in sequence. The excitation source provides an excitation signal of about 10 MHz, for example. After being filtered and power-amplified, the excitation signal is conducted to the human skin surface of the right wrist by the transmitting electrode TS.

[0061] When the right finger makes a gesture on the left palm or the back of the left hand, the first receiving electrode RS1 and the second receiving electrode RS2 of the second wearable device collect the response signal of the above excitation signal through the corresponding conductive circuits. In practical applications, the two groups of receiving electrodes can synchronously collect the bioelectric signals on the human skin surface in response to the collection command to obtain two groups of synchronous response signals. The two groups of receiving electrodes can also be used to continuously monitor the bioelectric signals on the human skin surface and synchronously collect them when the amplitude of the bioelectric signals on the human skin surface is greater than a certain value.

[0062] Step S120: Obtain two groups of surface electromyography signals to be recognized according to the two groups of electrode response signals, and recognize the gesture actions according to the time-domain waveform change characteristics of the two groups of surface electromyography signals.

[0063] After obtaining the two groups of response signals of the excitation signal through the two groups of receiving electrodes attached to the first limb, the two groups of response signals can be converted into two groups of surface electromyography signals through the Figure 4 signal processing branch shown in the figure, that is, the response signals in the analog domain are converted into surface electromyography signals in the digital domain that can be recognized by the machine, so that the processor of the second wearable device can recognize the gesture actions based on the surface electromyography signals in the digital domain.

[0064] As Figure 5 shown in the figure, the first wearable device is worn on the right wrist, and the second wearable device is worn on the left wrist. When the right finger touches the left hand, the human skin acts as a conductor between the transmitting electrode TS and the receiving electrodes RS1 and RS2, so that the transmitting electrode TS, the first receiving electrode RS1, and the human skin between them form a first conductive circuit, and the transmitting electrode TS, the second receiving electrode RS2, and the human skin between them form a second conductive circuit. During the conduction of the excitation signal on the human skin, it is conducted to the two groups of receiving electrodes of the second wearable device by the two conductive circuits. The inventors found during the research and development process that the skin impedance changes corresponding to the two conductive circuits are related to the gesture actions that occur, so that the gesture actions can be recognized based on the time-domain waveform change characteristics of the surface electromyography signals. Here, the time-domain waveform change characteristics include time characteristics and amplitude characteristics. The amplitude characteristics can be understood as the amplitude changes of the time-domain waveform corresponding to the surface electromyography signals, including the amplitude change trend. The time characteristics can be understood as the time interval information corresponding to the amplitude changes of the time-domain waveform.

[0065] The impedance of these two conductive circuits includes skin resistance, which is related to factors such as contact voltage, contact area, contact pressure, and skin surface condition. When the contact voltage, contact area, contact pressure, etc. remain unchanged, the larger the conductive path, the greater the skin impedance, and the smaller the voltage amplitude corresponding to the conductive circuit.

[0066] like Figure 5 As shown, when a tapping action such as a single click, double click, etc. occurs, or as Figure 9-1 and Figure 10-1 As shown, when a left-right sliding action occurs along the first direction (in the embodiment of the present application, the first direction refers to the direction of the perpendicular bisector of the two groups of receiving electrodes), the skin impedance corresponding to the two conductive loops changes in the same manner, and accordingly, the amplitude change trends of the two groups of surface electromyographic signals are also the same; Figure 11-1 and Figure 12-1 As shown, when an up and down sliding action occurs along the second direction (in the embodiment of the present application, the second direction refers to the connection direction of the two groups of receiving electrodes, and the second direction is perpendicular to the first direction), the skin impedance corresponding to the two conductive circuits changes in the opposite direction, and accordingly, the amplitude change trends of the two groups of surface electromyography signals are also opposite.

[0067] The gestures in the embodiments of the present application include first-type actions and second-type actions. The time characteristics of the two groups of surface electromyographic signals corresponding to the two types of actions are consistent in the time domain, that is, the two groups of surface electromyographic signals undergo obvious amplitude changes within the same time interval in the time domain. The first type of gesture refers to the tapping action performed by the second limb on the human skin of the first limb, and the second type of gesture refers to the continuous contact action performed by the second limb on the human skin of the first limb.

[0068] The amplitude variation trends of the two groups of surface electromyographic signals corresponding to the tapping action and some continuous contact actions in the time domain are the same, such as the amplitude of the two groups of surface electromyographic signals gradually increases during the amplitude maintenance time, or the amplitude gradually decreases, or the amplitude fluctuates slightly. The amplitude variation trends of the two groups of surface electromyographic signals corresponding to the other part of the continuous contact action in the time domain are opposite, such as the amplitude of the first group of surface electromyographic signals gradually increases during the amplitude maintenance time, while the amplitude of the second group of surface electromyographic signals gradually decreases during the amplitude maintenance time.

[0069] Therefore, the embodiment of the present application can identify the above-mentioned gesture actions based on the time domain waveform change characteristics of the two groups of surface electromyography signals, which can minimize the interference of individual differences on the recognition of surface electromyography signals and improve the accuracy of gesture recognition.

[0070] Step S130: sending the recognized gesture action to the head mounted device to control the head mounted device.

[0071] After obtaining the recognition result of the gesture action, the second wearable device according to the embodiment of the present application can send the recognition result to the head-mounted device in a wired or wireless manner. For example, the gesture recognition result is sent to the head-mounted device based on the Bluetooth wireless transmission method, so that the head-mounted device performs corresponding control based on the gesture recognition result.

[0072] The surface electromyogram signal in the embodiment of the present application refers to a reprocessed signal obtained by performing time-domain feature extraction on the electrode response signal. Specifically, the electrode response signals of two groups are sampled. For example, the electrode response signals are uniformly sampled at a sampling rate of 3.2 KHz to obtain sampling signals; according to the sliding time window, the sampling signals are respectively divided into multiple signal segments, where the window length of the sliding time window is greater than the sliding step of the sliding time window. For example, the window length of the sliding time window is 30 sampling points, and the sliding step is 20 sampling points. In this way, the window length of the incremental window formed between the sliding time windows at adjacent times is 10 sampling points; the signal feature value of each signal segment is obtained according to the amplitude of the sampling points in each signal segment; the surface electromyogram signal is obtained according to the signal feature values of each signal segment.

[0073] It should be noted that the inventor found during the research and development process that, based on any one of the methods such as the mean absolute value (MAV for short), integrated absolute value (IAV for short), and maximum absolute value (denoted as f max ) of the sampling point amplitudes in the signal segment to obtain the signal feature value of each signal segment, and performing filtering, smoothing, etc. on the signal feature values of all signal segments, the obtained surface electromyogram signal has the time-domain waveform change characteristics required by the embodiment of the present application, while the surface electromyogram signal obtained based on other feature values of the sampling point amplitudes does not have the time-domain waveform change characteristics required by the embodiment of the present application and cannot complete the recognition of gesture actions based on the embodiment of the present application.

[0074] In this embodiment, the time-domain feature extraction of the electrode response signal is performed to obtain the required surface electromyogram signal. On the one hand, the extracted time-domain features can better reflect the characteristics of the signal compared to the original electrode signal and can reduce the dimension of the signal to be recognized. On the other hand, the method of using a sliding time window + incremental window for time-domain feature extraction can ensure the continuity of the signal features, which is convenient for subsequent recognition of gesture actions based on the time-domain waveform change characteristics of the surface electromyogram signal.

[0075] In an embodiment of the present application, the recognition of gesture actions according to the time-domain waveform change characteristics of the two groups of surface electromyogram signals in step S120 above includes:

[0076] After obtaining the peak holding time when the amplitude of any group of surface electromyography (sEMG) signals rises above the signal amplitude reference value and then drops below the signal amplitude reference value; if the peak holding time is less than or equal to the first time period threshold, the gesture action is recognized as the first type of gesture action; if the peak holding time is greater than the first time period threshold, the gesture action is recognized as the second type of gesture action.

[0077] Here, the signal amplitude reference value refers to the minimum value of the signal amplitude corresponding to the occurrence of the gesture action. As long as the amplitude of the sEMG signal to be recognized is greater than this minimum value, it can indicate that a gesture action has occurred. For example Figure 6-2 、 Figure 7-2 、 Figure 8-2 、 Figure 9-2 、 Figure 10-3 、 Figure 11-3 、 Figure 11-5 、 Figure 12-3 and Figure 12-5 、when the amplitude of the sEMG signal is greater than 2V0, a significant amplitude change occurs, that is, a relatively high target peak appears in the above figure. This target peak corresponds to a gesture action, and 2V0 is the signal amplitude reference value of this embodiment.

[0078] In the embodiment of the present application, the signal amplitude reference value 2V0 is obtained based on the reference amplitude V0. The reference amplitude V0 refers to the signal characteristic value of the electrode response signal when the first limb contacts the second limb but the gesture action has not started. Taking the absolute value maximum f max as an example, the reference amplitude V0 can be understood as follows: when the transmitting electrode releases an excitation signal to the human skin of the right wrist, the absolute value maximum f of the electrode response signal when the right finger touches the left palm (or the back of the left hand) but the gesture action has not started max . Among them, when the two groups of receiving electrodes are attached to the human skin on the palm side of the left wrist, the right finger touches the left palm; when the two groups of receiving electrodes are attached to the human skin on the back side of the left wrist, the right finger touches the back of the left hand.

[0079] Optionally, the reference amplitude V0 can be obtained by the second wearable device collecting the electrode response signal of the user's hand during the initialization process of the wearable device.

[0080] Of course, the reference amplitude V0 can also be obtained based on statistical data. For example, multiple sample population sets can be set according to different ages, genders, and occupations of people. After collecting the motion signals of each person in each sample population set, the above-mentioned absolute value maximum f corresponding to all people in the sample population set is obtained max , and the above-mentioned absolute value maximum f corresponding to all people max is subjected to statistical processing such as clustering, and the absolute value maximum f corresponding to the clustering center maxThe reference amplitude V0 of the sample population set. When starting gesture recognition, determine the sample population set to which the user belongs, and use the reference amplitude V0 of the sample population set to which the user belongs as the reference amplitude V0 required in this embodiment. For another example, it is also possible to only construct a sample population set in advance and use the reference amplitude V0 of this sample population set as the reference amplitude V0 required in this embodiment. Optionally, the reference amplitude V0 = 850 mV.

[0081] To facilitate the understanding of the following related embodiments of the present application, the present application embodiment first describes the relevant constraint regulations for each gesture action.

[0082] The gesture actions in the embodiments of the present application include the first type of gesture actions and the second type of gesture actions. Among them, the first type of gesture actions refers to the short-time (for example, the contact time less than 100 ms can be understood as short time) tapping actions performed by the second limb on the human skin of the first limb, including single-click actions and double-click actions; the second type of gesture actions refers to the long-time (for example, the contact time greater than 100 ms can be understood as long time) continuous contact actions performed by the second limb on the human skin of the first limb, including long-press actions, left-right sliding actions along the first direction, and up-down sliding actions along the second direction.

[0083] Among them, the left-right sliding actions include the left sliding along the first direction and the right sliding against the first direction. As Figure 9-1 shown, in this embodiment, it is set that the sliding from the palm center position 2 far from the two groups of receiving electrodes to the palm root position 1 close to the two groups of receiving electrodes is the left sliding along the first direction. As Figure 10-1 shown, the sliding from the palm root position 1 to the palm center position 2 is the right sliding against the first direction. The palm root position refers to the junction position of the outer edge of the palm and the arm.

[0084] The up-down sliding actions include sliding actions such as the upward sliding along the second direction and the downward sliding against the second direction. The relative distance between the first receiving electrode RS1 and the thumb position 3 is less than the relative distance between the second receiving electrode RS2 and the thumb position 3, and the relative distance between the first receiving electrode RS1 and the palm outer edge position 4 is greater than the relative distance between the second receiving electrode RS2 and the palm outer edge position 4. As Figure 11-1 shown, the sliding action from the palm outer edge position 4 to the thumb position 3 is set as the upward sliding along the second direction. As Figure 12-1 shown, the sliding action from the thumb position 3 to the palm outer edge position 4 is set as the downward sliding against the second direction.

[0085] When it is recognized that the gesture action is a first - type gesture action based on the peak - holding time being less than or equal to the first - period threshold, an embodiment of the present application further obtains the time - domain waveform change characteristics within the double - click observation period after the amplitude of this group of surface electromyogram signals drops below the signal amplitude reference value. If within the double - click observation period, the amplitude of this group of surface electromyogram signals continuously remains less than the signal amplitude reference value, the gesture action is recognized as a single - click action; if within the double - click observation period, the amplitude of this group of surface electromyogram signals rises above the signal amplitude reference value again and then drops below the signal amplitude reference value again within the first - period threshold and then remains below the signal amplitude reference value, the gesture action is recognized as a double - click action.

[0086] For example, when the time - domain waveforms of two groups of electrode response signals collected by two groups of receiving electrodes are as Figure 6-1 shown, after signal processing of the electrode response signals, the surface electromyogram signals to be recognized as shown in Figure 6-2 are obtained. The time inflection point when the amplitude of this surface electromyogram signal rises to the signal amplitude reference value 2V0 is t1, and it drops below the signal amplitude reference value 2V0 at the time inflection point t2. [t1, t2] is the peak - holding time. Since t2 - t1 < T1, and within the time [t2, t2 + T2], the amplitude of this surface electromyogram signal continuously remains less than the signal amplitude reference value 2V0, so the gesture action to be recognized is a single - click action. Here, T1 is the first - period threshold, optionally, T1 = 100ms, and T2 is the time - period length greater than T1, T2 = 500ms.

[0087] When the time - domain waveforms of two groups of electrode response signals collected by two groups of receiving electrodes are as Figure 7-1 shown, after signal processing of the electrode response signals, the surface electromyogram signals to be recognized as shown in Figure 7-2 are obtained. Since t2 - t1 < T1, and there is a subsequent target peak within the T2 time period, and t4 - t3 of the subsequent target peak < T1, so the gesture action to be recognized is a double - click action.

[0088] When it is recognized that the gesture action is a second - type gesture action based on the peak - holding time being greater than the first - period threshold, this embodiment further includes:

[0089] Respectively obtain the amplitude change trends of the two groups of surface electromyogram signals within the peak - holding time; if the amplitude change trends in the two groups of surface electromyogram signals are the same, the gesture action is recognized as a long - press action or a left - right swipe action; if the amplitude change trends in the two groups of surface electromyogram signals are different, the gesture action is recognized as an up - down swipe action.

[0090] As described above, the amplitude change trends of the surface electromyogram signals in the present application embodiment during the peak maintenance time include three change trends: the amplitude gradually increases, the amplitude gradually decreases, and the amplitude fluctuates slightly. The amplitude change trend of the target peak can be determined according to the amplitude change of the target peak at adjacent moments. For example, the peak maintenance time is equally divided into N parts, and the amplitude difference between adjacent time zones is calculated. If the amplitude of the current time zone is less than that of the next time zone and the amplitude difference is greater than the set value, it can be determined that the amplitude change trend of the target peak is gradually increasing; similarly, if the amplitude of the current time zone is greater than that of the next time zone and the amplitude difference is greater than the set value, it can be determined that the amplitude change trend of the target peak is gradually decreasing; if the difference between the amplitude of the current time zone and that of the next time zone is not greater than the set value, it can be determined that the amplitude change trend of the target peak is a slight amplitude fluctuation. In this way, it can be determined whether the amplitude change trends of the two sets of surface electromyogram signals during the peak maintenance time are the same.

[0091] Of course, in other embodiments, the amplitude change trend of the surface electromyogram signal during the peak maintenance time can also be judged based on the slope feature. For example, the fitting line corresponding to the target peak is determined, and the amplitude change trend of the surface electromyogram signal during the peak maintenance time is determined according to the slope of the fitting line.

[0092] When identifying the gesture action as a long - press action or a left - right sliding action, the waveform tilt angle θ of any set of surface electromyogram signals during the peak maintenance time can also be obtained. If θ < Ang th 1, Ang th 1 is the first inclination threshold, and it is also necessary to confirm whether the peak maintenance time of this set of surface electromyogram signals is greater than the long - press time threshold T3. If it is greater than the long - press time threshold T3, the gesture action is identified as a long - press action. Optionally, T3 = 1000ms.

[0093] If θ > Ang th 2, Ang th 2 is the second inclination threshold, and the gesture action is identified as a right - sliding action along the first direction; if Ang th 2 ≥ θ ≥ Ang th 1, the gesture action is identified as a left - sliding action against the first direction.

[0094] When identifying the gesture action as an up - down sliding action, the first waveform tilt angle θ1 of the first set of surface electromyogram signals during the peak maintenance time and the second waveform tilt angle θ2 of the second set of surface electromyogram signals during the peak maintenance time are respectively obtained; if Ang th 2 ≥ θ1 ≥ Ang th 1, and θ2 > Ang th 2, the gesture action is identified as an up - sliding action along the second direction; if θ1 > Ang th 2, and Angth 2 ≥ θ2 ≥ Ang th 1. Identify the gesture action as a downward sliding action. Optionally, Ang th 1 = 5°, Ang th 2 = 90°.

[0095] The waveform tilt angle in the above embodiments represents the degree of tilt of the time-domain waveform of the surface electromyogram signal during the peak maintenance time relative to the reference horizontal line. As Figure 8-2 , Figure 9-3 , Figure 10-3 , Figure 11-3 , Figure 11-5 , Figure 12-3 and Figure 12-5 shown, the waveform tilt angle can be represented by the angle between the fitting line corresponding to the time-domain waveform and the reference horizontal line. Of course, in other embodiments, the angle between the tangents of the time-domain waveform at adjacent sampling points can also be calculated, and the sum of the angles between all adjacent tangents can be used to represent the waveform tilt angle. The tangent refers to the straight line determined by the amplitudes of two adjacent sampling points on the time-domain waveform.

[0096] It should be noted that when determining whether the amplitude change trends of two sets of surface electromyogram signals are the same and when determining the amplitude change trend of the surface electromyogram signal, the same amplitude feature can be used to improve the recognition efficiency. For example, the judgment can be based on the waveform tilt angle. When 90° ≥ θ ≥ 5°, the amplitude change trend of the surface electromyogram signal is that the amplitude gradually increases. When θ > 90°, the amplitude change trend of the surface electromyogram signal is that the amplitude gradually decreases. When θ < 5°, the amplitude change trend of the surface electromyogram signal is that the amplitude fluctuates slightly. It can also be based on different amplitude features to improve the recognition accuracy. For example, based on a relatively simple calculation scheme (the amplitude difference between adjacent time zones), it is determined whether the amplitude change trends of two sets of surface electromyogram signals are the same, and based on the waveform tilt angle, the amplitude change of the surface electromyogram signal during the peak maintenance time is judged.

[0097] For example, when the time-domain waveforms of the two sets of electrode response signals collected by two sets of receiving electrodes are as Figure 8-1 shown, after the electrode response signals are processed through the foregoing embodiments, the surface electromyogram signal to be recognized as shown in Figure 8-2 is obtained. After the amplitude of the surface electromyogram signal rises above the signal amplitude reference value 2V0 and maintains for more than T3, it then drops below the signal amplitude reference value 2V0, and during the peak maintenance time, the waveform tilt angle θ < Ang th 1. Therefore, the gesture action to be recognized is a long press action.

[0098] When the right finger moves on the surface of the left palm as Figure 9-1When the gesture action shown occurs, the time-domain waveforms of the two groups of electrode response signals collected by the two groups of receiving electrodes are as follows Figure 9-2 shown. After signal processing of the electrode response signals, the surface electromyogram signals to be recognized are obtained as follows Figure 9-3 shown. Since the amplitude of the surface electromyogram signal rises above the signal amplitude reference value 2V0 and remains above T1, then drops below the signal amplitude reference value 2V0, and within the peak holding time, Ang th 1 ≤ θ ≤ Ang th 2, so the gesture action to be recognized is a left swipe action.

[0099] When the right finger makes the gesture action shown on the surface of the left palm, the time-domain waveforms of the two groups of electrode response signals collected by the two groups of receiving electrodes are as follows Figure 10-1 shown. After signal processing of the electrode response signals, the surface electromyogram signals to be recognized are obtained as follows Figure 10-2 shown. Since the amplitude of the surface electromyogram signal rises above the signal amplitude reference value 2V0 and remains above T1, then drops below the signal amplitude reference value 2V0, and within the peak holding time, the waveform tilt angle θ > Ang Figure 10-3 shown. Since the amplitude of the surface electromyogram signal rises above the signal amplitude reference value 2V0 and remains above T1, then drops below the signal amplitude reference value 2V0, and within the peak holding time, the waveform tilt angle θ > Ang th 2, so the gesture action to be recognized is a right swipe action.

[0100] When the right finger makes the gesture action shown on the surface of the left palm, the time-domain waveform of the first group of electrode response signals collected by the first group of receiving electrodes RS1 is as follows Figure 11-1 shown, and the time-domain waveform of the second group of electrode response signals collected by the second group of receiving electrodes RS2 is as follows Figure 11-2 shown. After signal processing of the first group of electrode response signals, the first group of surface electromyogram signals to be recognized are obtained as follows Figure 11-4 shown, and the time-domain waveform of the second group of electrode response signals collected by the second group of receiving electrodes RS2 is as follows Figure 11-3 shown. After signal processing of the second group of electrode response signals, the second group of surface electromyogram signals to be recognized are obtained as follows Figure 11-5 shown. Since in the two groups of surface electromyogram signals, the amplitude of the surface electromyogram signal rises above the signal amplitude reference value 2V0 and remains above T1, then drops below the signal amplitude reference value 2V0, and within the peak holding time, Ang th 1 ≤ θ1 ≤ Ang th 2, θ2 > Ang th 2, so the gesture action to be recognized is an upward swipe action.

[0101] When the right finger makes the gesture action shown on the surface of the left palm, the time-domain waveform of the first group of electrode response signals collected by the first group of receiving electrodes RS1 is as follows Figure 12-1 shown, and the time-domain waveform of the second group of electrode response signals collected by the second group of receiving electrodes RS2 is as follows Figure 12-2 shown, and the time-domain waveform of the second group of electrode response signals collected by the second group of receiving electrodes RS2 is as followsFigure 12-4 As shown, after signal processing of the first group of electrode response signals, the first group of surface electromyography signals to be recognized as shown in Figure 12-3 is obtained. After signal processing of the second group of electrode response signals, the second group of surface electromyography signals to be recognized as shown in Figure 12-5 is obtained. Since in the two groups of surface electromyography signals, after the amplitude of the surface electromyography signal rises above the signal amplitude reference value of 2V0 and remains above T1, it then drops below the signal amplitude reference value of 2V0, and within the peak holding time, θ1 > Ang th 2, Ang th 1 ≤ θ2 ≤ Ang th 2, so the gesture action to be recognized is a downward sliding action.

[0102] It should be noted that in the process of gesture action recognition in the above embodiments of the present application, six thresholds need to be preset, namely the reference amplitude V0, the first period threshold T1, the double-click observation time period T2, the long-press time threshold T3, the first inclination threshold Ang th 1 and the second inclination threshold Ang th 2. Among them, the setting method of the reference amplitude V0 refers to the relevant embodiments in the previous text. The setting methods of the first period threshold T1, the double-click observation time period T2, the long-press time threshold T3, the first inclination threshold Ang th 1 and the second inclination threshold Ang th 2 can refer to the setting method of the reference amplitude V0, and will not be elaborated here in this embodiment.

[0103] To facilitate the understanding of the above embodiments of the present application, the gesture recognition process is described in detail below. As Figure 13 shown, the gesture recognition process includes the following steps S1301 to step S1313:

[0104] Step S1301 samples the electrode response signals of the two groups of receiving electrodes at a sampling rate of 3.2 KHz. Step S1302 performs signal segmentation using the sliding time window + incremental window method. Optionally, the window length of the sliding time window is 30 sampling points, and the window length of the incremental window is 10 sampling points. Step S1303 extracts the signal feature values of each segmented signal, such as the average absolute value feature, the integral absolute value feature, or the absolute value maximum feature, and performs filtering, smoothing, etc. on the extracted signal feature values to obtain the two groups of surface electromyography signals to be recognized.

[0105] Step S1304 analyzes the time-domain waveform change characteristics of the two groups of surface electromyography signals, specifically including:

[0106] In step S1306, if the amplitude of any group of surface electromyogram signals rises above the signal amplitude reference value of 2V0, then drops below 2V0 within 100 ms, and remains below 2V0 for 500 ms, the gesture occurring on the surface of the left palm is recognized as a click action;

[0107] In step S1307, if the amplitude of any group of surface electromyogram signals rises above the signal amplitude reference value of 2V0, then drops below 2V0 within 100 ms, then rises above 2V0 again within 500 ms, drops below 2V0 again within 100 ms, and then remains below 2V0, the gesture occurring on the surface of the left palm is recognized as a double - click action;

[0108] In step S1309, if the amplitude change trends of two groups of surface electromyogram signals are the same, and the amplitude of any group of surface electromyogram signals rises above the signal amplitude reference value of 2V0, remains above for more than 1000 ms and then drops below 2V0, and the waveform tilt angle θ < 5°, the gesture occurring on the surface of the left palm is recognized as a long - press action;

[0109] In step S1310, if the amplitude change trends of two groups of surface electromyogram signals are the same, and the amplitude of any group of surface electromyogram signals rises above the signal amplitude reference value of 2V0, remains above for more than 100 ms and then drops below 2V0, and the waveform tilt angle 5° ≤ θ ≤ 90°, the gesture occurring on the surface of the left palm is recognized as a left - swipe action;

[0110] In step S1311, if the amplitude change trends of two groups of surface electromyogram signals are the same, and the amplitude of any group of surface electromyogram signals rises above the signal amplitude reference value of 2V0, remains above for more than 100 ms and then drops below 2V0, and the waveform tilt angle θ > 90°, the gesture occurring on the surface of the left palm is recognized as a right - swipe action;

[0111] In step S1312, if the amplitude change trends of two groups of surface electromyogram signals are different, the amplitude of the first group of surface electromyogram signals all rises above the signal amplitude reference value of 2V0, remains above for more than 100 ms and then drops below 2V0, and the first waveform tilt angle 90° ≥ θ1 ≥ 5°; while the amplitude of the second group of surface electromyogram signals all rises above the signal amplitude reference value of 2V0, remains above for more than 100 ms and then drops below 2V0, and the second waveform tilt angle θ2 > 90°, the gesture occurring on the surface of the left palm is recognized as an up - swipe action;

[0112] In step S1313, if the amplitude change trends of the two sets of surface electromyogram signals are different, the amplitudes of the first set of surface electromyogram signals all rise above the signal amplitude reference value of 2V0 and remain above 2V0 for more than 100 ms and then drop below 2V0, and the inclination angle θ1 of the first waveform is greater than 90°; while the amplitudes of the second set of surface electromyogram signals all rise above the signal amplitude reference value of 2V0 and remain above 2V0 for more than 100 ms and then drop below 2V0, and the inclination angle 90°≥θ2≥5° of the second waveform, the gesture action occurring on the surface of the left palm is recognized as a downward sliding action.

[0113] In order to improve the recognition efficiency in this embodiment, when analyzing the time-domain waveform change characteristics of the two sets of surface electromyogram signals, first perform step S1305 to calculate the peak holding time of any set of surface electromyogram signals. The peak holding time refers to the time period [t1, t2] between the time inflection point t1 when the amplitude of the surface electromyogram signal rises to 2V0 and the time inflection point t2 when the amplitude drops below 2V0. Determine whether t2 - t1 is less than or equal to 100 ms, and select the corresponding gesture recognition branch for gesture action recognition based on the judgment result.

[0114] In order to further improve the recognition efficiency in this embodiment, when it is determined that t2 - t1 > 100 ms, also perform step S1308 to determine whether the amplitude change trends of the two sets of surface electromyogram signals are the same within the peak holding time [t1, t2], and select the corresponding gesture recognition branch for gesture action recognition based on the judgment result.

[0115] Through the above embodiments, the present application can accurately recognize various set gesture actions. The recognition process does not require the aid of a handle, can free both hands, and achieve bare-handed interaction. Compared with the mainstream visual solutions, the present application is not affected by light, has no angular dead ends for gesture actions, and does not require the use of image acquisition devices such as cameras, which can save computing resources and electric energy. In addition, the present application performs gesture action recognition based on the time-domain waveform change characteristics of two sets of surface electromyogram signals, and improves the recognition accuracy and efficiency through at least two-dimensional features (i.e., amplitude feature and time feature) of the time-domain waveform change characteristics in the recognition process.

[0116] Belonging to the same technical concept as the control method based on surface electromyogram signals in the foregoing embodiments, the embodiment of the present application further provides a control device based on surface electromyogram signals for implementing the control method based on surface electromyogram signals in the foregoing embodiments.

[0117] Figure 14 The structural schematic diagram of the control device based on surface electromyogram signals according to an embodiment of the present application is shown, as Figure 14 shown, the control device 1400 based on surface electromyogram signals includes: a signal acquisition unit 1410, a signal processing unit 1420, and a gesture sending unit 1430;

[0118] A signal acquisition unit 1410, configured to receive two sets of electrode response signals of the excitation signal through two sets of receiving electrodes attached to the first limb when the transmitting electrode attached to the second limb of the human body releases the excitation signal and the second limb makes a gesture on the human skin of the first limb;

[0119] A signal processing unit 1420, configured to obtain two sets of surface electromyographic signals to be recognized according to the two sets of electrode response signals, and recognize the gesture according to the time-domain waveform change characteristics of the two sets of surface electromyographic signals;

[0120] A gesture sending unit 1430, configured to send the recognized gesture to the head-mounted device for controlling the head-mounted device.

[0121] In an embodiment of the present application, the signal acquisition unit 1410 is specifically configured to perform sampling processing on the two sets of electrode response signals to obtain two sets of sampling signals; divide the two sets of sampling signals into multiple signal segments according to a sliding time window respectively, where the window length of the sliding time window is greater than the sliding step of the sliding time window; obtain the signal feature value of each signal segment according to the amplitude of the sampling points in each signal segment; and obtain the surface electromyographic signal according to the signal feature value of each signal segment.

[0122] In an embodiment of the present application, the gesture includes a first type of gesture and a second type of gesture. The first type of gesture refers to a tapping action performed by the second limb on the human skin of the first limb, and the second type of gesture refers to a continuous contact action performed by the second limb on the human skin of the first limb. The signal processing unit 1420 is configured to obtain the peak holding time from above the signal amplitude reference value to below the signal amplitude reference value after the amplitude of any set of surface electromyographic signals rises above the signal amplitude reference value; if the peak holding time is less than or equal to a first time period threshold, recognize the gesture as a first type of gesture; if the peak holding time is greater than the first time period threshold, recognize the gesture as a second type of gesture.

[0123] The first type of gesture includes a single click action and a double click action. The signal processing unit 1420 is specifically configured to, if the peak holding time is less than or equal to the first time period threshold, obtain the time-domain waveform change characteristics within the double click observation time period after the amplitude of the set of surface electromyographic signals drops below the signal amplitude reference value; if within the double click observation time period, the amplitude of the set of surface electromyographic signals continuously remains less than the signal amplitude reference value, recognize the gesture as a single click action; if within the double click observation time period, the amplitude of the set of surface electromyographic signals rises above the signal amplitude reference value again and drops below the signal amplitude reference value again within the first time period threshold and then remains below the signal amplitude reference value, recognize the gesture as a double click action.

[0124] The second type of gesture actions includes long-press actions and sliding actions. The sliding actions include left-right sliding actions along the first direction and up-down sliding actions along the second direction. The signal processing unit 1420 is further configured to, if the crest maintenance time is greater than the first time period threshold, respectively obtain the amplitude change trends of the two sets of surface electromyogram signals during the crest maintenance time; if the amplitude change trends of the two sets of surface electromyogram signals are the same, identify the gesture action as a long-press action or a left-right sliding action; if the amplitude change trends of the two sets of surface electromyogram signals are different, identify the gesture action as an up-down sliding action.

[0125] In an embodiment of the present application, the signal processing unit 1420 is specifically configured to, if the amplitude change trends of the two sets of surface electromyogram signals are the same, obtain the waveform tilt angle of any one set of surface electromyogram signals during the crest maintenance time; if the waveform tilt angle is less than the first tilt angle threshold, further confirm whether the crest maintenance time of this set of surface electromyogram signals is greater than the long-press time threshold, and if it is greater than the long-press time threshold, identify the gesture action as a long-press action; if the waveform tilt angle is greater than the second tilt angle threshold, identify the gesture action as a rightward sliding action along the first direction; if the waveform tilt angle is greater than or equal to the first tilt angle threshold and less than or equal to the second tilt angle threshold, identify the gesture action as a leftward sliding action against the second direction;

[0126] And, if the amplitude change trends of the two sets of surface electromyogram signals are different, respectively obtain the first waveform tilt angle of the first set of surface electromyogram signals during the crest maintenance time and the second waveform tilt angle of the second set of surface electromyogram signals during the crest maintenance time; if the first waveform tilt angle is greater than or equal to the first tilt angle threshold and less than or equal to the second tilt angle threshold, and the second waveform tilt angle is greater than the second tilt angle threshold, identify the gesture action as an upward sliding action along the second direction; if the first waveform tilt angle is greater than the second tilt angle threshold, and the second waveform tilt angle is greater than or equal to the first tilt angle threshold and less than or equal to the second tilt angle threshold, identify the gesture action as a downward sliding action against the second direction.

[0127] It can be understood that the above control device based on surface electromyogram signals can implement each step of the control method based on surface electromyogram signals provided in the foregoing embodiments. The relevant explanations of the control method based on surface electromyogram signals are all applicable to the control device based on surface electromyogram signals, and will not be elaborated here.

[0128] Figure 15 A schematic structural diagram of a wearable device in an embodiment of the present application is provided. Please refer to Figure 15, at the hardware level, the wearable device includes a processor and a memory, and optionally also includes an internal bus and a network interface. Among them, the memory may include a memory, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory, such as at least one disk memory, etc. Of course, the wearable device may also include other hardware required for other services, and also includes two sets of receiving electrodes, each set of receiving electrodes including at least one electromyogram electrode. The present application does not limit the number of electromyogram electrodes in each set of receiving electrodes, the relative positions between the electromyogram electrodes, and the types of electromyogram electrodes. Optionally, two signal processing branches may also be included, and the input end of each signal processing branch is connected to a corresponding set of receiving electrode groups, and is used to convert the response signals received by the set of receiving electrode groups into surface electromyogram signals to be recognized and then output to the processor.

[0129] Reference Figure 4 As shown, each signal processing branch includes a filter circuit, an operational amplifier, and an analog-to-digital converter (Analog To Digital Converter, abbreviated as ADC) connected in sequence. Optionally, the two signal processing branches may share an analog-to-digital converter.

[0130] The processor, interface module, communication module, and memory may be interconnected through an internal bus, and the internal bus may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 15 only a two-way arrow is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0131] The memory is used to store computer-executable instructions. The memory provides computer-executable instructions to the processor through the internal bus.

[0132] The processor executes the computer-executable instructions stored in the memory, and is specifically used to implement the control method based on surface electromyogram signals in the foregoing embodiments.

[0133] As described above in the present application Figure 1The functions performed by the control method based on surface electromyogram signals disclosed in the illustrated embodiments can be applied to or implemented by a processor. The processor may be an integrated circuit chip with the ability to process signals. During implementation, the steps of the above method can be completed by the integrated logic circuit in hardware in the processor or by instructions in software form.

[0134] An embodiment of the present application also proposes a computer-readable storage medium that stores one or more programs, and when the one or more programs are executed by a processor, the control method based on surface electromyogram signals in the foregoing embodiments is implemented.

[0135] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0136] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device, and the instruction device implements the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 the functions specified in one block or multiple blocks.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 the steps of the functions specified in one block or multiple blocks.

[0138] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and a memory.

[0139] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM), and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

[0140] Computer-readable media include permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. As defined in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0141] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0142] It should be understood that although the terms first, second, third, etc. may be used to describe various information in the present invention, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information.

[0143] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A control method based on surface electromyogram signals, characterized in that, The method includes: When the emission electrode attached to the second limb of the human body releases an excitation signal and the second limb makes a gesture on the human skin of the first limb, two groups of electrode response signals of the excitation signal are obtained through two groups of receiving electrodes attached to the first limb. Two groups of surface electromyography signals to be recognized are obtained according to the two groups of electrode response signals, and the gesture is recognized according to the time-domain waveform change characteristics of the two groups of surface electromyography signals. The time-domain waveform change characteristics include time characteristics and amplitude characteristics. The amplitude characteristic is the amplitude change of the time-domain waveform corresponding to the surface electromyography signal, and the time characteristic is the time interval information corresponding to the amplitude change of the time-domain waveform. The recognized gesture is sent to the head-mounted device for controlling the head-mounted device. The obtaining of the two groups of surface electromyography signals to be recognized according to the two groups of electrode response signals includes: Sampling the two groups of electrode response signals to obtain two groups of sampling signals; dividing the two groups of sampling signals into multiple signal segments respectively according to a sliding time window, where the window length of the sliding time window is greater than the sliding step of the sliding time window; obtaining the signal characteristic value of each signal segment according to any one of the average absolute value, integral absolute value, and absolute maximum value of the sampling point amplitudes in each signal segment; and obtaining the surface electromyography signal according to the signal characteristic value of each signal segment.

2. The method according to claim 1, wherein The gesture includes a first type of gesture and a second type of gesture. The first type of gesture refers to a tapping action made by the second limb on the human skin of the first limb, and the second type of gesture refers to a continuous contact action made by the second limb on the human skin of the first limb. The recognizing of the gesture according to the time-domain waveform change characteristics of the two groups of surface electromyography signals includes: Obtaining the peak holding time after the amplitude of any group of surface electromyography signals rises above the signal amplitude reference value and then drops below the signal amplitude reference value. If the peak holding time is less than or equal to a first time period threshold, the gesture is recognized as a first type of gesture. If the peak holding time is greater than the first time period threshold, the gesture is recognized as a second type of gesture.

3. The method according to claim 2, characterized in that, The first type of gesture includes a single click action and a double click action. The recognizing of the gesture as a first type of gesture if the peak holding time is less than or equal to a first time period threshold includes: Obtaining the time-domain waveform change characteristics within a double click observation time period after the amplitude of this group of surface electromyography signals drops below the signal amplitude reference value. If within the double click observation time period, the amplitude of this group of surface electromyography signals continuously remains less than the signal amplitude reference value, the gesture is recognized as a single click action. If within the double click observation time period, the amplitude of this group of surface electromyography signals rises above the signal amplitude reference value again and then drops below the signal amplitude reference value again within the first time period threshold and then remains below the signal amplitude reference value, the gesture is recognized as a double click action.

4. The method according to claim 2, wherein The second type of gesture actions includes a long - press action and a sliding action. The sliding action includes a left - right sliding action along a first direction and an up - down sliding action along a second direction. If the crest duration is greater than the first time - period threshold, identifying the gesture action as a second - type gesture action includes: Obtaining the amplitude change trends of two sets of surface electromyography (sEMG) signals during the crest duration respectively; If the amplitude change trends of the two sets of sEMG signals are the same, identifying the gesture action as a long - press action or the left - right sliding action; If the amplitude change trends of the two sets of sEMG signals are different, identifying the gesture action as the up - down sliding action.

5. The method according to claim 4, characterized in that, The step of, if the amplitude change trends of the two sets of sEMG signals are the same, identifying the gesture action as a long - press action or the left - right sliding action, includes: Obtaining the waveform tilt angle of any one set of sEMG signals during the crest duration; If the waveform tilt angle is less than the first tilt - angle threshold, further confirming whether the crest duration of this set of sEMG signals is greater than the long - press time threshold. If it is greater than the long - press time threshold, identifying the gesture action as a long - press action; If the waveform tilt angle is greater than the second tilt - angle threshold, identifying the gesture action as a right - sliding action along the first direction; If the waveform tilt angle is greater than or equal to the first tilt - angle threshold and less than or equal to the second tilt - angle threshold, identifying the gesture action as a left - sliding action against the second direction.

6. The method according to claim 5, wherein The step of, if the amplitude change trends of the two sets of sEMG signals are different, identifying the gesture action as the up - down sliding action, includes: Obtaining the first waveform tilt angle of the first set of sEMG signals and the second waveform tilt angle of the second set of sEMG signals during the crest duration respectively; If the first waveform tilt angle is greater than or equal to the first tilt - angle threshold and less than or equal to the second tilt - angle threshold, and the second waveform tilt angle is greater than the second tilt - angle threshold, identifying the gesture action as an up - sliding action along the second direction; If the first waveform tilt angle is greater than the second tilt - angle threshold, and the second waveform tilt angle is greater than or equal to the first tilt - angle threshold and less than or equal to the second tilt - angle threshold, identifying the gesture action as a down - sliding action against the second direction.

7. A control device based on surface electromyogram signals, characterized in that, The device includes: A signal acquisition unit, configured to receive two sets of electrode response signals of the excitation signal through two sets of receiving electrodes attached to the first limb when the transmitting electrode attached to the second limb of the human body releases an excitation signal and the second limb makes a gesture action on the human skin of the first limb; A signal processing unit, configured to obtain two sets of sEMG signals to be recognized according to the two sets of electrode response signals, and identify the gesture action according to the time - domain waveform change characteristics of the two sets of sEMG signals. The time - domain waveform change characteristics include time characteristics and amplitude characteristics. The amplitude characteristic is the amplitude change of the time - domain waveform corresponding to the sEMG signal, and the time characteristic is the time - interval information corresponding to the amplitude change of the time - domain waveform; A gesture sending unit, configured to send the recognized gesture actions to a head-mounted device for controlling the head-mounted device; The signal processing unit is specifically configured to perform sampling processing on the two groups of electrode response signals to obtain two groups of sampling signals; divide the two groups of sampling signals into multiple signal segments according to a sliding time window, where the window length of the sliding time window is greater than the sliding step of the sliding time window; obtain the signal feature value of each signal segment according to any one of the average absolute value, integral absolute value, and absolute value maximum of the amplitudes of the sampling points in each signal segment; and obtain the surface electromyogram signal according to the signal feature value of each signal segment.

8. A wearable device, characterized in that, Comprising: A processor; And A memory arranged to store computer-executable instructions, which when executed cause the processor to execute a control method based on surface electromyogram signals, the method comprising: When the transmitting electrode attached to the second limb of the human body releases an excitation signal and the second limb makes a gesture action on the human skin of the first limb, two groups of electrode response signals of the excitation signal are obtained through two groups of receiving electrodes attached to the first limb; Obtain two groups of surface electromyogram signals to be recognized according to the two groups of electrode response signals, and recognize the gesture action according to the time-domain waveform change characteristics of the two groups of surface electromyogram signals, where the time-domain waveform change characteristics include time characteristics and amplitude characteristics, the amplitude characteristic is the amplitude change of the time-domain waveform corresponding to the surface electromyogram signal, and the time characteristic is the time interval information corresponding to the amplitude change of the time-domain waveform; Send the recognized gesture actions to a head-mounted device for controlling the head-mounted device; The obtaining two groups of surface electromyogram signals to be recognized according to the two groups of electrode response signals includes: Performing sampling processing on the two groups of electrode response signals to obtain two groups of sampling signals; dividing the two groups of sampling signals into multiple signal segments according to a sliding time window, where the window length of the sliding time window is greater than the sliding step of the sliding time window; obtaining the signal feature value of each signal segment according to any one of the average absolute value, integral absolute value, and absolute value maximum of the amplitudes of the sampling points in each signal segment; and obtaining the surface electromyogram signal according to the signal feature value of each signal segment.

9. The wearable device according to claim 8, wherein, The gesture actions include a first type of gesture action, which refers to a tapping action performed by the second limb on the human skin of the first limb, including a single-click action and a double-click action, and the processor is further configured to execute: Obtain the peak holding time after the amplitude of any group of surface electromyogram signals rises above the signal amplitude reference value and then drops below the signal amplitude reference value; If the peak holding time is less than or equal to a first time period threshold, recognize the gesture action as a first type of gesture action; Wherein, when recognizing the gesture action as a first type of gesture action, the processor is further configured to execute: After the amplitude of the obtained group of surface electromyography signals drops below the signal amplitude reference value, obtain the time-domain waveform change characteristics within the double-click observation time period; if within the double-click observation time period, the amplitude of the group of surface electromyography signals continuously remains less than the signal amplitude reference value, identify the gesture as a single-click action; if within the double-click observation time period, the amplitude of the group of surface electromyography signals rises above the signal amplitude reference value again and then drops below the signal amplitude reference value again within the first time period threshold and then remains below the signal amplitude reference value, identify the gesture as a double-click action.

10. The wearable device according to claim 9, wherein, The gesture further includes a second type of gesture, which refers to a continuous contact action of the second limb on the human skin of the first limb, including a long-press action and a sliding action. The sliding action includes a left-right sliding action along a first direction and an up-down sliding action along a second direction. The processor is further configured to execute: If the peak holding time is greater than the first time period threshold, identify the gesture as a second type of gesture. Obtain the amplitude change trends of two groups of surface electromyography signals within the peak holding time respectively; if the amplitude change trends of the two groups of surface electromyography signals are the same, identify the gesture as a long-press action or the left-right sliding action. If the amplitude change trends of the two groups of surface electromyography signals are different, identify the gesture as the up-down sliding action. Wherein, when identifying the gesture as a long-press action or the left-right sliding action, the processor is further configured to execute: Obtain the waveform tilt angle of any group of surface electromyography signals within the peak holding time; if the waveform tilt angle is less than the first tilt angle threshold, further confirm whether the peak holding time of this group of surface electromyography signals is greater than the long-press time threshold. If it is greater than the long-press time threshold, identify the gesture as a long-press action. If the waveform tilt angle is greater than the second tilt angle threshold, identify the gesture as a rightward sliding action along the first direction; if the waveform tilt angle is greater than or equal to the first tilt angle threshold and less than or equal to the second tilt angle threshold, identify the gesture as a leftward sliding action against the second direction. And, when identifying the gesture as the up-down sliding action, the processor is further configured to execute: Obtain the first waveform tilt angle of the first group of surface electromyography signals within the peak holding time and the second waveform tilt angle of the second group of surface electromyography signals within the peak holding time respectively; if the first waveform tilt angle is greater than or equal to the first tilt angle threshold and less than or equal to the second tilt angle threshold, and the second waveform tilt angle is greater than the second tilt angle threshold, identify the gesture as an upward sliding action along the second direction; if the first waveform tilt angle is greater than the second tilt angle threshold, and the second waveform tilt angle is greater than or equal to the first tilt angle threshold and less than or equal to the second tilt angle threshold, identify the gesture as a downward sliding action against the second direction.

11. The wearable device according to claim 8, wherein It includes two groups of receiving electrode groups, and each group of receiving electrode groups includes at least one electromyogram electrode.

12. The wearable device according to claim 11, wherein It further includes two signal processing branches. The input end of each signal processing branch is connected to a corresponding group of receiving electrode groups, and is used to convert the response signals received by this group of receiving electrode groups into surface electromyogram signals to be recognized and then output to the processor.

13. A computer-readable storage medium stores one or more programs. When the one or more programs are executed by a processor, a control method based on surface electromyogram signals is implemented. The method includes: When the transmitting electrode attached to the second limb of the human body releases an excitation signal and the second limb makes a gesture on the human skin of the first limb, two groups of electrode response signals of the excitation signal are obtained through two groups of receiving electrodes attached to the first limb. Two groups of surface electromyogram signals to be recognized are obtained according to the two groups of electrode response signals, and the gesture is recognized according to the time-domain waveform change characteristics of the two groups of surface electromyogram signals. The time-domain waveform change characteristics include time characteristics and amplitude characteristics. The amplitude characteristic is the amplitude change of the time-domain waveform corresponding to the surface electromyogram signal, and the time characteristic is the time interval information corresponding to the amplitude change of the time-domain waveform. The recognized gesture is sent to the head-mounted device for head-mounted device control. The obtaining of the two groups of surface electromyogram signals to be recognized according to the two groups of electrode response signals includes: Sampling the two groups of electrode response signals to obtain two groups of sampling signals. According to a sliding time window, the two groups of sampling signals are respectively segmented into multiple signal segments. The window length of the sliding time window is greater than the sliding step length of the sliding time window. The signal feature value of each signal segment is obtained according to any one of the average absolute value, integral absolute value, and maximum absolute value of the amplitudes of the sampling points in each signal segment. The surface electromyogram signal is obtained according to the signal feature values of each signal segment.

Images