Gesture interaction method, circuit, smart wearable device, storage medium and product

Abstract

The present application discloses a gesture interaction method, circuit, smart wearable device, storage medium and product, which relates to the field of virtual reality interaction technology. The gesture interaction method is applied to smart wearable devices, and the smart wearable devices are equipped with an electromyography monitoring unit and an inertia detection unit. The gesture interaction method includes: collecting the user's hand electromyography information through the electromyography monitoring unit, and collecting the user's hand inertia information through the inertia detection unit; identifying the corresponding gesture interaction action according to the hand electromyography information and the hand inertia information; and sending the interaction control instruction corresponding to the gesture interaction action to the head-mounted display device connected to the smart wearable device. The present application solves the technical problem that the existing gesture interaction method affects the battery life of the head-mounted display device.

Description

Technical Field

[0001] The present application relates to the field of virtual reality interaction technology, and in particular to a gesture interaction method, circuit, smart wearable device, storage medium and product. Background Art

[0002] As a rapidly developing technology field in recent years, virtual reality interaction technology has greatly enriched the user experience. As an important means of interaction in virtual reality interaction scenarios, gesture interaction technology has received widespread attention and research.

[0003] In current gesture interaction solutions, in order to identify the user's current gesture, the user's hand is generally captured in real time through a camera configured on a head-mounted display device (such as a virtual reality helmet, augmented display glasses, etc.), and the corresponding gesture action is obtained through image recognition. However, during the gesture interaction process, the camera of the head-mounted display device needs to be in working state all the time, which consumes a lot of power for the head-mounted display device and seriously affects the battery life of the head-mounted display device.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a gesture interaction method, circuit, smart wearable device, storage medium and product, aiming to solve the technical problem that the existing gesture interaction method affects the battery life of the head-mounted display device.

[0006] To achieve the above purpose, the present application proposes a gesture interaction method, which is applied to a smart wearable device. The smart wearable device is equipped with an electromyography monitoring unit and an inertia detection unit. The gesture interaction method includes: The user's hand electromyography information is collected through the electromyography monitoring unit, and the user's hand inertia information is collected through the inertia detection unit; According to the hand electromyography information and hand inertia information, the corresponding gesture interaction action is identified; The interactive control instruction corresponding to the gesture interaction action is sent to the head-mounted display device connected to the smart wearable device.

[0007] In one embodiment, the step of identifying the corresponding gesture interaction action according to the hand electromyography information and the hand inertia information includes: According to the hand electromyography information, the real-time hand posture is identified; According to the hand inertia information, the hand movement trajectory is identified; According to the real-time hand posture and hand movement trajectory, the corresponding gesture interaction action is determined.

[0008] In one embodiment, the step of determining the corresponding gesture interaction action according to the real-time hand posture and hand motion trajectory includes: After the real-time hand posture is a predetermined starting gesture, determining the gesture starting time; After the real-time hand posture is a predetermined ending gesture, determining the gesture ending time; Intercepting the target motion trajectory from the hand motion trajectory between the start time of the gesture and the end time of the gesture; According to the target motion trajectory, the corresponding gesture interaction action is determined.

[0009] In one embodiment, after the real-time hand gesture is a predetermined starting gesture, after the step of determining the gesture starting time, the method includes: Determine whether the hand movement trajectory represents that the hand has not moved after the gesture initiation moment; If the hand motion trajectory indicates that the hand has not moved after the gesture start time, the predetermined start gesture is used as the gesture interaction action.

[0010] In addition, to achieve the above purpose, the present application also proposes a gesture interaction circuit, which is applied to the above gesture interaction method, and the gesture interaction circuit includes a main control unit, and an electromyography monitoring unit and an inertia detection unit connected to the main control unit; The electromyography monitoring unit is used to collect the user's hand electromyography information and send it to the main control unit; An inertial detection unit, used to collect the user's hand inertial information and send it to the main control unit; The main control unit is used to identify the corresponding gesture interaction action based on the hand electromyography information and hand inertia information; and send the interaction control instruction corresponding to the gesture interaction action to the head-mounted display device connected to the smart wearable device.

[0011] In one embodiment, the electromyography monitoring unit includes an electromyography processing module and at least one electrode group connected to the electromyography processing module.

[0012] In one embodiment, the minimum diameter of the first electrode and the second electrode in the electrode group is 4 mm; The minimum straight-line distance between the first electrode and the second electrode in the electrode group is 7 mm.

[0013] In one embodiment, the first electrode is connected to the electromyographic processing module after passing through a first current limiting resistor, and the second electrode is connected to the electromyographic processing module after passing through a second current limiting resistor; The first electrode and the second electrode are connected to one end of the electrostatic protection device, and the other end of the electrostatic protection device is grounded respectively.

[0014] In one embodiment, the clock line pin, input line pin, output line pin and selection line pin of the electromyographic processing module are respectively connected to the main control unit; The positive pin of the first capacitor of the electromyography processing module is connected to one end of the first filter capacitor, and the other end of the first filter capacitor is connected to the negative pin of the first capacitor of the electromyography processing module; The positive pin of the second capacitor of the electromyography processing module is connected to one end of the second filter capacitor, and the other end of the second filter capacitor is connected to the negative pin of the second capacitor of the electromyography processing module; The positive pin of the third capacitor of the electromyography processing module is connected to one end of the third filter capacitor, and the other end of the third filter capacitor is connected to the negative pin of the third capacitor of the electromyography processing module; The positive pin of the fourth capacitor of the electromyography processing module is connected to one end of the fourth filter capacitor, and the other end of the fourth filter capacitor is connected to the negative pin of the fourth capacitor of the electromyography processing module; The reference voltage pin of the electromyography processing module is connected to one end of the fifth filter capacitor, and the other end of the fifth filter capacitor is grounded; The reference output pin of the electromyography processing module is connected to one end of the sixth filter capacitor, and the other end of the sixth filter capacitor is grounded; The interface power pin of the electromyography processing module is respectively connected to the first power supply end and one end of the seventh filter capacitor, and the other end of the seventh filter capacitor is grounded: One end of the eighth filter capacitor and the ninth filter capacitor are connected in parallel to the first power supply end and the digital power pin of the electromyography processing module respectively, and the other end of the eighth filter capacitor and the ninth filter capacitor are connected in parallel to the ground and the digital ground pin of the electromyography processing module respectively; One end of the tenth filter capacitor and the eleventh filter capacitor are connected in parallel and are respectively connected to one end of the filter device and the analog power pin of the electromyography processing module, the other end of the filter device is connected to the first power supply end, and the other end of the tenth filter capacitor and the eleventh filter capacitor are connected in parallel and are respectively grounded and connected to the analog ground pin of the electromyography processing module.

[0015] In one embodiment, the clock line pin, the input line pin, the output line pin and the selection line pin of the inertia detection unit are respectively connected to the main control unit; The first interrupt pin and the second interrupt pin of the inertia detection unit are respectively connected to the main control unit; The main power supply pin of the inertia detection unit is respectively connected to the second power supply terminal and one end of the twelfth filter capacitor, and the other end of the twelfth filter capacitor is grounded; The interface power pin of the inertia detection unit is respectively connected to the second power supply end and one end of the thirteenth filter capacitor, and the other end of the thirteenth filter capacitor is grounded.

[0016] In addition, to achieve the above-mentioned purpose, the present application also proposes a smart wearable device, which includes the gesture interaction circuit as above, and the electromyography monitoring unit in the gesture interaction circuit is arranged on the side of the smart wearable device close to the skin.

[0017] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the gesture interaction method as described above are implemented.

[0018] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the gesture interaction method as described above are implemented.

[0019] One or more technical solutions proposed in this application have at least the following technical effects: The present application collects the user's hand electromyography information through an electromyography monitoring unit, and collects the user's hand inertia information through an inertia detection unit; identifies the corresponding gesture interaction action based on the hand electromyography information and the hand inertia information; and sends the interaction control instruction corresponding to the gesture interaction action to the head-mounted display device connected to the smart wearable device. Since the muscles of a person's hand will move when he or she makes an action, and the movement generates electromyography signals, the electromyography signals generated by different actions are also different. Therefore, the present application identifies the user's current hand posture through the hand electromyography information, and combines the hand movement trajectory identified through the hand inertia information to obtain the gesture interaction action made by the user. Then, the interaction control instruction corresponding to the gesture interaction action is sent to the head-mounted display device connected to the smart wearable device. The present application separates the hand action recognition function from the head-mounted display device, saves the power consumption of the head-mounted display device, and effectively avoids the influence of the gesture interaction function on the battery life of the head-mounted display device. Moreover, compared with the method of realizing hand motion recognition based on a camera, the present application can realize hand motion recognition based on hand electromyography information and hand inertia information, which consumes less power and computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A flowchart of the first embodiment of the gesture interaction method of the present application is provided; Figure 2 A schematic diagram of a scene of a gesture interaction method according to an embodiment of the present application; Figure 3 This is a flowchart of a feasible embodiment of the gesture interaction method of the present application; Figure 4 This is a schematic diagram of the structure of the gesture interaction circuit of this application; Figure 5 This is a schematic diagram of the structure of the electromyography monitoring unit involved in the embodiment of the present application; Figure 6 A schematic diagram of a partial circuit of an electromyography monitoring unit according to an embodiment of the present application; Figure 7 A circuit diagram of an electromyographic processing module according to an embodiment of the present application; Figure 8 Schematic diagram of a circuit of an inertia detection unit according to an embodiment of the present application.

[0023] Description of Figure Numbers: 10. Main control unit; 20. EMG monitoring unit; 21. EMG processing module; 22. Electrode group; EL1. First electrode; EL2. Second electrode; 23. Electrostatic protection device; R1. First current limiting resistor; R2. Second current limiting resistor; V1. First power supply terminal; C1. First filter capacitor; C2. Second filter capacitor; C3. Third filter capacitor; C4. Fourth filter capacitor; C5. Fifth filter capacitor; C6. Sixth filter capacitor; C7. Seventh filter capacitor; C8. Eighth filter capacitor; C9. Ninth filter capacitor; C10. Tenth filter capacitor; C11. Eleventh filter capacitor; FB1. Filter device; 30, inertia detection unit; V2, second power supply terminal; C12, twelfth filter capacitor; C13, thirteenth filter capacitor.

[0024] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0025] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0026] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0027] The main solution of the embodiment of the present application is: collecting the user's hand electromyography information through the electromyography monitoring unit, and collecting the user's hand inertia information through the inertia detection unit; identifying the corresponding gesture interaction action based on the hand electromyography information and the hand inertia information; sending the interaction control instructions corresponding to the gesture interaction action to the head-mounted display device connected to the smart wearable device.

[0028] In this embodiment, for the convenience of description, the following description is made with the smart wearable device as the execution subject.

[0029] In current gesture interaction solutions, in order to identify the user's current gesture, the user's hand is generally captured in real time through a camera configured on a head-mounted display device (such as a virtual reality helmet, augmented display glasses, etc.), and the corresponding gesture action is obtained through image recognition. However, during the gesture interaction process, the camera of the head-mounted display device needs to be in working state all the time, which consumes a lot of power for the head-mounted display device and seriously affects the battery life of the head-mounted display device.

[0030] The present application provides a solution, which can identify the user's current hand posture through hand electromyography information, and identify the hand's motion trajectory through hand inertia information, so as to obtain the gesture interaction action made by the user. Then, the interactive control instruction corresponding to the gesture interaction action is sent to the head-mounted display device connected to the smart wearable device. The present application separates the hand motion recognition function from the head-mounted display device, saves the power consumption of the head-mounted display device, and effectively avoids the impact of the gesture interaction function on the battery life of the head-mounted display device. Moreover, compared with the method of realizing hand motion recognition based on the camera, the present application can realize hand motion recognition based on hand electromyography information and hand inertia information, which consumes less power and computing resources.

[0031] Based on this, the embodiment of the present application provides a gesture interaction method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the gesture interaction method of the present application.

[0032] In this embodiment, the gesture interaction method is applied to a smart wearable device, and the smart wearable device is equipped with an electromyography monitoring unit and an inertia detection unit. The gesture interaction method includes steps S10 to S30: Step S10, collecting the user's hand electromyography information through the electromyography monitoring unit, and collecting the user's hand inertia information through the inertia detection unit; It should be noted that the smart wearable device can be a smart ring, a smart watch, a smart bracelet and other devices. Figure 2 As shown, taking the smart wearable device as a smart ring as an example, the smart ring is provided with an inertia detection unit for collecting the user's hand inertia information, and the inertia detection unit can be set at any position on the smart wearable device. The electromyography monitoring unit can be set on the side of the smart wearable device close to the skin to realize the collection of electromyography signals.

[0033] In addition, it should be noted that hand electromyography information is information describing the electromyography signal characteristics of the user's hand, such as the electromyography signal itself, or signal characteristics extracted from the electromyography signal. Hand inertia information includes acceleration and angular velocity in the directions of three vertical axes (i.e., X-axis, Y-axis, and Z-axis).

[0034] In this embodiment, the electrical signals generated during the movement of hand muscles can be received by the electrodes of the electromyography monitoring unit. Since different muscles move in different actions and generate different electromyography signals, the user's hand electromyography information can be obtained by collecting the electromyography signals generated during the movement of the user's hand muscles. When the inertial detection unit is affected by acceleration and rotation, the internal sensitive elements of the inertial detection unit will produce a corresponding physical response, and then after converting this physical response into an electrical signal, the hand inertia information describing the acceleration and angular velocity in the three-axis direction is obtained.

[0035] Step S20, identifying corresponding gesture interaction actions according to the hand electromyography information and the hand inertia information; Since different hand postures require different muscle movements, the electromyographic signals generated by different muscle movements are also different. Therefore, this embodiment can identify the real-time hand posture based on the hand electromyographic information. And identify the hand motion trajectory based on the hand inertia information. This embodiment can realize the recognition of the hand motion trajectory through methods based on integration, attitude dead reckoning, sliding window method, machine learning, etc. Since gesture actions may include not only hand postures (such as clenching fists, pinching fingers, opening palms, etc.) but also the motion trajectory under this hand posture, this embodiment can determine the corresponding gesture interaction action based on the real-time hand posture and hand motion trajectory. Exemplarily, this embodiment can be set in advance with an action mapping relationship between gesture interaction actions and hand postures and motion trajectories, and then the action mapping relationship can be queried according to the real-time hand posture and hand motion trajectory to obtain the gesture interaction action corresponding to the real-time hand posture and hand motion trajectory.

[0036] In a feasible implementation, step S20 may include steps S21 to S23: Step S21, identifying the real-time hand posture according to the hand electromyography information; Step S22, identifying the hand motion trajectory according to the hand inertia information; Step S23, determining the corresponding gesture interaction action according to the real-time hand posture and hand movement trajectory.

[0037] Since the muscle movements required for different hand postures are different, the electromyographic signals generated by different muscle movements are also different. Therefore, this embodiment can collect the hand electromyographic signal features under each preset hand posture in advance, and construct a hand posture mapping table according to the hand electromyographic signal features corresponding to each preset hand posture. Then, this embodiment can query the hand posture mapping table according to the hand electromyographic information, and use the preset hand posture corresponding to the hand electromyographic signal features matched by the hand electromyographic information as the real-time hand posture. According to the hand inertia information, the hand motion trajectory is identified. This embodiment can realize the recognition of the hand motion trajectory through methods based on integration, attitude dead reckoning, sliding window method, machine learning, etc. Exemplarily, taking the integration-based method as an example, this embodiment determines the object posture (hand motion direction) based on the angular velocity (heading angle, pitch angle and roll angle) in the three vertical axis directions in the hand inertia information, and then integrates the acceleration in the three vertical axis directions twice to estimate the object position (hand position), thereby generating the hand motion trajectory according to the hand motion direction and hand position. Since the gesture action may include not only the hand posture (such as clenching a fist, pinching fingers, opening the palm, etc.) but also the motion trajectory under this hand posture, the present embodiment can determine the corresponding gesture interaction action according to the real-time hand posture and the hand motion trajectory. Exemplarily, the present embodiment can be provided with an action mapping relationship between the gesture interaction action and the hand posture and the motion trajectory in advance, and then the action mapping relationship can be queried according to the real-time hand posture and the hand motion trajectory to obtain the gesture interaction action corresponding to the real-time hand posture and the hand motion trajectory. In another example, after the real-time hand posture is a predetermined starting gesture, the gesture starting time is determined; after the real-time hand posture is a predetermined ending gesture, the gesture ending time is determined; the target motion trajectory between the gesture starting time and the gesture ending time is intercepted from the hand motion trajectory; and the corresponding gesture interaction action is determined according to the target motion trajectory. In another example, after the real-time hand posture is a predetermined starting gesture, after the gesture starting time is determined, it is determined whether the hand motion trajectory after the gesture starting time represents that the hand has not moved; if the hand motion trajectory after the gesture starting time represents that the hand has not moved, the predetermined starting gesture is used as the gesture interaction action.

[0038] In a feasible implementation, step S23 may include steps A10 to A40: Step A10, after the real-time hand posture is a predetermined starting gesture, determining the gesture starting time; Step A20, after the real-time hand posture is a predetermined end gesture, determining the gesture end time; Step A30, intercepting the target motion trajectory between the start time of the gesture and the end time of the gesture from the hand motion trajectory; Step A40, determining the corresponding gesture interaction action according to the target motion trajectory.

[0039] It should be noted that the predetermined starting gesture is a predetermined gesture representing the starting node of the motion trajectory, such as pinching fingers, making a fist, etc. The predetermined ending gesture is a predetermined gesture representing the ending node of the motion trajectory, such as releasing pinching fingers, opening the palm, etc.

[0040] This embodiment can use the current moment as the gesture start moment after detecting that the real-time hand posture is a predetermined start gesture, and use the current moment as the gesture end moment after detecting that the real-time hand posture is a predetermined end gesture. Then, the target motion trajectory between the gesture start moment and the gesture end moment can be intercepted from the hand motion trajectory, and then this embodiment can determine the gesture interaction action corresponding to the target motion trajectory based on the target motion trajectory. Therefore, this embodiment can accurately intercept the part of the motion trajectory that is expected to be recognized from the hand motion trajectory without traversing and comparing the entire hand motion trajectory, which effectively improves the accuracy and efficiency of gesture recognition.

[0041] In a feasible implementation manner, step A10 may include steps B10 to B20: Step B10, determining whether the hand motion trajectory indicates that the hand has not moved after the gesture start time; Step B20: If the hand motion trajectory indicates that the hand has not moved after the gesture start time, the predetermined start gesture is used as the gesture interaction action.

[0042] In some scenarios, the gesture interaction action displayed by the user is not related to the motion trajectory but the hand posture. In this embodiment, after the real-time hand posture is determined as the predetermined starting gesture to determine the gesture start time, it can be determined whether the hand motion trajectory represents that the hand has not moved after the gesture start time. Exemplarily, in this embodiment, whether the displacement distance of the hand motion trajectory is less than a predetermined threshold value (such as 3cm, 5cm, etc.) within a predetermined time length (such as 1s, 1.5s, 2s, etc.) after the gesture start time, it can be determined whether it represents that the hand has not moved. If the displacement distance of the hand motion trajectory is less than the predetermined threshold value within the predetermined time length after the gesture start time, it is determined that the hand motion trajectory represents that the hand has not moved after the gesture start time. If the displacement distance of the hand motion trajectory is not less than the predetermined threshold value within the predetermined time length after the gesture start time, it is determined that the hand motion trajectory represents that the hand has moved after the gesture start time. If the hand motion trajectory represents that the hand has not moved after the gesture start time, the predetermined starting gesture can be used as a gesture interaction action. If the hand motion trajectory represents that the hand has moved after the gesture start time, step A20 can be executed.

[0043] Step S30: sending the interaction control instruction corresponding to the gesture interaction action to a head mounted display device connected to the smart wearable device.

[0044] It should be noted that head-mounted display devices may include virtual reality helmets, virtual reality glasses, augmented reality glasses, mixed reality helmets and other devices used for virtual reality interaction.

[0045] In this embodiment, the head-mounted display device is connected to the smart wearable device in communication so that the head-mounted display device and the smart wearable device can realize data transmission. The communication connection mode can be a wireless connection mode such as Bluetooth connection, WIFI connection, or a wired connection mode. In this embodiment, the interactive control instruction corresponding to the gesture interaction action can be sent to the head-mounted display device connected to the smart wearable device, so that the head-mounted display device responds to the interactive control instruction corresponding to the gesture interaction action and performs the corresponding operation.

[0046] As a feasible embodiment, Figure 3 As shown, the pinching action is taken as the predetermined starting action and the finger releasing action is taken as the predetermined ending action. In this embodiment, after the action collection of the gesture interaction action starts, step C10 is executed: the hand electromyographic information collected by the electromyographic monitoring unit is used to identify whether there is a pinching action, and if there is no pinching action, the collection operation of the electromyographic monitoring unit is continued. If there is a pinching action, step C20 is executed: the corresponding hand motion trajectory is identified through the hand inertia information collected by the inertia detection unit. If there is no hand motion trajectory (that is, the hand motion trajectory indicates that the hand has not moved), step C30 is executed: the pinching action is taken as a gesture interaction action, and step C40 is executed: corresponding feedback is given according to the interactive control instruction corresponding to the pinching action. If there is a hand motion trajectory (that is, the hand motion trajectory indicates that the hand moves), step C50 is executed: whether there is a finger releasing action is identified based on the hand electromyographic information. After identifying the existence of the finger action, step C60 is executed: corresponding feedback is given according to the interactive control instruction corresponding to the hand motion trajectory between the pinching action and the finger releasing action. After no finger action is recognized, step C70 is executed: continue to respond to the pinch-slide action until a finger action is recognized.

[0047] The first embodiment of the present application provides a gesture interaction method, which is applied to a smart wearable device. The smart wearable device is equipped with an electromyography monitoring unit and an inertia detection unit. The electromyography monitoring unit collects the user's hand electromyography information, and the inertia detection unit collects the user's hand inertia information; according to the hand electromyography information and the hand inertia information, the corresponding gesture interaction action is identified; and the interaction control instruction corresponding to the gesture interaction action is sent to the head-mounted display device connected to the smart wearable device. Since the muscles of the hand will move when a person makes an action, and the movement generates electromyography signals, the electromyography signals generated by different actions are also different. Therefore, this embodiment identifies the user's current hand posture through the hand electromyography information, and combines the hand movement trajectory identified through the hand inertia information, so that the gesture interaction action made by the user can be obtained. Then the interaction control instruction corresponding to the gesture interaction action is sent to the head-mounted display device connected to the smart wearable device. This embodiment separates the hand action recognition function from the head-mounted display device, saves the power consumption of the head-mounted display device, and effectively avoids the influence of the gesture interaction function on the battery life of the head-mounted display device. Moreover, compared with the method of realizing hand motion recognition based on a camera, this embodiment can realize hand motion recognition based on hand electromyography information and hand inertia information, which consumes less power and computing resources.

[0048] In the second embodiment of the present application, Figure 4 As shown, this embodiment proposes a gesture interaction circuit, which is applied to the above gesture interaction method. The gesture interaction circuit includes a main control unit 10, and an electromyography monitoring unit 20 and an inertia detection unit 30 connected to the main control unit 10; The electromyography monitoring unit 20 is used to collect the electromyography information of the user's hand and send it to the main control unit 10; The inertia detection unit 30 is used to collect the user's hand inertia information and send it to the main control unit 10; The main control unit 10 is used to identify the corresponding gesture interaction action according to the hand electromyography information and the hand inertia information; and send the interaction control instruction corresponding to the gesture interaction action to the head-mounted display device connected to the smart wearable device.

[0049] It should be noted that the main control unit 10 is a component with a computing function such as a CPU (Central Processing Unit) or an MCU (Microcontroller Unit).

[0050] In this embodiment, the electromyography monitoring unit 20 collects the user's hand electromyography information and sends it to the main control unit 10; the inertia detection unit 30 collects the user's hand inertia information and sends it to the main control unit 10; the main control unit 10 is used to identify the corresponding gesture interaction action based on the hand electromyography information and the hand inertia information; and send the interaction control instructions corresponding to the gesture interaction action to the head-mounted display device connected to the smart wearable device.

[0051] Therefore, this embodiment can identify the user's current hand posture through hand electromyography information, and identify the hand movement trajectory through hand inertia information, so as to obtain the gesture interaction action made by the user. This embodiment separates the hand motion recognition function from the head-mounted display device, which saves the power consumption of the head-mounted display device and effectively avoids the impact of the gesture interaction function on the battery life of the head-mounted display device. Moreover, compared with the method of realizing hand motion recognition based on the camera, this embodiment can realize hand motion recognition based on hand electromyography information and hand inertia information, which consumes less power and computing resources.

[0052] like Figure 5 As shown, in some embodiments, the electromyography monitoring unit 20 includes an electromyography processing module 21 and at least one electrode group 22 connected to the electromyography processing module 21 .

[0053] Taking the smart wearable device as a smart watch as an example, the electrode group of the electromyography monitoring unit 20 is arranged on the surface of the smart watch close to the skin. Figure 5 As shown, an electrode group 22 is arranged on the watch body, and three electrode groups 22 are evenly arranged on the watch strap. After the smart wearable device is worn, the electrode group 22 in the electromyography monitoring unit 20 is close to the user's skin. When the user makes a hand movement, the muscles of the hand will move to generate electromyography signals. These electromyography signals are transmitted to the electromyography processing module 21 through the electrode group 22. The electromyography processing module 21 processes the received electromyography signals to obtain hand electromyography information. It can be understood that in order to improve the accuracy and collection efficiency of electromyography signals, the setting position of the electrode group 22 in this embodiment can be determined according to the muscle position involved in the hand posture. Setting the electrode group 22 close to the muscle position involved in the hand posture can improve the accuracy and collection efficiency of collecting electromyography signals.

[0054] In some embodiments, the minimum diameter of the first electrode EL1 and the second electrode EL2 in the electrode group 22 is 4 mm, so that there is sufficient contact area to receive the electromyographic signal. The minimum straight-line distance between the first electrode EL1 and the second electrode EL2 in the electrode group 22 is 7 mm to avoid interference between the positive and negative electrodes. Therefore, this embodiment ensures the accuracy of the collected electromyographic signal.

[0055] like Figure 6As shown, in some embodiments, the first electrode EL1 is connected to the electromyographic processing module 21 after passing through the first current limiting resistor R1, and the second electrode EL2 is connected to the electromyographic processing module 21 after passing through the second current limiting resistor R2; The first electrode EL1 and the second electrode EL2 are connected to one end of the electrostatic protection device 23 , and the other end of the electrostatic protection device 23 is grounded respectively.

[0056] In this embodiment, the electromyography monitoring unit 20 includes four electrode groups as an example. The first electrode EL1 in each electrode group 22 flows through the first current limiting resistor R1 and is connected to the electromyography processing module 21, and the second electrode EL2 flows through the second current limiting resistor R2 and is connected to the electromyography processing module 21. The first current limiting resistor R1 and the second current limiting resistor R2 play a current limiting role, thereby limiting the current intensity flowing into the electromyography processing module 21 to a certain range to protect the electromyography processing module 21. Electrostatic discharge can accumulate thousands of volts of energy on the interface or device surface of electronic equipment, which may cause equipment failure, performance degradation, or even directly burn components. In this embodiment, the first electrode EL1 and the second electrode EL2 are connected to one end of the electrostatic protection device 23, and the other end of the electrostatic protection device 23 is grounded respectively. The electrostatic protection device 23 can be an electrostatic resistor of the type of transient voltage suppressor diode, varistor, multilayer chip capacitor, etc. In this embodiment, with the help of the electrostatic protection device 23, the electrostatic on the user's body surface is prevented from destroying the electromyography processing module 21 after being introduced through the electrode. In this embodiment, the circuit safety and stability are improved by current limiting resistors and electrostatic protection.

[0057] like Figure 7 As shown, in some embodiments, the clock line pin, input line pin, output line pin and selection line pin of the electromyography processing module 21 are respectively connected to the main control unit; The positive pin of the first capacitor of the electromyography processing module 21 is connected to one end of the first filter capacitor C1, and the other end of the first filter capacitor C1 is connected to the negative pin of the first capacitor of the electromyography processing module 21; The positive pin of the second capacitor of the electromyographic processing module 21 is connected to one end of the second filter capacitor C2, and the other end of the second filter capacitor C2 is connected to the negative pin of the second capacitor of the electromyographic processing module 21; The positive pin of the third capacitor of the electromyographic processing module 21 is connected to one end of the third filter capacitor C3, and the other end of the third filter capacitor C3 is connected to the negative pin of the third capacitor of the electromyographic processing module 21; The positive pin of the fourth capacitor of the electromyographic processing module 21 is connected to one end of the fourth filter capacitor C4, and the other end of the fourth filter capacitor C4 is connected to the negative pin of the fourth capacitor of the electromyographic processing module 21; The reference voltage pin of the electromyography processing module 21 is connected to one end of the fifth filter capacitor C5, and the other end of the fifth filter capacitor C5 is grounded; The reference output pin of the electromyography processing module 21 is connected to one end of the sixth filter capacitor C6, and the other end of the sixth filter capacitor C6 is grounded; The interface power pin of the electromyography processing module 21 is respectively connected to the first power supply terminal V1 and one end of the seventh filter capacitor C7, and the other end of the seventh filter capacitor C7 is grounded: One end of the eighth filter capacitor C8 and the ninth filter capacitor C9 are connected in parallel to the first power supply terminal V1 and the digital power pin of the electromyography processing module 21, respectively, and the other end of the eighth filter capacitor C8 and the ninth filter capacitor C9 are connected in parallel to the ground and the digital ground pin of the electromyography processing module 21, respectively; One end of the tenth filter capacitor C10 and the eleventh filter capacitor C11 are connected in parallel and are respectively connected to one end of the filter device FB1 and the analog power pin of the electromyography processing module 21, the other end of the filter device FB1 is connected to the first power supply terminal V1, and the other end of the tenth filter capacitor C10 and the eleventh filter capacitor C11 are connected in parallel and are respectively grounded and connected to the analog ground pin of the electromyography processing module 21.

[0058] like Figure 8 As shown, the input pin of the electromyography processing module 21 (i.e. Figure 8 The pins 1 to 8 in the myoelectric processing module 21 are connected to each electrode group 22 to receive the myoelectric signal of the electrode group 22. The clock line pin (i.e. Figure 8 Pin 42 in), input line pin (i.e. Figure 8 Pin 43 in), output line pin (i.e. Figure 8 Pin 44 in the Figure 8 The pins 41 in the SPI interface are connected to the main control unit to realize the SPI communication between the myoelectric processing module 21 and the main control unit 10.

[0059] The positive pin of the first capacitor of the electromyography processing module 21 (i.e. Figure 8 The pin 14 in the figure is connected to one end of the first filter capacitor C1, and the other end of the first filter capacitor C1 is connected to the first capacitor negative pin (i.e. Figure 8 The positive pin of the second capacitor of the electromyography processing module 21 (i.e. Figure 8 The pin 16 in the middle is connected to one end of the second filter capacitor C2, and the other end of the second filter capacitor C2 is connected to the negative pin of the second capacitor (i.e. Figure 8 The third capacitor positive pin (i.e. Figure 8 The pin 18 in the middle is connected to one end of the third filter capacitor C3, and the other end of the third filter capacitor C3 is connected to the negative pin of the third capacitor of the electromyography processing module 21 (i.e. Figure 8The positive pin of the fourth capacitor of the electromyography processing module 21 (i.e. Figure 8 The pin 20 in the figure is connected to one end of the fourth filter capacitor C4, and the other end of the fourth filter capacitor C4 is connected to the negative pin of the fourth capacitor (i.e. Figure 8 The first filter capacitor C1, the second filter capacitor C2, the third filter capacitor C3, and the fourth filter capacitor C4 are used to simulate the input of the high-pass filter, and the capacitance value can be adjusted according to the frequency band of the high-frequency signal actually required.

[0060] The reference voltage pin of the electromyography processing module 21 (i.e. Figure 8 The pin 37 in the figure is connected to one end of the fifth filter capacitor C5, and the other end of the fifth filter capacitor C5 is grounded; the reference output pin (i.e. Figure 8 The pin 38 in the figure is connected to one end of the sixth filter capacitor C6, and the other end of the sixth filter capacitor C6 is grounded; the interface power pin of the electromyography processing module 21 (ie Figure 8 Pin 47 in the circuit is connected to the first power supply terminal V1 and one end of the seventh filter capacitor C7 respectively, and the other end of the seventh filter capacitor C7 is grounded. The fifth filter capacitor C5 is a filter capacitor for reference output, which is used to filter the reference output. The sixth filter capacitor C6 is a filter capacitor for reference voltage, which is used to filter the reference voltage. The seventh filter capacitor C7 is a filter capacitor for interface power supply, which is used to filter high-frequency noise of the interface power supply.

[0061] One end of the eighth filter capacitor C8 and the ninth filter capacitor C9 are connected in parallel to the first power supply terminal V1 and the digital power supply pin of the electromyographic processing module 21 (ie Figure 8 The other ends of the eighth filter capacitor C8 and the ninth filter capacitor C9 are connected in parallel to the ground and the digital ground pin (i.e. Figure 8 The eighth filter capacitor C8 and the ninth filter capacitor C9 are filter capacitors of the digital power supply. The capacitance of the ninth filter capacitor C9 is smaller than that of the eighth filter capacitor C8. The ninth filter capacitor C9 is used to filter out low-frequency clutter, and the eighth filter capacitor C8 is used to filter out high-frequency clutter.

[0062] The tenth filter capacitor C10 and the eleventh filter capacitor C11 are connected in parallel to one end of the filter device FB1 and the analog power pin (ie, Figure 8 The other end of the filter device FB1 is connected to the first power supply terminal V1, and the other ends of the tenth filter capacitor C10 and the eleventh filter capacitor C11 are connected in parallel to the ground and the analog ground pin (i.e. Figure 8The filter device FB1 is used to filter out the noise of the analog power supply, the capacitance of the eleventh filter capacitor C11 is smaller than the tenth filter capacitor C10, the eleventh filter capacitor C11 is used to filter out the low-frequency noise, and the tenth filter capacitor C10 is used to filter out the high-frequency noise.

[0063] like Figure 8 As shown, in some embodiments, the clock line pin, input line pin, output line pin and selection line pin of the inertia detection unit 30 are respectively connected to the main control unit 10; The first interrupt pin and the second interrupt pin of the inertia detection unit 30 are respectively connected to the main control unit 10; The main power supply pin of the inertia detection unit 30 is respectively connected to the second power supply terminal V2 and one end of the twelfth filter capacitor C12, and the other end of the twelfth filter capacitor C12 is grounded; The interface power pin of the inertia detection unit 30 is respectively connected to the second power supply terminal V2 and one end of the thirteenth filter capacitor C13, and the other end of the thirteenth filter capacitor C13 is grounded.

[0064] In this embodiment, the inertia detection unit 30 communicates with the main control module 10 via SPI (Serial Peripheral Interface) signals. The clock line pin ( Figure 8 Pin 13), input line pin ( Figure 8 Pin 14), output line pin ( Figure 8 pin 1) and the select line pin ( Figure 8 The first interrupt pin ( Figure 8 middle pin 4) and the second interrupt pin ( Figure 8 The main power pins (9) of the inertia detection unit 30 are connected to the main control unit 10 for outputting interrupt signals and notifying the main control unit 10 when an event occurs in the inertia detection unit 30. Figure 8 The middle pin 8 is connected to the second power supply terminal V2 and one end of the twelfth filter capacitor C12 respectively, and the other end of the twelfth filter capacitor C12 is grounded to filter the main power supply. The interface power pin ( Figure 8 The middle pin 5) is respectively connected to the second power supply terminal V2 and one end of the thirteenth filter capacitor C13, and the other end of the thirteenth filter capacitor C13 is grounded to filter the interface power supply.

[0065] The present application provides a smart wearable device, which includes the above gesture interaction circuit, and the myoelectric monitoring unit in the gesture interaction circuit is arranged on the side of the smart wearable device close to the skin. For example, the smart wearable device can be a device worn on the hand, such as a smart ring, a smart bracelet, or a smart watch.

[0066] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0067] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the gesture interaction method in the above-mentioned embodiment.

[0068] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0069] The computer-readable storage medium may be included in the gesture interaction device; or may exist independently without being assembled into the gesture interaction device.

[0070] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the gesture interaction device, the gesture interaction device: collects the user's hand electromyography information through the electromyography monitoring unit, and collects the user's hand inertia information through the inertia detection unit; identifies the corresponding gesture interaction action based on the hand electromyography information and the hand inertia information; and sends the interaction control instruction corresponding to the gesture interaction action to the head-mounted display device connected to the smart wearable device.

[0071] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0072] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0073] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0074] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned gesture interaction method, and can solve the technical problem that the existing gesture interaction method affects the battery life of the head-mounted display device. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the gesture interaction method provided by the above-mentioned embodiment, and will not be repeated here.

[0075] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned gesture interaction method when executed by a processor.

[0076] The computer program product provided by this application can solve the technical problem that the existing gesture interaction method affects the battery life of the head-mounted display device. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the gesture interaction method provided by the above embodiment, which will not be repeated here.

[0077] The above are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A gesture interaction method, characterized in that: Applied to a smart wearable device, the smart wearable device is equipped with an electromyography monitoring unit and an inertia detection unit, and the gesture interaction method includes: The user's hand electromyography information is collected through the electromyography monitoring unit, and the user's hand inertia information is collected through the inertia detection unit; According to the hand electromyography information and the hand inertia information, identifying a corresponding gesture interaction action; The interactive control instruction corresponding to the gesture interaction action is sent to a head mounted display device connected to the smart wearable device.

2. The gesture interaction method according to claim 1, characterized in that: The step of identifying the corresponding gesture interaction action according to the hand electromyography information and the hand inertia information includes: According to the hand electromyography information, identifying the real-time hand posture; identifying a hand motion trajectory according to the hand inertia information; A corresponding gesture interaction action is determined according to the real-time hand posture and the hand movement trajectory.

3. The gesture interaction method according to claim 2, characterized in that: The step of determining the corresponding gesture interaction action according to the real-time hand posture and the hand motion trajectory comprises: After the real-time hand posture is a predetermined starting gesture, determining a gesture starting time; After the real-time hand gesture is a predetermined ending gesture, determining a gesture ending time; intercepting a target motion trajectory between the start time of the gesture and the end time of the gesture from the hand motion trajectory; According to the target motion trajectory, a corresponding gesture interaction action is determined.

4. The gesture interaction method according to claim 3, characterized in that: After the real-time hand gesture is a predetermined starting gesture, after the step of determining the gesture starting time, the method includes: Determining whether the hand motion trajectory indicates that the hand has not moved after the gesture start time; If the hand motion trajectory indicates that the hand has not moved after the gesture start time, the predetermined start gesture is used as the gesture interaction action.

5. A gesture interaction circuit, characterized in that: Applied to the gesture interaction method according to any one of claims 1 to 4, the gesture interaction circuit comprises a main control unit, and an electromyography monitoring unit and an inertia detection unit connected to the main control unit; The electromyography monitoring unit is used to collect the electromyography information of the user's hand and send it to the main control unit; The inertia detection unit is used to collect the user's hand inertia information and send it to the main control unit; The main control unit is used to identify the corresponding gesture interaction action according to the hand electromyography information and the hand inertia information; and send the interaction control instruction corresponding to the gesture interaction action to the head-mounted display device connected to the smart wearable device.

6. The gesture interaction circuit as claimed in claim 5, characterized in that: The electromyographic monitoring unit includes an electromyographic processing module and at least one electrode group connected to the electromyographic processing module.

7. The gesture interaction circuit as claimed in claim 6, characterized in that: The minimum diameter of the first electrode and the second electrode in the electrode group is 4 mm; The minimum straight-line distance between the first electrode and the second electrode in the electrode group is 7 mm.

8. The gesture interaction circuit as claimed in claim 6, characterized in that: The first electrode is connected to the electromyographic processing module after passing through a first current limiting resistor, and the second electrode is connected to the electromyographic processing module after passing through a second current limiting resistor; The first electrode and the second electrode are connected to one end of the electrostatic protection device, and the other end of the electrostatic protection device is grounded respectively.

9. The gesture interaction circuit as claimed in claim 6, characterized in that: The clock line pin, input line pin, output line pin and selection line pin of the electromyographic processing module are respectively connected to the main control unit; The positive pin of the first capacitor of the electromyography processing module is connected to one end of the first filter capacitor, and the other end of the first filter capacitor is connected to the negative pin of the first capacitor of the electromyography processing module; The positive pin of the second capacitor of the electromyography processing module is connected to one end of the second filter capacitor, and the other end of the second filter capacitor is connected to the negative pin of the second capacitor of the electromyography processing module; The positive pin of the third capacitor of the electromyography processing module is connected to one end of the third filter capacitor, and the other end of the third filter capacitor is connected to the negative pin of the third capacitor of the electromyography processing module; The positive pin of the fourth capacitor of the electromyography processing module is connected to one end of the fourth filter capacitor, and the other end of the fourth filter capacitor is connected to the negative pin of the fourth capacitor of the electromyography processing module; The reference voltage pin of the electromyography processing module is connected to one end of a fifth filter capacitor, and the other end of the fifth filter capacitor is grounded; The reference output pin of the electromyography processing module is connected to one end of a sixth filter capacitor, and the other end of the sixth filter capacitor is grounded; The interface power pin of the electromyography processing module is respectively connected to the first power supply end and one end of the seventh filter capacitor, and the other end of the seventh filter capacitor is grounded: One end of the eighth filter capacitor and the ninth filter capacitor are connected in parallel to the first power supply end and the digital power pin of the electromyographic processing module, respectively, and the other end of the eighth filter capacitor and the ninth filter capacitor are connected in parallel to ground and the digital ground pin of the electromyographic processing module, respectively; One end of the tenth filter capacitor and the eleventh filter capacitor are connected in parallel and are respectively connected to one end of the filter device and the analog power pin of the electromyography processing module, the other end of the filter device is connected to the first power supply end, and the other end of the tenth filter capacitor and the eleventh filter capacitor are connected in parallel and are respectively grounded and connected to the analog ground pin of the electromyography processing module.

10. The gesture interaction circuit according to any one of claims 5 to 9, characterized in that: The clock line pin, input line pin, output line pin and selection line pin of the inertia detection unit are respectively connected to the main control unit; The first interrupt pin and the second interrupt pin of the inertia detection unit are respectively connected to the main control unit; The main power pin of the inertia detection unit is respectively connected to the second power supply end and one end of the twelfth filter capacitor, and the other end of the twelfth filter capacitor is grounded; The interface power pin of the inertia detection unit is respectively connected to the second power supply end and one end of the thirteenth filter capacitor, and the other end of the thirteenth filter capacitor is grounded.

11. A smart wearable device, characterized in that: The smart wearable device comprises the gesture interaction circuit as claimed in any one of claims 5 to 10, and the electromyography monitoring unit in the gesture interaction circuit is arranged on a side of the smart wearable device close to the skin.

12. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the gesture interaction method according to any one of claims 1 to 4 are implemented.

13. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the gesture interaction method according to any one of claims 1 to 4 are implemented.

Images