Gesture interaction method, circuit, smart wearable device, storage medium and product

Abstract

The present application discloses a gesture interaction method, circuit, smart wearable device, storage medium and product, which relates to the field of virtual reality interaction technology. The gesture interaction method is applied to smart wearable devices, which are equipped with a heart rate monitoring unit and an inertia detection unit. The gesture interaction method includes: collecting the user's hand vasoconstriction information through the heart rate monitoring unit, and collecting the user's hand inertia information through the inertia detection unit; identifying the corresponding gesture interaction action according to the hand vasoconstriction information and the hand inertia information; and sending the interaction control instruction corresponding to the gesture interaction action to the head-mounted display device connected to the smart wearable device. The present application solves the technical problem that the existing gesture interaction method affects the battery life of the head-mounted display device.

Description

Technical Field

[0001] The present application relates to the field of virtual reality interaction technology, and in particular to a gesture interaction method, circuit, smart wearable device, storage medium and product. Background Art

[0002] As a rapidly developing technology field in recent years, virtual reality interaction technology has greatly enriched the user experience. As an important means of interaction in virtual reality interaction scenarios, gesture interaction technology has received widespread attention and research.

[0003] In current gesture interaction solutions, in order to identify the user's current gesture, the user's hand is generally captured in real time through a camera configured on a head-mounted display device (such as a virtual reality helmet, augmented display glasses, etc.), and the corresponding gesture action is obtained through image recognition. However, during the gesture interaction process, the camera of the head-mounted display device needs to be in working state all the time, which consumes a lot of power for the head-mounted display device and seriously affects the battery life of the head-mounted display device.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a gesture interaction method, circuit, smart wearable device, storage medium and product, aiming to solve the technical problem that the existing gesture interaction method affects the battery life of the head-mounted display device.

[0006] To achieve the above purpose, the present application proposes a gesture interaction method, which is applied to a smart wearable device. The smart wearable device is equipped with a heart rate monitoring unit and an inertia detection unit. The gesture interaction method includes: Collecting the user's hand vasoconstriction information through the heart rate monitoring unit, and collecting the user's hand inertia information through the inertia detection unit; According to the hand vascular contraction information and hand inertia information, the corresponding gesture interaction action is identified; The interactive control instruction corresponding to the gesture interaction action is sent to the head-mounted display device connected to the smart wearable device.

[0007] In one embodiment, the step of identifying the corresponding gesture interaction action according to the hand vasoconstriction information and the hand inertia information includes: According to the information of hand vasoconstriction, the real-time hand posture is identified; According to the hand inertia information, the hand movement trajectory is identified; According to the real-time hand posture and hand movement trajectory, the corresponding gesture interaction action is determined.

[0008] In one embodiment, the step of determining the corresponding gesture interaction action according to the real-time hand posture and hand motion trajectory includes: After the real-time hand posture is a predetermined starting gesture, determining the gesture starting time; After the real-time hand posture is a predetermined ending gesture, determining the gesture ending time; Intercepting the target motion trajectory from the hand motion trajectory between the start time of the gesture and the end time of the gesture; According to the target motion trajectory, the corresponding gesture interaction action is determined.

[0009] In one embodiment, after the real-time hand gesture is a predetermined starting gesture, after the step of determining the gesture starting time, the method includes: Determine whether the hand movement trajectory represents that the hand has not moved after the gesture initiation moment; If the hand motion trajectory indicates that the hand has not moved after the gesture start time, the predetermined start gesture is used as the gesture interaction action.

[0010] In addition, to achieve the above purpose, the present application also proposes a gesture interaction circuit, which is applied to the above gesture interaction method, and the gesture interaction circuit includes a main control unit, and a heart rate monitoring unit and an inertia detection unit connected to the main control unit; A heart rate monitoring unit, used to collect the user's hand vasoconstriction information and send it to the main control unit; An inertial detection unit, used to collect the user's hand inertial information and send it to the main control unit; The main control unit is used to identify the corresponding gesture interaction action based on the hand vascular contraction information and the hand inertia information; and send the interaction control instruction corresponding to the gesture interaction action to the head-mounted display device connected to the smart wearable device.

[0011] In one embodiment, the heart rate monitoring unit includes a heart rate processing module, and at least one infrared light source and two green light sources connected to the heart rate processing module; One green light source is in the same straight line as the infrared light source, and the other green light source is parallel to the infrared light source.

[0012] In one embodiment, the minimum straight-line distance between the infrared light source and the green light source on the same straight line is 2 mm; The minimum vertical distance between the parallel infrared light source and the green light source is 2.4 mm, and the minimum vertical distance between the heart rate processing module and the green light source is 3.7 mm.

[0013] In one embodiment, the positive electrodes of the infrared light source and the green light source are connected to the first power supply terminal, and the negative electrodes of the infrared light source and the green light source are connected to the heart rate processing module respectively; The heart rate monitoring unit also includes a first filter capacitor, a second filter capacitor, a third filter capacitor and a fourth filter capacitor; One ends of the first filter capacitor, the second filter capacitor, the third filter capacitor and the fourth filter capacitor are connected in parallel to the positive electrodes of the infrared light source and the green light source, and the other ends of the first filter capacitor, the second filter capacitor, the third filter capacitor and the fourth filter capacitor are grounded respectively.

[0014] In one embodiment, the clock line pin, the data line pin and the interrupt pin of the heart rate processing module are respectively connected to the main control unit; Each light source output pin of the heart rate processing module is connected to the negative electrode of the infrared light source and the green light source respectively; The interface power pin of the heart rate processing module is respectively connected to the second power supply end and one end of the fifth filter capacitor, and the other end of the fifth filter capacitor is grounded; The capacitor power pin of the heart rate processing module is respectively connected to the second power supply end and one end of the sixth filter capacitor, and the other end of the sixth filter capacitor is grounded; The analog power pin and the digital power pin of the heart rate processing module are connected in parallel to the second power supply end, the seventh filter capacitor, the eighth filter capacitor, and one end of the ninth filter capacitor, respectively, and the other ends of the seventh filter capacitor, the eighth filter capacitor, and the ninth filter capacitor are grounded respectively.

[0015] In one embodiment, the clock line pin, the input line pin, the output line pin and the selection line pin of the inertia detection unit are respectively connected to the main control unit; The first interrupt pin and the second interrupt pin of the inertia detection unit are respectively connected to the main control unit; The main power supply pin of the inertia detection unit is respectively connected to the third power supply terminal and one end of the tenth filter capacitor, and the other end of the tenth filter capacitor is grounded; The interface power pin of the inertia detection unit is respectively connected to the third power supply end and one end of the eleventh filter capacitor, and the other end of the eleventh filter capacitor is grounded.

[0016] In addition, to achieve the above-mentioned purpose, the present application also proposes a smart wearable device, which includes the gesture interaction circuit as above, and the heart rate monitoring unit in the gesture interaction circuit is arranged on the side of the smart wearable device close to the skin.

[0017] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the gesture interaction method as described above are implemented.

[0018] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the gesture interaction method as described above are implemented.

[0019] One or more technical solutions proposed in this application have at least the following technical effects: The present application collects the user's hand vasoconstriction information through a heart rate monitoring unit, and collects the user's hand inertia information through an inertia detection unit; identifies the corresponding gesture interaction action based on the hand vasoconstriction information and the hand inertia information; and sends the interaction control instruction corresponding to the gesture interaction action to a head-mounted display device connected to the smart wearable device. Since people exert force when performing actions, it will also affect vasoconstriction, and different actions have different effects on blood vessels. Therefore, the present application identifies the user's current hand posture through the hand vasoconstriction information, and combines the hand movement trajectory identified through the hand inertia information to obtain the gesture interaction action made by the user. Then, the interaction control instruction corresponding to the gesture interaction action is sent to the head-mounted display device connected to the smart wearable device. The present application separates the hand action recognition function from the head-mounted display device, saves the power consumption of the head-mounted display device, and effectively avoids the influence of the gesture interaction function on the battery life of the head-mounted display device. Moreover, compared with the method of realizing hand motion recognition based on a camera, the present application can realize hand motion recognition based on hand vascular contraction information and hand inertia information, which consumes less power and computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A flowchart of the first embodiment of the gesture interaction method of the present application is provided; Figure 2 A schematic diagram of a scene of a gesture interaction method according to an embodiment of the present application; Figure 3 This is a flowchart of a feasible embodiment of the gesture interaction method of the present application; Figure 4 This is a schematic diagram of the structure of the gesture interaction circuit of this application; Figure 5This is a schematic diagram of the structure of a heart rate monitoring unit involved in an embodiment of the present application; Figure 6 A partial circuit diagram of a heart rate monitoring unit according to an embodiment of the present application; Figure 7 A circuit diagram of a heart rate processing module according to an embodiment of the present application; Figure 8 Schematic diagram of a circuit of an inertia detection unit according to an embodiment of the present application.

[0023] Description of Figure Numbers: 10. Main control unit; 20. Heart rate monitoring unit; 21. Heart rate processing module; L1. Infrared light source; L2. Green light source; V1. First power supply terminal; C1. First filter capacitor; C2. Second filter capacitor; C3. Third filter capacitor; C4. Fourth filter capacitor; V2. Second power supply terminal; C5. Fifth filter capacitor; C6. Sixth filter capacitor; C7. Seventh filter capacitor; C8. Eighth filter capacitor; C9. Ninth filter capacitor; 30, inertia detection unit; V3, third power supply terminal; C10, tenth filter capacitor; C11, eleventh filter capacitor.

[0024] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0025] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0026] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0027] The main solution of the embodiment of the present application is: collecting the user's hand vasoconstriction information through a heart rate monitoring unit, and collecting the user's hand inertia information through an inertia detection unit; identifying corresponding gesture interaction actions based on the hand vasoconstriction information and hand inertia information; and sending the interaction control instructions corresponding to the gesture interaction actions to a head-mounted display device connected to the smart wearable device.

[0028] In this embodiment, for the convenience of description, the following description is made with the smart wearable device as the execution subject.

[0029] In current gesture interaction solutions, in order to identify the user's current gesture, the user's hand is generally captured in real time through a camera configured on a head-mounted display device (such as a virtual reality helmet, augmented display glasses, etc.), and the corresponding gesture action is obtained through image recognition. However, during the gesture interaction process, the camera of the head-mounted display device needs to be in working state all the time, which consumes a lot of power for the head-mounted display device and seriously affects the battery life of the head-mounted display device.

[0030] The present application provides a solution, which can identify the user's current hand posture through hand vasoconstriction information, and identify the hand's motion trajectory through hand inertia information, so as to obtain the gesture interaction action made by the user. Then, the interactive control instruction corresponding to the gesture interaction action is sent to the head-mounted display device connected to the smart wearable device. The present application separates the hand motion recognition function from the head-mounted display device, saves the power consumption of the head-mounted display device, and effectively avoids the impact of the gesture interaction function on the battery life of the head-mounted display device. Moreover, compared with the method of realizing hand motion recognition based on the camera, the present application can realize hand motion recognition based on hand vasoconstriction information and hand inertia information, which consumes less power and computing resources.

[0031] Based on this, the embodiment of the present application provides a gesture interaction method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the gesture interaction method of the present application.

[0032] In this embodiment, the gesture interaction method is applied to a smart wearable device, and the smart wearable device is equipped with a heart rate monitoring unit and an inertia detection unit. The gesture interaction method includes steps S10 to S30: Step S10, collecting the user's hand vasoconstriction information through the heart rate monitoring unit, and collecting the user's hand inertia information through the inertia detection unit; It should be noted that the smart wearable device can be a smart ring, a smart watch, a smart bracelet and other devices. Figure 2 As shown, taking the smart wearable device as a smart ring as an example, the smart ring is provided with an inertia detection unit for collecting the user's hand inertia information, and the inertia detection unit can be set at any position on the smart wearable device. The heart rate monitoring unit can be set on the side of the smart wearable device close to the skin to collect vasoconstriction information.

[0033] In addition, it should be noted that the hand vasoconstriction information is information describing the vasoconstriction characteristics of the user's hand, such as a PPG image (Photo-Plethysmo-Gram), or image features extracted from a PPG image. The hand inertia information includes acceleration and angular velocity in the directions of three vertical axes (i.e., the X-axis, the Y-axis, and the Z-axis).

[0034] In this embodiment, the heart rate monitoring unit can emit light at a specific wavelength, and a certain proportion of the emitted light is absorbed by the blood flowing in the blood vessels. Since different actions will have different effects on the blood vessels, this change can be detected by collecting the reflected light, and the user's hand blood vessel contraction information can be obtained. When the inertial detection unit is affected by acceleration and rotation, the internal sensitive elements of the inertial detection unit will produce a corresponding physical response, and then after converting this physical response into an electrical signal, the hand inertial information describing the acceleration and angular velocity in the three-axis direction is obtained.

[0035] Step S20, identifying corresponding gesture interaction actions according to the hand vascular contraction information and the hand inertia information; Since different hand postures have different effects on blood vessels, this embodiment can identify the real-time hand posture according to the hand blood vessel contraction information. And identify the hand motion trajectory according to the hand inertia information. This embodiment can realize the recognition of the hand motion trajectory through methods based on integration, attitude dead reckoning, sliding window method, machine learning, etc. Since gesture actions may include not only hand postures (such as clenching fists, pinching fingers, opening palms, etc.) but also the motion trajectory under this hand posture, this embodiment can determine the corresponding gesture interaction action according to the real-time hand posture and hand motion trajectory. Exemplarily, this embodiment can be set in advance with an action mapping relationship between gesture interaction actions and hand postures and motion trajectories, and then the action mapping relationship can be queried according to the real-time hand posture and hand motion trajectory to obtain the gesture interaction action corresponding to the real-time hand posture and hand motion trajectory.

[0036] In a feasible implementation, step S20 may include steps S21 to S23: Step S21, identifying the real-time hand posture according to the hand vascular contraction information; Step S22, identifying the hand motion trajectory according to the hand inertia information; Step S23, determining the corresponding gesture interaction action according to the real-time hand posture and hand movement trajectory.

[0037] Since different hand postures have different effects on blood vessels, this embodiment can collect the hand vascular contraction features under each preset hand posture in advance, and construct a hand posture mapping table according to the hand vascular contraction features corresponding to each preset hand posture. In addition, this embodiment can query the hand posture mapping table according to the hand vascular contraction information, and use the preset hand posture corresponding to the hand vascular contraction features matching the hand vascular contraction information as the real-time hand posture. According to the hand inertia information, the hand motion trajectory is identified. This embodiment can realize the recognition of the hand motion trajectory through methods based on integration, attitude dead reckoning, sliding window method, machine learning, etc. Exemplarily, taking the integration-based method as an example, this embodiment determines the object posture (hand motion direction) based on the angular velocity (heading angle, pitch angle and roll angle) in the three vertical axis directions in the hand inertia information, and then integrates the acceleration in the three vertical axis directions twice to estimate the object position (hand position), thereby generating the hand motion trajectory according to the hand motion direction and hand position. Since the gesture action may include not only the hand posture (such as clenching a fist, pinching fingers, opening the palm, etc.) but also the motion trajectory under this hand posture, the present embodiment can determine the corresponding gesture interaction action according to the real-time hand posture and the hand motion trajectory. Exemplarily, the present embodiment can be provided with an action mapping relationship between the gesture interaction action and the hand posture and the motion trajectory in advance, and then the action mapping relationship can be queried according to the real-time hand posture and the hand motion trajectory to obtain the gesture interaction action corresponding to the real-time hand posture and the hand motion trajectory. In another example, after the real-time hand posture is a predetermined starting gesture, the gesture starting time is determined; after the real-time hand posture is a predetermined ending gesture, the gesture ending time is determined; the target motion trajectory between the gesture starting time and the gesture ending time is intercepted from the hand motion trajectory; and the corresponding gesture interaction action is determined according to the target motion trajectory. In another example, after the real-time hand posture is a predetermined starting gesture, after the gesture starting time is determined, it is determined whether the hand motion trajectory after the gesture starting time represents that the hand has not moved; if the hand motion trajectory after the gesture starting time represents that the hand has not moved, the predetermined starting gesture is used as the gesture interaction action.

[0038] In a feasible implementation, step S23 may include steps A10 to A40: Step A10, after the real-time hand posture is a predetermined starting gesture, determining the gesture starting time; Step A20, after the real-time hand posture is a predetermined end gesture, determining the gesture end time; Step A30, intercepting the target motion trajectory between the start time of the gesture and the end time of the gesture from the hand motion trajectory; Step A40, determining the corresponding gesture interaction action according to the target motion trajectory.

[0039] It should be noted that the predetermined starting gesture is a predetermined gesture representing the starting node of the motion trajectory, such as pinching fingers, making a fist, etc. The predetermined ending gesture is a predetermined gesture representing the ending node of the motion trajectory, such as releasing pinching fingers, opening the palm, etc.

[0040] This embodiment can use the current moment as the gesture start moment after detecting that the real-time hand posture is a predetermined start gesture, and use the current moment as the gesture end moment after detecting that the real-time hand posture is a predetermined end gesture. Then, the target motion trajectory between the gesture start moment and the gesture end moment can be intercepted from the hand motion trajectory, and then this embodiment can determine the gesture interaction action corresponding to the target motion trajectory based on the target motion trajectory. Therefore, this embodiment can accurately intercept the part of the motion trajectory that is expected to be recognized from the hand motion trajectory without traversing and comparing the entire hand motion trajectory, which effectively improves the accuracy and efficiency of gesture recognition.

[0041] In a feasible implementation manner, step A10 may include steps B10 to B20: Step B10, determining whether the hand movement trajectory indicates that the hand has not moved after the gesture start time; Step B20: If the hand motion trajectory indicates that the hand has not moved after the gesture start time, the predetermined start gesture is used as the gesture interaction action.

[0042] In some scenarios, the gesture interaction action displayed by the user is not related to the motion trajectory but the hand posture. In this embodiment, after the real-time hand posture is determined as the predetermined starting gesture to determine the gesture start time, it can be determined whether the hand motion trajectory represents that the hand has not moved after the gesture start time. Exemplarily, in this embodiment, whether the displacement distance of the hand motion trajectory is less than a predetermined threshold value (such as 3cm, 5cm, etc.) within a predetermined time length (such as 1s, 1.5s, 2s, etc.) after the gesture start time, it can be determined whether it represents that the hand has not moved. If the displacement distance of the hand motion trajectory is less than the predetermined threshold value within the predetermined time length after the gesture start time, it is determined that the hand motion trajectory represents that the hand has not moved after the gesture start time. If the displacement distance of the hand motion trajectory is not less than the predetermined threshold value within the predetermined time length after the gesture start time, it is determined that the hand motion trajectory represents that the hand has moved after the gesture start time. If the hand motion trajectory represents that the hand has not moved after the gesture start time, the predetermined starting gesture can be used as a gesture interaction action. If the hand motion trajectory represents that the hand has moved after the gesture start time, step A20 can be executed.

[0043] Step S30: sending the interaction control instruction corresponding to the gesture interaction action to a head mounted display device connected to the smart wearable device.

[0044] It should be noted that head-mounted display devices may include virtual reality helmets, virtual reality glasses, augmented reality glasses, mixed reality helmets and other devices used for virtual reality interaction.

[0045] In this embodiment, the head-mounted display device is connected to the smart wearable device in communication so that the head-mounted display device and the smart wearable device can realize data transmission. The communication connection mode can be a wireless connection mode such as Bluetooth connection, WIFI connection, or a wired connection mode. In this embodiment, the interactive control instruction corresponding to the gesture interaction action can be sent to the head-mounted display device connected to the smart wearable device, so that the head-mounted display device responds to the interactive control instruction corresponding to the gesture interaction action and performs the corresponding operation.

[0046] As a feasible embodiment, take the pinching action as the predetermined starting action and the finger releasing action as the predetermined ending action as an example. In this embodiment, after the action collection of the gesture interaction action starts, step C10 is executed: the hand vasoconstriction information collected by the heart rate monitoring unit is used to identify whether there is a pinching action, and if there is no pinching action, the collection operation of the heart rate monitoring unit is continued. If there is a pinching action, step C20 is executed: the corresponding hand motion trajectory is identified through the hand inertia information collected by the inertia detection unit. If there is no hand motion trajectory (that is, the hand motion trajectory indicates that the hand has not moved), step C30 is executed: the pinching action is used as a gesture interaction action, and step C40 is executed: corresponding feedback is given according to the interactive control instruction corresponding to the pinching action. If there is a hand motion trajectory (that is, the hand motion trajectory indicates that the hand moves), step C50 is executed: based on the hand vasoconstriction information, it is identified whether there is a finger releasing action. After identifying the existence of the finger action, step C60 is executed: corresponding feedback is given according to the interactive control instruction corresponding to the hand motion trajectory between the pinching action and the finger releasing action. After no finger action is recognized, step C70 is executed: continue to respond to the pinch-slide action until a finger action is recognized.

[0047] The first embodiment of the present application provides a gesture interaction method, which is applied to a smart wearable device. The smart wearable device is equipped with a heart rate monitoring unit and an inertia detection unit. The heart rate monitoring unit collects the user's hand vasoconstriction information, and the inertia detection unit collects the user's hand inertia information; according to the hand vasoconstriction information and the hand inertia information, the corresponding gesture interaction action is identified; and the interaction control instruction corresponding to the gesture interaction action is sent to the head-mounted display device connected to the smart wearable device. Because people use force when doing actions, it will also affect vasoconstriction, and different actions have different effects on blood vessels. Therefore, this embodiment identifies the user's current hand posture through the hand vasoconstriction information, and combines the hand movement trajectory identified through the hand inertia information, so that the gesture interaction action made by the user can be obtained. Then the interaction control instruction corresponding to the gesture interaction action is sent to the head-mounted display device connected to the smart wearable device. This embodiment separates the hand action recognition function from the head-mounted display device, saves the power consumption of the head-mounted display device, and effectively avoids the influence of the gesture interaction function on the battery life of the head-mounted display device. Moreover, compared with the method of realizing hand motion recognition based on a camera, this embodiment can realize hand motion recognition based on hand vascular contraction information and hand inertia information, which consumes less power and computing resources.

[0048] In the second embodiment of the present application, Figure 4 As shown, this embodiment proposes a gesture interaction circuit, which is applied to the above gesture interaction method. The gesture interaction circuit includes a main control unit 10, and a heart rate monitoring unit 20 and an inertia detection unit 30 connected to the main control unit 10; A heart rate monitoring unit 20, used to collect the user's hand vasoconstriction information and send it to the main control unit 10; The inertia detection unit 30 is used to collect the user's hand inertia information and send it to the main control unit 10; The main control unit 10 is used to identify the corresponding gesture interaction action according to the hand vasoconstriction information and the hand inertia information; and send the interaction control instruction corresponding to the gesture interaction action to the head-mounted display device connected to the smart wearable device.

[0049] It should be noted that the main control unit 10 is a component with computing functions such as a CPU (Central Processing Unit) and an MCU (Microcontroller Unit).

[0050] In this embodiment, the heart rate monitoring unit 20 collects the user's hand vasoconstriction information and sends it to the main control unit 10; the inertia detection unit 30 collects the user's hand inertia information and sends it to the main control unit 10; the main control unit 10 is used to identify the corresponding gesture interaction action according to the hand vasoconstriction information and the hand inertia information; and send the interaction control instruction corresponding to the gesture interaction action to the head-mounted display device connected to the smart wearable device.

[0051] Therefore, this embodiment can identify the user's current hand posture through the hand vasoconstriction information, and identify the hand movement trajectory through the hand inertia information, so as to obtain the gesture interaction action made by the user. This embodiment separates the hand motion recognition function from the head-mounted display device, saves the power consumption of the head-mounted display device, and effectively avoids the impact of the gesture interaction function on the battery life of the head-mounted display device. Moreover, compared with the method of realizing hand motion recognition based on the camera, this embodiment can realize hand motion recognition based on the hand vasoconstriction information and the hand inertia information, and consumes less power and computing resources.

[0052] like Figure 5 As shown, in some embodiments, the heart rate monitoring unit 20 includes a heart rate processing module 21, and at least one infrared light source L1 and two green light sources L2 connected to the heart rate processing module 21; One green light source L2 is on the same straight line as the infrared light source L1 , and the other green light source L2 is parallel to the infrared light source L1 .

[0053] It should be noted that the heart rate monitoring unit 20 includes at least one infrared light source L1 and two green light sources L2. Of course, depending on the size of the monitoring area, it can also include more infrared light sources L1 and green light sources L2, or add a red light source that can emit red light to increase the detection function of other physiological parameters such as blood oxygen.

[0054] The heart rate processing module 21 is a component that processes the light emitted by the infrared light source L1 and the two green light sources L2 and reflected from the user's skin to obtain the hand vasoconstriction information. The infrared light source L1 and the green light source L2 can be OLED lights, LED lights, and other light-emitting components that can generate infrared light and green light.

[0055] In this embodiment, the heart rate monitoring unit 20 includes a heart rate processing module 21, and at least one infrared light source L1 and two green light sources L2 connected to the heart rate processing module 21. Since infrared light has a longer wavelength and stronger penetrating power, it can penetrate deeper layers of the skin, and green light is more easily absorbed by blood, so it can more accurately detect changes in blood flow in blood vessels. Therefore, the use of two different wavelengths of light can combine their advantages to improve the accuracy and reliability of the collected hand vascular contraction information. One green light source L2 is in the same straight line as the infrared light source L1, and the other green light source L2 is parallel to the infrared light source L1. Therefore, under this distribution, the infrared light emitted by the infrared light source L1 can assist the green light emitted by the green light source L2 to penetrate deeper into the skin.

[0056] In some embodiments, the minimum straight-line distance between the infrared light source L1 and the green light source L2 on the same straight line is 2 mm; The minimum vertical distance between the parallel infrared light source L1 and the green light source L2 is 2.4 mm, and the minimum vertical distance between the heart rate processing module 21 and the green light source L2 is 3.7 mm.

[0057] In this embodiment, the minimum straight-line distance between the infrared light source L1 and the green light source L2 on the same straight line is 2 mm, the minimum vertical distance between the parallel infrared light source L1 and the green light source L2 is 2.4 mm, and the minimum vertical distance between the heart rate processing module 21 and the green light source L2 is 3.7 mm. This can avoid that the light emitted by different light sources will not be blocked or interfered.

[0058] like Figure 6 As shown, in some embodiments, the positive electrodes of the infrared light source L1 and the green light source L2 are commonly connected to the first power supply terminal V1, and the negative electrodes of the infrared light source L1 and the green light source L2 are respectively connected to the heart rate processing module 21; The heart rate monitoring unit 20 further includes a first filter capacitor C1, a second filter capacitor C2, a third filter capacitor C3 and a fourth filter capacitor C4; One ends of the first filter capacitor C1, the second filter capacitor C2, the third filter capacitor C3 and the fourth filter capacitor C4 are connected in parallel to the positive electrodes of the infrared light source L1 and the green light source L2, and the other ends of the first filter capacitor C1, the second filter capacitor C2, the third filter capacitor C3 and the fourth filter capacitor C4 are grounded respectively.

[0059] In this embodiment, the infrared light source L1 and the two green light sources L2 share a common anode, the positive electrodes of the infrared light source L1 and the green light source L2 are connected to the first power supply terminal V1, and the negative electrodes of the infrared light source L1 and the green light source L2 are connected to the light source output pins of the heart rate processing module 21, that is, through Figure 7The lines of PPG_LED0, PPG_LED1, and PPG_LED3 transmit data with the heart rate processing module 21, so that the brightness and the frequency of the infrared light source L1 and the green light source L2 are controlled by the heart rate processing module 21. The heart rate monitoring unit 20 also includes a first filter capacitor C1, a second filter capacitor C2, a third filter capacitor C3, and a fourth filter capacitor C4; one end of the first filter capacitor C1, the second filter capacitor C2, the third filter capacitor C3, and the fourth filter capacitor C4 are connected in parallel to the positive electrodes of the infrared light source L1 and the green light source L2, and the other ends of the first filter capacitor C1, the second filter capacitor C2, the third filter capacitor C3, and the fourth filter capacitor C4 are grounded respectively. The first filter capacitor C1, the second filter capacitor C2, and the third filter capacitor C3 correspond to the infrared light source L1 and the two green light sources L2 respectively, and are placed close to the corresponding light sources. The fourth filter capacitor C4 is shared by the infrared light source L1 and the two green light sources L2, and is placed close to the infrared light source L1 and the two green light sources L2 to ensure the filtering effect of the first filter capacitor C1, the second filter capacitor C2, the third filter capacitor C3, and the fourth filter capacitor C4 on each light source.

[0060] like Figure 7 As shown, in some embodiments, the clock line pin, data line pin and interrupt pin of the heart rate processing module 21 are respectively connected to the main control unit 10; Each light source output pin of the heart rate processing module 21 is connected to the negative electrode of the infrared light source L1 and the green light source L2 respectively; The interface power pin of the heart rate processing module 21 is respectively connected to the second power supply terminal V2 and one end of the fifth filter capacitor C5, and the other end of the fifth filter capacitor C5 is grounded; The capacitor power pin of the heart rate processing module 21 is respectively connected to the second power supply terminal V2 and one end of the sixth filter capacitor C6, and the other end of the sixth filter capacitor C6 is grounded; The analog power pin and the digital power pin of the heart rate processing module 21 are connected in parallel to the second power supply terminal V2, the seventh filter capacitor C7, the eighth filter capacitor C8, and one end of the ninth filter capacitor C9, respectively, and the other ends of the seventh filter capacitor C7, the eighth filter capacitor C8, and the ninth filter capacitor C9 are grounded respectively.

[0061] In this embodiment, the heart rate processing module 21 communicates with the main control module 10 via I2C (Inter-Integrated Circuit) signals. The clock line pin ( Figure 7 Middle pin 6), data line pin ( Figure 7 pin 24) and the interrupt pin ( Figure 7The middle pin 4) is connected to the main control unit 10 respectively, and the interrupt pin of the heart rate processing module 21 is used to output an interrupt signal to notify the main control unit 10 when an event occurs in the heart rate processing module 21. Figure 7 The middle pins 14, 15, 16) are connected to the negative electrodes of the infrared light source L1 and the green light source L2 respectively; the interface power pin of the heart rate processing module 21 ( Figure 7 The pin 25 is connected to the second power supply terminal V2 and one end of the fifth filter capacitor C5 respectively, and the other end of the fifth filter capacitor C5 is grounded to filter the interface power supply. Figure 7 The analog power pin (18) of the heart rate processing module 21 is connected to the second power supply terminal V2 and one end of the sixth filter capacitor C6, and the other end of the sixth filter capacitor C6 is grounded to filter the capacitor power supply. Figure 7 Pin 7) and the digital power pin ( Figure 7 The middle pin 5) is connected in parallel to the second power supply terminal V2, the seventh filter capacitor C7, the eighth filter capacitor C8, and one end of the ninth filter capacitor C9, and the other ends of the seventh filter capacitor C7, the eighth filter capacitor C8, and the ninth filter capacitor C9 are grounded respectively to filter the analog power supply. The seventh filter capacitor C7 is smaller than the eighth filter capacitor C8. Since the analog power pin of the heart rate processing module 21 needs to power the analog part and has high requirements for the power supply, two capacitors are placed. The eighth filter capacitor C8 with a larger capacitance is used to filter out the interference of the low-frequency part, and the seventh filter capacitor C7 with a smaller capacitance is used to filter out the interference of the high-frequency part. The digital power pin of the heart rate processing module 21 needs to power the digital part, and the ninth filter capacitor C9 is used to filter the digital power pin. These filter capacitors need to be placed close to the corresponding pins.

[0062] like Figure 8 As shown, in some embodiments, the clock line pin, input line pin, output line pin and selection line pin of the inertia detection unit 30 are respectively connected to the main control unit 10; The first interrupt pin and the second interrupt pin of the inertia detection unit 30 are respectively connected to the main control unit 10; The main power pin of the inertia detection unit 30 is respectively connected to the third power supply terminal V3 and one end of the tenth filter capacitor C10, and the other end of the tenth filter capacitor C10 is grounded; The interface power pin of the inertia detection unit 30 is respectively connected to the third power supply terminal V3 and one end of the eleventh filter capacitor C11 , and the other end of the eleventh filter capacitor C11 is grounded.

[0063] In this embodiment, the inertia detection unit 30 communicates with the main control module 10 via SPI (Serial Peripheral Interface) signals. The clock line pin ( Figure 8 Pin 13), input line pin ( Figure 8 Pin 14), output line pin ( Figure 8 pin 1) and the select line pin ( Figure 8 The first interrupt pin ( Figure 8 middle pin 4) and the second interrupt pin ( Figure 8 The main power pins (9) of the inertia detection unit 30 are connected to the main control unit 10 for outputting interrupt signals and notifying the main control unit 10 when an event occurs in the inertia detection unit 30. Figure 8 The pin 8 is connected to the third power supply terminal V3 and one end of the tenth filter capacitor C10 respectively, and the other end of the tenth filter capacitor C10 is grounded to filter the main power supply. Figure 8 The middle pin 5) is respectively connected to the third power supply terminal V3 and one end of the eleventh filter capacitor C11, and the other end of the eleventh filter capacitor C11 is grounded to filter the interface power supply.

[0064] The present application provides a smart wearable device, which includes the above gesture interaction circuit, and the heart rate monitoring unit in the gesture interaction circuit is arranged on the side of the smart wearable device close to the skin. For example, the smart wearable device can be a device worn on the hand such as a smart ring, a smart bracelet, or a smart watch.

[0065] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0066] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the gesture interaction method in the above-mentioned embodiment.

[0067] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0068] The computer-readable storage medium may be included in the gesture interaction device; or may exist independently without being assembled into the gesture interaction device.

[0069] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the gesture interaction device, the gesture interaction device: collects the user's hand vasoconstriction information through the heart rate monitoring unit, and collects the user's hand inertia information through the inertia detection unit; identifies the corresponding gesture interaction action based on the hand vasoconstriction information and the hand inertia information; and sends the interaction control instruction corresponding to the gesture interaction action to the head-mounted display device connected to the smart wearable device.

[0070] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0071] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0072] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0073] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned gesture interaction method, and can solve the technical problem that the existing gesture interaction method affects the battery life of the head-mounted display device. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the gesture interaction method provided by the above-mentioned embodiment, and will not be repeated here.

[0074] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned gesture interaction method when executed by a processor.

[0075] The computer program product provided by this application can solve the technical problem that the existing gesture interaction method affects the battery life of the head-mounted display device. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the gesture interaction method provided by the above embodiment, which will not be repeated here.

[0076] The above are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A gesture interaction method, characterized in that: Applied to a smart wearable device, the smart wearable device is equipped with a heart rate monitoring unit and an inertia detection unit, and the gesture interaction method includes: Collecting the user's hand vasoconstriction information through the heart rate monitoring unit, and collecting the user's hand inertia information through the inertia detection unit; identifying corresponding gesture interaction actions according to the hand vasoconstriction information and the hand inertia information; The interactive control instruction corresponding to the gesture interaction action is sent to a head mounted display device connected to the smart wearable device.

2. The gesture interaction method according to claim 1, characterized in that: The step of identifying the corresponding gesture interaction action according to the hand vasoconstriction information and the hand inertia information comprises: Recognizing the real-time hand posture according to the hand vasoconstriction information; Identifying a hand motion trajectory according to the hand inertia information; A corresponding gesture interaction action is determined according to the real-time hand posture and the hand movement trajectory.

3. The gesture interaction method according to claim 2, characterized in that: The step of determining the corresponding gesture interaction action according to the real-time hand posture and the hand motion trajectory comprises: After the real-time hand posture is a predetermined starting gesture, determining a gesture starting time; After the real-time hand gesture is a predetermined ending gesture, determining a gesture ending time; intercepting a target motion trajectory between the start time of the gesture and the end time of the gesture from the hand motion trajectory; According to the target motion trajectory, a corresponding gesture interaction action is determined.

4. The gesture interaction method according to claim 3, characterized in that: After the real-time hand gesture is a predetermined starting gesture, after the step of determining the gesture starting time, the method includes: Determining whether the hand motion trajectory indicates that the hand has not moved after the gesture start time; If the hand motion trajectory indicates that the hand has not moved after the gesture start time, the predetermined start gesture is used as the gesture interaction action.

5. A gesture interaction circuit, characterized in that: Applied to the gesture interaction method according to any one of claims 1 to 4, the gesture interaction circuit comprises a main control unit, and a heart rate monitoring unit and an inertia detection unit connected to the main control unit; The heart rate monitoring unit is used to collect the user's hand vasoconstriction information and send it to the main control unit; The inertia detection unit is used to collect the user's hand inertia information and send it to the main control unit; The main control unit is used to identify the corresponding gesture interaction action according to the hand vasoconstriction information and the hand inertia information; and send the interaction control instruction corresponding to the gesture interaction action to the head-mounted display device connected to the smart wearable device.

6. The gesture interaction circuit as claimed in claim 5, characterized in that: The heart rate monitoring unit includes a heart rate processing module, and at least one infrared light source and two green light sources connected to the heart rate processing module; One green light source is on the same straight line as the infrared light source, and the other green light source is parallel to the infrared light source.

7. The gesture interaction circuit as claimed in claim 6, characterized in that: The minimum straight-line distance between the infrared light source and the green light source on the same straight line is 2 mm; The minimum vertical distance between the parallel infrared light source and the green light source is 2.4 mm, and the minimum vertical distance between the heart rate processing module and the green light source is 3.7 mm.

8. The gesture interaction circuit as claimed in claim 6, characterized in that: The positive electrodes of the infrared light source and the green light source are connected to the first power supply terminal, and the negative electrodes of the infrared light source and the green light source are connected to the heart rate processing module respectively; The heart rate monitoring unit further includes a first filter capacitor, a second filter capacitor, a third filter capacitor and a fourth filter capacitor; One ends of the first filter capacitor, the second filter capacitor, the third filter capacitor and the fourth filter capacitor are connected in parallel to the positive electrodes of the infrared light source and the green light source, and the other ends of the first filter capacitor, the second filter capacitor, the third filter capacitor and the fourth filter capacitor are grounded respectively.

9. The gesture interaction circuit as claimed in claim 6, characterized in that: The clock line pin, data line pin and interrupt pin of the heart rate processing module are respectively connected to the main control unit; Each light source output pin of the heart rate processing module is connected to the negative electrode of the infrared light source and the negative electrode of the green light source respectively; The interface power pin of the heart rate processing module is respectively connected to the second power supply end and one end of the fifth filter capacitor, and the other end of the fifth filter capacitor is grounded; The capacitor power pin of the heart rate processing module is respectively connected to the second power supply end and one end of the sixth filter capacitor, and the other end of the sixth filter capacitor is grounded; The analog power pin and the digital power pin of the heart rate processing module are connected in parallel and are respectively connected to the second power supply end, the seventh filter capacitor, the eighth filter capacitor, and one end of the ninth filter capacitor, and the other ends of the seventh filter capacitor, the eighth filter capacitor, and the ninth filter capacitor are respectively grounded.

10. The gesture interaction circuit according to any one of claims 5 to 9, characterized in that: The clock line pin, input line pin, output line pin and selection line pin of the inertia detection unit are respectively connected to the main control unit; The first interrupt pin and the second interrupt pin of the inertia detection unit are respectively connected to the main control unit; The main power pin of the inertia detection unit is respectively connected to the third power supply terminal and one end of the tenth filter capacitor, and the other end of the tenth filter capacitor is grounded; The interface power pin of the inertia detection unit is respectively connected to the third power supply end and one end of the eleventh filter capacitor, and the other end of the eleventh filter capacitor is grounded.

11. A smart wearable device, characterized in that: The smart wearable device comprises the gesture interaction circuit as claimed in any one of claims 5 to 10, and the heart rate monitoring unit in the gesture interaction circuit is arranged on a side of the smart wearable device close to the skin.

12. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the gesture interaction method according to any one of claims 1 to 4 are implemented.

13. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the gesture interaction method according to any one of claims 1 to 4 are implemented.

Images