Gesture recognition method and apparatus

Abstract

The present application relates to the technical field of terminals, and discloses a gesture recognition method and apparatus, capable of improving the accuracy of gesture recognition. The method is applied to a first electronic device, the first electronic device comprises a first sensor, the first sensor is located on a side of the first electronic device, and the side refers to the side of the first electronic device, worn by a user, closer to the user's hand. The method comprises: in response to a first gesture of the user, the first sensor collects a first signal; and the first electronic device recognizes the first gesture on the basis of the first signal, and then the first electronic device executes a corresponding operation on the basis of a recognition result of the first gesture.

Description

Gesture recognition method and device

[0001] This application claims priority from the Chinese patent application No. 202411111141.0 filed on August 13, 2024, and entitled "Gesture recognition method and device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of terminals, and in particular to a gesture recognition method and device. BACKGROUND

[0003] With the development of terminal technology, electronic devices such as smartwatches are applied. In some scenarios, a user can use some functions in a smartwatch through gestures, or the user can use a smartwatch to control other devices through gestures. However, the recognition effect of the smartwatch on the user's gestures is poor, which affects the user experience. SUMMARY

[0004] The present application provides a gesture recognition method and device, which can improve the accuracy of gesture recognition.

[0005] To achieve the above purpose, the present application provides the following technical solutions:

[0006] In a first aspect, the embodiments of the present application provide a gesture recognition method applied to a first electronic device, the first electronic device comprising a first sensor, the first sensor being located at a side edge of the first electronic device, and the side edge being a side close to a user's hand when the user wears the first electronic device, the method comprising: in response to a first gesture of the user, the first sensor can collect a first signal, and the first electronic device can recognize the first gesture based on the first signal.

[0007] For example, as shown in FIG. 2A, the first sensor (side edge PPG sensor) is located at the side edge of the first electronic device, and the side edge is the side close to the user's hand when the user wears the first electronic device. The user makes a first gesture, in response to the first gesture of the user, the first sensor can collect a first signal, and the first electronic device can recognize the first gesture (the gesture made by the user) based on the first signal. The first electronic device can also be a smart ring as shown in FIG. 2B or smart glasses as shown in FIG. 2C, etc.

[0008] The first sensor is located at the side edge of the first electronic device, and can collect the first signal from the user's hand. When the user's hand moves, the first signal also changes, and the first signal can represent the movement characteristics of the user's hand. In this way, the gesture is recognized by the first sensor at the side edge of the first electronic device, which reduces the interference of external factors such as the wearing position, wearing tightness or wrist characteristics of the user of the first electronic device, and improves the accuracy of gesture recognition.

[0009] In a possible implementation, the first sensor includes a side PPG sensor. The side PPG sensor can emit a light signal and collect a reflected light signal (the first signal). In this way, the first electronic device can identify the first gesture according to a size change of the light signal collected by the side PPG sensor.

[0010] Optionally, the first sensor can further include an infrared sensor, an ultrasonic sensor, or a radar sensor, and the embodiments of the present application do not make a specific limitation.

[0011] In a possible implementation, identifying the first gesture based on the first signal includes: obtaining a waveform feature of the first signal based on the first signal, and identifying the first gesture based on the waveform feature of the first signal.

[0012] For example, as shown in (a) of FIG. 10, the waveform of the first signal is shown, and the waveform feature of the first signal can be obtained based on the waveform of the first signal. The processor can identify the first gesture based on the waveform feature of the first signal. For example, the waveform feature of the first signal matches the waveform feature of the first signal of the second gesture (gesture A) in the preset gesture library, and it can be determined that the first gesture is the second gesture.

[0013] In this way, the first electronic device can accurately identify the first gesture based on the first signal.

[0014] In a possible implementation, the waveform feature of the first signal includes one or more of the following: a peak value (also referred to as a peak value) of the waveform of the first signal, a peak width, a number of peaks, a trough value of the waveform, a trough width, a number of troughs, a mean value of the waveform, a variance of the waveform, a peak-to-peak value of the waveform, or a gradient of the waveform.

[0015] The gradient of the waveform can indicate the amplitude of the first gesture and the completion time of the first gesture. When the amplitude of the gesture is large and the user completes the gesture in a short time, the gesture is identified, which can prevent gesture recognition caused by user mis-touch.

[0016] In a possible implementation, the first electronic device further includes a second sensor, and identifying the first gesture based on the first signal includes: the second sensor collects a second signal; and identifying the first gesture based on the first signal and the second signal.

[0017] Optionally, the second sensor can be an acceleration (ACC) sensor and / or a gyroscope sensor, etc., for collecting acceleration information (the second signal) of the user's hand.

[0018] In this way, the first electronic device can identify the first gesture based on the first signal collected by the side PPG sensor and the second signal collected by the ACC sensor and / or the gyroscope sensor, and the gesture recognition result is more accurate.

[0019] In an embodiment, the first electronic device can obtain a waveform feature of the second signal based on the second signal, and identify the first gesture based on the waveform feature of the first signal and the waveform feature of the second signal.

[0020] In a possible implementation, identifying the first gesture based on the first signal includes: if the first signal satisfies a first feature and the second signal satisfies a second feature, the first gesture is a single-hand gesture; the second feature indicates that a waveform change of the second signal is greater than a first preset threshold.

[0021] For example, the waveform feature of the first signal matches (satisfies the first feature) the waveform feature of the first signal of a second gesture in a preset gesture library, and the waveform change of the second signal is greater than the first preset threshold (satisfies the second feature), and it is determined that the first gesture is a single-hand gesture.

[0022] For another example, the waveform feature of the first signal matches (satisfies the first feature) the waveform feature of the first signal of a second gesture in a preset gesture library, and the waveform feature of the second signal matches (satisfies the second feature) the waveform feature of the second signal of a single-hand gesture in the preset gesture library, and it is determined that the first gesture is a single-hand gesture.

[0023] In a possible implementation, identifying the first gesture based on the first signal includes: if the first signal satisfies a third feature and the second signal satisfies a fourth feature, the first gesture is a double-hand gesture; the fourth feature indicates that a waveform change of the second signal is less than or equal to a first preset threshold.

[0024] For example, the waveform feature of the first signal matches (satisfies the third feature) the waveform feature of a second gesture in a preset gesture library, and the waveform change of the second signal is less than or equal to the first preset threshold (satisfies the fourth feature), and it is determined that the first gesture is a double-hand gesture.

[0025] For another example, the waveform feature of the first signal matches (satisfies the second feature) the waveform feature of the first signal of a second gesture in a preset gesture library, and the waveform feature of the second signal matches (satisfies the fourth feature) the waveform feature of the second signal of a double-hand gesture in the preset gesture library, and it is determined that the first gesture is a single-hand gesture.

[0026] The first feature can be the same as or different from the third feature, and the embodiments of the present application do not make specific limitations thereto.

[0027] In this way, the first electronic device can not only recognize a gesture made by a single hand of the user, but also recognize a gesture made by both hands of the user, the gesture type is expanded, the gesture recognition is more flexible, and the user experience is improved.

[0028] In a possible implementation, the first gesture is recognized based on the first signal and the second signal, including: determining direction information of the first electronic device based on the second signal; and recognizing the first gesture based on the direction information of the first electronic device and the first signal.

[0029] For example, as shown in FIG. 15, the first electronic device can determine the direction information of the first electronic device, such as the orientation of the first electronic device or the orientation of the wrist of the user, based on the second signal collected by the ACC sensor and / or the gyroscope sensor, and recognize the first gesture based on the orientation of the first electronic device and the first signal.

[0030] In this way, the first electronic device can recognize simple gestures in multiple directions according to the waveform characteristics of the first signal and the waveform characteristics of the second signal, the gesture type is expanded, and it is also convenient for the user to learn gestures.

[0031] In a possible implementation, the first electronic device further includes a third sensor, and the first gesture is recognized based on the first signal, including: collecting a third signal; the third signal is used to represent a physiological feature of a hand of the user; the first gesture is recognized based on the first signal, the second signal and the third signal; or the first gesture is recognized based on the first signal and the third signal.

[0032] Optionally, the third sensor can be a bottom PPG sensor, which is used to detect a physiological signal of the user, and the third signal collected by the third sensor is used to represent the physiological feature of the hand of the user.

[0033] The first electronic device can recognize the first gesture based on the first signal and the third signal. For example, the priority of the first signal is set to be higher than the priority of the third signal, the first electronic device first recognizes the gesture according to the first signal, and when the recognition fails (for example, no corresponding gesture is matched) according to the first signal, the gesture is recognized according to the third signal.

[0034] Alternatively, the first electronic device can recognize the first gesture based on the first signal, the second signal and the third signal. For example, the priority of the first signal is set to be higher than the priority of the third signal, the first electronic device first recognizes the gesture according to the first signal and the second signal, and when the recognition fails (for example, no corresponding gesture is matched) according to the first signal, the gesture is recognized according to the third signal and the second signal.

[0035] Therefore, the gesture recognition is more accurate, the application scenarios are more extensive, the types of recognizable gestures are increased, and user experience is improved.

[0036] In a possible implementation, the first gesture is identified based on the first signal, the second signal and the third signal, including: weighting the waveform feature of the first signal, the waveform feature of the second signal and the waveform feature of the third signal.

[0037] The weight of the waveform feature of the second signal is unchanged.

[0038] The waveform variation amplitude in the waveform feature of the first signal is greater than the waveform variation amplitude in the waveform feature of the third signal, and the weight of the waveform feature of the first signal is greater than the weight of the waveform feature of the third signal.

[0039] Alternatively,

[0040] The waveform variation amplitude in the waveform feature of the first signal is less than the waveform variation amplitude in the waveform feature of the third signal, and the weight of the waveform feature of the first signal is less than the weight of the waveform feature of the third signal.

[0041] Alternatively,

[0042] The waveform variation amplitude in the waveform feature of the first signal is less than the waveform variation amplitude in the waveform feature of the third signal, and the weight of the waveform feature of the first signal is the same as the weight of the waveform feature of the third signal.

[0043] The first gesture is identified based on the weighted waveform feature of the first signal, the waveform feature of the second signal and the waveform feature of the third signal.

[0044] Therefore, the first signal, the second signal and the third signal are weighted and fused to identify the gesture, so that the accuracy of gesture recognition can be improved.

[0045] In a possible implementation, the first gesture is identified based on the first signal, including: controlling the first electronic device based on the first gesture; or, controlling the second electronic device through the first electronic device based on the first gesture.

[0046] For example, the smart watch (the first electronic device) can control the smart watch based on the first gesture; or, the smart watch can control the smart air conditioner (the second electronic device) by controlling the smart watch.

[0047] Therefore, the application scenarios of the embodiments of the present application are more extensive, and the user experience is further improved.

[0048] In a possible implementation, the first gesture includes one or more of the following: turning the palm upward, continuously turning the palm upward, turning the palm downward, continuously turning the palm downward, turning the palm to the left, turning the palm to the right, keeping the hand wearing the first electronic device still, moving another hand closer to or farther away from the first electronic device, tapping a finger, pinching a finger, or rubbing the palm.

[0049] In a possible implementation, before the first sensor collects the first signal in response to the first gesture of the user, the method further includes: the gesture recognition function in the first electronic device is closed; the first electronic device detects a trigger condition, and the gesture recognition function in the first electronic device is opened; the trigger condition includes one or more of the following: a time condition, a location condition, or an event condition.

[0050] In this way, the gesture recognition function of the first electronic device is in a normally closed state, and the gesture recognition function is triggered passively only when a trigger event is detected, which can reduce the power consumption of the first electronic device.

[0051] In a possible implementation, before the first sensor collects the first signal in response to the first gesture of the user, the method further includes: the gesture recognition function in the first electronic device is closed; the first electronic device detects that the user performs a pre-action, and the gesture recognition function in the first electronic device is opened; the pre-action includes one or more of the following: the user raising the wrist or the user turning the wrist, a touch operation of the user, a voice operation of the user, or the user wearing the first electronic device touching the service card.

[0052] In this way, the gesture recognition function of the first electronic device is in a normally closed state and is set with a pre-action, and the gesture recognition function is opened only when the user actively performs the pre-action, which can reduce the misrecognition rate of the user's unconscious action and can reduce the power consumption of the first electronic device.

[0053] In a possible implementation, the first electronic device further includes a function of determining a physiological feature of the user through the first sensor, and the method further includes: closing the function of determining the physiological feature of the user through the first sensor in the first electronic device; and opening the function of gesture recognition through the first sensor in the first electronic device.

[0054] In this way, the first signal collected by the first electronic device can be prevented from being used to determine the physiological feature and used for gesture recognition, which can affect the result of gesture recognition or the measurement result of the physiological feature.

[0055] In a second aspect, the present application provides a gesture recognition device applied to a first electronic device, the first electronic device comprising a first sensor, the first sensor being located at a side edge of the first electronic device, and the side edge being a side close to a user's hand when the user wears the first electronic device, the device comprising: a collection module configured to collect a first signal in response to a first gesture of the user; and a processing module configured to recognize the first gesture based on the first signal.

[0056] In a third aspect, the present application provides an electronic device, comprising: a processor and a memory, the memory being coupled to the processor, the memory being configured to store computer program codes, the computer program codes comprising computer instructions, and the processor being configured to read the computer instructions from the memory, so that the first electronic device executes the method of the first aspect or any one of the implementation manners of the first aspect.

[0057] In a fourth aspect, the present application provides a computer readable storage medium, the computer readable storage medium comprising computer programs or instructions, and the computer programs or instructions being configured to cause the first electronic device of the second aspect to execute the method of the first aspect when the computer programs or instructions are run on the first electronic device.

[0058] In a fifth aspect, the present application provides a computer program product, the computer program product comprising: computer programs or instructions, and the computer programs or instructions being configured to cause a computer to execute the method of the first aspect when the computer programs or instructions are run on the computer.

[0059] In a sixth aspect, the present application provides a chip system, comprising: a processor, the processor being configured to call and run computer programs stored in a memory to execute any one of the methods provided in the implementation manners of the first aspect.

[0060] The technical effects of the second aspect to the sixth aspect and any one of the implementation manners of the second aspect to the sixth aspect can be referred to the technical effects of the first aspect and any one of the implementation manners of the first aspect, which will not be described herein. BRIEF DESCRIPTION OF DRAWINGS

[0061] FIG. 1 is a schematic diagram of a gesture recognition scene according to an embodiment of the present application;

[0062] FIG. 2A is a structural schematic diagram of an electronic device according to an embodiment of the present application;

[0063] FIG. 2B is a structural schematic diagram of another electronic device according to an embodiment of the present application;

[0064] FIG. 2C is a structural schematic diagram of another electronic device according to an embodiment of the present application;

[0065] FIG. 3 is a structural schematic diagram of a gesture recognition device according to an embodiment of the present application;

[0066] FIG. 4 is a flow diagram of a gesture recognition method according to an embodiment of the present application;

[0067] FIG. 5 is a schematic diagram of a gesture recognition device according to an embodiment of the present application;

[0068] FIG. 6 is a gesture diagram according to an embodiment of the present application;

[0069] FIG. 7A is a waveform diagram of a gesture according to an embodiment of the present application;

[0070] FIG. 7B is a waveform diagram of a gesture according to an embodiment of the present application;

[0071] FIG. 8 is a gesture diagram according to an embodiment of the present application;

[0072] FIG. 9A is a waveform diagram of a gesture according to an embodiment of the present application;

[0073] FIG. 9B is a waveform diagram of a gesture according to an embodiment of the present application;

[0074] FIG. 10 is a waveform diagram of a gesture according to an embodiment of the present application;

[0075] FIG. 11 is a flow diagram of a gesture recognition method according to an embodiment of the present application;

[0076] FIG. 12 is a flow diagram of a gesture recognition method according to an embodiment of the present application;

[0077] FIG. 13 is a waveform diagram of a gesture according to an embodiment of the present application;

[0078] FIG. 14 is a scene diagram of a gesture recognition according to an embodiment of the present application;

[0079] FIG. 15 is a gesture diagram according to an embodiment of the present application;

[0080] FIG. 16 is a flow diagram of a gesture recognition method according to an embodiment of the present application;

[0081] FIG. 17 is a flow diagram of a gesture recognition method according to an embodiment of the present application;

[0082] FIG. 18 is a flow diagram of a gesture recognition wake-up method according to an embodiment of the present application;

[0083] FIG. 19 is a flow diagram of a gesture recognition wake-up method according to an embodiment of the present application;

[0084] FIG. 20 is a flow diagram of a gesture recognition wake-up method according to an embodiment of the present application;

[0085] FIG. 21 is a flow diagram of another gesture recognition method according to an embodiment of the present application;

[0086] FIG. 22 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application;

[0087] FIG. 23 is a schematic diagram of a chip system according to an embodiment of the present application. DETAILED DESCRIPTION

[0088] In the embodiments of the present application, the word "exemplary" or "for example" is used to mean serving as an example, instance, or illustration. Any implementation or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being preferred or advantageous over other implementations or design solutions. In fact, the word "exemplary" or "for example" is used to present concepts in a concrete manner in the embodiments of the present application.

[0089] In the embodiments of the present application, the terms "first", "second", etc. are used only for the purpose of description, and should not be interpreted or implied to indicate or imply relative importance or imply the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specified.

[0090] In the present application, the term "at least one" means one or more, and the term "plurality" means two or more, for example, a plurality of second packets means two or more second packets. The terms "system" and "network" are often used interchangeably in the present application.

[0091] It should be understood that the terms used in the description of various described examples herein are only for the purpose of describing specific examples, and are not intended to be limiting.

[0092] It should also be understood that the term "and / or" used herein means and encompasses any and all possible combinations of one or more of the associated listed items. The term "and / or", is a description of the associated relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there can be three cases: A exists alone, A and B exist together, and B exists alone. In addition, the character " / " in the present application generally represents an "or" relationship between the front and rear associated objects.

[0093] It should also be understood that in various embodiments of the present application, the size of the serial number of each process does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0094] It should be understood that every mention of "an embodiment," "one embodiment," "a possible implementation," or the like throughout this specification means that a particular feature, structure, or characteristic described in connection with the embodiment or implementation is included in at least one embodiment of the application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in a possible implementation" in various places throughout this specification are not necessarily referring to the same embodiment. Furthermore, the particular features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0095] Firstly, some technical terms related to the embodiments of the present application are introduced:

[0096] Photo plethysmo graphy (PPG):

[0097] PPG is a non-invasive optical measurement technology, which belongs to infrared non-destructive testing technology, and can be used to detect gestures of a user, and can also be used to detect physiological parameters such as heart rate, blood pressure and / or blood oxygen saturation.

[0098] A PPG sensor is a sensor based on PPG technology, which can include a light source and a photoelectric detector. The light source is used to emit a light signal, and the light source can include a light emitting diode (LED). The photoelectric detector is used to collect a reflected light signal (or a transmitted light signal), and the photoelectric detector can include a photo diode (PD) and / or a photo transistor, and the PD can convert the reflected light signal into a voltage signal.

[0099] For example, after the light signal emitted by the light source reaches the user's hand, part of the light is absorbed by the skin and blood of the hand, and another part of the light signal is reflected back to the PPG sensor. The photoelectric detector in the PPG sensor can collect the reflected light signal. Subsequently, the size change of the reflected light signal can be used to determine the motion characteristics of the hand. The motion characteristics can include the direction of the hand motion, the speed of the motion, the distance of the motion, or the frequency of the motion, etc. Alternatively, the size change of the reflected light signal can be used to determine physiological characteristics such as changes in blood flow caused by compression of blood vessels in the hand (such as the wrist), and physiological parameters such as heart rate, blood pressure and / or blood oxygen saturation of the user can be obtained according to the physiological characteristics.

[0100] With the advancement of terminal technology, the functions integrated in terminal devices are also increasing. For example, in addition to the original time display function, a smart watch also includes functions such as controlling other home devices, health detection, and message notification. For example, as shown in Figure 1(b), a user can control a smart air conditioner through a smart watch.

[0101] With the increase of functions in the smart watch, in order to improve the user experience, it is necessary to further increase the interaction mechanism between the user and the smart watch, and improve the interaction efficiency between the user and the smart watch.

[0102] In the related art, the user can operate on the touch screen (display screen) of the smart watch. For example, the user can perform touch and / or sliding operations on the touch screen by fingers, and the smart watch can implement corresponding functions in response to the above operations. When the user cannot effectively or quickly operate on the touch screen; for example, there is water stain on the touch screen, the user's hand has water stain, or the interface currently displayed by the smart watch is not the interface that the user wants to access or operate. The user can also control the smart watch through the physical keys in the smart watch; for example, the user can press the power key, rotate the crown or press the function key, or press the key in a self-defined manner, to control the smart watch.

[0103] In some scenarios, the user is not convenient to press the physical keys in the smart watch. At this time, the user can control the smart watch through gestures, and when the user makes a gesture, the smart watch can recognize the gesture through the bottom PPG sensor (located at the bottom of the watch dial of the smart watch, the PPG sensor that is attached to the user's skin when worn) and the acceleration (ACC) sensor, and execute corresponding operations according to the recognized gesture, to control the smart watch.

[0104] For example, when the user makes a gesture, the smart watch can collect reflected light signals based on the bottom PPG sensor at the wrist part, to obtain physiological characteristics of the user's wrist part; the smart watch can also collect signals based on the ACC sensor and / or gyroscope sensor, to obtain motion characteristics of the user's hand. Then the smart watch can input the obtained physiological characteristics and motion characteristics into a machine learning model, to predict the gesture, and execute the operation corresponding to the gesture.

[0105] However, the reflected light signals collected by the bottom PPG sensor are greatly disturbed by external factors, for example, including the wearing position of the smart watch, the wearing tightness or the wrist characteristics of the user, etc., wherein the wrist characteristics of the user can refer to the geometric shape of the user's wrist. When the external factors change, the accuracy of the physiological signals collected by the bottom PPG sensor is low, and the accuracy of gesture recognition is also low.

[0106] To solve the above problems, an embodiment of the present application provides a gesture recognition method. In the method, a first electronic device can collect reflected light signals through a side PPG sensor, and recognize a gesture based on the reflected light signals collected by the side PPG sensor. Then the first electronic device can execute corresponding operations based on the gesture recognition result. Wherein, the side PPG sensor can be located at one side of the side of the first electronic device close to the hand.

[0107] The first electronic device can be a smart watch, a smart bracelet, a smart helmet, a smart ring, or a smart glass, etc. The embodiments of the present application do not specially limit the specific form of the first electronic device. The gesture recognition of the embodiments of the present application can include any hand-related action recognition. For example, the gesture recognition can include recognition of any one or more of the following actions: wrist action, palm action, finger action, or hand occlusion, etc.

[0108] It can be understood that, regardless of the form of the first electronic device, the arrangement of the side PPG sensor in the first electronic device can meet the requirement of gesture recognition.

[0109] The possible forms of the first electronic device being a smart watch, a smart ring, or a smart glass will be described below with reference to FIGS. 2A-2C.

[0110] As shown in FIG. 2A, the first electronic device can be a smart watch, and the side PPG sensor can be located at the side of the first electronic device, which is close to the user's hand when the user wears the first electronic device. The side of the first electronic device can include the side of the right side of the dial of the first electronic device or the side of the left side of the dial of the first electronic device, etc.

[0111] For example, as shown in (a) of FIG. 2A, the side PPG sensor is located at the right side of the dial of the first electronic device. As shown in (a) of FIG. 2A, the side PPG sensor can be located at the side of the smart watch close to the hand, i.e., the right side of the dial of the smart watch. In this way, when the user wears the smart watch with the left hand, the side PPG sensor can collect the light signal reflected by the user's hand, and does not need to collect the light signal reflected by the user's wrist. Therefore, the interference of external factors such as the wearing position, the tightness of wearing, or the wrist characteristics of the user of the smart watch can be reduced, and the accuracy of gesture recognition can be improved.

[0112] Optionally, the side PPG sensor can be located at the right side of the first electronic device as shown in (a) of FIG. 2A; or the side PPG sensor can also be located at the upper right side of the first electronic device; or the side PPG sensor can also be located at the lower right side of the first electronic device, and the embodiments of the present application do not specially limit this.

[0113] Fig. 2A (b) shows an implementation where the side PPG sensor is located on the left side of the first electronic device watch face. As shown in Fig. 2A (b), the side PPG sensor can be located on the side of the smart watch close to the hand, i.e., on the left side of the smart watch watch face. In this way, when the user wears the smart watch on the left hand, the side PPG sensor can collect the light signal reflected by the user's hand, rather than the light signal reflected by the user's wrist, thus reducing the interference of external factors such as the wearing position, wearing tightness of the smart watch, or the user's wrist characteristics, and improving the accuracy of gesture recognition.

[0114] Optionally, the side PPG sensor can be located on the left side of the first electronic device as shown in Fig. 2A (b); or the side PPG sensor can also be located on the left side of the first electronic device; or the side PPG sensor can also be located on the left side of the first electronic device, and the embodiments of the present application do not make specific limitations thereto.

[0115] It should be noted that the number of side PPG sensors can be multiple, and the multiple side PPG sensors can be distributed on the sides of the left and right sides of the first electronic device watch face (not shown in the figure), so that when the user wears the first electronic device on the right hand, the first electronic device can select the side PPG sensor on the left side of the watch face for gesture recognition, and when the user wears the first electronic device on the left hand, the first electronic device can select the side PPG sensor on the right side of the watch face for gesture recognition. It can be understood that the habits of users wearing smart watches can be different, and therefore the specific positions of the side PPG sensors in the smart watch can be flexibly selected based on the habits of users wearing the smart watch.

[0116] As shown in Fig. 2B, the first electronic device can be a smart ring. The side PPG sensor can be located on the side of the first electronic device, and the side is close to the user's fingertips and / or the back of the hand when the user wears the first electronic device. The side of the first electronic device can include the side of the right side of the first electronic device or the side of the left side of the first electronic device, etc.

[0117] Fig. 2B (a) shows an implementation where the side PPG sensor is located on the right side of the first electronic device. As shown in Fig. 2B (a), the side PPG sensor can be located on the side of the smart ring close to the fingertips, i.e., on the right side of the smart ring. In this way, when the user wears the smart ring on the left hand, the side PPG sensor can collect the light signal reflected when the user's fingers are bent, and can reduce the interference of external factors such as the wearing position, wearing tightness of the smart ring, or the user's finger characteristics, and improve the accuracy of gesture recognition. The finger characteristics of the user can refer to the geometric shape of the user's fingers.

[0118] Optionally, the side PPG sensor can be located on the right side of the first electronic device as shown in FIG. 2B(a); or the side PPG sensor can also be located on the upper right side of the first electronic device; or the side PPG sensor can also be located on the lower right side of the first electronic device, and the embodiments of the present application do not make specific limitations.

[0119] FIG. 2B(b) shows that the side PPG sensor is located on the left side of the first electronic device. As shown in FIG. 2B(b), the side PPG sensor can be located on the side of the smart ring close to the fingertips, i.e. the left side of the smart ring. In this way, when the user wears the smart ring with the right hand, the side PPG sensor can collect the reflected light signal when the user's finger is bent, which can reduce the interference of external factors such as the wearing position, wearing tightness of the smart ring or the user's finger characteristics, and improve the accuracy of gesture recognition.

[0120] Optionally, the side PPG sensor can be located on the right side of the first electronic device as shown in FIG. 2B(a); or the side PPG sensor can also be located on the upper right side of the first electronic device; or the side PPG sensor can also be located on the lower right side of the first electronic device, and the embodiments of the present application do not make specific limitations.

[0121] It can be understood that the above FIG. 2B(a) and FIG. 2B(b) are both described with the side PPG sensor located on the side close to the user's fingertips when the user wears the first electronic device. In some implementations, the side PPG sensor can also be located on the side close to the user's back of the hand when the user wears the first electronic device (not shown in the figure), and then the side PPG sensor can collect the reflected light signal when the user's back of the hand, wrist and / or arm moves, thereby realizing the detection of gestures, which will not be described.

[0122] It should be noted that the number of side PPG sensors can be multiple, and the multiple side PPG sensors can be distributed on the side of the first electronic device on the left and right sides (not shown in the figure), so that when the user wears the right hand, the first electronic device can select the left side PPG sensor for gesture recognition, and when the user wears the left hand, the first electronic device can select the right side PPG sensor for gesture recognition. It can be understood that the position of the user wearing the smart ring can be different, such as the user can wear one or more of the index finger, middle finger, ring finger, little finger, or thumb, or rotate the smart ring. Therefore, the specific position of the side PPG sensor in the smart ring can be flexibly selected based on the position of the smart ring worn by the user. As shown in FIG. 2C, the first electronic device can be smart glasses. The side PPG sensor can be located on the side of the first electronic device, and the side is the side close to the hand when the user wears the first electronic device. Among them, the side of the first electronic device can include the glasses legs and / or the glasses frame of the smart glasses. The side close to the hand when the user wears the first electronic device can be understood as the side not attached to the skin of the face.

[0123] FIG. 2C shows that the side PPG sensor can be located on the right side of the glasses legs of the first electronic device (the right side is the right side relative to the user). As shown in FIG. 2C, the side PPG sensor can be located on the side of the first electronic device that is not attached to the skin of the face. In this way, when the user wears smart glasses, the side PPG sensor can collect the light signal reflected when the user's hand is close to, away from, and / or touches the right side of the glasses legs of the first electronic device, which can reduce the interference of external factors such as the wearing position, wearing tightness, etc. of the first electronic device, and improve the accuracy of gesture recognition.

[0124] Optionally, the side PPG sensor can be located at the middle position of the right side of the glasses legs of the first electronic device as shown in FIG. 2C; the side PPG sensor can also be located at the position close to or away from the glasses frame of the right side glasses leg; the side PPG sensor can also be located at the middle position of the left side glasses leg; the side PPG sensor can also be located at the position close to or away from the glasses frame of the left side glasses leg, etc. The embodiments of the present application do not make specific limitations thereto.

[0125] As described above, the possible forms of the first electronic device as smart watches, smart rings, or smart glasses are exemplarily described. It can be understood that the form of the first electronic device is not limited by the embodiments of the present application, and regardless of the form of the first electronic device, the working principle of the side PPG sensor in the first electronic device can be similar, and gesture recognition can be performed by transmitting a light signal and receiving a reflected signal of the light signal. The possible forms and working principles of the side PPG sensor in the first electronic device are described below.

[0126] In one embodiment, the side PPG sensor can be integrated with the signal processing unit as a side PPG module.

[0127] Optionally, the side PPG module can collect a light signal reflected by the user's hand, and the signal processing unit can obtain a waveform feature of the reflected light signal based on the reflected light signal.

[0128] The waveform feature includes, but is not limited to, one or more of the following: a peak value of the waveform, a peak width, a number of peaks, a trough value of the waveform, a trough width, a number of troughs, a mean value of the waveform, a variance of the waveform, a peak-to-peak value of the waveform, or a gradient of the waveform, etc.

[0129] In another embodiment, the side PPG sensor can also be integrated with other sensors as a module.

[0130] For example, the side PPG sensor can be integrated with an electrocardiogram (ECG) sensor as a module, which can collect a light signal reflected by the hand through the side PPG sensor and collect an electrical signal of the hand through the ECG sensor. Subsequently, the processor can identify a gesture based on the collected light signal reflected by the hand and the electrical signal.

[0131] For another example, the side PPG sensor can be integrated with a pressure sensor as a module, which can collect a light signal reflected by the hand through the side PPG sensor and collect a pressure signal of the hand through the pressure sensor. Subsequently, the processor can identify a gesture based on the collected light signal reflected by the hand and the pressure signal. For another example, the side PPG sensor can be integrated with an ECG sensor and a pressure sensor as a module, which can collect a light signal reflected by the hand through the side PPG sensor, collect an electrical signal of the hand through the ECG sensor, and collect a pressure signal of the hand through the pressure sensor. Subsequently, the processor can identify a gesture based on the collected light signal reflected by the hand, the electrical signal, and the pressure signal.

[0132] It should be understood that the above side PPG sensor integrated with the ECG sensor and the pressure sensor as a module is only an example, and the side PPG sensor can also be integrated with other sensors or devices as a module, which is not specifically limited in the embodiments of the present application.

[0133] The side PPG sensor can also be replaced by other infrared sensors that can determine the motion characteristics of the user, or the side PPG sensor can also be replaced by ultrasonic sensors or radar sensors that can determine the motion characteristics of the user, etc., which is not specifically limited in the embodiments of the present application.

[0134] The above describes the possible forms and working principles of the first electronic device. The first electronic device can control the function of the first electronic device itself through gesture recognition, and can also control the second electronic device. The second electronic device can be, for example, a television, a PC, a mobile phone, a tablet computer, a VR terminal device, an AR terminal device, a wireless terminal in industrial control, a wireless terminal in unmanned driving, a wireless terminal in remote medical treatment, a wireless terminal in smart power grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, a wearable device, or a vehicle-mounted device, etc. The embodiments of the present application do not specially limit the specific form of the second electronic device.

[0135] The use scenario of the first electronic device is described below with reference to FIG. 1.

[0136] As shown in (a) of FIG. 1, the first electronic device can control the first electronic device through gesture recognition. For example, when the user wears the first electronic device, the user can make a gesture action, and the side PPG sensor in the first electronic device can perform gesture recognition based on the above principle to obtain a gesture recognition result. The first electronic device can store a correspondence between the gesture recognition result and a first operation instruction. The first operation instruction can be an instruction for controlling the quality of the first electronic device. The first electronic device can control the first electronic device based on the first operation instruction corresponding to the gesture recognition result. The correspondence between the gesture recognition result and the first operation instruction will be described in detail in subsequent embodiments, and will not be described here.

[0137] As shown in (b) of FIG. 1, the first electronic device can control the second electronic device through gesture recognition. For example, the side PPG sensor in the first electronic device can perform gesture recognition based on the above principle to obtain a gesture recognition result. The first electronic device can store a correspondence between the gesture recognition result and a second operation instruction. The second operation instruction can be an instruction for controlling the second electronic device. When the operation instruction corresponding to the gesture recognition result is the second operation instruction for the second electronic device, the first electronic device can control the second electronic device based on the second operation instruction corresponding to the gesture recognition result. The first electronic device and the second electronic device can communicate in any manner.

[0138] Referring to FIG. 3, a hardware structure diagram of the first electronic device is provided. The method in the following embodiments can be implemented in the first electronic device with the above hardware structure.

[0139] As shown in (a) of FIG. 3, the first electronic device can include a processor 110 and a side PPG sensor 101.

[0140] Optionally, the first electronic device can further include one or more of the accelerometer sensor 102, the bottom PPG sensor 103, or the display screen 104.

[0141] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the first electronic device. In other embodiments of the present application, the first electronic device can include more or fewer components than illustrated, or combine certain components, or split certain components, or different arrangement of components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0142] The side PPG sensor 101 is configured to collect a first signal. For example, the side PPG sensor 101 emits a light signal, and receives a reflected light signal of the light signal to obtain the first signal. The first signal can be used to identify a gesture of a user. The first signal can also be used to determine a physiological parameter of the user, such as heart rate, blood pressure, and / or blood oxygen saturation. The side PPG sensor 101 can also be configured to send the first signal to the processor 110.

[0143] The accelerometer sensor 102 is configured to collect a second signal. The second signal is acceleration information of the first electronic device in each direction (generally three-axis direction). The accelerometer sensor 102 is also configured to detect a gravity acceleration signal and determine direction information of the first electronic device, such as an orientation of the first electronic device or an orientation of a wrist of the user. The accelerometer sensor 102 can also be used for gesture recognition. For example, the accelerometer sensor 102 can detect a change in the acceleration signal of the first electronic device in the X-axis direction, and identify a gesture as horizontal left and right shaking of the arm. The accelerometer sensor 102 can also be configured to send the second signal to the processor 110.

[0144] The bottom PPG sensor 103 is configured to collect a third signal. The third signal is a reflected light signal of a light signal emitted by the bottom PPG sensor 103. The third signal can be used for gesture recognition, or the third signal can also be used to determine a physiological parameter of the user, such as heart rate, blood pressure, and / or blood oxygen saturation. The bottom PPG sensor 103 can also be configured to send the third signal to the processor 110. The bottom PPG sensor 103 is located at the bottom of the first electronic device and is a PPG sensor that is in contact with the skin of the user when worn.

[0145] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a controller, a memory, a digital signal processor (DSP), and / or a neural-network processing unit (NPU), etc. The NPU is a neural-network (NN) computing processor, which is derived from the structure of a biological neural network, for example, the transmission mode between human brain neurons, and can also constantly self-learn by quickly processing input information. Through the NPU, intelligent cognition applications of the first electronic device can be realized, such as gesture recognition, image recognition, face recognition, speech recognition, text understanding, etc. Different processing units can be independent devices or integrated in one or more processors.

[0146] In the embodiments of the present application, the processor 110 is configured to receive a first signal from the side PPG sensor 101 and identify a gesture based on the first signal. The processor 110 is also configured to receive the first signal from the side PPG sensor 101 and a second signal from the accelerometer sensor 102, and identify the gesture based on the first signal and the second signal. The processor 110 is also configured to receive the first signal from the side PPG sensor 101 and a third signal from the bottom PPG sensor 103, and identify the gesture based on the first signal and the third signal. The processor 110 is also configured to receive the first signal from the side PPG sensor 101, the second signal from the accelerometer sensor 102, and the third signal from the bottom PPG sensor 103, and identify the gesture based on the first signal, the second signal, and the third signal.

[0147] The controller can be the nerve center and command center of the first electronic device. The controller can generate operation control signals according to instruction operation codes and timing signals to complete the control of instruction fetching and instruction execution.

[0148] The processor 110 can also be provided with a memory for storing instructions and data.

[0149] The display screen 104 is configured to display interfaces, images, videos, and the like. The display screen 104 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, or the like.

[0150] It should be understood that the above collection of the second signal by the acceleration sensor is only an example, and the embodiments of the present application can also collect the second signal by other sensors such as a gyroscope sensor, and the like, and the embodiments of the present application do not make specific limitations thereto.

[0151] In an embodiment, as shown in (b) of FIG. 3, the first electronic device can include a processor 110 and a side PPG module 111.

[0152] Optionally, the first electronic device can further include one or more of an accelerometer module 112, a bottom PPG module 113, or a display screen 104.

[0153] The side PPG module 111 is configured to collect a first signal and process a waveform feature of the first signal. The waveform feature of the first signal is used to identify a user's posture and / or gesture and the like motion parameters. The waveform feature of the first signal is also used to determine a user's heart rate, blood pressure, and / or blood oxygen saturation and the like physiological parameters. The side PPG module 111 can also be configured to send the waveform feature of the first signal to the processor 110.

[0154] The accelerometer module 112 is configured to collect a second signal and obtain a waveform feature of the second signal. The accelerometer module 112 can also be configured to send the waveform feature of the second signal to the processor 110.

[0155] The bottom PPG module 113 is configured to collect a third signal and obtain a waveform feature of the third signal. The waveform feature of the third signal is used to determine a user's heart rate, blood pressure, and / or blood oxygen saturation and the like physiological parameters. The bottom PPG module 113 can also be configured to send the waveform feature of the third signal to the processor 110. The bottom PPG module 113 is located at the bottom of the first electronic device and is a PPG module that is in contact with the user's skin when worn.

[0156] The processor 110 is configured to receive a waveform feature of a first signal from the side PPG module 111, and identify a gesture based on the waveform feature of the first signal. The processor 110 is also configured to receive a waveform feature of a second signal from the accelerometer module 112, and identify a gesture based on the waveform feature of the first signal and the waveform feature of the second signal. The processor 110 is also configured to receive a waveform feature of a third signal from the bottom PPG module 113, and identify a gesture based on the waveform feature of the first signal, the waveform feature of the second signal, and the waveform feature of the third signal.

[0157] The display screen 104 is configured to display interfaces, images, videos, and the like.

[0158] Based on FIG. 3(a) and FIG. 3(b), the first electronic device can further include one or more of a pressure sensor, a gyroscope sensor, a proximity light sensor, a touch sensor, or a key, and the like (not shown in the figures).

[0159] The pressure sensor is configured to sense a pressure signal, convert the pressure signal into an electrical signal, and send the electrical signal to the processor 110. In some embodiments, the pressure sensor can be disposed on the display screen 104. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, and the like. A capacitive pressure sensor can include at least two parallel plates made of conductive material. When a force is applied to the pressure sensor, the capacitance between the electrodes changes. The first electronic device determines the size of the pressure according to the change in capacitance. When a touch operation is applied to the display screen 104, the first electronic device detects the size of the touch operation according to the pressure sensor. The first electronic device can also calculate the position of the touch according to the detection signal of the pressure sensor.

[0160] The gyroscope sensor can be used to determine the angular velocity of the first electronic device around three axes (i.e., x, y, and z axes) to determine the motion posture of the first electronic device. In some embodiments, the gyroscope sensor can be used for gesture recognition. For example, the gyroscope sensor detects the rotation and inclination of the first electronic device by measuring the angular velocity of the first electronic device, thereby helping to determine the posture and motion state of the hand wearing the first electronic device in three-dimensional space. The gyroscope sensor can also be used to determine the direction information of the first electronic device. In the embodiments of the present application, the combination of the gyroscope sensor and the accelerometer sensor 102 can further improve the functionality and user experience of the first electronic device.

[0161] The key includes a power-on key and the like. The key can be a mechanical key. It can also be a touch key. The first electronic device can receive a key input, generate a key signal input related to the user settings and function control of the first electronic device.

[0162] In an embodiment, after the user makes a gesture, the first electronic device can collect a first signal by the side PPG sensor, and then obtain a waveform feature of the first signal. The first electronic device can compare the waveform feature of the first signal with waveform features of first signals of gestures in a preset gesture library, to identify the gesture.

[0163] As shown in FIG. 4, it is a flowchart of a gesture recognition method provided by an embodiment of the present application. Referring to FIG. 4, the flowchart includes S101-S107:

[0164] S101, the side PPG sensor in the first electronic device collects a first signal.

[0165] For example, as shown in FIG. 5, the side PPG sensor includes an LED and a PD. The LED in the side PPG sensor can emit a light signal, and the light signal is reflected by the user's hand to form the first signal. The PD in the side PPG sensor can collect the first signal.

[0166] In an example, the side PPG sensor can continuously emit a light signal and collect a first signal. Alternatively, when the gesture recognition function of the first electronic device is turned on, the side PPG sensor starts to emit a light signal and collect a first signal. For specific description, please refer to S500-S504, S601-S604 and S701-S705 below, which will not be described here.

[0167] Optionally, when the user makes a gesture, the size of the first signal collected by the side PPG sensor also changes.

[0168] For example, when the user makes a gesture of turning up the palm, the size of the first signal collected changes over time, first increases and then decreases, and the waveform of the first signal appears a peak. As shown in FIG. 6, it is a schematic diagram of the gesture of turning up the palm of the user; FIG. 7A(a) is a waveform diagram of the corresponding first signal when the user makes the gesture, wherein the abscissa of the waveform diagram is time, and the ordinate is the size of the first signal. FIG. 7A(b) is a waveform diagram of the gradient of the waveform of the first signal when the user makes the gesture, wherein the abscissa of the waveform diagram is time, and the ordinate is the gradient of the waveform of the first signal.

[0169] Optionally, the unit of the abscissa in the above waveform diagram can be second s, millisecond ms or nanosecond ns; the unit of the ordinate can be millivolt mV, microvolt μV, etc. The unit of the abscissa in the waveform gradient change diagram can be second s, millisecond ms or nanosecond ns; the unit of the ordinate can be millivolt mV / ns or microvolt μV / ns. The embodiments of the present application do not make specific limitations in this regard.

[0170] In an embodiment, the waveform ordinate of the first signal of (a) of FIG. 7A can also be the amplitude of the first signal, and the embodiments of the present application do not make specific limitations thereto.

[0171] When the user's palm is kept horizontal, as shown in (a) of FIG. 6. At this time, the user's hand is away from the side PPG sensor of the first electronic device, and the reflection area of the light signal by the hand is small. The waveform of the first signal can be shown in region a of (a) of FIG. 7A, the first signal collected by the side PPG sensor is small, for example, the first signal is less than the first signal threshold. The waveform gradient of the first signal can be shown in region d of (b) of FIG. 7A, the waveform of the first signal is unchanged, and the waveform gradient of the first signal is 0. Alternatively, the waveform of the first signal changes little, and the waveform gradient of the first signal is less than the first signal gradient threshold and greater than the second signal gradient threshold. The first signal threshold, the first signal gradient threshold and the second signal gradient threshold can be customized by the user or by the developer before the first electronic device is shipped, and the embodiments of the present application do not make specific limitations thereto.

[0172] When the user's palm is kept horizontal, as shown in (a) of FIG. 6. At this time, the user's hand is away from the side PPG sensor of the first electronic device, and the reflection area of the light signal by the hand is small. The waveform of the first signal can be shown in region a of (a) of FIG. 7A, the first signal collected by the side PPG sensor is small, for example, the first signal is less than the first signal threshold. The waveform gradient of the first signal can be shown in region d of (b) of FIG. 7A, the waveform of the first signal is unchanged, and the waveform gradient of the first signal is 0. Alternatively, the waveform of the first signal changes little, and the waveform gradient of the first signal is less than the first signal gradient threshold and greater than the second signal gradient threshold. The first signal threshold, the first signal gradient threshold and the second signal gradient threshold can be customized by the user or by the developer before the first electronic device is shipped, and the embodiments of the present application do not make specific limitations thereto.

[0173] When the user's palm is kept horizontal, as shown in (a) of FIG. 6. At this time, the user's hand is away from the side PPG sensor of the first electronic device, and the reflection area of the light signal by the hand is small. The waveform of the first signal can be shown in region a of (a) of FIG. 7A, the first signal collected by the side PPG sensor is small, for example, the first signal is less than the first signal threshold. The waveform gradient of the first signal can be shown in region d of (b) of FIG. 7A, the waveform of the first signal is unchanged, and the waveform gradient of the first signal is 0. Alternatively, the waveform of the first signal changes little, and the waveform gradient of the first signal is less than the first signal gradient threshold and greater than the second signal gradient threshold. The first signal threshold, the first signal gradient threshold and the second signal gradient threshold can be customized by the user or by the developer before the first electronic device is shipped, and the embodiments of the present application do not make specific limitations thereto.

[0174] The first signal gradient threshold and the second signal gradient threshold can be opposite numbers of each other. The embodiments of the present application do not make specific limitations thereto.

[0175] The peak value of the first signal is greater than the first signal threshold. The waveform gradient of the first signal also changes in positive and negative directions, wherein the peak value of the first signal gradient is greater than the first signal gradient threshold, and the trough value of the first signal gradient is less than the second signal gradient threshold.

[0176] Optionally, when the user continuously turns the palm upward, the waveform diagram of the first signal can be as shown in FIG. 7B (a), the waveform of the first signal can continuously appear multiple wave peaks, wherein each wave peak value is greater than the first signal threshold value, the region a is a region after the end of the previous waveform and before the start of the next waveform, and the time of the region a is less than the first time threshold value. The waveform diagram of the waveform gradient of the first signal can be as shown in FIG. 7B (b), the waveform gradient of the first signal also appears multiple positive and negative changes, wherein each wave peak value of the first signal gradient is greater than the first signal gradient threshold value, and each wave trough value of the first signal gradient is less than the second gradient signal threshold value. The region d is a region after the end of the previous gradient waveform and before the start of the next gradient waveform.

[0177] Optionally, the first time threshold value can be used to limit the duration of the region a. For example, in FIG. 7B (a), if the time of the region a is less than the first time threshold value, the waveforms on both sides of the region a can be valid waveforms, and the subsequent related calculation of the waveform gradient can be performed. If in some cases, the time of the region a is greater than the first time threshold value (not shown in the figure), the waveforms on both sides of the region a can be invalid waveforms, and the invalid waveforms can be removed when the related calculation of the waveform gradient is performed. The first time threshold value can be customized by the user or by the developer before the first electronic device is shipped, and the embodiments of the present application do not make specific limitations thereto.

[0178] Further exemplarily, when the user wears the first electronic device to make a downward palm gesture, the size of the first signal collected by the side PPG sensor changes with time, first decreases and then increases. The first signal appears a wave trough. As shown in FIG. 8, it is a schematic diagram of the user's downward palm gesture; FIG. 9A (a) is a waveform diagram of the first signal corresponding to the gesture made by the user, wherein the abscissa of the waveform diagram is time, and the ordinate is the size of the first signal. FIG. 9A (b) is a waveform diagram of the waveform gradient of the first signal corresponding to the gesture made by the user, wherein the abscissa of the waveform diagram is time, and the ordinate is the waveform gradient of the first signal.

[0179] Optionally, the unit of the abscissa in the above waveform diagram can be second s, millisecond ms or nanosecond ns; the unit of the ordinate can be millivolt mV, microvolt μV, etc. The unit of the abscissa in the waveform gradient change diagram can be second s, millisecond ms or nanosecond ns; the unit of the ordinate can be millivolt mV / ns or microvolt μV / ns. The embodiments of the present application do not make specific limitations thereto.

[0180] In an embodiment, the ordinate of the waveform of the first signal in FIG. 8 (a) can also be the amplitude of the first signal, and the embodiments of the present application do not make specific limitations thereto.

[0181] When the user's palm is kept horizontal, as shown in (a) of FIG. 8. At this time, the user's hand is away from the side PPG sensor of the first electronic device, and the reflection area of the light signal is small. The waveform of the first signal can be shown as area a' in (a) of FIG. 9A, the first signal collected by the side PPG sensor is small, for example, the first signal is less than the second signal threshold. The waveform gradient of the first signal can be shown as area d' in (b) of FIG. 9A, the waveform of the first signal is unchanged, and the waveform gradient of the first signal is 0. Or, the waveform of the first signal changes little, and the waveform gradient of the first signal is greater than the third signal gradient threshold and less than the fourth signal gradient threshold. Wherein, the second signal threshold, the third signal gradient threshold and the fourth signal gradient threshold can be customized by the user, or can be customized by the developer before the first electronic device is shipped, and the first signal threshold and the second signal threshold can be the same or different; the third signal gradient threshold and the second signal gradient threshold can be the same or different, the fourth signal gradient threshold and the first signal gradient threshold can be the same or different, and the third signal gradient threshold and the fourth signal gradient threshold can be opposite numbers, and the embodiments of the present application do not make specific limitations.

[0182] When the user's palm is kept horizontal, as shown in (a) of FIG. 8. At this time, the user's hand is away from the side PPG sensor of the first electronic device, and the reflection area of the light signal is small. The waveform of the first signal can be shown as area a' in (a) of FIG. 9A, the first signal collected by the side PPG sensor is small, for example, the first signal is less than the second signal threshold. The waveform gradient of the first signal can be shown as area d' in (b) of FIG. 9A, the waveform of the first signal is unchanged, and the waveform gradient of the first signal is 0. Or, the waveform of the first signal changes little, and the waveform gradient of the first signal is greater than the third signal gradient threshold and less than the fourth signal gradient threshold. Wherein, the second signal threshold, the third signal gradient threshold and the fourth signal gradient threshold can be customized by the user, or can be customized by the developer before the first electronic device is shipped, and the first signal threshold and the second signal threshold can be the same or different; the third signal gradient threshold and the second signal gradient threshold can be the same or different, the fourth signal gradient threshold and the first signal gradient threshold can be the same or different, and the third signal gradient threshold and the fourth signal gradient threshold can be opposite numbers, and the embodiments of the present application do not make specific limitations.

[0183] When the user's palm is kept horizontal, as shown in (a) of FIG. 8. At this time, the user's hand is away from the side PPG sensor of the first electronic device, and the reflection area of the light signal is small. The waveform of the first signal can be shown as area a' in (a) of FIG. 9A, the first signal collected by the side PPG sensor is small, for example, the first signal is less than the second signal threshold. The waveform gradient of the first signal can be shown as area d' in (b) of FIG. 9A, the waveform of the first signal is unchanged, and the waveform gradient of the first signal is 0. Or, the waveform of the first signal changes little, and the waveform gradient of the first signal is greater than the third signal gradient threshold and less than the fourth signal gradient threshold. Wherein, the second signal threshold, the third signal gradient threshold and the fourth signal gradient threshold can be customized by the user, or can be customized by the developer before the first electronic device is shipped, and the first signal threshold and the second signal threshold can be the same or different; the third signal gradient threshold and the second signal gradient threshold can be the same or different, the fourth signal gradient threshold and the first signal gradient threshold can be the same or different, and the third signal gradient threshold and the fourth signal gradient threshold can be opposite numbers, and the embodiments of the present application do not make specific limitations.

[0184] Wherein, the first signal trough value is less than the second signal threshold. The waveform gradient of the first signal also changes positively and negatively, wherein the first signal gradient peak value is greater than the fourth signal gradient threshold, and the first signal gradient trough value is less than the third signal gradient threshold.

[0185] Optionally, when the user continuously flips the palm downward, the waveform schematic diagram of the first signal can be as shown in FIG. 9B (a), the waveform of the first signal can continuously appear a plurality of wave troughs, wherein the wave trough value of each wave trough is less than the second signal threshold value, the region a' is a region before the next waveform starts after the previous waveform ends, and the time of the region a' is less than the second time threshold value. The waveform schematic diagram of the waveform gradient of the first signal can be as shown in FIG. 9B (b), the waveform gradient of the first signal also appears a plurality of positive and negative changes, wherein each wave peak value of the first signal gradient is greater than the fourth signal gradient threshold value, and each wave trough value of the first signal gradient is less than the third signal gradient threshold value, and the region d' is a region before the next gradient waveform starts after the previous gradient waveform ends.

[0186] Optionally, the second time threshold value can be the same as or different from the first time threshold value, and the second time threshold value has a similar function as the first time threshold value, the difference is that the second time threshold value is used to limit the effective and ineffective of the waveforms on both sides of the region a', and details are not described herein. The second time threshold value can be customized by the user or by the developer before the first electronic device is shipped, and embodiments of the present application do not make specific limitations.

[0187] Further exemplarily, the user wearing the first electronic device can also make a gesture of flipping the palm upward and downward, and the waveform of the corresponding first signal appears a wave peak and a wave trough. Or the user wearing the first electronic device can also make a gesture of continuously flipping the palm upward and downward, and the waveform of the corresponding first signal appears a wave peak and a wave trough alternately.

[0188] It should be understood that the above gestures of flipping the palm upward and downward by the user are only one example, and the user can also make other gestures, and the side PPG sensor collects the corresponding first signal, and embodiments of the present application do not make specific limitations.

[0189] S102, the side PPG sensor sends the first signal to the processor.

[0190] Optionally, the side PPG sensor can collect the first signal in real time and send the first signal to the processor in real time.

[0191] S103, the processor obtains the waveform feature of the first signal.

[0192] Optionally, the processor can obtain the waveform of the first signal according to the first signal collected in real time.

[0193] As shown in FIG. 10(a), a waveform diagram of the first signal is shown. The horizontal axis of the waveform diagram represents time, and the vertical axis represents the size of the first signal. In the waveform diagram, t1 represents the time at which the waveform of the first signal starts, and t2 represents the time at which the waveform of the first signal ends. h0 represents the minimum value of the first signal in the waveform, and h1 represents the maximum value of the first signal in the waveform. FIG. 10(b) is a waveform diagram of the gradient of the corresponding first signal, in which the horizontal axis of the waveform diagram represents time, and the vertical axis represents the gradient of the waveform of the first signal. In the waveform diagram, t1 represents the time at which the waveform of the first signal starts in FIG. 10(a), and t2 represents the time at which the waveform of the first signal ends in FIG. 10(a). h3 represents the minimum value of the first signal in the waveform of the gradient of the first signal, and h4 represents the maximum value of the first signal in the waveform of the gradient of the first signal.

[0194] Optionally, the processor can obtain the waveform feature of the first signal according to the waveform of the first signal.

[0195] As an example, the waveform feature of the first signal can be that the peak value of the waveform is h1, the peak width is (t2-t1), the number of peaks is 1, and the peak-to-peak value of the waveform is (h1-h0). The peak value of the gradient waveform is h4.

[0196] In an embodiment, the processor can obtain the waveform of the first signal according to the first signal collected in real time, and the waveform feature of the first signal can be the waveform feature of the peak with the maximum peak value in the waveform. As an example, the waveform feature of the first signal can be one or more of the following: the peak value of the peak with the maximum peak value in the waveform of the first signal, the peak width of the peak with the maximum peak value in the waveform of the first signal, the number of peaks, the peak-to-peak value of the waveform, the peak value of the gradient with the maximum gradient value in the gradient of the first signal, the peak width of the gradient with the maximum gradient value in the gradient of the first signal, and the like. The embodiments of the present application do not make specific limitations on this.

[0197] In S104, the processor identifies the gesture based on the waveform feature of the first signal.

[0198] The processor can determine whether to identify the gesture based on the waveform feature of the first signal.

[0199] As an example, the processor can determine whether to identify the gesture based on the total width of the waveform of the first signal and the peak-to-peak value of the waveform.

[0200] The total width of the waveform of the first signal includes the total width of the waveform corresponding to the first signal threshold value when at least one continuous peak is greater than the first signal threshold value; or the total width of the waveform of the first signal includes the total width of the waveform corresponding to the second signal threshold value when at least one continuous valley is less than the second signal threshold value.

[0201] When the total width of the waveform of the first signal is less than the first threshold value and the peak-to-peak value of the waveform is greater than the second threshold value, the processor identifies the gesture based on the waveform feature of the first signal. That is, when the user completes the gesture in a short time and the amplitude of the gesture is large, the first electronic device can identify the gesture, and thus, gesture recognition caused by user mis-touch can be prevented.

[0202] Optionally, the first threshold value and the second threshold value can be values defined according to experience or values calculated according to an algorithm, and embodiments of the present application do not make specific limitations thereto.

[0203] Optionally, the processor can further determine whether to identify the gesture according to the peak value of the waveform gradient of the first signal and the total width of the waveform gradient of the first signal.

[0204] The total width of the waveform gradient of the first signal includes the total width of the gradient waveform corresponding to the first signal gradient threshold value and / or the fourth signal gradient threshold value when at least one continuous peak is greater than the first signal gradient threshold value and / or the fourth signal gradient threshold value. Alternatively, the total width of the waveform of the first signal includes the total width of the gradient waveform corresponding to the second signal gradient threshold value and / or the third signal gradient threshold value when at least one continuous trough is less than the second signal gradient threshold value and / or the third signal gradient threshold value.

[0205] When the peak value of the waveform gradient of the first signal is greater than the third threshold value, it indicates that the speed of the rising of the peak of the first signal is fast, and the amplitude of the gesture is large. When the total width of the waveform gradient of the first signal is less than the fourth threshold value, it indicates that the peak width of the first signal is small, and the user completes the gesture in a short time, and thus, the first electronic device can identify the gesture, and thus, gesture recognition caused by user mis-touch can be prevented.

[0206] Optionally, the third threshold value and the fourth threshold value can be values defined according to experience or values calculated according to an algorithm, and embodiments of the present application do not make specific limitations thereto.

[0207] It should be understood that the above embodiments only take the total width of the first signal and the peak-to-peak value of the waveform of the first signal or the peak value of the waveform gradient of the first signal and the total width of the waveform gradient of the first signal as examples to determine whether to identify the gesture, and embodiments of the present application can also determine whether to identify the gesture through other waveform features of the first signal or other waveform features of the waveform gradient of the first signal, and embodiments of the present application do not make specific limitations thereto.

[0208] Optionally, the processor can compare the waveform feature of the first signal with the waveform feature of the first signal of a plurality of gestures in a preset gesture library, and if the waveform feature of the first signal matches the waveform feature of the first signal of one gesture in the preset gesture library, it is determined that the gesture is the gesture made by the user, and thus, the gesture is identified.

[0209] In an embodiment, the processor can identify the gesture in a manner of a neural network model. For example, the gesture matching the waveform feature of the first signal is determined in a manner of a random forest, so as to identify the gesture.

[0210] For example, the preset gesture library can include the waveform feature of the first signal of gesture A (turning up the palm) and gesture B (turning down the palm). The waveform feature of the first signal of gesture A is as follows: the peak value of the waveform is h1, the width of the peak is (t2-t1), the number of the peak is 1, the peak-to-peak value of the waveform is (h1-h0), and the peak value of the gradient waveform is h4. The waveform feature of the first signal of gesture B is as follows: the trough value of the waveform is h5, the width of the trough is (t7-t6), the number of the trough is 1, the peak-to-peak value of the waveform is (h6-h5), and the peak value of the gradient waveform is h8.

[0211] Then, the processor can determine, according to the random forest, that the waveform feature of the first signal of the gesture made by the user matches the waveform feature of the first signal of gesture A, and determine that the gesture made by the user is gesture A. Embodiments of the present application take gesture A as an example to introduce.

[0212] Optionally, the waveform feature of the first signal of the gesture made by the user matching the waveform feature of the first signal of gesture A can be that the waveform feature of the first signal of the gesture made by the user is the same as the waveform feature of the first signal of gesture A.

[0213] Alternatively, the waveform feature of the first signal of the gesture made by the user matching the waveform feature of the first signal of gesture A can also be that the waveform feature of the first signal of the gesture made by the user is similar to the waveform feature of the first signal of gesture A, and embodiments of the present application do not make specific limitations thereon.

[0214] Wherein, when the waveform feature of the first signal of the gesture made by the user meets certain conditions compared with the waveform feature of the first signal of gesture A, the first electronic device can determine that the waveform feature of the first signal of the gesture made by the user is similar to the waveform feature of the first signal of gesture A, identify the gesture made by the user as gesture A, and the condition can be that the absolute value of the difference between the waveform feature of the first signal of the gesture made by the user and the corresponding waveform feature in the first signal of gesture A is less than the corresponding error threshold.

[0215] Specifically, the condition can include one or more of the following: an absolute value of a difference between a waveform peak value of the first signal of the gesture made by the user and a waveform peak value of the first signal of gesture A is less than a first error threshold; an absolute value of a difference between a waveform width of the first signal of the gesture made by the user and a waveform width of the first signal of gesture A is less than a second error threshold; an absolute value of a difference between a number of wave peaks of the first signal of the gesture made by the user and a number of wave peaks of the first signal of gesture A is less than a third error threshold; an absolute value of a difference between a peak-to-peak value of the waveform of the first signal of the gesture made by the user and a peak-to-peak value of the waveform of the first signal of gesture A is less than a fourth error threshold, or an absolute value of a difference between a wave peak value of a waveform gradient of the first signal of the gesture made by the user and a wave peak value of the waveform gradient of the first signal of gesture A is less than a fifth error threshold.

[0216] The first error threshold, the second error threshold, the third error threshold, the fourth error threshold, and the fifth error threshold can be customized by the user or by a developer before the first electronic device is shipped, and embodiments of the present application do not make specific limitations thereto.

[0217] For example, if the waveform feature of the first signal of the gesture made by the user is that the waveform peak value is h1, the wave peak width is (t2-t1), the number of wave peaks is 1, and the peak-to-peak value of the waveform is (h1-h0), and the wave peak value of the gradient waveform is h4.

[0218] The waveform feature of the first signal of gesture A is that the waveform peak value is h1', the wave peak width is (t2-t1), the number of wave peaks is 1, the peak-to-peak value of the waveform is (h1-h0), and the wave peak value of the gradient waveform is h4.

[0219] The waveform feature of the first signal of the gesture made by the user is different from the waveform feature of the first signal of gesture A in the waveform peak value, and the same in other waveform features. When the absolute value of the difference between the waveform peak value of the first signal of the gesture made by the user and the waveform peak value of the first signal of gesture A is less than the first error threshold, it is determined that the waveform feature of the first signal of the gesture made by the user is similar to the waveform feature of the first signal of gesture A, and the gesture made by the user is recognized as gesture A.

[0220] It should be understood that the above waveform feature of the first signal of the gesture made by the user and the waveform feature of the first signal of gesture A, the first threshold, the second threshold, and the first error threshold are only an example, and the similarity between the waveform feature of the first signal of the gesture made by the user and the waveform feature of the first signal of gesture A can also be determined in other manners, and embodiments of the present application do not make specific limitations thereto.

[0221] It should be noted that the processor can identify other gestures in the preset gesture library in a manner similar to the process of identifying gesture A, and details are not repeated. For example, the processor can identify gesture B, gesture C, or gesture D in a manner similar to the process of identifying gesture A. In the process of identifying gesture B, gesture C, or gesture D, the waveform features of gesture B, gesture C, or gesture D are compared with the waveform features of the first signal collected by the first electronic device.

[0222] S105, the processor determines an operation instruction corresponding to the gesture.

[0223] In an embodiment of the present application, the operation instruction can be an operation instruction for controlling the first electronic device, or an operation instruction for controlling the second electronic device. For example, the processor stores a correspondence between a gesture recognition result and an operation instruction. Different gestures can correspond to different operation instructions.

[0224] For example, the processor can store a correspondence between a first gesture recognition result and a first operation instruction. The first operation instruction is an instruction for controlling the first electronic device. For example, the first gesture recognition result can include gesture C, and the first operation instruction corresponding to gesture C can include "return to the previous interface". Alternatively, the first gesture recognition result can include gesture D, and the first operation instruction corresponding to gesture D can include "take a photo", etc. The specific implementation of the first operation instruction to control the first electronic device can refer to the description of S108 (not shown in the figure), and details are not repeated here.

[0225] For another example, the processor can store a correspondence between a second gesture recognition result and a second operation instruction. The second operation instruction is an instruction for controlling the second electronic device. For example, when the second electronic device is a smart air conditioner, the second gesture recognition result can include gesture A, and the second operation instruction corresponding to gesture A can include "blow upward". Alternatively, the second gesture recognition result can include gesture B, and the second operation instruction corresponding to gesture B can include "blow downward". The specific implementation of the second operation instruction to control the second electronic device can refer to the description of S106-S107, and details are not repeated here.

[0226] It can be understood that the specific form of the second electronic device and the specific control content of the second operation instruction can be set in combination with the actual application scenario, and the embodiments of the present application are not limited. For example, the second electronic device can also be any smart home such as a sound box, a television, a refrigerator, or a smart toilet, or can be a smart phone or an autonomous vehicle, and the second operation instruction can be any operation instruction for controlling the second electronic device.

[0227] S106, the processor sends the second operation instruction to the second electronic device.

[0228] Optionally, the processor can send the second operation instruction corresponding to the gesture A to the second electronic device through star flash, Bluetooth (BT) (for example, traditional Bluetooth or Bluetooth low energy (BLE)), wireless local area networks (WLAN) (for example, a wireless fidelity (Wi-Fi) network), near field communication (NFC), frequency modulation (FM), infrared (IR), or the like. For example, the processor can send the second operation instruction "blowing upwards" corresponding to the gesture A to the second electronic device through Bluetooth.

[0229] S107, the second electronic device executes the second operation instruction.

[0230] Optionally, after receiving the second operation instruction from the first electronic device, the second electronic device can execute the second operation instruction. For example, in response to the second operation instruction "blowing upwards", the second electronic device can execute blowing upwards.

[0231] S108, the first electronic device executes the first operation instruction.

[0232] After obtaining the first operation instruction, the first electronic device can execute the first operation instruction. For example, the first operation instruction is "returning to the previous interface", and in response to the first operation instruction "returning to the previous interface", the first electronic device returns to the previous interface. For another example, the first operation instruction is "page turning", and in response to the first operation instruction "page turning", the display interface in the first electronic device turns the page. For another example, the first operation instruction is "playing music", and in response to the first operation instruction "playing music", the first electronic device plays music.

[0233] In this way, in the above S106, S107, and S108, the "turning-up palm" gesture corresponds to the second operation instruction "blowing upwards"; the "turning-down palm" gesture corresponds to the second operation instruction "blowing downwards"; or the "turning-up palm" gesture corresponds to the first operation instruction "returning to the previous interface"; or the "turning-up palm" gesture corresponds to the first operation instruction "page turning"; or the "turning-up palm" gesture corresponds to the first operation instruction "playing music", which improves the pointing nature of the gesture action, makes it easier for users to understand the function corresponding to the gesture, reduces the cost of users learning the gesture, and improves the ease of use of the gesture and the first electronic device and the ease of use of the gesture and the second electronic device (such as a smart air conditioner, a mobile phone, a smart screen, or an extended reality (XR) device).

[0234] And, the above embodiment adopts the side PPG sensor installed on the side of the first electronic device close to the user's hand when the user wears the first electronic device, and identifies the gesture based on the first signal, which can reduce the interference of external factors such as the wearing position, wearing tightness of the first electronic device, or the wrist characteristics of the user, and improve the accuracy of gesture recognition.

[0235] As in the above embodiment corresponding to FIG. 4, the example illustrates the process of gesture recognition of the first electronic device through the side PPG sensor. In some embodiments, the side PPG sensor in the embodiment corresponding to FIG. 4 can be replaced by a side PPG module. For example, FIG. 11 illustrates the process of gesture recognition of the first electronic device through the side PPG module. In other embodiments, on the basis of the embodiment corresponding to FIG. 4, a second sensor can also be added to the first electronic device. For example, FIG. 12 illustrates the process of gesture recognition of the first electronic device through the side PPG sensor and the second sensor. The second sensor can be a sensor for detecting the motion state of the first electronic device, for example, the second sensor can include an ACC sensor and / or a gyroscope sensor, etc., and is specifically used to collect acceleration information of the user's hand. The embodiments of the present application do not make specific limitations in this regard.

[0236] Referring to FIG. 11, the flow includes S201-S203 and S104-S107, or the flow includes S201-S203 and S104-S105, S108: S201, the side PPG module in the first electronic device collects the first signal.

[0237] Optionally, the side PPG module can integrate the side PPG sensor and the signal processing unit. The side PPG sensor in the side PPG module can collect the first signal, and the signal processing unit can obtain the waveform feature of the first signal based on the first signal.

[0238] The specific process in which the side PPG sensor collects the first signal can be referred to S101 above.

[0239] S202, the side PPG module obtains the waveform feature of the first signal.

[0240] Optionally, the signal processing unit is integrated in the side PPG module. The signal processing unit can obtain the waveform of the first signal according to the first signal collected in real time, and obtain the waveform feature of the first signal based on the waveform of the first signal.

[0241] S203, the side PPG module sends the waveform feature of the first signal to the processor.

[0242] Optionally, S104-S107 or S104-S105, S108 can be referred to the above, which will not be repeated here.

[0243] In this way, the waveform characteristics of the first signal are obtained by using the side PPG module, the burden of the processor of the first electronic device is reduced, the workloads of various components of the first electronic device are optimized, and the side PPG module is in a modular form, so that the development of a system or a product is more flexible and efficient.

[0244] Referring to FIG. 12, the flow includes S301-S304, S101-S103, and S105-S107, or includes S301-S304, S101-S103, S105, and S108:

[0245] Optionally, S101-S103 can refer to the above description and will not be repeated here.

[0246] S301, the second sensor in the first electronic device collects a second signal.

[0247] The second signal can be a signal for reflecting the motion of the first electronic device. For example, the second sensor is used to collect acceleration information of a user's hand, and the second signal can be an acceleration signal of a wrist wearing the first electronic device in X-Y-Z directions. For example, the second signal can include X signal, Y signal, and Z signal.

[0248] When the wrist of the user wearing the first electronic device moves, the acceleration of the first electronic device changes in X-Y-Z directions, and the sizes of the X signal, the Y signal, and the Z signal collected by the second sensor also change.

[0249] S302, the second sensor sends the second signal to the processor.

[0250] Optionally, the second sensor can collect the second signal in real time and send the second signal to the processor in real time.

[0251] S303, the processor obtains the waveform characteristics of the second signal.

[0252] Optionally, the processor can obtain the waveform of the second signal according to the second signal collected in real time.

[0253] For example, the waveform diagram of the second signal can be as shown in FIG. 13(a). In the waveform diagram, the abscissa represents time, and the ordinate represents the size of the second signal. Optionally, the unit of the abscissa in the above waveform diagram of the second signal can be second s, millisecond ms, or nanosecond ns; the unit of the ordinate can be m / s 2 , m / ms 2 or m / ns 2 , and the embodiments of the present application do not make specific limitations.

[0254] Optionally, the processor obtains a waveform feature of the second signal according to the waveform of the second signal.

[0255] For example, the waveform feature of the second signal can include:

[0256] The waveform feature of the X signal is that the waveform peak value is hx1, the number of wave peaks is 4, and the peak-to-peak value of the waveform is (hx1-hx0); the waveform feature of the Y signal is that the waveform peak value is hy1, the number of wave peaks is 4, and the peak-to-peak value of the waveform is (hy1-hy0); and the waveform feature of the Z signal is that the waveform peak value is hz1, the number of wave peaks is 2, and the peak-to-peak value of the waveform is (hz1-hz0).

[0257] For another example, the waveform diagram of the second signal can be as shown in Figure 13(b). In the waveform diagram, the abscissa represents time, and the ordinate represents the size of the second signal. Optionally, the unit of the abscissa in the above-mentioned waveform diagram of the second signal can be second (s), millisecond (ms) or nanosecond (ns); the unit of the ordinate can be m / s, m / ms or m / ns, which is not limited in the embodiments of the present application. 2 2 2

[0258] The waveform feature of the second signal can include:

[0259] The waveform feature of the X signal is that the waveform peak value is hx3, the number of wave peaks is 8, and the peak-to-peak value of the waveform is (hx3-hx2); the waveform feature of the Y signal is that the waveform peak value is hy3, the number of wave peaks is 6, and the peak-to-peak value of the waveform is (hy3-hy2); and the waveform feature of the Z signal is that the waveform peak value is hz3, the number of wave peaks is 3, and the peak-to-peak value of the waveform is (hz3-hz2).

[0260] Optionally, the sequence of S101-S103 and S301-S303 is not limited. For example, S101-S103 can be executed first and then S301-S303. Alternatively, S101 and S301 can be executed first, then S102 and S302, and then S103 and S303. Alternatively, S101-S103 and S301-S303 can be executed synchronously. The embodiments of the present application do not make specific limitations in this regard.

[0261] S304, the processor identifies the gesture based on the waveform feature of the first signal and the waveform feature of the second signal.

[0262] ​​​In the embodiments of the present application, the processor can take the recognition result of the waveform feature of the first signal and the recognition result of the waveform feature of the second signal as the basis for recognizing the gesture. It can be understood that the processor can recognize the gesture of a single hand, or can recognize the gesture of two hands. The following will be illustrated by taking case one and case two as examples:

[0263] Case one: the processor recognizes the gesture of a single hand. In the embodiments of the present application, the processor can determine whether the first electronic device is moved according to the size of the peak-to-peak value of the waveform in the waveform feature of the second signal, and the processor can recognize whether there is an effective gesture based on the waveform feature of the first signal, and then determine whether the gesture is the gesture of a single hand and what the gesture of the single hand is. For example, if the first signal satisfies the first feature and the second signal satisfies the second feature, it can be determined that the gesture made by the user is the gesture of a single hand, and the gesture of the single hand is recognized.

[0264] For example, the processor can determine whether the second sensor moves in the X-Y-Z direction based on the waveform feature of the second signal. For example, when the X signal fluctuates, such as the value of the X signal changes, the peak-to-peak value (hx1-hx0) of the waveform of the X signal is greater than the first preset threshold, which indicates that the second sensor moves in the X-axis direction; when the Y signal fluctuates, such as the value of the Y signal changes, the peak-to-peak value (hy1-hy0) of the waveform of the Y signal is greater than the first preset threshold, which indicates that the second sensor moves in the Y-axis direction; when the Z signal fluctuates, such as the value of the Z signal changes, the peak-to-peak value (hz1-hz0) of the waveform of the Z signal is greater than the first preset threshold, which indicates that the second sensor moves in the Z-axis direction.

[0265] In a possible implementation, the second signal satisfying the second feature can include that the waveform change of the second signal is greater than the first preset threshold. For example, the second signal satisfying the second feature can be that the peak-to-peak value of the waveform of the wrist of the user wearing the first electronic device in at least one of the X-axis direction, the Y-axis direction and the Z-axis direction is greater than the first preset threshold. For example, when the user makes a gesture, the second sensor detects that the peak-to-peak value of the waveform of the wrist of the user wearing the first electronic device in at least one of the X-axis direction, the Y-axis direction and the Z-axis direction is greater than the first preset threshold, and it can be determined that the user makes a gesture with a single hand (for example, the hand wearing the first electronic device).

[0266] For example, when the waveform of the second signal is as shown in FIG. 13(a), the peak-to-peak value of the waveform of the X signal is (hx1-hx0), (hx1-hx0) is greater than the first preset threshold; the peak-to-peak value of the waveform of the Y signal is (hy1-hy0), (hy1-hy0) is greater than the first preset threshold; the peak-to-peak value of the waveform of the Z signal is (hz1-hz0), (hz1-hz0) is greater than the first preset threshold. The second signal satisfies the second feature. The processor can determine that the second sensor moves in the X-Y-Z direction based on the waveform feature of the second signal. That is, the wrist wearing the first electronic device moves in the X-Y-Z direction.

[0267] The second signal satisfying the second feature described above is only an example. When the peak-to-peak value of the waveform of the second signal is greater than the first preset threshold for the waveform of the X signal, and the peak-to-peak value of the waveform of the Y signal and the peak-to-peak value of the waveform of the Z signal are less than the first preset threshold. At this time, the second signal also satisfies the second feature. Embodiments of the present application do not make specific limitations on this.

[0268] In another possible implementation, the second signal satisfying the second feature can also be that the waveform feature of the second signal matches the preset waveform feature of the second signal.

[0269] For example, when the waveform feature of the second signal matches the waveform feature of the second signal of a single-hand gesture in the preset gesture library, it can be determined that the user makes a single-hand gesture.

[0270] Further, it can be determined whether the first signal satisfies the first feature. The first signal satisfying the first feature can be that the waveform feature of the first signal matches the waveform feature of the first signal of a gesture in the preset gesture library. When the waveform feature of the first signal matches the waveform feature of the first signal of gesture A in the preset gesture library, gesture A can be recognized, and gesture A is a single-hand gesture.

[0271] In this way, the first signal satisfies the first feature, and the second signal satisfies the second feature. The processor can determine that the gesture made by the user is a single-hand gesture A.

[0272] It can be understood that the second signal satisfies the second feature, and the first signal satisfies different first features, which can correspond to different gestures. When the waveform feature of the first signal matches the waveform feature of the first signal of different gestures in the preset gesture library, different single-hand gestures are corresponded.

[0273] For example, when the second signal satisfies the second feature, the waveform feature of the first signal matches the waveform feature of the first signal of gesture M in the preset gesture library, which corresponds to single-hand gesture M; and when the waveform feature of the first signal matches the waveform feature of the first signal of gesture N in the preset gesture library, which corresponds to single-hand gesture N.

[0274] Case two: the processor identifies the gesture of two hands. In the embodiments of the present application, if the first signal satisfies the third feature and the second signal satisfies the fourth feature, it can be determined that the gesture made by the user is the gesture of two hands.

[0275] In a possible implementation, the fourth feature satisfied by the second signal can indicate that the waveform variation of the second signal is less than a first preset threshold.

[0276] The third feature satisfied by the first signal can be that the waveform feature of the first signal matches the waveform feature of the first signal of a gesture in the preset gesture library. The first feature and the third feature can be the same or different.

[0277] The embodiments of the present application take the case where the first feature and the third feature are different as an example for introduction. For example, the waveform feature of the first signal matches the waveform feature of the first signal of gesture C in the preset gesture library.

[0278] The fourth feature can be that the peak-to-peak values of the waveforms of the wrist of the user wearing the first electronic device in the X-axis direction, the Y-axis direction and the Z-axis direction are all less than or equal to a first preset threshold. For example, when the user makes a gesture, the ACC sensor detects that the peak-to-peak values of the waveforms of the wrist of the user wearing the first electronic device in the X-axis direction, the Y-axis direction and the Z-axis direction are all less than or equal to the first preset threshold, it can be determined that the user makes a gesture with two hands, the hand wearing the first electronic device is stationary, and the hand not wearing the first electronic device moves.

[0279] For example, when the waveform of the second signal is as shown in (b) of FIG. 13, the peak-to-peak value of the waveform of the X signal is (hx3-hx2), (hx3-hx2) is less than the first preset threshold; the peak-to-peak value of the waveform of the Y signal is (hy3-hy2), (hy3-hy2) is less than the first preset threshold; the peak-to-peak value of the waveform of the Z signal is (hz3-hz2), (hz3-hz2) is less than the first preset threshold. The second signal satisfies the fourth feature. The processor can determine that the second sensor moves less (or does not move) in the X-Y-Z direction based on the waveform feature of the second signal. That is, the wrist wearing the first electronic device hardly moves in the X-Y-Z direction.

[0280] In this way, the first signal satisfies the third feature and the second signal satisfies the fourth feature, and the processor can determine that the gesture made by the user is the gesture of two hands C.

[0281] For example, the gesture made by the user can be as shown in FIG. 14, the left hand of the user wears the first electronic device, and the right hand does not wear the first electronic device. The left hand of the user remains stationary, and the right hand approaches or moves away from the side PPG sensor of the first electronic device.

[0282] It can be understood that the second signal satisfies the fourth feature, and the first signal satisfies a different first feature, which can correspond to different gestures of the hands.

[0283] For example, the second signal satisfies the fourth feature, and the waveform feature of the first signal matches the waveform feature of the first signal of gesture P in the preset gesture library, which corresponds to gesture P of the hands; and the waveform feature of the first signal matches the waveform feature of the first signal of gesture Q in the preset gesture library, which corresponds to gesture Q of the hands.

[0284] In another possible implementation, the second signal satisfying the fourth feature can also be that the waveform feature of the second signal matches the waveform feature of another preset second signal.

[0285] For example, the waveform feature of the second signal matches the waveform feature of the second signal of the gesture of the hands in the preset gesture library, and it can be determined that the user makes the gesture of the hands.

[0286] For example, the waveform feature of the first signal matches the waveform feature of the first signal of gesture C in the preset gesture library; and the waveform feature of the second signal matches the waveform feature of another second signal in the preset gesture library, which corresponds to the gesture of the hands. It can be determined that the user makes gesture C, which is the gesture of the hands.

[0287] In an application scenario, when the first electronic device is wetted by water, the user's fingers are wet, or the user wears the first electronic device in water (such as swimming), the user can control the first electronic device through a gesture of a hand or gestures of the hands.

[0288] For example, when the user swims, the touch function of the first electronic device fails, and the user can control the first electronic device through gestures of the hands. For example, when danger is encountered while swimming, the first electronic device can be controlled to alarm through the gesture shown in FIG. 14. For example, the user can keep the hand wearing the first electronic device still, and the hand not wearing the first electronic device performs an action of approaching or moving away from the first electronic device, to control the first electronic device.

[0289] S105-S107 or S104-S105, S108 can refer to the foregoing, and will not be repeated here.

[0290] In this way, the first electronic device can not only recognize a gesture made by a hand of the user, but also recognize a gesture made by the hands of the user, expand the type of gestures, make gesture recognition more flexible, and improve the user experience.

[0291] As the corresponding embodiment of FIG. 12, a user wears a first electronic device, and gesture recognition is implemented by the side PPG sensor and the second sensor in the first electronic device to recognize whether the gesture is a single-hand gesture or a double-hand gesture. In some embodiments, the user can also wear the first electronic device and a third electronic device with one hand or both hands, and gesture recognition is implemented by the side PPG sensor and / or the second sensor in the first electronic device and the first PPG sensor and / or the fourth sensor in the third electronic device to recognize whether the gesture is a single-hand gesture or a double-hand gesture. The first electronic device and the third electronic device can communicate in any manner. The third electronic device can be a smart watch, a smart bracelet, a smart helmet, a smart ring, or a smart glasses, etc. for gesture recognition. The first PPG sensor in the third electronic device has similar structure and function to the side PPG sensor in the first electronic device, and can refer to the side PPG sensor in the first electronic device; or the first PPG sensor in the third electronic device has similar structure and function to the bottom PPG sensor in the first electronic device, and can refer to the bottom PPG sensor in the first electronic device, which is not described herein. The third electronic device can include one or more first PPG sensors, which are not limited in the embodiments of the present application. The fourth sensor in the third electronic device has similar structure and function to the second sensor in the first electronic device, and can refer to the second sensor in the first electronic device, which is not described herein.

[0292] For example, a user wears a first electronic device with the left hand and a third electronic device with the right hand. The first electronic device includes a side PPG sensor and a second sensor, and the third electronic device includes a fourth sensor.

[0293] Optionally, the first electronic device can send the information collected by the side PPG sensor and the information collected by the second sensor to the third electronic device, and the third electronic device can recognize whether the gesture made by the user is a single-hand gesture or a double-hand gesture based on the information collected by the side PPG sensor in the first electronic device, the information collected by the second sensor in the first electronic device, and the information collected by the fourth sensor in the third electronic device. Alternatively, the third electronic device can send the information collected by the fourth sensor to the first electronic device, and the first electronic device can recognize whether the gesture made by the user is a single-hand gesture or a double-hand gesture based on the information collected by the side PPG sensor in the first electronic device, the information collected by the second sensor in the first electronic device, and the information collected by the fourth sensor in the third electronic device.

[0294] Optionally, the first electronic device can send the gesture recognition result to the third electronic device, and the third electronic device determines whether the gesture made by the user is a single-hand gesture or a double-hand gesture based on the gesture recognition result of the first electronic device and the gesture recognition result of the third electronic device; or the third electronic device can send the gesture recognition result to the first electronic device, and the first electronic device determines whether the gesture made by the user is a single-hand gesture or a double-hand gesture based on the gesture recognition result of the first electronic device and the gesture recognition result of the third electronic device.

[0295] The gesture recognition result of the third electronic device can be used as an auxiliary to verify whether the gesture recognition result of the first electronic device is accurate. For example, the gesture recognition result of the first electronic device is the same as the gesture recognition result of the third electronic device, for example, both are single-hand gestures, and it can be determined that the gesture made by the user is a single-hand gesture. Or, the gesture recognition result of the first electronic device is more accurate, and the gesture made by the user is determined to be a single-hand gesture or a double-hand gesture based on the gesture recognition result of the first electronic device. It can be understood that the gesture recognition result of the first electronic device can also be used as an auxiliary to verify whether the gesture recognition result of the third electronic device is accurate. Or, the gesture recognition result of the third electronic device is more accurate, and the gesture made by the user is determined to be a single-hand gesture or a double-hand gesture based on the gesture recognition result of the third electronic device. The embodiments of the present application do not make specific limitations on this. It can be understood that the gesture recognition result of the first electronic device or the gesture recognition result of the third electronic device can also be used as an auxiliary based on the priority of the first electronic device and the third electronic device. The gesture recognition result of the electronic device with low priority is used as an auxiliary to verify whether the gesture recognition result of the electronic device with high priority is accurate. Optionally, the priority of the first electronic device and the third electronic device can be pre-set, and the embodiments of the present application do not make specific limitations on this.

[0296] It can be understood that the above embodiments take the user wearing the first electronic device on the left hand and the third electronic device on the right hand as an example. The first electronic device includes a side PPG sensor and a second sensor, and the third electronic device includes a fourth sensor. This is only an example, and the embodiments of the present application do not make specific limitations on whether the first electronic device or the third electronic device is worn on the left hand or the right hand and the number. The first electronic device can also include one or more of a side PPG sensor, a second sensor, or a bottom PPG sensor, and the third electronic device can also include one or more of a first PPG sensor and a fourth sensor. The embodiments of the present application do not make specific limitations on this.

[0297] Optionally, when the gesture recognition succeeds, an operation instruction corresponding to the gesture can be determined. If the operation instruction instructs the first electronic device to execute the operation instruction, the first electronic device executes the operation instruction; or the third electronic device sends the operation instruction to the first electronic device, and the first electronic device executes the operation instruction.

[0298] If the operation instruction instructs the second electronic device to execute the operation instruction, the first electronic device sends the operation instruction to the second electronic device, and the second electronic device executes the operation instruction; or the third electronic device sends the operation instruction to the second electronic device, and the second electronic device executes the operation instruction; or the third electronic device sends the operation instruction to the first electronic device, the first electronic device sends the operation instruction to the second electronic device, and the second electronic device executes the operation instruction.

[0299] As described above, when the first electronic device performs gesture recognition of single-hand gestures and double-hand gestures, it is illustrated that the specific gesture is recognized according to the hand posture. In a possible implementation, the first electronic device can perform gesture recognition in combination with the direction information of the first electronic device and the posture of the user's hand. For example, the first electronic device calculates the direction information of the first electronic device according to the value of the second signal, and performs gesture recognition in combination with the direction information of the first electronic device and the specific posture of the hand. Illustratively, the second sensor can be used to detect the direction information of the first electronic device, such as the orientation of the dial of the first electronic device. When the user makes a gesture, the processor can calculate the orientation of the dial of the first electronic device according to the value of the second signal.

[0300] In this case, the greater the average value of the signal in a certain direction in the second signal, the greater the angle of the first electronic device in the positive direction of the direction.

[0301] For example, the greater the average value of the Z signal compared to the average value of the X signal and the average value of the Y signal, the greater the angle of the first electronic device in the positive direction of the Z axis.

[0302] For example, if the average value of the Z signal is hz4, hz4 is greater than a first preset average value, the average value of the X signal and the average value of the Y signal are hz5, and hz5 is close to 0, it can be determined that the first electronic device is oriented upward. If the average value of the Z signal is hz6, hz6 is less than a second preset average value, the average value of the X signal and the average value of the Y signal are hz5, and hz5 is close to 0, it can be determined that the first electronic device is oriented downward. In this case, the first preset average value is greater than 0, and the second preset average value is less than 0.

[0303] Specifically, the processor can calculate the orientation of the dial of the first electronic device according to the average value of the X signal, the average value of the Y signal, and the average value of the Z signal in the second signal.

[0304] It should be understood that the above calculation of the dial orientation of the first electronic device based on the mean value of the X signal, the mean value of the Y signal and the mean value of the Z signal in the second signal is only an example, and the dial orientation of the first electronic device can also be calculated based on other values of the second signal, such as the maximum value of each direction signal in the second signal, and the like, which is not limited in the embodiments of the present application.

[0305] Then, the processor can determine different gestures based on the dial orientation of the first electronic device and the gesture of the user, so as to realize the recognition of multi-direction gestures.

[0306] For example, the user can customize gestures corresponding to both the dial orientation of the first electronic device and the hand posture, so that the gestures corresponding to both the dial orientation of the first electronic device and the hand posture are stored in the first electronic device, and the first electronic device can recognize specific gestures in combination with the dial orientation of the first electronic device and the hand posture in the future. Of course, the first electronic device can also have some default gestures corresponding to both the dial orientation of the first electronic device and the hand posture, which is not limited in the embodiments of the present application.

[0307] For example, when the dial is upward, the user's palm is turned outward, which indicates that the gesture is upward palm turning; and the user's palm is turned inward, which indicates that the gesture is downward palm turning. When the dial is downward, the user's palm is turned outward, which indicates that the gesture is downward palm turning; and the user's palm is turned inward, which indicates that the gesture is upward palm turning. When the dial is leftward, the user's palm is turned outward, which indicates that the gesture is leftward palm turning; and the user's palm is turned inward, which indicates that the gesture is rightward palm turning. When the dial is rightward, the user's palm is turned outward, which indicates that the gesture is rightward palm turning; and the user's palm is turned inward, which indicates that the gesture is leftward palm turning.

[0308] It should be understood that the above different gestures corresponding to different dial orientations of the first electronic device are only an example, which is not limited in the embodiments of the present application.

[0309] In another embodiment, the first electronic device can also detect the direction information of the first electronic device through a gyroscope sensor; or the first electronic device can also detect the direction information of the first electronic device through other sensors, which is not limited in the embodiments of the present application.

[0310] Optionally, the user can also customize other gestures, such as shaking left and right after turning the palm outward, and the like, which is not limited in the embodiments of the present application.

[0311] Optionally, the processor can determine multiple gestures in different directions based on the dial orientation of the first electronic device and the gesture of the user.

[0312] For example, when the processor determines that the first electronic device is face up, i.e. the back of the hand is upward, according to the waveform feature of the second signal. The first electronic device compares the waveform feature of the first signal with the waveform feature of the first signal when the user's palm is pronated or supinated, and can determine whether the user's palm is pronated or supinated. As shown in Fig. 15(a), when the user's palm is pronated, the user's gesture is recognized as upward palm. When the user's palm is supinated, the user's gesture is recognized as downward palm.

[0313] For another example, when the first electronic device determines that the first electronic device is face left, i.e. the back of the hand is left, according to the waveform feature of the second signal. The first electronic device compares the waveform feature of the first signal with the waveform feature of the first signal when the user's palm is pronated or supinated, and can determine whether the user's palm is pronated or supinated. As shown in Fig. 15(b), when the user's palm is pronated, the user's gesture is recognized as left palm. When the user's palm is supinated, the user's gesture is recognized as right palm.

[0314] For another example, when the first electronic device determines that the first electronic device is face down, i.e. the back of the hand is downward, according to the waveform feature of the second signal. The first electronic device compares the waveform feature of the first signal with the waveform feature of the first signal when the user's palm is pronated or supinated, and can determine whether the user's palm is pronated or supinated. As shown in Fig. 15(c), when the user's palm is pronated, the user's gesture is recognized as downward palm. When the user's palm is supinated, the user's gesture is recognized as upward palm.

[0315] For another example, when the first electronic device determines that the first electronic device is face right, i.e. the back of the hand is right, according to the waveform feature of the second signal. The first electronic device compares the waveform feature of the first signal with the waveform feature of the first signal when the user's palm is pronated or supinated, and can determine whether the user's palm is pronated or supinated. As shown in Fig. 15(d), when the user's palm is pronated, the user's gesture is recognized as right palm. When the user's palm is supinated, the user's gesture is recognized as left palm.

[0316] It should be noted that the above upward palm, downward palm, left palm and right palm can be relative to the user's up, down, left and right, which are defined for the purpose of clear distinction of gestures, and the first electronic device can not distinguish the directions, but can classify according to the waveform feature of the signal.

[0317] Of course, the first electronic device can also divide the space corresponding to the categories of upward turning palm, downward turning palm, left turning palm, and right turning palm. For example, for a single-hand gesture, the arm (such as the forearm) wearing the first electronic device divides the space. When the watch face is upward or downward, the space is divided into two spaces by the forearm, so when the palm turns upward in the upper space, the first electronic device can recognize the gesture as upward turning palm, where the palm turning upward in the upper space can correspond to the palm turning outward when the watch face is upward, or the palm turning inward when the watch face is downward. Similarly, when the palm turns downward in the lower space, the first electronic device can recognize the gesture as downward turning palm, where the palm turning downward in the lower space can correspond to the palm turning inward when the watch face is upward, or the palm turning outward when the watch face is downward. When the watch face is left or right, the space is divided into two spaces by the forearm, so when the palm turns left in the left space, the first electronic device can recognize the gesture as left turning palm, where the palm turning left in the left space can correspond to the palm turning outward when the watch face is left, and the palm turning inward when the watch face is right. Similarly, when the palm turns right in the right space, the first electronic device can recognize the gesture as right turning palm, where the palm turning right in the right space can correspond to the palm turning inward when the watch face is left, and the palm turning outward when the watch face is right.

[0318] Optionally, the above embodiments only exemplify the watch face orientations of the four directions of up, down, left, and right corresponding to the gestures, and the user can also customize other directions and watch face orientations at any angle and corresponding gestures as needed. The first electronic device can compare the waveform characteristics of the first signal with the waveform characteristics corresponding to the user-defined gestures after determining the watch face orientation according to the waveform characteristics of the second signal, and recognize the gesture. The embodiments of the present application do not make specific limitations in this regard.

[0319] In this way, the first electronic device can recognize multiple simple gestures in different directions according to the waveform characteristics of the first signal and the waveform characteristics of the second signal, expand the types of gestures, and also facilitate user learning of gestures.

[0320] As in the above embodiment corresponding to FIG. 12, the process of the first electronic device side PPG sensor and the second sensor implementing gesture recognition is exemplarily illustrated. In some embodiments, on the basis of the embodiment corresponding to FIG. 12, the first electronic device can also increase a bottom PPG sensor. Exemplarily, FIG. 17 shows the process of the first electronic device implementing gesture recognition through the side PPG sensor, the second sensor, and the bottom PPG sensor. The bottom PPG sensor collects a third signal, and the third signal is used to represent the physiological characteristics of the user's hand.

[0321] For example, after the user makes a gesture, the first electronic device can collect a first signal through the side PPG sensor, and the processor obtains a waveform feature of the first signal based on the first signal. The first electronic device can collect a second signal through the second sensor, and the processor obtains a waveform feature of the second signal based on the second signal. The first electronic device can also collect a third signal through the bottom PPG sensor, and the processor obtains a waveform feature of the third signal based on the second signal. The processor in the first electronic device can identify the gesture based on the waveform feature of the first signal, the waveform feature of the second signal, and the waveform feature of the third signal.

[0322] Optionally, when the user makes a gesture with a large amplitude (which can also be referred to as a gesture with strong directivity), such as a palm-up, palm-down, palm-left, or palm-right gesture, the bottom PPG sensor is greatly affected by external factors such as the tightness of wearing the first electronic device, the wearing position, and the wrist characteristics of the user. The waveform feature of the third signal collected by the bottom PPG sensor has low accuracy. The first signal collected by the side PPG sensor has less external influence, the waveform feature of the first signal is obvious, and the accuracy is high. The processor can identify the gesture through the first signal collected by the side PPG sensor.

[0323] Optionally, when the user makes a gesture with a small amplitude (which can also be referred to as a micro-gesture), such as tapping a finger, pinching a finger, or rubbing a palm, the first signal collected by the side PPG sensor changes little, and the waveform feature of the first signal changes little. The bottom PPG sensor is close to the wrist, and the third signal collected by the bottom PPG sensor can be used to determine physiological characteristics such as blood flow changes caused by blood vessel compression at the user's wrist. The processor can identify the gesture through the third signal collected by the bottom PPG sensor.

[0324] In one example, as shown in FIG. 16, the processor can fuse the waveform feature of the first signal, the waveform feature of the second signal, and the waveform feature of the third signal to improve the accuracy of gesture recognition. The specific process of feature fusion can refer to FIG. 17:

[0325] As shown in FIG. 17, it is a flowchart of a gesture recognition method provided by an embodiment of the present application. Referring to FIG. 17, the flow includes multiple S401-S304, S301-S303, S101-S103, and S105-S107 or multiple S401-S304, S301-S303, S101-S103, and S105-S105, S108:

[0326] Optionally, S301-S303 and S101-S103 can refer to the above description and will not be described here.

[0327] S401, the bottom PPG sensor in the first electronic device collects a third signal.

[0328] Optionally, the bottom PPG sensor can collect a third signal when the user makes a gesture.

[0329] Optionally, the bottom PPG sensor can be located at the bottom of the dial of the first electronic device. When the user wears the first electronic device, the bottom PPG sensor is in contact with the user's wrist, and the bottom PPG sensor can emit a light signal to the wrist and receive a third signal.

[0330] Optionally, the LED in the bottom PPG sensor emits a light signal to the wrist, and the light signal irradiates to the bottom of the blood vessels through the skin surface. The pigment of the blood (such as hemoglobin) will absorb the light signal, and the surrounding tissues (such as skin and bone) will reflect or scatter the light signal. When the heart beats, the amount of blood in the blood vessels changes, causing the amount of light signal absorbed by the blood to also change. And the amount of light signal absorbed by the blood is different when the oxygen content in the blood is different. The distance between the bottom PPG sensor and the wrist does not change, so the third signal received by the PD in the bottom PPG sensor changes with the slight movement of the wrist causing the blood to change.

[0331] S402, the bottom PPG sensor sends the third signal to the processor.

[0332] Optionally, the bottom PPG sensor can collect the third signal in real time, and send the third signal to the processor in real time.

[0333] S403, the processor obtains the waveform feature of the third signal.

[0334] Optionally, the processor can obtain the waveform of the third signal according to the third signal received in real time, and obtain the waveform feature of the first signal according to the waveform of the third signal.

[0335] Exemplarily, the waveform feature of the third signal can be: the waveform peak value is h10, the peak width is (t10-t11), the number of peaks is 5, and the peak-to-peak value of the waveform is (h10-h00).

[0336] Optionally, the sequence of S101-S103, S301-S303 and S401-S403 is not limited.

[0337] S404, the processor identifies the gesture based on the waveform feature of the first signal, the waveform feature of the second signal and the waveform feature of the third signal.

[0338] Optionally, the processor can determine the gesture recognition method according to the magnitude of the waveform change amplitude in the waveform feature of the first signal and the waveform change amplitude in the waveform feature of the third signal.

[0339] When the processor detects that the waveform variation amplitude in the waveform feature of the first signal is greater than the waveform variation amplitude in the waveform feature of the third signal, it can be determined that the gesture made by the user is a gesture with strong directivity, and the processor can identify the gesture through the waveform feature of the first signal. Alternatively, the processor identifies the gesture through the waveform feature of the first signal and the waveform feature of the second signal.

[0340] When the processor detects that the waveform variation amplitude in the waveform feature of the first signal is less than the waveform variation amplitude in the waveform feature of the third signal, it can be determined that the gesture made by the user is a micro-gesture, and the processor can identify the gesture through the waveform feature of the third signal. Alternatively, the processor identifies the gesture through the waveform feature of the third signal and the waveform feature of the second signal.

[0341] Optionally, the processor can fuse the waveform feature of the first signal, the waveform feature of the second signal, and the waveform feature of the third signal.

[0342] For example, the processor can weight the waveform feature of the first signal, the waveform feature of the second signal, and the waveform feature of the third signal, and identify the gesture made by the user according to the weighted waveform feature of the first signal, the waveform feature of the second signal, and the waveform feature of the third signal through random forest.

[0343] For example, when the processor detects that the waveform variation amplitude in the waveform feature of the first signal is greater than the waveform variation amplitude in the waveform feature of the third signal, the weight of the waveform feature of the first signal is increased, and the weight of the waveform feature of the third signal is decreased; the weight of the waveform feature of the first signal is greater than the weight of the waveform feature of the third signal. When the processor detects that the waveform variation amplitude in the waveform feature of the first signal is less than the waveform variation amplitude in the waveform feature of the third signal, the weight of the waveform feature of the first signal is decreased, and the weight of the waveform feature of the third signal is increased; the weight of the waveform feature of the first signal is less than the weight of the waveform feature of the third signal. When the processor detects that the waveform variation amplitude in the waveform feature of the first signal is equal to the waveform variation amplitude in the waveform feature of the third signal, the weight of the waveform feature of the first signal is kept consistent with the weight of the waveform feature of the third signal.

[0344] The processor identifies the gesture through random forest according to the weight of the adjusted waveform feature of each signal.

[0345] It should be understood that the above waveform feature fusion manner is only an example, and other fusion manners can also be used. For example, the priority of the waveform feature of the first signal and the waveform feature of the third signal is set, and the priority of the waveform feature of the first signal is greater than the priority of the waveform feature of the third signal. The processor first identifies the gesture according to the waveform feature of the first signal, and then identifies the gesture according to the waveform feature of the third signal when the identification according to the waveform feature of the first signal fails (for example, no corresponding gesture is matched). Alternatively, the processor first identifies the gesture according to the waveform feature of the first signal and the waveform feature of the second signal, and then identifies the gesture according to the waveform feature of the third signal and the waveform feature of the second signal when the identification according to the waveform feature of the first signal and the waveform feature of the second signal fails (for example, no corresponding gesture is matched).

[0346] For another example, the processor can identify the gesture made by the user based on the first signal and the second signal through a random forest, and then calibrate the identification result according to the third signal. Alternatively, the processor can also identify the gesture made by the user based on the third signal and the second signal through a random forest, and then calibrate the identification result according to the first signal. The embodiments of the present application do not make specific limitations in this regard.

[0347] In an embodiment, in the above S304, the processor identifies whether the gesture made by the user is a single-hand gesture or a double-hand gesture according to the waveform feature of the first signal and the waveform feature of the second signal. In an embodiment, the processor can determine whether the gesture made by the user is a single-hand gesture or a double-hand gesture according to the waveform feature of the first signal and the waveform feature of the third signal.

[0348] For example, the third signal satisfies a fifth feature, and the gesture of the user is a single-hand gesture. The fifth feature can indicate that the waveform change of the third signal is greater than a second preset threshold. For example, the peak-to-peak value of the waveform in the waveform feature of the third signal is greater than the second preset threshold. The third signal satisfies a sixth feature, and the gesture of the user is a double-hand gesture. The sixth feature can indicate that the waveform change of the third signal is less than or equal to the second preset threshold. For example, the peak-to-peak value of the waveform in the waveform feature of the third signal is less than or equal to the second preset threshold. The second preset threshold can be customized by the user or customized by the developer before the first electronic device is shipped, and the embodiments of the present application do not make specific limitations in this regard.

[0349] The above fifth feature that is satisfied can be matching of the waveform feature of the third signal corresponding to a preset single-hand gesture, the sixth feature that is satisfied can be matching of the waveform feature of the third signal corresponding to a preset double-hand gesture, and the embodiments of the present application do not make specific limitations in this regard.

[0350] Thus, when the first signal satisfies the first feature and the third signal satisfies the fifth feature, the processor identifies the gesture made by the user as the single-hand gesture. When the first signal satisfies the third feature and the third signal satisfies the sixth feature, the processor identifies the gesture made by the user as the double-hand gesture.

[0351] In an embodiment, the processor can determine whether the gesture made by the user is the single-hand gesture or the double-hand gesture according to the waveform feature of the first signal, the waveform feature of the second signal, and the waveform feature of the third signal.

[0352] For example, when the first signal satisfies the first feature, the second signal satisfies the second feature, and the third signal satisfies the fifth feature, the processor identifies the gesture made by the user as the single-hand gesture. When the first signal satisfies the third feature, the second signal satisfies the fourth feature, and the third signal satisfies the sixth feature, the processor identifies the gesture made by the user as the double-hand gesture.

[0353] S105-S107 or S104-S105, S108 can refer to the above, and will not be repeated here.

[0354] Thus, the gesture recognition result is more accurate, the applicable scenarios are more extensive, the types of recognizable gestures are increased, and the user experience is improved by using the signals collected by multiple sensors and multiple gesture recognition manners.

[0355] In some other embodiments, on the basis of the embodiment corresponding to FIG. 4, a bottom PPG sensor can be further added to the first electronic device, and the first electronic device can implement the gesture recognition process through the side PPG sensor and the bottom PPG sensor. The third signal is collected by the bottom PPG sensor, and the third signal is used to represent the physiological feature of the hand of the user.

[0356] That is, the processor can also identify the gesture made by the user based on the first signal and the third signal.

[0357] For example, the processor can identify the gesture made by the user based on the waveform feature of the first signal, and then calibrate the identification result according to the waveform feature of the third signal.

[0358] The waveform feature of the first signal matches the waveform feature of the first signal of gesture A and gesture A' in the preset gesture library, and the waveform feature of the third signal matches the waveform feature of the third signal of gesture A in the preset gesture library. Then the processor can determine that the gesture made by the user is gesture A.

[0359] For another example, the processor can identify the gesture made by the user based on the waveform feature of the first signal and the waveform feature of the third signal.

[0360] For example, the waveform feature of the first signal matches the waveform feature of the first signal of gesture A in the preset gesture library, and the waveform feature of the third signal matches the waveform feature of the third signal of gesture A in the preset gesture library. The processor can determine that the gesture made by the user is gesture A.

[0361] For another example, the waveform feature of the first signal matches the waveform feature of the first signal of gesture A in the preset gesture library, and the waveform feature of the third signal matches the waveform feature of the third signal of gesture H in the preset gesture library.

[0362] When the processor detects that the waveform change amplitude in the waveform feature of the first signal is greater than the waveform change amplitude in the waveform feature of the third signal, the processor can determine that the gesture made by the user is gesture A. When the processor detects that the waveform change amplitude in the waveform feature of the first signal is less than the waveform change amplitude in the waveform feature of the third signal, the processor can determine that the gesture made by the user is gesture H.

[0363] The specific way in which the processor identifies the gesture made by the user based on the first signal and the third signal is only an example, and the way in which the processor identifies the gesture made by the user based on the first signal and the third signal can also be other, and the embodiments of the present application do not specifically limit this.

[0364] In some scenarios, because the user is always in a non-stationary state, when the user wears the first electronic device, the waveform of the first signal of the side PPG sensor of the first electronic device fluctuates, and the gesture recognition function can be triggered by mistake, resulting in misrecognition of the gesture.

[0365] To solve the above problems, the embodiments of the present application first wake up the gesture recognition function before gesture recognition.

[0366] In some embodiments, the side PPG sensor of the first electronic device can be in a normal open state (or referred to as an operating state) and always collect the first signal, at this time the gesture recognition function of the first electronic device can be turned on or turned off. For example, as shown in FIG. 18, when the gesture recognition function is turned on, the process of turning on the gesture recognition function when the side PPG sensor of the first electronic device is in a normal open state and the gesture recognition function is turned off.

[0367] In some embodiments, the side PPG sensor of the first electronic device can be in a normally closed state (or referred to as a dormant state), and in the absence of external intervention or signals, the side PPG sensor is in a closed state by default and does not collect the first signal. At this time, the gesture recognition function of the first electronic device is also in a normally closed state and does not perform gesture recognition. For example, as shown in FIGS. 19 and 20, the process of starting the gesture recognition function when the side PPG sensor of the first electronic device is in a normally closed state is shown. FIG. 19 shows the process of passively waking up the side PPG sensor of the first electronic device, causing the side PPG sensor to collect the first signal and starting the gesture recognition function of the first electronic device. FIG. 20 shows the process of actively waking up the side PPG sensor of the first electronic device, causing the side PPG sensor to collect the first signal and starting the gesture recognition function of the first electronic device.

[0368] Referring to FIG. 18, the side PPG sensor is in a normally open state. The side PPG sensor can be used for gesture recognition, and the side PPG sensor can also be used to obtain physiological parameters such as heart rate, blood pressure, and / or blood oxygen saturation of the user.

[0369] When the side PPG sensor is used for gesture recognition, the first signal received by the side PPG sensor is used to identify the gesture. As shown in FIG. 18, a gesture recognition function wake-up method flowchart is shown, including S500-S504:

[0370] S500, determining whether the first signal collected by the side PPG sensor in the first electronic device is used to determine physiological characteristics.

[0371] Optionally, if the first signal collected by the side PPG sensor in the first electronic device is used to determine physiological characteristics, S501 is performed, and if the first signal collected by the side PPG sensor in the first electronic device is not used to determine physiological characteristics, S502 is performed.

[0372] S501, if the first signal collected by the side PPG sensor in the first electronic device is used to determine physiological characteristics, the gesture recognition function is closed.

[0373] In one embodiment, the priority of the first signal collected by the side PPG sensor for determining physiological characteristics is higher than that for gesture recognition, and when the first signal collected by the side PPG sensor is used to measure physiological characteristics of the user, the gesture recognition function is closed.

[0374] In this way, the first signal collected by the side PPG sensor for determining physiological characteristics and for gesture recognition can be prevented from conflicting, affecting the results of gesture recognition or the measurement results of physiological characteristics.

[0375] S502, if the first signal collected by the side PPG sensor in the first electronic device is not used to determine the physiological feature, the gesture recognition function is started.

[0376] S503, when the user makes a gesture, the first electronic device recognizes the gesture.

[0377] For details, refer to S103, S104, or S303, S304, or S403, S404, which are not repeated here.

[0378] S504, the first electronic device executes the operation instruction corresponding to the gesture.

[0379] For details, refer to S105-S107 or S104-S105, S108, which are not repeated here.

[0380] In this way, the first signal collected by the side PPG sensor can be used for gesture recognition and can also be used to determine the physiological feature of the user, expanding the use scenario of the side PPG sensor; and determining the physiological feature of the user is performed in a time period separated from gesture recognition, preventing conflicts.

[0381] It should be noted that in FIG. 18, S500 and S502 are optional steps. Specifically, the first electronic device can not include a physiological function measurement implementation, and the first electronic device can not perform the steps of S500 and S502 for determining whether the physiological function measurement and the gesture recognition conflict, and closing or starting the gesture recognition function. The first electronic device can execute the steps of S503 and S504 for recognizing gestures and executing operation instructions corresponding to gestures when the side PPG sensor is in an always-on state, and no further description is provided.

[0382] In another embodiment, the side PPG sensor is in a normally closed state, and the gesture recognition function is in a normally closed state. The gesture recognition function of the first electronic device can be triggered by a trigger condition to trigger the side PPG sensor to start, to wake up the gesture recognition function of the first electronic device. In this method, the gesture recognition function of the first electronic device is passively woken up.

[0383] As shown in FIG. 19, a gesture recognition function wake-up method flowchart includes S601-S604:

[0384] S601, the gesture recognition function of the first electronic device is closed.

[0385] Optionally, the first electronic device gesture recognition function off can be a gesture recognition function off after the first electronic device is initialized, or the first electronic device initiatively closes the gesture recognition function based on scene recognition, for example, recognizing a sleep scene or other scene that does not need gesture recognition, to save power consumption, or the first electronic device closes the gesture recognition function in response to a user operation, and the embodiments of the present application do not make specific limitations thereto.

[0386] S602, the first electronic device judges whether a trigger condition is detected, and if the trigger condition is detected, the gesture recognition function is turned on.

[0387] In the embodiments of the present application, the trigger condition can be a condition that needs to turn on the gesture recognition function, for example, the trigger condition can include one or more of the following: time condition, location condition or event condition. The various cases of the trigger condition will be described below.

[0388] Optionally, the trigger condition can be a time condition.

[0389] For example, the user wears the first electronic device to sleep, and the first electronic device can detect the current time, and when the preset first time is reached, the first electronic device is triggered to turn on the gesture recognition function. Alternatively, the trigger condition can be a wake-up alarm ring event, and when the user sets the wake-up alarm to ring, the first electronic device is triggered to turn on the gesture recognition function.

[0390] In an embodiment, the first electronic device can detect the current time, and when the preset second time is reached, the first electronic device can also close the gesture recognition function.

[0391] The first time and the second time can be customized by the user, such as the first time being a preset wake-up time and the second time being a preset sleep time, or the first time and the second time can be customized by the R&D personnel before the first electronic device is shipped, or the first time and the second time can also be updated based on big data learning, and the embodiments of the present application do not make specific limitations thereto.

[0392] Optionally, the trigger condition can also be a location condition.

[0393] For example, when the first electronic device detects that the first electronic device enters a first area, the first electronic device is triggered to turn on the gesture recognition function. For example, the first area is a residential house, and when the user wears the first electronic device to enter the residential house, the first electronic device is triggered to turn on the gesture recognition function, and the user can control the first electronic device through gestures, thereby controlling the smart home devices in the residential house.

[0394] For example, when the first electronic device detects that the first electronic device is close to the first device, the first electronic device is triggered to start the gesture recognition function. For example, the first device is a vehicle, and when the user wearing the first electronic device is within a first distance (e.g., 5 m) from the vehicle, the first electronic device is triggered to start the gesture recognition function, and the user can control the first electronic device through gestures, and then control the vehicle.

[0395] The first electronic device can determine whether the user enters the first area or the first distance between the user and the first device through one or more of global positioning system (GPS), Bluetooth, and / or wireless fidelity (WiFi) technology, ultra-wideband (UWB), or time of flight (ToF) technology, and then control the second electronic device such as a smart home device through one or more of Bluetooth, WiFi, FM, IR, or NFC.

[0396] Optionally, the trigger condition can also be an event condition.

[0397] The event condition can be used to indicate a user state event or a first electronic device state event.

[0398] For example, the user state event can include a getting-up event or an exercise event. For example, when the first electronic device detects a getting-up event, the first electronic device is triggered to start the gesture recognition function. For example, the first electronic device detects that the user gets up at 7:00 in the morning according to the alarm clock, and the first electronic device is triggered to start the gesture recognition function. In this way, the gesture recognition function of the first electronic device is started in the state of the user being awake, which can prevent the gesture recognition from being triggered by the user's unconscious actions when the user is sleeping.

[0399] The first electronic device state event can include one or more of the following: an application state event of the first electronic device, a connection establishment event between the first electronic device and the second electronic device, a message receiving event of the first electronic device from the second electronic device, or a signal receiving event of the first electronic device from the second electronic device.

[0400] For example, the application state event can include that the application receives a message and / or some function of the application is turned on, etc. When the first electronic device detects the application state event of the first electronic device, the first electronic device can be triggered to turn on the gesture recognition function. For example, the first electronic device detects that the social application receives a message, and the first electronic device can be triggered to turn on the gesture recognition function. In this way, after the user receives the message of the social application, the user can operate the first electronic device through a gesture to reply to the message or ignore the message or expand the message, which can save the user time, quickly and efficiently process the message, and improve the user experience.

[0401] For another example, the first electronic device detects that the alarm application is running and the alarm is ringing, and the first electronic device can be triggered to turn on the gesture recognition function. In this way, after the user hears the sound of the alarm ringing, the user can operate the first electronic device through a gesture to cancel the alarm or delay the alarm, which can save the user time and operation steps, quickly and efficiently complete the operation, and improve the user experience.

[0402] For another example, when the first electronic device detects that the first electronic device establishes a connection event with the second electronic device, the first electronic device can be triggered to turn on the gesture recognition function. For example, after the first electronic device establishes a connection with the second electronic device, the first electronic device can be triggered to turn on the gesture recognition function for controlling the second electronic device through a gesture. In this way, after the first electronic device establishes a connection with the second electronic device, the user can control the second electronic device through a gesture and the first electronic device, which improves the user experience.

[0403] For another example, the first electronic device receives a message event of the second electronic device, which can refer to an event that the first electronic device receives a message from the second device, and when the first electronic device receives the message event from the second electronic device, the first electronic device can be triggered to turn on the gesture recognition function. For example, the call application of the first electronic device receives a telephone call message of the second electronic device, etc., and the first electronic device can be triggered to turn on the gesture recognition function. In this way, after the user receives the call message, the user can operate the first electronic device through a gesture to answer the call or hang up the call, which can save the user time, quickly and efficiently process the message, and improve the user experience.

[0404] For another example, the signal event of the second electronic device can refer to an event that the first electronic device receives a signal from the second electronic device, and the gesture recognition function of the first electronic device can be triggered when the first electronic device receives the signal event from the second electronic device. For example, the signal from the second electronic device can be an ultra-wideband (UWB) signal, and the second electronic device can be a vehicle. When the user wearing the first electronic device approaches the vehicle, the UWB signal emitted by the vehicle can be detected, triggering the first electronic device to start the gesture recognition function. The user can control the first electronic device through gestures, and then control the vehicle.

[0405] For another example, the signal from the second electronic device can be a Bluetooth signal, and the second electronic device can be a smart home device (such as a smart speaker). When the user wearing the first electronic device approaches the smart home device, the Bluetooth signal emitted by the smart home device can be detected, triggering the first electronic device to start the gesture recognition function. The user can control the first electronic device through gestures, and then control the smart home device. When the user does not touch the second electronic device, the user can also control the second electronic device through gestures, improving the user experience.

[0406] The signal from the second electronic device can also be one or more of a WiFi signal, a GPS signal, or an NFC signal, which is not limited in the embodiments of the present application.

[0407] In another scenario, if the first electronic device does not detect the trigger condition, the first electronic device can continue to detect whether there is a trigger condition until the trigger condition is detected, and then the gesture recognition function is started.

[0408] S603, when the user makes a gesture, the first electronic device recognizes the gesture

[0409] For details, refer to S103 and S104, or S303 and S304, or S403 and S404, which are not repeated here.

[0410] S604, the first electronic device executes the operation instruction corresponding to the gesture.

[0411] For details, refer to S105-S107 or S104-S105 and S108, which are not repeated here.

[0412] In this way, the gesture recognition function of the first electronic device is in a normally closed state, and the gesture recognition function is triggered passively only when the trigger condition is detected, which can reduce the power consumption of the first electronic device.

[0413] In another embodiment, the side PPG sensor is in a closed state, and the gesture recognition function is in a closed state. The user can actively wake up the gesture recognition function through a pre-action.

[0414] Optionally, when the side PPG sensor in the first electronic device is in a closed state, the second sensor can be in an open state. When the second sensor identifies that the user has made a pre-action, the gesture recognition function of the side PPG sensor can be turned on.

[0415] As shown in FIG. 20, it is a flowchart of a gesture recognition function wake-up method, including S701-S705:

[0416] S701, the gesture recognition function of the first electronic device is closed.

[0417] That is, the side PPG sensor of the first electronic device is in a closed state, and the gesture recognition function is closed.

[0418] S702, the first electronic device determines whether a pre-action is detected. If the pre-action is detected, the gesture recognition function is turned on.

[0419] Optionally, the pre-action can include an interaction action between the user and the first electronic device.

[0420] For example, the pre-action can include that the user raises the wrist wearing the first electronic device and / or the user turns over the wrist wearing the first electronic device, etc. When the user raises the wrist wearing the first electronic device and / or the user turns over the wrist wearing the first electronic device, the first electronic device is triggered to turn on the gesture recognition function.

[0421] In an embodiment, the first electronic device can detect whether the user makes a pre-action through a second sensor.

[0422] For example, when the user makes an action, the waveform of the second signal collected by the second sensor can fluctuate. The processor can compare the waveform characteristics of the second signal with the waveform characteristics of the second signal when the user raises the wrist wearing the first electronic device. If the waveform characteristics of the second signal are the same or similar to the waveform characteristics of the second signal when the user raises the wrist wearing the first electronic device, it is determined that the user makes a pre-action.

[0423] Optionally, when the first electronic device detects that the user raises the wrist wearing the first electronic device through the second sensor, the first electronic device can turn on the screen and turn on the gesture recognition function of the side PPG sensor.

[0424] For example, when the first electronic device receives a call prompt, the user can raise the wrist wearing the first electronic device, the first electronic device displays the call information on the screen, and the gesture recognition function of the first electronic device is started. Subsequently, the user can operate the first electronic device to answer or hang up the call through gestures.

[0425] For another example, when the mobile phone receives a call prompt, and the user is inconvenient to operate the mobile phone (for example, is in a meeting). The user can raise the wrist wearing the first electronic device, the first electronic device displays the call information on the screen, and the gesture recognition function of the first electronic device is started. Subsequently, the user can operate the mobile phone to answer or hang up the call through gestures and the first electronic device.

[0426] For another example, the pre-action can also include a touch operation or a click operation of the user. The user can touch the display screen of the first electronic device, or click the display screen of the first electronic device, or click the physical button on the first electronic device, or click the physical button on the smart phone, to trigger the first electronic device to start the gesture recognition function. For example, the user clicks the power button twice, and the first electronic device starts the gesture recognition function in response to the operation.

[0427] Optionally, the first electronic device can detect the touch operation or the click operation of the user acting on the display screen through the pressure sensor. For example, when the pressure signal of the pressure sensor is greater than the first pressure threshold, the gesture recognition function is started.

[0428] The first pressure value can be user-defined, and the embodiments of the present application do not make specific limitations.

[0429] For another example, the pre-action can also include a voice operation of the user. The user can also trigger the first electronic device to start the gesture recognition function through a voice instruction. For example, the user can issue a voice instruction "start gesture recognition function", and the first electronic device receives the voice instruction from the user, and starts the gesture recognition function in response to the instruction.

[0430] For another example, the pre-action can also include the user wearing the first electronic device touching the service card. The service card can be a virtual or physical card in a smart home device. The user can touch the service card through the first electronic device, and the first electronic device and the service card establish a connection through NFC, Bluetooth or WiFi, to trigger the first electronic device to start the gesture recognition function. For example, the user can wear the first electronic device to touch the service card in the smart home device, and the first electronic device and the service card in the smart home device establish a connection through NFC, Bluetooth or WiFi, to trigger the first electronic device to start the gesture recognition function. The user can control the first electronic device through gestures, and further control the smart home device.

[0431] Optionally, the above embodiments take the smart home device as an example for introduction, and the embodiments of the present application can also be applied to other smart devices, and the embodiments of the present application do not make specific limitations in this regard.

[0432] In another scenario, if the first electronic device does not detect the pre-action, the first electronic device can continue to detect whether there is a pre-action until the pre-action is detected, and then the gesture recognition function is started.

[0433] S703, when the user makes a gesture, the first electronic device recognizes the gesture.

[0434] In an embodiment, after starting the gesture recognition, and before specific gesture recognition is performed, it can also be judged whether the gesture recognition is triggered.

[0435] For example, when the size of the first signal collected by the side PPG sensor is greater than a preset threshold, it can be indicated that the intensity of the reflected light received by the side PPG sensor is greater than the threshold, and then the gesture recognition can be triggered. For example, when the peak-to-peak value of the waveform of the first signal is greater than a second threshold, the gesture recognition is triggered, and subsequently the gesture is recognized through the waveform whose peak-to-peak value of the waveform of the first signal is greater than the second threshold. In this way, the single-hand gesture or the double-hand gesture can not be distinguished, and when there is an obvious gesture signal, the gesture recognition is triggered in time.

[0436] For another example, when it is detected based on the second signal collected by the second sensor and the third signal collected by the bottom PPG sensor that there is no obvious hand signal, it can be determined that it is a single-hand gesture, and then it is judged whether the size of the first signal is greater than a preset threshold, and the gesture recognition is triggered. In this way, the recognition of the single-hand gesture can be more accurate and timely.

[0437] The gesture recognition can refer to S103, S104, or S303, S304, or S403, S404, which will not be repeated here.

[0438] S704, the first electronic device executes the operation instruction corresponding to the gesture.

[0439] For details, refer to S105-S107 or S104-S105, S108, which will not be repeated here.

[0440] S705, the first electronic device detects whether there is a gesture action within a first time length, and if there is no gesture action within the first time length, the gesture recognition function is closed.

[0441] In an embodiment, the first electronic device can detect whether there is a gesture action within a first time length, and if there is no gesture action within the first time length, the gesture recognition function is closed. This prevents the user from being mistaken and reduces the consumption of resources of the first electronic device.

[0442] Optionally, the first time length can be a time length customized by the user. For example, the first time length can be 30 seconds or 60 seconds, and the embodiments of the present application do not make specific limitations in this regard.

[0443] For example, if the first time length is 30 seconds, when the first electronic device executes the operation instruction corresponding to the gesture, if the gesture is not detected again within 30 seconds, the gesture recognition function is closed.

[0444] For example, if the gesture action is detected within the first time length, the gesture is recognized and the operation corresponding to the gesture is executed.

[0445] In a possible implementation, the gesture action detected within the first time length can be a gesture action for setting a do-not-disturb mode. After the first electronic device hangs up an incoming call, in order not to be disturbed by other incoming calls, the user can set the first electronic device to the do-not-disturb mode through the gesture action. Within the first time length, after the first electronic device detects the gesture action for setting the do-not-disturb mode, the first electronic device can be set to the do-not-disturb mode. It can be understood that the do-not-disturb mode can also be replaced by any do-not-disturb mode such as the airplane mode, and the embodiments of the present application do not make specific limitations.

[0446] In another embodiment, S705 can be replaced by that the first electronic device can detect whether a preset time is reached, and if the first electronic device detects that the preset time is reached, the gesture recognition function is closed.

[0447] For example, the preset time is 10:00 pm, and when the first electronic device detects that 10:00 pm is reached, the gesture recognition function is closed. At this time, the user is less active, and the probability of using the gesture recognition function is small, so the gesture recognition function is closed to reduce the resource consumption of the first electronic device.

[0448] In another embodiment, the first electronic device can detect whether the user is asleep, and if the first electronic device detects that the user is asleep, the gesture recognition function is closed. This avoids the gesture recognition caused by the user's accidental touch when the user is sleeping, and saves power consumption.

[0449] Optionally, after the first electronic device closes the gesture recognition function, when the first electronic device detects the pre-action again, the gesture recognition function can be started again.

[0450] In this way, the gesture recognition function of the first electronic device is in a normally closed state and a pre-action is set, and the gesture recognition function is started only when the user actively makes the pre-action, which can reduce the misrecognition rate of the user's unintentional action. When the user does not make the gesture again within the first time length, the gesture recognition function is closed, which can reduce the power consumption of the first electronic device.

[0451] It can be understood that the method after gesture recognition is closed in FIG. 18 can be combined with the method after gesture recognition is closed in FIG. 19. For example, in FIG. 18, when the gesture recognition function of the first electronic device is closed, a trigger event can be further recognized. When the first electronic device detects the trigger event, the gesture recognition can be started. Details can be referred to the description of S602 in FIG. 19, and details are not described herein.

[0452] The method after gesture recognition is closed in FIG. 18 can be combined with the method after gesture recognition is closed in FIG. 20. When the gesture recognition function of the first electronic device is closed, a pre-action can be recognized. When the first electronic device recognizes the pre-action, the gesture recognition can be started. Details can be referred to the description of S702 in FIG. 20. The embodiment of the present application does not make specific limitation.

[0453] It should be noted that, for ease of description, in the above embodiments, the side PPG sensor and the bottom PPG sensor are exemplarily described. In some implementations, the side PPG sensor can be replaced by a first sensor, and the first sensor can further include an infrared sensor, an ultrasonic sensor, or a radar sensor, or any device capable of achieving the functions of the side PPG sensor. The bottom PPG sensor can be replaced by a third sensor, and the third sensor can further include an infrared sensor or an ECG sensor, or any device capable of achieving the functions of the bottom PPG sensor. The embodiment of the present application does not make specific limitation.

[0454] The following is another gesture recognition method flowchart provided by the embodiment of the present application. As shown in FIG. 21, the method includes S801-S802:

[0455] S801, the first electronic device collects a first signal through a first sensor in response to a first gesture of a user.

[0456] The first sensor is located on the side of the first electronic device, and the side is close to the hand of the user when the user wears the first electronic device.

[0457] Optionally, the first sensor can be a side PPG sensor. The side PPG sensor can collect the first signal from the hand of the user. When the hand of the user moves, the first signal also changes. The first signal can represent the movement characteristics of the hand of the user.

[0458] Details can be referred to the content in S101 in FIG. 4, which is not described herein.

[0459] S802, the first electronic device recognizes the first gesture based on the first signal.

[0460] Optionally, the processor in the first electronic device can receive the first signal from the first sensor, and identify the first gesture based on the first signal.

[0461] For example, the processor can obtain a waveform feature of the first signal based on the first signal, and identify the first gesture based on the waveform feature of the first signal. For example, when the waveform feature of the first signal matches a waveform feature of a second gesture in a preset gesture library, the first gesture is identified as the second gesture.

[0462] For details, refer to the content in S102-S104 in FIG. 4, which will not be repeated here.

[0463] In an embodiment, the first sensor can be integrated in a first module, and the first module is a module including the first sensor, such as a side PPG module. The first module can collect the first signal, and obtain a waveform feature of the first signal based on the first signal, and send the waveform feature of the first signal to the processor in the first electronic device. The processor in the first electronic device identifies the first gesture based on the waveform feature of the first signal. For details, refer to the content in S201-S203 in FIG. 11 and S104, which will not be repeated here. The first electronic device can further include a second sensor, and collect a second signal through the second sensor. The second sensor can be an ACC sensor and / or a gyroscope sensor or other sensors that can be used to collect acceleration information of the user's hand, and the present application does not make specific limitations here.

[0464] Optionally, the first electronic device can further identify the first gesture based on the first signal and the second signal.

[0465] For example, the processor can obtain a waveform feature of the second signal based on the second signal, and identify the first gesture based on the waveform feature of the second signal and the waveform feature of the first signal.

[0466] For details, refer to the content in S301-S304 in FIG. 12 and S101-S103, which will not be repeated here. The first electronic device can further include a third sensor, and collect a third signal through the third sensor. The third sensor can be a bottom PPG sensor.

[0467] Optionally, the first electronic device can further identify the first gesture based on the first signal and the third signal.

[0468] For example, the processor can obtain a waveform feature of the third signal based on the third signal, and identify the first gesture based on the waveform feature of the first signal and the waveform feature of the third signal.

[0469] The waveform characteristics of the first signal and the waveform characteristics of the third signal can be obtained according to the content in S401-S403 and S101-S103 in FIG. 17, and the gesture can be recognized according to the waveform characteristics of the first signal and the waveform characteristics of the third signal.

[0470] Optionally, the first electronic device can also recognize the first gesture based on the first signal, the second signal and the third signal.

[0471] For example, the processor can obtain the waveform characteristics of the third signal based on the third signal, and recognize the first gesture based on the waveform characteristics of the first signal, the waveform characteristics of the second signal and the waveform characteristics of the third signal.

[0472] The content in S401-S404, S101-S103 and S301-S303 in FIG. 17 can be referred to, and details are not repeated here.

[0473] Optionally, after the gesture recognition is completed, the first electronic device can perform a corresponding operation based on the gesture.

[0474] For example, the first electronic device can control the first electronic device based on the gesture.

[0475] The content in S108 can be referred to, and details are not repeated here.

[0476] Alternatively, the first electronic device can control the second electronic device (such as a smart air conditioner) based on the gesture.

[0477] The content in S106 and S107 in FIG. 4 can be referred to, and details are not repeated here.

[0478] In this way, the gesture is recognized by the first sensor on the side of the first electronic device, which reduces the interference of external factors such as the wearing position, wearing tightness or wrist characteristics of the user of the first electronic device, and improves the accuracy of gesture recognition.

[0479] It should be understood that some operations in the processes of the above-mentioned method embodiments are optionally combined, and / or the order of some operations is optionally changed. Moreover, the execution order between the steps of each process is only exemplary and does not constitute a limitation on the execution order between the steps, and other execution orders between the steps can also be used. It is not intended to indicate that the execution order is the only execution order in which the operations can be performed. A person of ordinary skill in the art can think of various ways to reorder the operations described herein. In addition, it should be pointed out that the process details involved in some embodiments herein are also applicable in a similar manner to other embodiments, or different embodiments can be combined for use.

[0480] In addition, some steps in the method embodiments can be replaced by other possible steps. Alternatively, some steps in the method embodiments can be optional and can be deleted in some use scenarios. Alternatively, other possible steps can be added in the method embodiments.

[0481] In addition, the above method embodiments can be implemented individually or in combination.

[0482] Some other embodiments of the present application provide an apparatus, which can be the electronic device or a component (such as a chip system) in the electronic device.

[0483] The apparatus can include a memory and one or more processors. The memory and the processor are coupled. The memory is configured to store computer program code including computer instructions. When the processor executes the computer instructions, the electronic device can perform each function or step performed by the mobile phone in the above method embodiments.

[0484] In the above embodiments, the core structure of the electronic device can be represented as the hardware structure shown in FIG. 22, and the electronic device includes at least one processor 2201, a communication line 2202, a memory 2203, and at least one communication interface 2204. The memory 2203 can also be included in the processor 2201.

[0485] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In some other embodiments of the present application, the electronic device can include more or fewer components than those illustrated, or combine some components, or split some components, or different arrangement of components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0486] The processor 2201 can be a general central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of programs of the present application. Alternatively, the processor 2201 can perform the steps performed by the processor in the above FIG. 4, FIG. 11, FIG. 12, and FIG. 17.

[0487] The communication line 2202 can include a path for transmitting information between the above components.

[0488] The communication interface 2204 is configured to communicate with other devices. In the embodiments of the present application, the communication interface can be a module, a circuit, a bus, an interface, a transceiver or other device capable of realizing the communication function, and is configured to communicate with other devices. Optionally, when the communication interface is a transceiver, the transceiver can be a separately arranged transmitter, which is configured to send information to other devices. The transceiver can also be a separately arranged receiver, which is configured to receive information from other devices. The transceiver can also be a component integrating the functions of the transmitter and the receiver. The specific implementation of the transceiver is not limited in the embodiments of the present application. Optionally, the communication interface 2204 is configured to communicate between the first electronic device and the second electronic device.

[0489] The memory 2203 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, a magneto-optical disk, a magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to this. The memory can exist independently and be connected to the processor through the communication line 2202. The memory can also be integrated with the processor. Optionally, the memory 2203 can store the waveform features of at least one gesture in the preset gesture library.

[0490] The memory 2203 is configured to store computer-executable instructions for implementing the solutions of the present application, and the processor 2201 is configured to control the execution of the computer-executable instructions. The processor 2201 is configured to execute the computer-executable instructions stored in the memory 2203, so as to implement the gesture recognition method provided in the embodiments of the present application.

[0491] Optionally, the computer-executable instructions in the embodiments of the present application can also be referred to as application program codes, instructions, computer programs or other names, and the embodiments of the present application are not limited to this.

[0492] In a specific implementation, as an embodiment, the processor 2201 can include one or more CPUs, such as CPU0 and CPU1 in FIG. 22.

[0493] In a particular implementation, as an example, the electronic device can include a plurality of processors, such as processor 2201 and processor 2205 in FIG. 22. Each of the processors can be a single-CPU processor or a multi-CPU processor. The processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0494] The embodiments of the present application also provide a chip system, as shown in FIG. 23, which includes at least one processor 161 and at least one interface circuit 162. The processor 161 and the interface circuit 162 can be interconnected through a line. For example, the interface circuit 162 can be used to receive signals from other devices (e.g., a memory of the electronic device). For another example, the interface circuit 162 can be used to send signals to other devices (e.g., the processor 161). Illustratively, the interface circuit 162 can read instructions stored in the memory and send the instructions to the processor 161. When the instructions are executed by the processor 161, the electronic device can perform various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which are not limited in the embodiments of the present application.

[0495] The embodiments of the present application also provide a computer storage medium, which includes computer instructions, when the computer instructions are run on the above electronic device, the electronic device performs various functions or steps performed by the mobile phone in the above method embodiments.

[0496] The embodiments of the present application also provide a computer program product, when the computer program product is run on a computer, the computer performs various functions or steps performed by the mobile phone in the above method embodiments.

[0497] From the above description of the embodiments, those skilled in the art can clearly understand that, for the convenience and brevity, only the division of the above functional modules is taken as an example for illustration, and in actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. The specific working process of the above described system, device and module can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0498] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are only illustrative, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another apparatus, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical, mechanical or other forms.

[0499] The units described as separate components can or can not be physically separate, and the components shown as units can be one physical unit or a plurality of physical units, that is, can be located in one place, or can be distributed to a plurality of different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0500] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0501] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application essentially or the parts that make contributions to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing an apparatus (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the method described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various storage program codes.

[0502] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any change or replacement within the technical scope disclosed in the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A gesture recognition method, characterized by, The method is applied to a first electronic device, the first electronic device comprises a first sensor, the first sensor is located at a side edge of the first electronic device, and the side edge is a side close to a user's hand when the user wears the first electronic device, and the method comprises: In response to a first gesture of the user, the first sensor collects a first signal; The first gesture is identified based on the first signal.

2. The method of claim 1, wherein, The first electronic device further comprises a second sensor, and the first gesture is identified based on the first signal, comprising: The second sensor collects a second signal; The first gesture is identified based on the first signal and the second signal.

3. The method according to claim 1 or 2, characterized in that, The first gesture is identified based on the first signal, comprising: If the first signal satisfies a first feature and the second signal satisfies a second feature, the first gesture is a single-hand gesture; the second feature indicates that a waveform change of the second signal is greater than a first preset threshold.

4. The method according to any one of claims 1 to 3, characterized in that, The first gesture is identified based on the first signal, comprising: If the first signal satisfies a third feature and the second signal satisfies a fourth feature, the first gesture is a double-hand gesture; the fourth feature indicates that a waveform change of the second signal is less than or equal to a first preset threshold.

5. The method according to any one of claims 2-4, characterized in that, The first gesture is identified based on the first signal and the second signal, comprising: Direction information of the first electronic device is determined based on the second signal; The first gesture is identified based on the direction information of the first electronic device and the first signal.

6. The method according to any one of claims 1 to 5, characterized in that, The first electronic device further comprises a third sensor, and the first gesture is identified based on the first signal, comprising: A third signal is collected; the third signal is used to represent a physiological feature of the user's hand; The first gesture is identified based on the first signal, the second signal and the third signal, or the first gesture is identified based on the first signal and the third signal.

7. The method according to any one of claims 1 to 6, characterized in that, The method further comprises: The first electronic device is controlled based on the first gesture; or a second electronic device is controlled by the first electronic device based on the first gesture. The first sensor comprises a side edge PPG sensor.

8. The method according to any one of claims 1 to 7, characterized in that, The second sensor comprises an ACC sensor and / or a gyroscope sensor, and is used to collect acceleration information of the user's hand.

9. The method according to any one of claims 2-8, characterized in that, The third sensor comprises a bottom PPG sensor.

10. The method according to any one of claims 6-9, characterized in that, The first gesture comprises one or more of the following: turning up a palm, continuously turning up a palm, turning down a palm, continuously turning down a palm, turning a palm to the left, turning a palm to the right, keeping a hand wearing the first electronic device still, moving another hand close to or away from the first electronic device, tapping a finger, pinching a finger or rubbing a palm.

11. The method according to any one of claims 1 to 10, characterized in that, Before the first sensor collects the first signal in response to the first gesture of the user, the method further comprises:

12. The method according to any one of claims 1 to 11, characterized in that, A gesture recognition function in the first electronic device is turned off; The first electronic device detects a trigger condition to turn on the gesture recognition function in the first electronic device; the trigger condition comprises one or more of the following: a time condition, a location condition or an event condition. Before the first sensor collects the first signal in response to the first gesture of the user, the method further comprises:

13. The method according to any one of claims 1 to 12, characterized in that, A gesture recognition function in the first electronic device is turned off; ​ The first electronic device detects a user making a pre-action to turn on a gesture recognition function in the first electronic device; the pre-action includes one or more of the following: the user raising a wrist or the user turning over a wrist, a touch operation of the user, a voice operation of the user, or the user wearing the first electronic device touching a service card.

14. An electronic device, comprising: Comprise: a processor and a memory coupled to the processor, the memory configured to store computer program code comprising computer instructions that, when read by the processor from the memory, cause the first electronic device to perform the method of any one of claims 1-13.

15. A computer-readable storage medium having stored therein instructions, the computer-readable storage medium comprising: When the instructions are run on the first electronic device, the first electronic device is caused to perform the method of any one of claims 1-13.

Images