Unlocking and gesture control of electronic devices

By combining thermal sensors and image sensors, the problem of image sensors identifying living bodies and environmental interference in electronic device unlocking and gesture control is solved, achieving higher unlocking security and gesture control accuracy.

CN118940240APending Publication Date: 2024-11-12PIXART IMAGING INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410123580.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-05-11
Filing Date
2024-01-29
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In the prior art, image sensors are difficult to directly identify living bodies in electronic device unlocking and gesture control, and are susceptible to ambient light and interference, reducing user experience and security.

Method used

A system combining thermal sensors and image sensors is adopted to detect human body temperature and image sensors through thermal sensors to obtain image frames. The processor performs live recognition and gesture recognition, eliminates environmental interference, and improves the accuracy of unlocking and gesture control.

Benefits of technology

It effectively reduces the complexity and environmental interference of biometric unlocking, improves the unlocking security of electronic devices and the accuracy of gesture control, and enhances the user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118940240A_ABST
    Figure CN118940240A_ABST
Patent Text Reader

Abstract

The invention discloses an electronic device capable of improving the strength of a digital electronic lock and gesture control correctness. The electronic device is unlocked according to a gesture of a single hand or two hands of a user, an operator, a gesture position, gesture dwell time and gesture change. The electronic device also improves the accuracy of gesture control according to the accessory pattern, temperature detection and the spatial information of the wearable device. The invention further provides an electronic device for identifying the movement legitimacy according to the change sequence of the three-axis acceleration of the accelerometer.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to unlocking and gesture control of an electronic device, and more particularly to an electronic device that is unlocked without using biometrics and is gesture controlled in combination with temperature detection and a wearable device. Background Art

[0002] Image sensors have been widely used in portable electronic devices as a way to unlock the device. However, image sensors are sometimes difficult to directly identify living bodies and are easily affected by ambient light. In order to solve these problems, more complex algorithms are usually required.

[0003] For example, biometrics can be used to unlock digital electronic locks. However, in a system involving many users, it is necessary to register all users' biometrics, such as fingerprints or facial information, in advance, which is very troublesome.

[0004] In addition, although the use of gestures to control electronic devices has been proposed in the industry, it is easily affected by the environment. For example, other human bodies in the scene or objects with a similar shape to the hand will reduce the recognition accuracy, thereby reducing the user's willingness to use gesture control.

[0005] In view of this, the present invention provides an electronic device that can reduce the risk of being cracked without using biometric features for unlocking and an electronic device that can eliminate environmental interference in gesture recognition. Summary of the invention

[0006] The present invention provides an electronic device and an unlocking method thereof that are unlocked according to the user's single-hand or double-hand gestures (including predetermined gestures and user-defined gestures), operating hand (left hand or right hand), gesture position (or hand position), gesture dwell time and gesture changes (such as dynamic gestures).

[0007] The present invention also provides an electronic device and a gesture recognition method thereof that improves the accuracy of gesture control by combining accessory patterns, temperature detection, and spatial information of a wearable device.

[0008] The invention also provides an electronic device and a mobile warning method thereof for judging the legitimacy of a mobile device according to the change sequence of the three-axis acceleration values ​​of the three-axis accelerometer.

[0009] The present invention provides an electronic device comprising an image sensor, a gesture sensor and a processor. The image sensor is used to output an image frame for detecting a face image. The gesture sensor is used to generate a gesture detection signal. The processor is used to identify whether a face is blocked according to the face image, and only when the face is not blocked, identify a gesture combination according to the gesture detection signal, and unlock the electronic device when the gesture combination meets a predetermined condition.

[0010] The present invention also provides an electronic device including an image sensor and a processor. The image sensor is used to obtain an image frame including a hand image. The processor is used to identify a specific pattern related to an accessory in the image frame, perform gesture recognition based on the hand image and the specific pattern in the image frame, and control the electronic device to perform a corresponding operation of the recognized gesture.

[0011] The present invention also provides an electronic device comprising a three-axis accelerometer and a processor. The three-axis accelerometer is used to output acceleration values ​​of the three axes. The processor is used to receive the acceleration values ​​of the three axes, determine whether the changes of the acceleration values ​​of the three axes conform to a predetermined sequence, and turn off the alarm function when the predetermined sequence is conformed.

[0012] In order to make the above and other purposes, features and advantages of the present invention more obvious, the following will be described in detail with reference to the accompanying drawings. In addition, in the description of the present invention, the same components are represented by the same symbols, which are hereby described together. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a block diagram of an identification system according to an embodiment of the present invention;

[0014] Figure 2A is a schematic diagram of the operation of the face recognition system according to the first embodiment of the present invention;

[0015] Figure 2B is a schematic diagram of a face image acquired by a face recognition system according to a first embodiment of the present invention;

[0016] Figure 2C is a schematic diagram of temperature distribution of a thermal image acquired by the face recognition system according to the first embodiment of the present invention;

[0017] Figure 3A is a schematic diagram of the operation of a gesture recognition system according to a second embodiment of the present invention;

[0018] Figure 3B is a schematic diagram of an image frame acquired by a gesture recognition system according to a second embodiment of the present invention;

[0019] Figure 4 is a schematic diagram of a medical monitoring system according to a third embodiment of the present invention;

[0020] Figure 5 is a schematic diagram of a body temperature monitoring device according to a fourth embodiment of the present invention;

[0021] Figure 6 is a block diagram of an electronic device according to another embodiment of the present invention;

[0022] Figure 7is a flow chart of a method for unlocking an electronic device according to another embodiment of the present invention;

[0023] FIG. 8A to FIG. 8F yes Figure 7 A schematic diagram of a gesture combination of an unlocking method according to an embodiment of the present invention;

[0024] Fig. 9 is a flow chart of a gesture recognition method for an electronic device according to another embodiment of the present invention;

[0025] FIG. 10A to FIG. 10C yes Fig. 9 A schematic diagram of confirming a hand through a thermal sensor in an unlocking method of an embodiment;

[0026] Fig.11 yes Fig. 9 A schematic diagram of confirming a hand through a specific pattern of an accessory in a gesture recognition method of an embodiment;

[0027] Fig.12 yes Fig. 9 A schematic diagram of confirming a hand through spatial information of a wearable device in a gesture recognition method of an embodiment;

[0028] Fig.13 is a flow chart of a mobile alert method of an electronic device according to another embodiment of the present invention;

[0029] Fig.14A A schematic diagram of four faces and four corners of an electronic device according to another embodiment of the present invention; and

[0030] Fig. 14B yes Fig.14A A schematic diagram of three-axis values ​​when the electronic device of an embodiment is moved.

[0031] Description of Reference Numerals

[0032] 600 Electronic Devices

[0033] 61 Thermal Sensor

[0034] 62 Image Sensor

[0035] 63 Processor

[0036] 64 Storage

[0037] 65 Gesture Sensor

[0038] 66 Wireless Communicator

[0039] 67 Triaxial Accelerometer

[0040] Ih Thermal Image

[0041] Im Image Frame

[0042] Sg gesture detection signal

[0043] Swls wireless signal

[0044] S_3ax three-axis acceleration value DETAILED DESCRIPTION

[0045] Please refer to Figure 1 , which is a block diagram of a recognition system 100 according to an embodiment of the present invention. The recognition system 100 can be applied to portable devices, such as electronic devices including mobile phones, tablet computers, and laptop computers; and can be applied to wearable devices, such as electronic devices including watches, bracelets, and armbands, but not limited thereto. The recognition system 100 includes a thermal sensor 11, an image sensor 12, a processor 13, and a storage 14. The recognition system 100 can be used for at least face recognition and gesture recognition.

[0046] The thermal sensor 11 includes, for example, a pyroelectric infrared (PIR) type, a thermopile type, or a bolometer type sensor, which can detect infrared rays and output an electrical signal (e.g., a voltage or current signal) or a digital signal to reflect the detected temperature. The thermal sensor 11 preferably outputs a two-dimensional thermal image to correspond to the two-dimensional image frame acquired by the image sensor 12. For example, the detection value of each pixel of the two-dimensional thermal image represents the temperature of the detected area, and the detected area is partially overlapped or non-overlapped with each other relative to adjacent pixels of the two-dimensional thermal image according to the microlens configuration thereon.

[0047] Compared to conventional temperature sensors that are in contact with the detected object when performing heat detection or temperature detection, the thermal sensor 11 of the present invention can detect temperature without contacting the object because the thermal sensor 11 uses a thermopile type or a thermistor type sensor. In other words, the thermal sensor 11 of the present invention can still detect the target temperature even if the target (such as a human body) is covered by clothes or cloth, thereby improving reliability and applicability.

[0048] The image sensor 12 includes, for example, a CCD image sensor or a CMOS image sensor, which has pixels arranged in an array for outputting a two-dimensional image frame.

[0049] The processor 13 is, for example, a digital signal processor (DSP), a microcontroller (MCU), a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a graphics processing unit (GPU), which is electrically connected to the thermal sensor 11 and the image sensor 12 to receive the thermal image Ih and the image frame Im respectively and perform post-processing using software and / or hardware. The processor 13 also controls the opening and closing of the thermal sensor 11 and the image sensor 12 and the operation of their pixels.

[0050] The storage 14 includes, for example, volatile storage and / or non-volatile storage, which is used to record in advance the algorithms, thresholds and parameters used by the processor 13 for post-processing. In different applications, the storage 14 can also temporarily store the data of the thermal image Ih and / or image frame Im when the device is operating.

[0051] Please refer to Figure 2A , which is a schematic diagram of a face recognition system 200 according to a first embodiment of the present invention. Figure 2A The face recognition system 200 is shown to be disposed on a side of the portable device facing the user. The face recognition system 200 includes: Figure 1 The thermal sensor 11, image sensor 12, processor 13 and storage 14 are shown. The thermal sensor 11 is used to obtain a thermal image Ih (such as Figure 2C The image sensor 12 is used to obtain an image frame Im (such as Figure 2B The first field of view FOV1 and the second field of view FOV2 overlap at least in part to obtain corresponding features, so that different sensors can be used to obtain information of the same area or surface. The thermal sensor 11 and the image sensor 12 can obtain images simultaneously or in different time periods without any specific limitation.

[0052] The processor 13 is used to perform face recognition and / or material recognition according to the image frame Im, wherein the processor 13 may use a known face recognition algorithm to recognize a face image (such as Figure 2B The processor 13 also performs live body recognition based on the thermal distribution of the area in the thermal image Ih corresponding to the facial features of the human face image in the image frame Im.

[0053] In the image unlocking system, in order to prevent unscrupulous persons from using a photo or video of a registered face to unlock, the face recognition system 200 of the first embodiment of the present invention can distinguish the imitations based on the thermal image Ih obtained by the thermal sensor 11 and the skin texture of the face in the image frame Im. Therefore, the living body in the present invention refers to a real person rather than a photo or video.

[0054] For example, in a non-limiting implementation, when the processor 13 recognizes that the image frame Im does not contain a registered face image or the registered face image does not have skin texture, the thermal sensor 11 is not turned on to reduce energy consumption. That is, the processor 13 turns on the thermal sensor 11 only when it is recognized that the image frame Im contains a registered face image or the registered face image has skin texture, but the present invention is not limited thereto.

[0055] In another non-limiting embodiment, the processor 13 can also reversely control the thermal sensor 11 and the image sensor 12, that is, the processor 13 does not turn on the image sensor 12 before sensing that the area of ​​the object image in the thermal image Ih is greater than the threshold. Only when the thermal image Ih contains a valid face image (that is, the area of ​​the object is greater than the threshold), the processor 13 turns on the image sensor 12 for face recognition. In other embodiments, during the unlocking process, the thermal sensor 11 and the image sensor 12 can also be turned on at the same time.

[0056] In the first embodiment, the first field of view FOV1 of the thermal sensor 11 preferably covers a larger range than the second field of view FOV2 of the image sensor 12. In addition, since the processor 13 performs liveness recognition based on the thermal image Ih, the processor 13 only needs to perform two-dimensional face recognition based on the image frame Im, and does not need to perform three-dimensional face recognition, so as to reduce energy consumption. Although it is known that three-dimensional face recognition can be used to distinguish between photos and real people, it requires a higher amount of calculation.

[0057] In addition, in order to further prevent unscrupulous elements from using heated photos to unlock, the processor 13 is not simply used to confirm whether the thermal image Ih contains an image of an object with a temperature greater than a predetermined temperature, but is also used to identify the regional thermal distribution in the thermal image Ih. Figure 2C As shown, the thermal image Ih includes a high temperature area and a low temperature area. For example, the high temperature area corresponds to the face image in the image frame Im (eg Figure 2B The low temperature area corresponds to the nose area of ​​the face image in the image frame Im, while the low temperature area corresponds to the eye and mouth areas of the face image in the image frame Im.

[0058] At this time, the storage 14 is used to pre-record the temperature distribution of each area of ​​the face, which can be obtained by entering the setting mode (such as executing application software or pressing a button) to detect and store the registered user, or can be pre-stored according to statistical results before leaving the factory. The processor 13 compares (such as calculating similarity or correlation) the regional thermal distribution in the current thermal image (such as the thermal image Ih obtained during the unlocking process) with the pre-stored temperature distribution for live body recognition. In addition, the processor 13 calculates the temperature difference between the high temperature area and the low temperature area to confirm that the face recognition system 200 is currently detecting a human body, not other disguises.

[0059] In another embodiment, the regional thermal distribution in the current thermal image is compared with the positions of the facial features (e.g., eyes, nose, mouth) identified in the acquired image frame Im to confirm whether the regional thermal distribution matches the corresponding facial features. In this way, it is still possible to distinguish between a real face and a disguised object without using the temperature distribution pre-recorded in the memory 14.

[0060] When the processor 13 completes the registered face confirmation according to the face recognition, it turns on the operating function of the electronic device configured with the face recognition system 200, such as turning on the display screen.

[0061] Please refer to Figure 3A , which is a schematic diagram of a gesture recognition system 300 according to a second embodiment of the present invention. Figure 3A The gesture recognition system 300 is disposed on a side of the portable device facing the user. The gesture recognition system 300 also includes Figure 1 The thermal sensor 11, image sensor 12, processor 13 and storage 14 are shown. The thermal sensor 11 is used to obtain a thermal image Ih with a first field of view FOV1 and output it to the processor 13. The image sensor 12 is used to obtain an image frame Im with a second field of view FOV2 (such as Figure 3B as shown) and output to processor 13.

[0062] The processor 13 is used to determine the image processing range WOI according to the thermal image Ih, and perform gesture recognition only according to the image area in the image frame Im corresponding to the image processing range WOI determined in the thermal image Ih, so as to eliminate the influence of ambient light. Therefore, in order to enable the processor 13 to correctly determine the image processing range WOI in the image frame Im, in the second embodiment, the first field of view FOV1 of the thermal sensor 11 is preferably the same as the second field of view FOV2 of the image sensor 12, and the size of the thermal image Ih and the image frame Im is preferably substantially the same. For example, the corresponding image processing range WOI in the image frame Im can be determined according to the pixel address or position within the image processing range WOI determined in the thermal image Ih.

[0063] For example, the processor 13 first determines the range in the thermal image Ih where the temperature is greater than the temperature threshold (determined according to the human body temperature) as the image processing range WOI, which is the image area in the thermal image Ih. Since the first field of view FOV1 and the second field of view FOV2 are configured to correspond, the processor 13 can confirm the corresponding image processing range WOI in the image frame Im, such as Figure 3B As shown, the size of the corresponding image processing range WOI is smaller than the size of the image frame Im. It must be noted that the image processing range WOI is not limited to Figure 3B The displayed rectangle may be of other appropriate shapes as long as it can include the object region in the image frame Im. In some implementations, two image processing areas WOI may be defined based on two object regions.

[0064] In other embodiments, the first field of view FOV1 and the second field of view FOV2 may not be completely the same but have an angle difference. In this case, as long as the storage 14 pre-stores the space conversion algorithm or matrix conversion algorithm of the thermal image Ih and the image frame Im, the processor 13 can confirm the corresponding image processing range WOI in the image frame Im according to the stored algorithm after confirming the image processing range WOI in the thermal image Ih.

[0065] In another non-limiting implementation, the processor 13 may also first identify the object image in the image frame Im, which may include the ambient light image. The processor 13 then uses the thermal image Ih to exclude non-human images, such as object images outside the image processing range WOI are not used for gesture recognition, thereby effectively improving recognition accuracy and eliminating interference.

[0066] More specifically, in the second embodiment, the processor 13 performs gesture recognition based on a partial image region in the image frame Im, and the thermal image Ih is used to provide a denoising function.

[0067] In addition, in order to save energy, the processor 13 turns on the image sensor 12 only after recognizing that the image processing range WOI in the thermal image Ih is larger than a predetermined size. In this embodiment, the entire sensing array of the image sensor 12 is turned on, or only a portion of the sensing array corresponding to the image processing range WOI is turned on. In other words, when the thermal image Ih does not contain an area greater than a predetermined temperature threshold, the processor 13 only turns on the thermal sensor 11 to continuously acquire the thermal image Ih at a predetermined frequency; or, although the thermal image Ih contains an area greater than a predetermined temperature threshold, if the area is smaller than a predetermined size (which is determined, for example, according to the hand size within the detectable distance of the system), the processor 13 still only turns on the thermal sensor 11 to continuously acquire the thermal image Ih at a predetermined frequency, but the present invention is not limited thereto.

[0068] In other embodiments, during the gesture recognition process, the thermal sensor 11 and the image sensor 12 may be turned on at the same time. For example, only some pixels of the thermal sensor 11 are turned on, and the pixels correspond to the pixel area where the image sensor 14 detects the object. In more detail, the sensing arrays of the thermal sensor 11 and the image sensor 12 do not need to be turned on in their entirety, but only some pixels are turned on to reduce energy consumption.

[0069] In another embodiment, the processor 13 first performs material recognition based on the image frame Im acquired by the image sensor 11, and then performs gesture recognition based on the thermal image Ih acquired by the thermal sensor 11. For example, if the processor 13 does not determine the image of an object with skin material in the image frame Im, the thermal sensor 11 is not turned on. The thermal sensor 11 is turned on only after the skin material is determined in the image frame Im. In addition, the processor 13 also determines the image processing range WOI in the thermal image Ih based on the skin material area in the image frame Im, that is, the processor 13 first determines the skin material area in the image frame Im, and then determines the image processing range WOI corresponding to the skin material area in the thermal image Ih. Gesture recognition is performed only based on the object image within the image processing range WOI in the thermal image Ih.

[0070] It must be noted that, although the identification system 100 is described as being applied to a portable device in the first and second embodiments, the present invention is not limited thereto. The identification system 100 of the first and second embodiments can also be applied to a wearable device, a door access control or a vehicle security system and / or a control system. The processor 13 can use the thermal image Ih for live body identification and denoising to improve the judgment accuracy and safety.

[0071] Please refer to Figure 4 , which is a schematic diagram of a medical monitoring system 400 according to a third embodiment of the present invention. The medical monitoring system 400 is applied to, for example, a medical institution or a nursing institution to solve the problem of insufficient manpower. The medical monitoring system 400 mainly includes a wearable accessory 41 and a central computer system 43 coupled to each other. The wearable accessory 41 is used to be worn on the human body, such as Figure 4 The wearable accessory 41 is shown as being worn on an arm, but is not limited thereto, and can also be worn on other parts of the body suitable for measuring body temperature. The central computer system 43 reacts accordingly to the detection result of the wearable accessory 41, such as providing a warning.

[0072] For example, the wearable accessory 41 may be a customized accessory, a smart watch, a smart armband or a smart bracelet, etc., which at least includes a thermal sensor 411, a processor 412 and a transmitter 413. The thermal sensor 411 may output a two-dimensional thermal image similar to the thermal sensors of the first and second embodiments described above, and the processor 412 calculates its average temperature. In addition, in the third embodiment, the thermal sensor 411 may only include a single detection unit (such as a photodiode) and output an electrical signal or a digital signal at a time to represent the detected temperature, instead of outputting a two-dimensional thermal image. The processor 412 may also be a digital signal processor, a microcontroller, a central processing unit, a specific application integrated circuit or a graphics processor.

[0073] In the example where the thermal sensor 411 is disposed on other electronic devices that are not in direct contact with the user (for example, the electronic device is disposed on a wall or ceiling), the thermal sensor 411 monitors the user's whole body temperature. When the temperature difference between the core temperature and the limb temperature is greater than a predetermined threshold, the electronic device provides a warning message Sw.

[0074] When measuring body temperature, the thermal sensor 411 directly detects the skin surface temperature as the body temperature, or detects the temperature difference between the room temperature and the body temperature (that is, the room temperature and the body temperature are detected simultaneously using the same or different sensors), and deducts the temperature difference from the room temperature (for example, using the processor 412) to obtain the body temperature.

[0075] The thermal sensor 411 is used to detect the body temperature of the human body and output an electrical signal or a digital signal to the processor 412. After the processor 412 determines the temperature according to the received signal, it controls the transmitter 413 ( Figure 4 For example, the antenna sends the temperature information St related to the human body temperature and the number information S of the wearable accessory 41 ID (For example, the medical monitoring system 400 may include multiple wearable accessories 41 at the same time and each accessory has its own number), for example, wireless communication methods such as Bluetooth communication, Zigbee, microwave communication, etc. are used for transmission.

[0076] In one embodiment, the central computer system 43 is configured to receive the temperature information St and the number information S ID In another embodiment, the central computer system 43 may include a plurality of receivers configured in different spaces for receiving temperature information St and number information S from different patients. ID , the host of the central computer system 43 is electrically connected to these receivers.

[0077] When the human body temperature represented by the received temperature information St exceeds the predetermined range, the central computer system 43 sends a message corresponding to the number information S ID Related warning information Sw, wherein the number information S ID Correlation means that the warning information Sw is related to the number information S sent by the wearer. ID In one embodiment, the warning information Sw is transmitted by means of a light signal, a broadcast, etc. In another embodiment, the central computer system 43 further includes a transmitter (not shown) to wirelessly send the warning information Sw to a portable device 49, which is carried by medical personnel, for example.

[0078] In a non-limiting embodiment, the central computer system 43 further includes a display 431 (such as a liquid crystal display or a plasma display) for displaying the time distribution of the human body temperature for the convenience of medical staff. The display 431, for example, displays or indicates a high temperature threshold TH1 and a low temperature threshold TH2. When the central computer system 43 determines that the human body temperature detected by the thermal sensor 411 exceeds a predetermined range (for example, exceeds the high temperature threshold TH1 or the low temperature threshold TH2), the warning message Sw is issued. The thresholds TH1 and TH2 can be set or adjusted according to different detected users.

[0079] In a non-limiting embodiment, the central computer system 43 further includes a camera 45. When it is determined that the body temperature of the human body exceeds the predetermined range, the central computer system 43 turns on the camera 45 to monitor the patient. The camera 45 is usually turned off to maintain the privacy of the patient. The image obtained by the camera 45 can also be displayed on the display 431.

[0080] In a non-limiting embodiment, the central computer system 43 further includes a drug dispensing device 47. When it is determined that the human body temperature exceeds the predetermined range, the central computer system 43 turns on the drug dispensing device 47 to automatically dispense the drug. The operating state of the drug dispensing device 47 can also be displayed on the display 431 (if included). The display 431 can simultaneously display the relationship between the dosage and the temperature change. The central computer system 43 can also control the drug dispensing device 47 to stop dispensing the drug when the body temperature returns to the predetermined range.

[0081] Although the above description indicates that the medical monitoring system 400 is applied to an operating organization, the present invention is not limited thereto. The medical monitoring system 400 of the third embodiment can also be used for home care, as long as a tablet computer, desktop computer or laptop computer replaces the operation of the central computer system 43.

[0082] Although the third embodiment is described by taking the wearable accessory 41 as an example, it is only for illustration and not for limiting the present invention. In other embodiments, the wearable accessory 41 can be replaced by other monitoring devices that do not directly contact the human body. The thermal sensor 411 detects body temperature in a non-contact manner.

[0083] Please refer to Figure 5 , which is a schematic diagram of a body temperature monitoring device 500 according to a fourth embodiment of the present invention. The body temperature monitoring device 500 comprises a wearable accessory 51, a thermal sensor 511, a processor 512, a display 513 and a storage 514. The processor 512 can also be a digital signal processor, a microcontroller, a central processing unit, an application-specific integrated circuit or a graphics processor.

[0084] The wearable accessory 51 is used to be worn on the human body, such as a watch, a bracelet or an armband, which can be roughly attached to and fixed to the detected skin surface, without any specific restrictions. The thermal sensor 511 is configured on the wearable accessory 51, and is used to detect the basal body temperature (BBT) of the human body, and output an electrical signal or a digital signal to the processor 512. The processor 512 is used to record the basal body temperature every day, and when the temperature change of the basal body temperature exceeds the temperature change threshold (for example, 0.3 to 0.5 degrees, which is pre-recorded in the storage 514), the display 513 is controlled to prompt.

[0085] For example, the processor 512 controls the thermal sensor 511 to automatically detect the basal body temperature at a fixed time every day, for example, based on the system clock. Alternatively, the body temperature monitoring device 500 further includes a button 515. When the processor 512 receives a pressing signal of the button 515, the thermal sensor 511 is controlled to detect the basal body temperature and display it on the display 513 and store it in the storage 514 for long-term monitoring.

[0086] The display 513 can provide various information in the form of icons or numbers, such as the ovulation period or date (e.g., displayed as yyyy / m / d), the high temperature period (e.g., Figure 5 Displays the number of days when the basal body temperature is around 36.8 degrees, which varies according to different users) and during low temperature periods (e.g. Figure 5 Displays the number of days when the basal body temperature is around 36.4 degrees, which depends on different users) and any other information that helps understand the physiological period.

[0087] Since the basal body temperature is preferably measured when the user wakes up but before getting out of bed, in order to achieve the purpose of automatic measurement, the body temperature monitoring device 500 also includes an acceleration detector (such as a G-sensor) for detecting whether the user gets up from the bed. For example, before getting up, the acceleration detector only detects acceleration in two dimensions (such as the XY axis), but after getting up from the bed, the acceleration in the third dimension (such as the Z axis) can be detected. The processor 512 is also used to determine the time to get up (but not get out of bed) based on the acceleration signal detected by the acceleration detector, and control the thermal sensor 511 to automatically detect the basal body temperature at the time to get up. The so-called detection of acceleration refers to the detection of an acceleration value greater than a predetermined threshold.

[0088] In a non-limiting embodiment, when the processor 512 detects that the user is lying in bed (for example, no Z-axis or other acceleration is detected within a predetermined period), the thermal sensor 511 is controlled to detect the temperature once every predetermined period (for example, 1 to several minutes). Only the detected temperature before the acceleration in the Z-axis is detected is regarded as the basal body temperature by the processor 512 and temporarily stored in the storage 514. In order to increase the judgment accuracy, when the processor 512 does not detect another Z-axis acceleration again within a predetermined period after detecting an acceleration in the Z-axis, it means that the user just changes the posture in bed, and the temporarily stored detected temperature is not regarded as the basal body temperature.

[0089] In a non-limiting embodiment, the body temperature monitoring device 500 can be wirelessly connected to another thermal sensor, which includes a wireless transceiver and a processor (e.g., a digital processor). The other thermal sensor is arranged near the user or the bed. When the body temperature monitoring device 500 detects the acceleration in the Z-axis direction, it sends a request signal to notify the other thermal sensor, and the processor of the other thermal sensor can identify (e.g., use the hardware and / or software therein to determine the change of the high temperature area in the acquired data) whether the user gets up. If so, the other thermal sensor sends a response signal to the body temperature monitoring device 500, so that it regards the body temperature measured before the user gets up as the basal body temperature. If not, the other thermal sensor does not send a response signal or sends a response signal to indicate that the body temperature does not need to be measured at this time.

[0090] In a non-limiting embodiment, when the processor 512 detects that the user shakes the body temperature monitoring device 500 in a predetermined pattern (for example, shaking it left to right or up and down several times), it starts measuring the body temperature and records it as the basal body temperature.

[0091] It is understood that the numerical values ​​of the above embodiments are only used for illustration rather than for limiting the present invention.

[0092] It must be noted that, although the above-mentioned recognition system 100 is described by taking the example of including both the thermal sensor 11 and the image sensor 12, the present invention is not limited thereto. In other embodiments, the recognition system 100 includes only one of the thermal sensor 11 and the image sensor 12, and receives another signal (image frame Im or thermal image Ih) from the external sensor through its input / output interface. For example, the recognition system 100 includes the thermal sensor 11 but receives the image frame Im from the external image sensor; or, the recognition system 100 includes the image sensor 12 but receives the thermal image Ih from the external thermal sensor.

[0093] Please refer to Figure 6, which is a block diagram of an electronic device 600 according to another embodiment of the present invention. The electronic device 600 can be controlled by gestures and is, for example, an electronic door lock, a car door lock, an electronic safe, and a personal electronic product that can use a digital electronic lock, but is not limited to the examples in the present invention.

[0094] The electronic device 600 includes a thermal sensor 61 , an image sensor 62 , a processor 63 and a storage 64 , which are respectively the same as the thermal sensor 11 , the image sensor 12 , the processor 13 and the storage 14 described above, and thus will not be described in detail herein.

[0095] The processor 63 is used to perform unlocking judgment and gesture judgment according to the image frame Im acquired by the image sensor 62. In the present invention, the processor 63 can also perform gesture judgment according to other methods, such as millimeter wave, flight time, etc. At this time, the electronic device 600 also includes a gesture sensor 65 for outputting a gesture detection signal Sg.

[0096] In one implementation, when the processor 63 performs gesture determination based on the image frame Im acquired by the image sensor 62 , the gesture sensor 65 is the image sensor 62 and the gesture detection signal Sg is the image frame Im.

[0097] In another embodiment, when the processor 63 performs gesture determination based on millimeter waves, the gesture sensor 65 is a millimeter wave radar and the gesture detection signal Sg is an output signal of the millimeter wave radar.

[0098] In another embodiment, when the processor 63 performs gesture determination based on the time of flight, the gesture sensor 65 is a time of flight sensor (eg, including a single photon collapse diode array) and the gesture detection signal Sg is an output signal of the time of flight sensor, such as a surge count value.

[0099] The method of using millimeter waves and flight time to determine gestures is well known and will not be described in detail here.

[0100] Please refer to Figure 7As shown, it is a flowchart of an unlocking method of an electronic device 600 according to another embodiment of the present invention, comprising the following steps: motion detection (step S70); face detection (step S71); face occlusion detection (step S72); gesture detection (step S73) and whether it meets the predetermined gesture combination (step S74); if it meets, unlock (step S75); if it does not meet, store / transmit the face image (step S76). It can be understood that in order to execute the present unlocking method, the processor 63 has built-in (software, firmware and / or hardware) face recognition algorithm and gesture recognition algorithm, which are known and will not be described here. In one embodiment, the face recognition algorithm only needs to identify whether it is a face, and it is not necessary to identify the face identity.

[0101] Step S70: To save energy, the processor 63 does not execute the face recognition algorithm and the gesture recognition algorithm before the image sensor 62 detects motion. In one embodiment, the processor 63 can detect motion based on the image frames Im sequentially output by the image sensor 62. In another embodiment, the pixel circuit of the image sensor 62 can be used directly to detect motion. The method of using the pixel circuit to detect motion can be, for example, referred to U.S. Patent Application No. 17 / 009,417, which is jointly owned by the same assignee in this case, and the application date is September 1, 2020, and the invention name is "Pixel circuit capable of outputting time signals and performing analog calculations", all of which are incorporated herein by reference.

[0102] When motion is detected, the process proceeds to step S71.

[0103] For example, before detecting a motion, the image sensor 62 acquires image frames Im at a first frame rate; after detecting a motion, the image sensor 62 acquires image frames Im at a second frame rate higher than the first frame rate.

[0104] Step S71: Next, the processor 63 may identify (for example, using a face detection algorithm) a face image according to the image frame Im output by the image sensor 62. When it is identified that the image frame Im contains a face image, the process proceeds to step S72. When no face image is detected, the process returns to step S70.

[0105] Step S72: In the present invention, a complete facial image is set as a condition for unlocking. Therefore, the processor 63 then determines whether the facial image in the image frame Im is blocked, such as by sunglasses, masks, etc., covering a part of the face, such as the eyes, nose, mouth, etc. When the face is blocked, the processor 63 records the facial image for user confirmation, such as storing it in the memory 64 or transmitting it to the user's mobile device, that is, directly entering step S76; at the same time, the processor 63 does not recognize gestures and gesture combinations, that is, does not enter steps S73 and S74. When the face is not blocked, it enters step S73.

[0106] Step S73: Next, the processor 63 determines whether a gesture is detected according to the gesture detection signal Sg of the gesture sensor 65. As described above, the gesture sensor 65 is the image sensor 62 or a millimeter wave radar or a time-of-flight sensor is used, and there is no specific limitation. That is, when the image sensor 62 is not used to detect gestures, the image sensor 62 is only used to detect actions and obtain facial images. If no gesture is detected, the process proceeds to step S76 to store / transmit the facial image for confirmation by the device owner. If a gesture is detected, the process proceeds to step S74.

[0107] Step S74: In the present invention, the processor 63 determines whether to unlock the electronic device 600 by whether a series of gestures conform to a predetermined sequence / variation, which is referred to as a gesture combination. When the gesture combination conforms to a predetermined condition (e.g., sequence, variation, etc.), the process proceeds to step S75 to unlock the electronic device 600. The functions that are restricted from being executed by the electronic device 600 in the locked state depend on different operating systems (OS) and are not specifically limited, but at least the functions that can be executed must be able to be executed. Figure 7 Unlock method.

[0108] In one embodiment, the gesture combination includes a static single-hand gesture, an operating hand (left hand or right hand), a gesture position, and a gesture dwell time. Fig. 8A The display gesture combination includes: a first time interval in which a user's right hand (displaying R) extends a single finger and remains stationary in the upper left corner pixel area for 3 seconds; a second time interval in which a user's left hand (displaying L) extends three fingers and remains stationary in the central pixel area for 1 second. The lengths of the first time interval and the second time interval are preferably determined in advance.

[0109] In another embodiment, the gesture combination includes a combination of a static two-hand gesture, a static one-hand gesture, an operating hand, a gesture position, and a gesture dwell time. Figure 8B The display gesture combination includes: in the first time interval, the user's right hand (displaying R) extends one finger and the left hand (displaying L) extends three fingers in different gestures and remains still for 3 seconds (the pixel area may be limited or not limited, and there is no specific restriction); in the second time interval, the user's right hand (displaying R) extends four fingers and the left hand (displaying L) extends two fingers and remains still for 1 second after changing the gesture (the pixel area may be limited or not limited, and there is no specific restriction); in the third time interval, the user's right hand (displaying R) extends the thumb and remains still in the pixel area in the lower right corner for 5 seconds. The lengths of the first time interval, the second time interval, and the third time interval are preferably determined in advance.

[0110] The gestures, positions and times of the above embodiments may be factory defined or set by the device owner, for example, Figure 8CThe gesture combination customized by the user is displayed, including: in the first time interval, the thumb and little finger of the user's left hand (displayed L) are extended and remain still in the middle pixel area on the left side for 3 seconds; in the second time interval, the user's right hand (displayed R) indicates an OK gesture and remains still in the middle pixel area on the right side for 3 seconds; in the third time interval, the right and left hands of the user are combined in a gesture, with the thumbs touching and the little fingers touching to form a heart shape and remain still for 5 seconds (the pixel area may be limited or not, and there is no specific restriction).

[0111] In this embodiment, the number of pixel regions divided by the field of view of the image sensor 62 is not limited to 9, and may be determined according to the size of the field of view and the operating distance.

[0112] In another embodiment, the gesture combination includes the dynamic change, position change, movement direction and movement speed of a single-hand gesture. Fig.8D and Fig. 8E The display gesture combination includes: in the first time interval T1, the user's hand (the operating hand may be limited or not limited, and there is no specific restriction) is located in the pixel area in the upper right corner with five fingers open ( Fig.8D ) or only partially open your fingers ( Fig. 8E ); in the second time interval T2, the user's hand is located in the upper left pixel area and the five fingers are combined into a fist; in the third time interval T3, the user's hand is located in the lower left pixel area and the five fingers are combined into a fist; and in the fourth time interval T4, the user's hand is located in the lower right pixel area and the five fingers are combined into a fist, and the reverse can also be done. In addition, it can also be as follows FIG. 8A to FIG. 8C The gesture dwell time in different pixel areas is combined to form other implementations.

[0113] For example, Figure 8F The displayed gesture combination includes the rotation direction (clockwise or counterclockwise), number of rotations, and rotation speed of the predetermined gesture (here only the index finger is extended), etc. This embodiment can be configured to distinguish the operating hand or not, without specific limitation.

[0114] It is understood that static gestures, dynamic gestures, operating hands, gesture positions, and gesture dwelling times are not limited to those listed in the present invention.

[0115] At the same time, in order to allow the user to know the gesture position and the gesture dwell time, the electronic device 600 of this embodiment may also include a display or be wirelessly coupled to a display (such as a display on a mobile device, wirelessly connected to the electronic device 600) for displaying the gesture position and the gesture dwell time.

[0116] When the gesture combination recognized by the processor 63 does not meet the predetermined condition, the process proceeds to step S76 to store / transmit the face image acquired by the image sensor 62 for confirmation by the device owner.

[0117] In order to further enhance the strength of the digital electronic lock, the electronic device 600 of this embodiment further includes a thermal sensor 61 for outputting a thermal image Ih, and the processor 63 is further used to determine whether the temperature of the face and the temperature of the hands meet the expected standards based on the thermal image Ih, that is, to perform liveness recognition.

[0118] For example Figure 7 In step S72, when the processor 63 recognizes that the face is not obstructed, it first determines whether the face temperature is within a predetermined range (for example, determined according to the results of previous measurements) based on the thermal image Ih. If the face temperature is not within the predetermined range, it proceeds to step S76; if the face temperature is within the predetermined range, it proceeds to step S73. For example, in step S73, when the processor 63 detects a gesture, it first determines whether the hand temperature is within a predetermined range (for example, determined according to the results of previous measurements) based on the thermal image Ih. If the hand temperature is not within the predetermined range, it proceeds to step S76; if the hand temperature is within the predetermined range, it proceeds to step S74. The method of determining the temperature of an object based on a thermal image is known, so it will not be described in detail here.

[0119] Please refer to Fig. 9 As shown, it is a flowchart of a gesture recognition method of an electronic device 600 according to another embodiment of the present invention, comprising the following steps: action detection (step S90); hand image detection (step S91); hand confirmation (step S92); gesture judgment (step S93); feedback detection prompt (step S94); and executing the corresponding operation of the gesture (step S95).

[0120] Similarly, the image sensor 62 is used to detect actions and obtain image frames Im containing hand images. Fig. 10A As shown, the image frame Im includes not only the hand image but also other environmental images, which may be recognized by the processor 63. Therefore, the gesture recognition method of this embodiment increases the accuracy of gesture recognition and eliminates environmental interference by combining thermal sensors, spatial information of the wearable device and / or specific patterns of accessories.

[0121] Step S90: To save energy, before the image sensor 62 detects any motion, the processor 63 does not execute the gesture recognition algorithm and turns off the illumination light source, such as an infrared light emitting diode (not shown), but not limited thereto. Motion detection has been described above, so it will not be repeated here. When the processor 63 confirms that there is motion (as described above, it can be calculated based on the image frame or based on the calculation result of the pixel array), it enters step S91.

[0122] Step S91: Since this method is mainly used for gesture control, the processor 63 then determines whether there is a hand image in the image frame Im (for example, using a hand recognition algorithm, implemented in software, firmware and / or hardware). In this step, the processor 63 does not recognize gestures but only determines the hand object in the image frame Im. In order to improve the accuracy of the hand object determination, in this step, the processor 63 can control the illumination light source to light up. If the hand image is not detected or acquired, return to step S90; if the hand image is detected or acquired, proceed to step S92.

[0123] Steps S92 to S93: As described above, since gesture control is easily affected by environmental interference, the gesture recognition method of the present invention uses at least one of a thermal sensor, a wearable device, and an accessory to determine a region of interest (WOI) in the image frame Im, that is, to perform hand confirmation.

[0124] Please refer to Fig. 10B As shown, the processor 63 receives the thermal image Ih from the thermal sensor 61 and determines the image processing area in the thermal image Ih and confirms the corresponding image processing range WOI in the image frame Im. The way in which the processor 63 determines the image processing area and the corresponding image processing range WOI has been described in Figure 3B And the related instructions, so no more details are given here.

[0125] In one embodiment, when the processor 63 determines two image processing ranges according to the thermal image Ih, for example Fig. 10C The image processing range WOI_1 corresponding to the hand and the image processing range WOI_2 corresponding to the face are displayed, and the processor 63 is configured to select one of them for gesture recognition. Since the hand is usually closer to the electronic device 600 when performing gesture operation, the temperature represented by the image processing range WOI_1 is usually higher than the temperature represented by the image processing range WOI_2. The processor 63 is configured to perform gesture recognition according to the image processing range corresponding to the higher temperature in the image frame Im. In addition, the motion vector of the image processing range WOI_1 is smaller and the motion vector of the image processing range WOI_2 is larger. The processor 63 can also select one of the image processing ranges according to the temperature and the motion vector at the same time.

[0126] Please refer to Fig.11 As shown, when performing gesture control, the user can wear accessories with specific patterns on their hands. For example, Fig.11The display accessory is a ring 111A on a finger, a bracelet or a watch 111B on a wrist, but is not limited thereto. The specific pattern may be a pattern or shape on an accessory, or an arrangement pattern formed when the light source thereon emits light, and there is no specific limitation. Therefore, the processor 63 may recognize the specific pattern according to the image frame Im, and perform gesture recognition according to the hand image of the image frame Im and the specific pattern (for example, using a built-in gesture recognition algorithm).

[0127] Also refer to Fig. 10C and Fig.11 As shown, when two image processing ranges, such as WOI_1 and WOI_2, are determined based on the thermal image Ih, since the accessories 111A and / or 111B are worn on the hands, the processor 63 is configured to perform gesture recognition based on the image processing range (in this case WOI_1) in the image frame Im that is closer to the specific pattern.

[0128] Please refer to Fig.12 As shown, when performing gesture recognition, the user can wear a wearable device 121 that can emit wireless signals, wherein the wearable device 121 can communicate wirelessly with the electronic device 600, such as Bluetooth communication, Wi-Fi communication, wireless radio frequency identification (RFID), etc., without specific restrictions. In this embodiment, the electronic device 600 also includes a wireless communicator 66 for wireless connection, such as a Bluetooth chip, a Wi-Fi chip, or an RFID chip. The wireless communicator 66 is used to receive spatial information from the wearable device 121.

[0129] The processor 63 may perform the gesture recognition based on the hand image, the specific pattern and the spatial information. The spatial information may be, for example, three-dimensional position information (in an implementation mode in which the processor 63 can determine the three-dimensional position based on a wireless signal) or distance (displayed as depth) information (in an implementation mode in which the processor 63 can determine the distance based on the strength of a wireless signal).

[0130] Step S94: In an embodiment where the user wears the wearable device 121, the processor 63 also notifies the wearable device 121 of the completion information through the wireless communicator 66 after completing the gesture recognition. The wearable device 121 executes the feedback prompt after receiving the completion information. For example, when the wearable device 121 is a watch, bracelet, glove, ring, etc., the feedback prompt may be a vibration; when the wearable device 121 is glasses, contact lenses, virtual reality equipment, etc., the feedback prompt may be an image or text message; when the wearable device 121 is a headset, necklace, virtual reality equipment, etc., the feedback prompt may be a sound message.

[0131] In addition, the wearable device 121 is not limited to performing feedback prompts by itself, but can also perform feedback prompts through other devices wirelessly connected to it. For example, the wearable device 121 can be wirelessly connected to a vehicle device, such as a steering wheel, a driving central system, a head-up display, a car audio system, etc., and can generate vibrations, images, texts, sounds, etc. as prompts by sending signals to the vehicle device.

[0132] In the present invention, step S94 can be implemented selectively, which is determined based on whether the user wears the wearable device 121 on his hand and whether the wearable device 121 performs wireless communication with the electronic device 600.

[0133] Step S95: Finally, the processor 63 controls the electronic device 600 to execute a corresponding operation of the recognized gesture or the above-mentioned gesture combination. The corresponding operation is determined according to the operating system configuration of the electronic device 600 and is not specifically limited.

[0134] Since the embodiment of the present invention adds a hand confirmation mechanism (steps S92 to S93), environmental interference can be greatly reduced.

[0135] For example, in an implementation manner in which the user wears accessory 111A or 111B, when the processor 63 determines that the pixel distance between the hand image in the image frame Im and the specific pattern exceeds a predetermined distance, it indicates that the hand image is not the hand of the current user, and therefore the corresponding operation of step S95 is not performed; or, when the processor 63 determines that the positional relationship between the hand image in the image frame Im and the specific pattern is inconsistent (for example, the moving direction of the hand image is different from the moving direction of the specific pattern, or the size of the hand image and the specific pattern are displayed at different distances), it indicates that the hand image is not the hand of the current user, and therefore the corresponding operation of step S95 is not performed.

[0136] For example, in an embodiment where the user wears and uses the wearable device 121, the processor 63 does not perform the corresponding operation of step S95 when it is determined that the positional relationship between the hand image and the spatial information is inconsistent. The processor 63 can use conventional technology to calculate the spatial information (including three-dimensional position or depth, etc.) using the wireless signal provided by the wearable device 121. The present invention is to compare whether the positional relationship between the spatial information and the hand image is consistent, such as whether the moving direction is the same and whether the distance from the electronic device 600 is consistent (because it is worn on the hand), to distinguish between the user's hand and environmental interference.

[0137] In one embodiment, when accessory 111A or 111B is also capable of wireless communication with the electronic device 600 (transmitting wireless signals to the electronic device 600), the processor 63 simultaneously determines whether the hand image is the hand of the current user based on the specific pattern and the positional relationship to further increase the accuracy of the gesture.

[0138] The current user mentioned in the above embodiment refers to the user who is using gestures to control the electronic device 600 .

[0139] Please refer to Fig.13 As shown, it is a flow chart of a mobile alert method for an electronic device 600 according to another embodiment of the present invention, which is mainly used to issue an alarm and record a facial image for subsequent use when the movable electronic device 600 is moved by a person in an unexpected manner. For example, the owner of the electronic device 600 can know who is trying to move the electronic device 600.

[0140] The motion warning method of this embodiment includes the following steps: detecting the three-axis values ​​(step S130); whether the value changes conform to a predetermined sequence (step S131); if they conform, entering step S132 to turn off the alarm; if they do not conform, sounding an alarm and notifying the user (step S135). In the embodiment where the electronic device 600 further includes an image sensor 62, the motion warning method includes the following steps: determining whether a face is detected (step S133) and storing / transmitting a face image (step S134).

[0141] Step S130: In this embodiment, a triaxial accelerometer 67 is mainly configured in the electronic device 600 to output a triaxial acceleration value S_3ax, for example Fig. 14B When the processor 63 receives the three-axis acceleration value S_3ax and determines that the acceleration value of at least one axis exceeds the threshold, it means that the electronic device 600 is moved, and then enters step S131. That is, if the electronic device 600 is not moved, the algorithm for determining the change order in the processor 63 is not executed.

[0142] Step S131: In order to detect the change sequence, the processor 63 needs to determine the acceleration values ​​S_3ax of the three axes in at least two time periods. Fig.14A The electronic device 600 is shown to have four faces A to D and four corners 1 to 4; Fig. 14B (a) shows the state of the three-axis values ​​when the electronic device 600 is not moved; Fig. 14B (b) shows the surface D ( Fig. 14B The changes of the three-axis values ​​when the rear side of the vehicle is lifted; Fig. 14B (c) shows the changes in the three-axis values ​​when the surface C of the electronic device 600 is lifted; Fig. 14B (d) shows the change of the three-axis values ​​when the corner 4 of the electronic device 600 is lifted. Fig.14A and Fig. 14B , when the three-axis value changes of other faces or angles being lifted can be understood, so it is not repeated here. The predetermined order can be any appropriate order defined in advance, such as Fig. 14B First, surface D is lifted, then surface C is lifted, and finally corner 4 is lifted, but the present invention is not limited to this.

[0143] Step S132: When the processor 63 determines that the change of the three-axis acceleration value S_3ax complies with the predetermined sequence, it means that the current user is a legitimate user, and therefore the alarm is closed (ie, no alarm is issued).

[0144] Step S135: When the processor 63 determines that the change of the three-axis acceleration value S_3ax does not conform to the predetermined sequence, it means that the current user is not a legitimate user, so an alarm is issued and the user is notified. The alarm can be issued by sound, light, etc., without specific restrictions. In some systems, the door is automatically locked when the alarm is issued. Notifying the user can, for example, send a message to the user's mobile device, without specific restrictions, as long as the user can be informed that the electronic device 600 is currently being moved.

[0145] Step S133: In the embodiment where the electronic device 600 includes the image sensor 62, the processor 63 further determines (eg, with a built-in face recognition algorithm) whether the image frame Im acquired by the image sensor 62 contains a face image. The processor 63 only determines the face and does not need to determine the identity of the face.

[0146] Step S134: If a face image is included, the processor 63 stores the face image in the memory 64 for later confirmation, or transmits the face image to the user's mobile device, such as a mobile phone. After storing / transmitting the face image, the processor 63 then controls the electronic device 600 to sound an alarm, that is, enter step S135. If a face image is not included, the processor 63 controls the electronic device 600 to directly sound an alarm and notify the user, that is, enter step S135. Steps S134 and S135 can be performed simultaneously.

[0147] This embodiment provides the device owner with a facial image of an illegal user to increase the security of the device. In this embodiment, steps S133 and S134 can be implemented selectively.

[0148] In summary, the recognition system and monitoring system using only image sensors have operational limitations, and more complex algorithms must be used to overcome such limitations. Therefore, the present invention further provides a face recognition system ( Figure 2A )、Gesture Recognition System( Figure 3A ), medical monitoring system( Figure 4 ) and body temperature monitoring device ( Figure 5 ), which overcomes the limitations of systems that only use image sensors by using thermal sensors, and can effectively improve the accuracy of the recognition system and the applicable scenarios of the monitoring system.

[0149] Although the present invention has been disclosed through the above examples, they are not intended to limit the present invention. Any person skilled in the art with ordinary knowledge in the art to which the present invention belongs can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the appended claims.

Claims

1. An electronic device, comprising: An image sensor, the image sensor is used to output an image frame for detecting a face image; a gesture sensor, the gesture sensor being configured to generate a gesture detection signal; and processor, which is used to Identify whether the face is covered according to the face image, When the face is not covered, the gesture combination is identified according to the gesture detection signal, and When the gesture combination meets a predetermined condition, the electronic device is unlocked.

2. The electronic device according to claim 1, further comprising a thermal sensor for outputting a thermal image, wherein: The processor is also used to determine whether the temperature of the face and the temperature of the hand meet the predetermined temperature according to the thermal image.

3. The electronic device according to claim 1, wherein: The gesture sensor is the image sensor, millimeter wave radar or time of flight sensor.

4. The electronic device according to claim 1, wherein: When the face is blocked, the processor is further configured to record the face image and not recognize the gesture combination.

5. The electronic device according to claim 1, wherein: When the gesture combination does not meet the predetermined condition, the processor is further configured to record the facial image.

6. The electronic device according to claim 1, wherein: The gesture combination includes a combination of a static single-hand gesture, an operating hand, a gesture position, and a gesture dwell time, and The electronic device further includes or is wirelessly coupled to a display, which is used to display the gesture position and the gesture dwell time.

7. The electronic device according to claim 1, wherein: The gesture combination includes a static two-hand gesture, a static one-hand gesture, a combination of an operating hand, a gesture position, and a gesture dwell time, and The electronic device further includes or is wirelessly coupled to a display, which is used to display the gesture position and the gesture dwell time.

8. The electronic device according to claim 1, wherein: The gesture combination includes a combination of dynamic changes, position changes, moving directions and moving speeds of a single-hand gesture.

9. An electronic device, comprising: an image sensor for acquiring an image frame including an image of a hand; and processor, which is used to identifying a specific pattern of the accessory in the image frame, performing gesture recognition according to the hand image and the specific pattern of the image frame, and The electronic device is controlled to perform operations corresponding to the recognized gesture.

10. The electronic device according to claim 9, further comprising a thermal sensor for acquiring a thermal image including a hand, wherein: The processor is further configured to determine an image processing range of the image frame according to the thermal image.

11. The electronic device according to claim 10, wherein: The processor is further configured to perform the gesture recognition according to the image processing range corresponding to the higher temperature in the image frame when two image processing ranges are determined according to the thermal image.

12. The electronic device according to claim 10, wherein: The processor is further configured to perform the gesture recognition according to the image processing range in the image frame that is closer to the specific pattern when two image processing ranges are determined according to the thermal image.

13. The electronic device according to claim 9 further comprises a wireless communicator for receiving spatial information of the wearable device, wherein: The processor is further configured to perform the gesture recognition according to the hand image, the specific pattern and the spatial information.

14. The electronic device according to claim 13, wherein: The processor is further configured to notify the wearable device of completion information through the wireless communicator after completing the gesture recognition.

15. The electronic device according to claim 13, wherein: The processor is further configured to not control the electronic device to perform the corresponding operation when it is determined that the positional relationship between the hand image and the spatial information is inconsistent.

16. The electronic device according to claim 9, wherein: The processor is further configured to not control the electronic device to perform the corresponding operation when it is determined that the positional relationship between the hand image in the image frame and the specific pattern is inconsistent.

17. The electronic device according to claim 9, wherein: The processor is also used for not controlling the electronic device to perform the corresponding operation when it is determined that the pixel distance between the hand image and the specific pattern in the image frame exceeds a predetermined distance.

18. The electronic device according to claim 9, wherein: The accessory is a watch, a bracelet, or a ring.

19. An electronic device, comprising: A three-axis accelerometer, which is used to output acceleration values ​​of three axes; and processor, which is used to receiving the acceleration values ​​of the three axes, Determine whether the changes in the acceleration values ​​of the three axes conform to a predetermined sequence, and When the predetermined sequence is met, the alarm function is turned off.

20. The electronic device according to claim 19, further comprising an image sensor for acquiring image frames, wherein When the changes in the acceleration values ​​of the three axes do not conform to the predetermined sequence, the processor is further configured to: When the image frame does not contain a face image, the electronic device is controlled to sound an alarm and notify the user, and When the image frame includes a face image, the face image is recorded.