Gesture recognition system using thermal sensor

Abstract

The invention discloses an identification system applied to a portable device or a wearable device. The identification system utilizes a heat sensor to detect heat energy of a human body, and has the functions of living body identification, image denoising, body temperature prompting and the like according to a detection result.

Description

[0001] This application is a divisional application of the Chinese invention patent application with application number 201911213625.5, application date December 2, 2019, and name “Identification system using thermal sensor”. Technical Field

[0002] The present invention relates to an identification system and device, and more particularly to an identification system and device that uses a thermal sensor to perform functions such as living body identification, image denoising, and body temperature monitoring. Background Art

[0003] Image sensors have been widely used in portable electronic devices as a way to unlock the device. However, image sensors are sometimes difficult to directly identify living bodies and are easily affected by ambient light. In order to solve these problems, more complex algorithms are usually required.

[0004] In addition, with the aging of the population, the care of the elderly and young children has gradually increased the burden on society. Due to the shortage of manpower, it is impossible to rely entirely on manpower. It is necessary to use the power of technology to achieve the purpose of automated monitoring to reduce manpower requirements and social costs.

[0005] In view of this, the present invention provides an identification system using a temperature sensor to assist in the shortcomings of a system using only an image sensor. Summary of the invention

[0006] The invention provides a face recognition system capable of performing living body recognition.

[0007] The invention also provides a gesture recognition system capable of eliminating the influence of ambient light.

[0008] The present invention also provides a medical monitoring system applicable to a medical system, such as a hospital or a nursing home.

[0009] The present invention also provides a body temperature monitoring device which can be applied to home care.

[0010] The present invention provides a gesture recognition system comprising a thermal sensor, an image sensor and a processor. The thermal sensor is used to acquire and output a thermal image with a first field of view. The image sensor is used to acquire and output an image frame with a second field of view. The processor is used to determine an image processing range in the thermal image, and perform gesture recognition only based on an image area in the image frame corresponding to the image processing range in the thermal image.

[0011] The present invention also provides a gesture recognition system comprising a thermal sensor, an image sensor and a processor. The thermal sensor is used to acquire and output a thermal image with a first field of view. The image sensor is used to acquire and output an image frame with a second field of view. The processor is used to determine an image processing range in the thermal image according to a skin material area in the image frame, and perform gesture recognition only based on an object image within the image processing range in the thermal image.

[0012] The face recognition system and gesture recognition system of the embodiment of the present invention can be applied to the same portable device at the same time. The face recognition system is used as an unlocking device, for example, and the gesture recognition system is used for device operation, for example. Therefore, the recognition system first turns on the face recognition function, and only turns on the gesture recognition function when the registered face is recognized.

[0013] In the present invention, the denoising method of the gesture recognition system can also be applied to the face recognition system to improve the recognition accuracy of the system.

[0014] In the present invention, the body temperature monitoring device can also be applied to a medical monitoring system. For example, the same wearable accessory can perform different functions such as fever detection and basal body temperature detection by selecting different modes.

[0015] In order to make the above and other purposes, features and advantages of the present invention more obvious, the following will be described in detail with reference to the accompanying drawings. In addition, in the description of the present invention, the same components are represented by the same symbols, which are hereby described together. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a block diagram of an identification system according to an embodiment of the present invention;

[0017] Figure 2A is a schematic diagram of the operation of the face recognition system according to the first embodiment of the present invention;

[0018] Figure 2B is a schematic diagram of a face image acquired by a face recognition system according to a first embodiment of the present invention;

[0019] Figure 2C is a schematic diagram of temperature distribution of a thermal image acquired by the face recognition system according to the first embodiment of the present invention;

[0020] Figure 3A is a schematic diagram of the operation of a gesture recognition system according to a second embodiment of the present invention;

[0021] Figure 3B is a schematic diagram of an image frame acquired by a gesture recognition system according to a second embodiment of the present invention;

[0022] Figure 4 is a schematic diagram of a medical monitoring system according to a third embodiment of the present invention; and

[0023] Figure 5 Schematic diagram of a body temperature monitoring device according to a fourth embodiment of the present invention.

[0024] Description of Reference Numerals

[0025] 100 Identification System

[0026] 11 Thermal Sensor

[0027] 12 Image Sensor

[0028] 13 Processor

[0029] 14 Storage

[0030] Ih Thermal Image

[0031] Im Image Frame DETAILED DESCRIPTION

[0032] Please refer to Figure 1 , which is a block diagram of a recognition system 100 according to an embodiment of the present invention. The recognition system 100 can be applied to portable devices, such as electronic devices including mobile phones, tablet computers, and laptop computers; and can be applied to wearable devices, such as electronic devices including watches, bracelets, and armbands, but not limited thereto. The recognition system 100 includes a thermal sensor 11, an image sensor 12, a processor 13, and a storage 14. The recognition system 100 can be used for at least face recognition and gesture recognition.

[0033] The thermal sensor 11 includes, for example, a pyroelectric infrared (PIR) type, a thermopile type, or a bolometer type sensor, which can detect infrared rays and output an electrical signal (e.g., a voltage or current signal) or a digital signal to reflect the detected temperature. The thermal sensor 11 preferably outputs a two-dimensional thermal image to correspond to the two-dimensional image frame acquired by the image sensor 12. For example, the detection value of each pixel of the two-dimensional thermal image represents the temperature of the detected area, and the detected area is partially overlapped or non-overlapped with each other relative to adjacent pixels of the two-dimensional thermal image according to the microlens configuration thereon.

[0034] Compared to conventional temperature sensors that are in contact with the detected object when performing heat detection or temperature detection, the thermal sensor 11 of the present invention can detect temperature without contacting the object because the thermal sensor 11 uses a thermopile type or a thermistor type sensor. In other words, the thermal sensor 11 of the present invention can still detect the target temperature even if the target (such as a human body) is covered by clothes or cloth, thereby improving reliability and applicability.

[0035] The image sensor 12 includes, for example, a CCD image sensor or a CMOS image sensor, which has pixels arranged in an array for outputting a two-dimensional image frame.

[0036] The processor 13 is, for example, a digital signal processor (DSP), a microcontroller (MCU), a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a graphics processing unit (GPU), which is electrically connected to the thermal sensor 11 and the image sensor 12 to receive the thermal image Ih and the image frame Im respectively and perform post-processing using software and / or hardware. The processor 13 also controls the opening and closing of the thermal sensor 11 and the image sensor 12 and the operation of their pixels.

[0037] The storage 14 includes, for example, volatile storage and / or non-volatile storage, which is used to record in advance the algorithms, thresholds and parameters used by the processor 13 for post-processing. In different applications, the storage 14 can also temporarily store the data of the thermal image Ih and / or image frame Im when the device is operating.

[0038] Please refer to Figure 2A , which is a schematic diagram of a face recognition system 200 according to a first embodiment of the present invention. Figure 2A The face recognition system 200 is shown to be disposed on a side of the portable device facing the user. The face recognition system 200 includes: Figure 1 The thermal sensor 11, image sensor 12, processor 13 and storage 14 are shown. The thermal sensor 11 is used to obtain a thermal image Ih (such as Figure 2C The image sensor 12 is used to obtain an image frame Im (such as Figure 2B The first field of view FOV1 and the second field of view FOV2 overlap at least in part to obtain corresponding features, so that different sensors can be used to obtain information of the same area or surface. The thermal sensor 11 and the image sensor 12 can obtain images simultaneously or in different time periods without any specific limitation.

[0039] The processor 13 is used to perform face recognition and / or material recognition according to the image frame Im, wherein the processor 13 may use a known face recognition algorithm to recognize a face image (such as Figure 2B The processor 13 also performs live body recognition based on the thermal distribution of the area in the thermal image Ih corresponding to the facial features of the human face image in the image frame Im.

[0040] In the image unlocking system, in order to prevent unscrupulous persons from using a photo or video of a registered face to unlock, the face recognition system 200 of the first embodiment of the present invention can distinguish the imitations based on the thermal image Ih obtained by the thermal sensor 11 and the skin texture of the face in the image frame Im. Therefore, the living body in the present invention refers to a real person rather than a photo or video.

[0041] For example, in a non-limiting implementation, when the processor 13 recognizes that the image frame Im does not contain a registered face image or the registered face image does not have skin texture, the thermal sensor 11 is not turned on to reduce energy consumption. That is, the processor 13 turns on the thermal sensor 11 only when it is recognized that the image frame Im contains a registered face image or the registered face image has skin texture, but the present invention is not limited thereto.

[0042] In another non-limiting embodiment, the processor 13 can also reversely control the thermal sensor 11 and the image sensor 12, that is, the processor 13 does not turn on the image sensor 12 before sensing that the area of ​​the object image in the thermal image Ih is greater than the threshold. Only when the thermal image Ih contains a valid face image (that is, the area of ​​the object is greater than the threshold), the processor 13 turns on the image sensor 12 for face recognition. In other embodiments, during the unlocking process, the thermal sensor 11 and the image sensor 12 can also be turned on at the same time.

[0043] In the first embodiment, the first field of view FOV1 of the thermal sensor 11 preferably covers a larger range than the second field of view FOV2 of the image sensor 12. In addition, since the processor 13 performs liveness recognition based on the thermal image Ih, the processor 13 only needs to perform two-dimensional face recognition based on the image frame Im, and does not need to perform three-dimensional face recognition, so as to reduce energy consumption. Although it is known that three-dimensional face recognition can be used to distinguish between photos and real people, it requires a higher amount of calculation.

[0044] In addition, in order to further prevent unscrupulous elements from using heated photos to unlock, the processor 13 is not simply used to confirm whether the thermal image Ih contains an image of an object with a temperature greater than a predetermined temperature, but is also used to identify the regional thermal distribution in the thermal image Ih. Figure 2C As shown, the thermal image Ih includes a high temperature area and a low temperature area. For example, the high temperature area corresponds to the face image in the image frame Im (eg Figure 2B The low temperature area corresponds to the nose area of ​​the face image in the image frame Im, while the low temperature area corresponds to the eye and mouth areas of the face image in the image frame Im.

[0045] At this time, the storage 14 is used to pre-record the temperature distribution of each area of ​​the face, which can be obtained by entering the setting mode (such as executing application software or pressing a button) to detect and store the registered user, or can be pre-stored according to statistical results before leaving the factory. The processor 13 compares (such as calculating similarity or correlation) the regional thermal distribution in the current thermal image (such as the thermal image Ih obtained during the unlocking process) with the pre-stored temperature distribution for live body recognition. In addition, the processor 13 calculates the temperature difference between the high temperature area and the low temperature area to confirm that the face recognition system 200 is currently detecting a human body, not other disguises.

[0046] In another embodiment, the regional thermal distribution in the current thermal image is compared with the positions of the facial features (e.g., eyes, nose, mouth) identified in the acquired image frame Im to confirm whether the regional thermal distribution matches the corresponding facial features. In this way, it is still possible to distinguish between a real face and a disguised object without using the temperature distribution pre-recorded in the memory 14.

[0047] When the processor 13 completes the registered face confirmation according to the face recognition, it turns on the operating function of the electronic device configured with the face recognition system 200, such as turning on the display screen.

[0048] Please refer to Figure 3A , which is a schematic diagram of a gesture recognition system 300 according to a second embodiment of the present invention. Figure 3A The gesture recognition system 300 is disposed on a side of the portable device facing the user. The gesture recognition system 300 also includes Figure 1 The thermal sensor 11, image sensor 12, processor 13 and storage 14 are shown. The thermal sensor 11 is used to obtain a thermal image Ih with a first field of view FOV1 and output it to the processor 13. The image sensor 12 is used to obtain an image frame Im with a second field of view FOV2 (such as Figure 3B as shown) and output to processor 13.

[0049] The processor 13 is used to determine the image processing range WOI according to the thermal image Ih, and perform gesture recognition only according to the image area in the image frame Im corresponding to the image processing range WOI determined in the thermal image Ih, so as to eliminate the influence of ambient light. Therefore, in order to enable the processor 13 to correctly determine the image processing range WOI in the image frame Im, in the second embodiment, the first field of view FOV1 of the thermal sensor 11 is preferably the same as the second field of view FOV2 of the image sensor 12, and the size of the thermal image Ih and the image frame Im is preferably substantially the same. For example, the corresponding image processing range WOI in the image frame Im can be determined according to the pixel address or position within the image processing range WOI determined in the thermal image Ih.

[0050] For example, the processor 13 first determines the range in the thermal image Ih where the temperature is greater than the temperature threshold (determined according to the human body temperature) as the image processing range WOI, which is the image area in the thermal image Ih. Since the first field of view FOV1 and the second field of view FOV2 are configured to correspond, the processor 13 can confirm the corresponding image processing range WOI in the image frame Im, such as Figure 3B As shown, the size of the corresponding image processing range WOI is smaller than the size of the image frame Im. It must be noted that the image processing range WOI is not limited to Figure 3B The displayed rectangle may be of other appropriate shapes as long as it can include the object region in the image frame Im. In some implementations, two image processing areas WOI may be defined based on two object regions.

[0051] In other embodiments, the first field of view FOV1 and the second field of view FOV2 may not be completely the same but have an angle difference. In this case, as long as the storage 14 pre-stores the space conversion algorithm or matrix conversion algorithm of the thermal image Ih and the image frame Im, the processor 13 can confirm the corresponding image processing range WOI in the image frame Im according to the stored algorithm after confirming the image processing range WOI in the thermal image Ih.

[0052] In another non-limiting implementation, the processor 13 may also first identify the object image in the image frame Im, which may include the ambient light image. The processor 13 then uses the thermal image Ih to exclude non-human images, such as object images outside the image processing range WOI are not used for gesture recognition, thereby effectively improving recognition accuracy and eliminating interference.

[0053] More specifically, in the second embodiment, the processor 13 performs gesture recognition based on a partial image region in the image frame Im, and the thermal image Ih is used to provide a denoising function.

[0054] In addition, in order to save energy, the processor 13 turns on the image sensor 12 only after recognizing that the image processing range WOI in the thermal image Ih is larger than a predetermined size. In this embodiment, the entire sensing array of the image sensor 12 is turned on, or only a portion of the sensing array corresponding to the image processing range WOI is turned on. In other words, when the thermal image Ih does not contain an area greater than a predetermined temperature threshold, the processor 13 only turns on the thermal sensor 11 to continuously acquire the thermal image Ih at a predetermined frequency; or, although the thermal image Ih contains an area greater than a predetermined temperature threshold, if the area is smaller than a predetermined size (which is determined, for example, according to the hand size within the detectable distance of the system), the processor 13 still only turns on the thermal sensor 11 to continuously acquire the thermal image Ih at a predetermined frequency, but the present invention is not limited thereto.

[0055] In other embodiments, during the gesture recognition process, the thermal sensor 11 and the image sensor 12 may be turned on at the same time. For example, only some pixels of the thermal sensor 11 are turned on, and the pixels correspond to the pixel area where the image sensor 14 detects the object. In more detail, the sensing arrays of the thermal sensor 11 and the image sensor 12 do not need to be turned on in their entirety, but only some pixels are turned on to reduce energy consumption.

[0056] In another embodiment, the processor 13 first performs material recognition based on the image frame Im acquired by the image sensor 11, and then performs gesture recognition based on the thermal image Ih acquired by the thermal sensor 11. For example, if the processor 13 does not determine the image of an object with skin material in the image frame Im, the thermal sensor 11 is not turned on. The thermal sensor 11 is turned on only after the skin material is determined in the image frame Im. In addition, the processor 13 also determines the image processing range WOI in the thermal image Ih based on the skin material area in the image frame Im, that is, the processor 13 first determines the skin material area in the image frame Im, and then determines the image processing range WOI corresponding to the skin material area in the thermal image Ih. Gesture recognition is performed only based on the object image within the image processing range WOI in the thermal image Ih.

[0057] It must be noted that, although the identification system 100 is described as being applied to a portable device in the first and second embodiments, the present invention is not limited thereto. The identification system 100 of the first and second embodiments can also be applied to a wearable device, a door access control or a vehicle security system and / or a control system. The processor 13 can use the thermal image Ih for live body identification and denoising to improve the judgment accuracy and safety.

[0058] Please refer to Figure 4 , which is a schematic diagram of a medical monitoring system 400 according to a third embodiment of the present invention. The medical monitoring system 400 is applied to, for example, a medical institution or a nursing institution to solve the problem of insufficient manpower. The medical monitoring system 400 mainly includes a wearable accessory 41 and a central computer system 43 coupled to each other. The wearable accessory 41 is used to be worn on the human body, such as Figure 4 The wearable accessory 41 is shown as being worn on an arm, but is not limited thereto, and can also be worn on other parts of the body suitable for measuring body temperature. The central computer system 43 reacts accordingly to the detection result of the wearable accessory 41, such as providing a warning.

[0059] For example, the wearable accessory 41 may be a customized accessory, a smart watch, a smart armband or a smart bracelet, etc., which at least includes a thermal sensor 411, a processor 412 and a transmitter 413. The thermal sensor 411 may output a two-dimensional thermal image similar to the thermal sensors of the first and second embodiments described above, and the processor 412 calculates its average temperature. In addition, in the third embodiment, the thermal sensor 411 may only include a single detection unit (such as a photodiode) and output an electrical signal or a digital signal at a time to represent the detected temperature, instead of outputting a two-dimensional thermal image. The processor 412 may also be a digital signal processor, a microcontroller, a central processing unit, a specific application integrated circuit or a graphics processor.

[0060] In the example where the thermal sensor 411 is disposed on other electronic devices that are not in direct contact with the user (for example, the electronic device is disposed on a wall or ceiling), the thermal sensor 411 monitors the user's whole body temperature. When the temperature difference between the core temperature and the limb temperature is greater than a predetermined threshold, the electronic device provides a warning message Sw.

[0061] When measuring body temperature, the thermal sensor 411 directly detects the skin surface temperature as the body temperature, or detects the temperature difference between the room temperature and the body temperature (that is, the room temperature and the body temperature are detected simultaneously using the same or different sensors), and deducts the temperature difference from the room temperature (for example, using the processor 412) to obtain the body temperature.

[0062] The thermal sensor 411 is used to detect the body temperature of the human body and output an electrical signal or a digital signal to the processor 412. After the processor 412 determines the temperature according to the received signal, it controls the transmitter 413 ( Figure 4 For example, the antenna sends the temperature information St related to the human body temperature and the number information S of the wearable accessory 41 ID (For example, the medical monitoring system 400 may include multiple wearable accessories 41 and each accessory has its own serial number), for example, wireless communication methods such as Bluetooth communication, Zigbee, microwave communication, etc. are used for transmission.

[0063] In one embodiment, the central computer system 43 is configured to receive the temperature information St and the number information S ID In another embodiment, the central computer system 43 may include a plurality of receivers configured in different spaces for receiving temperature information St and number information S from different patients. ID , the host of the central computer system 43 is electrically connected to these receivers.

[0064] When the human body temperature represented by the received temperature information St exceeds the predetermined range, the central computer system 43 sends a message corresponding to the number information S IDRelated warning information Sw, wherein the number information S ID Correlation means that the warning information Sw is related to the number information S sent by the wearer. ID In one embodiment, the warning information Sw is transmitted by means of a light signal, a broadcast, etc. In another embodiment, the central computer system 43 further includes a transmitter (not shown) to wirelessly send the warning information Sw to a portable device 49, which is carried by medical personnel, for example.

[0065] In a non-limiting embodiment, the central computer system 43 further includes a display 431 (such as a liquid crystal display or a plasma display) for displaying the time distribution of the human body temperature for the convenience of medical staff. The display 431, for example, displays or indicates a high temperature threshold TH1 and a low temperature threshold TH2. When the central computer system 43 determines that the human body temperature detected by the thermal sensor 411 exceeds a predetermined range (for example, exceeds the high temperature threshold TH1 or the low temperature threshold TH2), the warning message Sw is issued. The thresholds TH1 and TH2 can be set or adjusted according to different detected users.

[0066] In a non-limiting embodiment, the central computer system 43 further includes a camera 45. When it is determined that the body temperature of the human body exceeds the predetermined range, the central computer system 43 turns on the camera 45 to monitor the patient. The camera 45 is usually turned off to maintain the privacy of the patient. The image obtained by the camera 45 can also be displayed on the display 431.

[0067] In a non-limiting embodiment, the central computer system 43 further includes a drug dispensing device 47. When it is determined that the human body temperature exceeds the predetermined range, the central computer system 43 turns on the drug dispensing device 47 to automatically dispense the drug. The operating state of the drug dispensing device 47 can also be displayed on the display 431 (if included). The display 431 can simultaneously display the relationship between the dosage and the temperature change. The central computer system 43 can also control the drug dispensing device 47 to stop dispensing the drug when the body temperature returns to the predetermined range.

[0068] Although the above description indicates that the medical monitoring system 400 is applied to an operating organization, the present invention is not limited thereto. The medical monitoring system 400 of the third embodiment can also be used for home care, as long as a tablet computer, desktop computer or laptop computer replaces the operation of the central computer system 43.

[0069] Although the third embodiment is described by taking the wearable accessory 41 as an example, it is only for illustration and not for limiting the present invention. In other embodiments, the wearable accessory 41 can be replaced by other monitoring devices that do not directly contact the human body. The thermal sensor 411 detects body temperature in a non-contact manner.

[0070] Please refer to Figure 5 , which is a schematic diagram of a body temperature monitoring device 500 according to a fourth embodiment of the present invention. The body temperature monitoring device 500 comprises a wearable accessory 51, a thermal sensor 511, a processor 512, a display 513 and a storage 514. The processor 512 can also be a digital signal processor, a microcontroller, a central processing unit, an application-specific integrated circuit or a graphics processor.

[0071] The wearable accessory 51 is used to be worn on the human body, such as a watch, a bracelet or an armband, which can be roughly attached to and fixed to the detected skin surface, without any specific restrictions. The thermal sensor 511 is configured on the wearable accessory 51, and is used to detect the basal body temperature (BBT) of the human body, and output an electrical signal or a digital signal to the processor 512. The processor 512 is used to record the basal body temperature every day, and when the temperature change of the basal body temperature exceeds the temperature change threshold (for example, 0.3 to 0.5 degrees, which is pre-recorded in the storage 514), the display 513 is controlled to prompt.

[0072] For example, the processor 512 controls the thermal sensor 511 to automatically detect the basal body temperature at a fixed time every day, for example, based on the system clock. Alternatively, the body temperature monitoring device 500 further includes a button 515. When the processor 512 receives a pressing signal of the button 515, the thermal sensor 511 is controlled to detect the basal body temperature and display it on the display 513 and store it in the storage 514 for long-term monitoring.

[0073] The display 513 can provide various information in the form of icons or numbers, such as the ovulation period or date (e.g., displayed as yyyy / m / d), the high temperature period (e.g., Figure 5 Displays the number of days when the basal body temperature is around 36.8 degrees, which varies according to different users) and during low temperature periods (e.g. Figure 5 Displays the number of days when the basal body temperature is around 36.4 degrees, which depends on different users) and any other information that helps understand the physiological period.

[0074] Since the basal body temperature is preferably measured when the user wakes up but before getting out of bed, in order to achieve the purpose of automatic measurement, the body temperature monitoring device 500 also includes an acceleration detector (such as a G-sensor) for detecting whether the user gets up from the bed. For example, before getting up, the acceleration detector only detects acceleration in two dimensions (such as the XY axis), but after getting up from the bed, the acceleration in the third dimension (such as the Z axis) can be detected. The processor 512 is also used to determine the time to get up (but not get out of bed) based on the acceleration signal detected by the acceleration detector, and control the thermal sensor 511 to automatically detect the basal body temperature at the time to get up. The so-called detection of acceleration refers to the detection of an acceleration value greater than a predetermined threshold.

[0075] In a non-limiting embodiment, when the processor 512 detects that the user is lying in bed (for example, no Z-axis or other acceleration is detected within a predetermined period), the thermal sensor 511 is controlled to detect the temperature once every predetermined period (for example, 1 to several minutes). Only the detected temperature before the acceleration in the Z-axis is detected is regarded as the basal body temperature by the processor 512 and temporarily stored in the storage 514. In order to increase the judgment accuracy, when the processor 512 does not detect another Z-axis acceleration again within a predetermined period after detecting an acceleration in the Z-axis, it means that the user just changes the posture in bed, and the temporarily stored detected temperature is not regarded as the basal body temperature.

[0076] In a non-limiting embodiment, the body temperature monitoring device 500 can be wirelessly connected to another thermal sensor, which includes a wireless transceiver and a processor (e.g., a digital processor). The other thermal sensor is arranged near the user or the bed. When the body temperature monitoring device 500 detects the acceleration in the Z-axis direction, it sends a request signal to notify the other thermal sensor, and the processor of the other thermal sensor can identify (e.g., use the hardware and / or software therein to determine the change of the high temperature area in the acquired data) whether the user gets up. If so, the other thermal sensor sends a response signal to the body temperature monitoring device 500, so that it regards the body temperature measured before the user gets up as the basal body temperature. If not, the other thermal sensor does not send a response signal or sends a response signal to indicate that the body temperature does not need to be measured at this time.

[0077] In a non-limiting embodiment, when the processor 512 detects that the user shakes the body temperature monitoring device 500 in a predetermined pattern (for example, shaking it left to right or up and down several times), it starts measuring the body temperature and records it as the basal body temperature.

[0078] It is understood that the numerical values ​​of the above embodiments are only used for illustration rather than for limiting the present invention.

[0079] It must be noted that, although the above-mentioned recognition system 100 is described by taking the example of including both the thermal sensor 11 and the image sensor 12, the present invention is not limited thereto. In other embodiments, the recognition system 100 includes only one of the thermal sensor 11 and the image sensor 12, and receives another signal (image frame Im or thermal image Ih) from the external sensor through its input / output interface. For example, the recognition system 100 includes the thermal sensor 11 but receives the image frame Im from the external image sensor; or, the recognition system 100 includes the image sensor 12 but receives the thermal image Ih from the external thermal sensor.

[0080] In summary, the recognition system and monitoring system using only image sensors have operational limitations, and more complex algorithms must be used to overcome such limitations. Therefore, the present invention further provides a face recognition system ( Figure 2A )、Gesture Recognition System( Figure 3A ), medical monitoring system( Figure 4 ) and body temperature monitoring device ( Figure 5 ), which overcomes the limitations of systems that only use image sensors by using thermal sensors, and can effectively improve the accuracy of the recognition system and the applicable scenarios of the monitoring system.

[0081] Although the present invention has been disclosed through the above examples, they are not intended to limit the present invention. Any person skilled in the art with ordinary knowledge in the art to which the present invention belongs can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the appended claims.

Claims

1. A gesture recognition system, the gesture recognition system comprising: A thermal sensor, the thermal sensor being used to acquire and output a thermal image in a first field of view; an image sensor configured to acquire and output an image frame with a second field of view; and A processor is used to determine an image processing range in the thermal image and perform gesture recognition only based on an image area in the image frame that corresponds to the image processing range in the thermal image. 2 . The gesture recognition system according to claim 1 , wherein the first field of view is the same as the second field of view. 3 . The gesture recognition system according to claim 1 , wherein the processor determines a range in the thermal image where the temperature is greater than a temperature threshold as the image processing range. 4 . The gesture recognition system according to claim 1 , wherein the processor turns on the image sensor only after first recognizing that the image processing range in the thermal image is larger than a predetermined size. 5 . The gesture recognition system according to claim 1 , wherein the processor first recognizes the object image in the image frame and then excludes non-human images within the image processing range of the thermal image. 6 . The gesture recognition system according to claim 5 , wherein the object images outside the image processing range are not used for performing the gesture recognition. 7 . The gesture recognition system according to claim 1 , wherein the image area corresponding to the image processing range is smaller than a size of the image frame.

8. The gesture recognition system according to claim 1, wherein the processor first performs material recognition based on the image frame acquired by the image sensor, and activates the thermal sensor only after the skin material is determined in the image frame.

9. A gesture recognition system, the gesture recognition system comprising: A thermal sensor, the thermal sensor being used to acquire and output a thermal image in a first field of view; an image sensor configured to acquire and output an image frame with a second field of view; and A processor is used to determine an image processing range in the thermal image according to a skin material area in the image frame, and perform gesture recognition only based on an object image within the image processing range in the thermal image. 10 . The gesture recognition system of claim 9 , wherein the first field of view is the same as the second field of view.

Images