Electronic system and physiological monitoring method

Abstract

An electronic system and a physiological monitoring method are provided that can monitor a user's physiological state and issue a warning if necessary. 【Solution means】The electronic system 1 includes a camera 10, an auxiliary feature sensor 11, and an accelerometer 12. The camera is configured to capture a face image of the user. The auxiliary feature sensor is configured to sense auxiliary features of the user. The accelerometer is configured to calculate the emotional state of the user according to the face image. When it is determined that the emotional state satisfies a predetermined emotional state, the accelerometer is configured to determine the emotional level of the user by analyzing the auxiliary features and determine whether to issue a warning signal accordingly.

Description

Technical Field

[0001] The present invention relates to an electronic system and a physiological monitoring method, and more particularly to an electronic system and a physiological monitoring method configured to monitor a health state.

Background Art

[0002] In modern life, people are often exposed to high-pressure and long-hour working conditions. In such a situation, how to appropriately remind users so that they can respond or improve immediately has become one of the most important issues to be addressed.

Summary of the Invention

Problems to be Solved by the Invention

[0003] The present invention provides an electronic system and a physiological monitoring method that can monitor a user's physiological state and issue a warning if necessary.

Means for Solving the Problems

[0004] The electronic system of the present invention includes a camera, an auxiliary feature sensor, and an accelerometer. The camera is configured to capture a face image of the user. The auxiliary feature sensor is configured to sense auxiliary features of the user. The accelerometer is configured to calculate an emotional state of the user according to the face image. When it is determined that the emotional state satisfies a predetermined emotional state, the accelerometer is configured to determine an emotional level of the user by analyzing the auxiliary features and determine whether to issue a warning signal accordingly.

[0005] The physiological monitoring method of the present invention includes the following operations. A face image of the user is captured. Auxiliary features of the user are sensed. The emotional state of the user is calculated according to the face image. When it is determined that the emotional state satisfies a predetermined emotional state, by analyzing the auxiliary features, the emotional level of the user is determined, and it is determined whether to issue a warning signal accordingly.

Effect of the Invention

[0006] Based on the above, the electronic system and the physiological monitoring method of the present invention can continuously monitor the emotional and physiological states of the user and determine whether to issue a warning signal accordingly.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Mode for Carrying Out the Invention

[0008] FIG. 1 shows a schematic block diagram of an electronic system 1 according to an embodiment of the present invention. As shown in FIG. 1, the electronic system 1 includes a camera 10, an auxiliary feature sensor 11, and an accelerometer 12. The camera 10 may be configured to capture a face image of the user. The auxiliary feature sensor 11 may be configured to sense auxiliary features of the same user. The accelerometer 12 is coupled to the camera 10 and the auxiliary feature sensor 11. The accelerometer 12 may be configured to calculate the emotional state of the user according to the face image. When it is determined that the emotional state satisfies a predetermined emotional state, by analyzing the auxiliary features, the emotional level of the user is determined, and it is determined whether to issue a warning signal accordingly.

[0009] Briefly speaking, in some embodiments, the accelerator 12 of the electronic system 1 can first determine the user's emotional state according to the face image, then assist in determining the user's emotional level according to the auxiliary features, and determine whether to issue a warning signal according to the emotional level in the emotional state.

[0010] FIG. 2 shows a flowchart of a physiological monitoring method according to an embodiment of the present invention. The physiological monitoring method shown in FIG. 2 can be applied to and executed on the electronic system 1 of FIG. 1. Specifically, the physiological monitoring method in FIG. 2 includes steps S20 to S22.

[0011] In step S20, the electronic system 1 can capture the user's face image through the camera 10 and provide it to the accelerator 12. In step S21, the auxiliary feature sensor 11 can sense the user's auxiliary features. Finally, in step S22, the accelerator 12 of the electronic system 1 can calculate the user's emotional state according to the face image. When it is determined that the emotional state satisfies a predetermined emotional state, the user's emotional level is determined by analyzing the auxiliary features, and it is determined whether to issue a warning signal accordingly.

[0012] Specifically, in step S22, after receiving the face image, the accelerator 12 can determine the user's emotional state through image recognition. In some embodiments, the accelerator 12 pre-stores at least one image feature of a predetermined emotional state and determines whether the user's face image belongs to a specific predetermined emotional state by identifying whether the face image satisfies any of the image features. For example, for monitoring the user's health status, the image features stored by the accelerator 12 can be, for example, excitement, fatigue, anxiety, anger, etc., or other more excited emotional states. The accelerator 12 can determine whether the user's emotion corresponds to one of the more excited emotional states by recognizing the face image.

[0013] Furthermore, when the accelerator 12 determines that the user corresponds to a predetermined emotional state, the accelerator 12 can further obtain the auxiliary features sensed by the auxiliary feature sensor 11 and determine the current emotional level of the user in that emotional state. In some embodiments, the auxiliary feature sensor 11 may be, for example, an infrared camera configured to sense auxiliary features such as a facial temperature change signal. Alternatively, the auxiliary feature sensor 11 may be configured to sense auxiliary features such as at least one of blood pressure, blood oxygen, pulse, body temperature, or other similar parameters, and may be other wearable or non-wearable sensing devices.

[0014] Taking the facial temperature change as an example, when the accelerator 12 determines that there is a high possibility that the user is in an excited state according to the face image, the accelerator 12 can further refer to the facial temperature change signal to determine whether the user's facial temperature change is too high, and further determine whether the emotional level of the user in the excited emotion is too high, that is, whether it is an overly excited emotional state. Specifically, the accelerator 12 can sense the temperature change of a part or all of the user's face, such as the forehead and cheeks, or sense the average temperature change of the entire face. In some embodiments, when the accelerator 12 determines that the user's emotional state has changed to a more excited emotional state such as excitement or anger according to the face image, the user's facial temperature generally increases. Therefore, the accelerator 12 can determine whether the upward change in the user's facial temperature is still within the normal range through the facial temperature change signal. In contrast, when the accelerator 12 determines that the user's emotional state has changed to a more negative emotional state such as depression or fatigue, the user's facial temperature generally decreases, and the accelerator 12 can determine whether the decrease change in the user's facial temperature is within the normal range through the facial temperature change signal.

[0015] In some embodiments, the normal range of the facial temperature change signal may have a fixed or variable temperature threshold. For example, the variable temperature threshold can take into account changes in the reference room temperature or other variables. For example, the accelerometer 12 may refer to an increase or decrease in the change of the room temperature. After subtracting the change in the facial temperature change caused by the change in the room temperature, by determining the emotional level according to the subtracted facial temperature change, the determination accuracy of the emotional level is improved. In this way, the accelerometer 12 can determine the emotional state of the user according to the facial image and determine the emotional level of the user according to the auxiliary features.

[0016] In some embodiments, the auxiliary feature used by the accelerometer 12 to determine the emotional level may be one of the physiological change signals such as blood pressure, blood oxygen, pulse, body temperature, etc. Specifically, similar to the above process of determining the emotional level according to the facial temperature change, the accelerometer 12 may determine the emotional level of the user according to whether the physiological change signal exceeds a predetermined range. For example, the normal range of the user's pulse rate is between 60 and 100 beats per minute. The accelerometer 12 may determine the emotional level of the user by determining whether the user's pulse exceeds the normal range or whether the amount of pulse change increases or decreases excessively.

[0017] And when the accelerometer 12 determines that the user's emotional level is too high or too low according to the auxiliary features, the accelerometer 12 may make a corresponding determination and issue a warning signal. In some embodiments, the accelerometer 12 can remind the user in the form of, for example, image display, sound, vibration, etc. Alternatively, the warning signal may be sent to the user's emergency contact via the network by the electronic system 1. In this way, the electronic system 1 can determine whether the user's physiological state is harmful to health according to the calculated emotional state and the corresponding emotional level, and issue a corresponding warning signal according to the urgency.

[0018] In some embodiments, the accelerator 12 may be integrated on the motherboard, for example, or may be separately disposed on a keyboard, a mouse, a connector, or a peripheral device. In some embodiments, the accelerator 12 may be, for example, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Arithmetic Logic Unit (ALU), a Field Programmable Gate Array (FPGA), any other type of integrated circuit, a state machine, an Advanced RISC Machine (ARM)-based processor, other similar elements, or a combination of the above elements. The accelerator 12 can store a pre-trained artificial intelligence (AI) model. The artificial intelligence model may be trained by a dataset including a plurality of face images having pre-labeled emotional states. As a result, the artificial intelligence model can classify subsequent input face images and determine whether the face image corresponds to any of the predetermined emotional states. The accelerator 12 can determine the emotional state of the user by executing the artificial intelligence model and performing image recognition on the face image. For example, the artificial intelligence model stored in the accelerator 12 may be a machine learning model with supervised learning or unsupervised learning, or may be a deep learning model based on the above classification, such as the Region-based Convolutional Neural Networks (RCNN) series, which may be, for example, RCNN, Fast RCNN, Mask RCNN, and Faster RCNN, SSD, and the YOLO (You only look once, YOLO) series (e.g., YOLO, YOLOv2, YOLO 9000, and YOLOv3) and other models. Further, the above artificial intelligence model may be constructed based on a decision tree, logistic regression, SVM (support vector machine), naive Bayes, kNN, or other suitable algorithms.Furthermore, for the aforementioned deep learning model, aspects such as image preprocessing, model architecture, model depth, convolutional layer parameters, pooling layer, activation function, loss function, the specific architecture of each model, design concept, accuracy, and execution speed can also be considered. By combining, adding, adjusting, and modifying these aspects, a customized deep learning model can be created.

[0019] In some embodiments, the auxiliary feature sensor 11 may have one or more sensors according to the requirements of the electronic system 1. For example, when the accelerometer 12 needs to evaluate the emotion level based on the facial temperature change signal, the auxiliary feature sensor 11 may include, for example, an infrared camera for capturing and displaying the user's facial temperature change. When the accelerometer 12 determines the emotion level according to, for example, blood pressure, blood oxygen, pulse, body temperature, or other similar physiological change signals, the auxiliary feature sensor 11 may be, for example, a wearable sensing device such as a watch or a bracelet, or a non-wearable sensing device arranged on a mouse, a mouse pad, or earphones.

[0020] FIG. 3 is a flowchart of a physiological monitoring method according to an embodiment of the present invention. The physiological monitoring method shown in FIG. 3 can be applied to and executed on the electronic system 1 of FIG. 1. Specifically, the physiological monitoring method in FIG. 3 includes steps S30 to S37.

[0021] In step S30, the camera 10 can capture a face image of the user and provide it to the accelerator 12. In step S31, the auxiliary feature sensor 11 may be an infrared temperature sensor configured to sense, for example, the facial temperature of the user and provide a facial temperature change signal to the accelerator 12 accordingly. In step S32, in addition to the above-described infrared temperature sensor, the auxiliary feature sensor 11 may further include other wearable or non-wearable sensing devices for sensing physiological change signals of the user, such as at least one of blood pressure, blood oxygen, pulse, body temperature, or other similar parameters, and provide them to the accelerator 12.

[0022] In step S33, the accelerator 12 can determine the emotional state of the user according to the face image. For example, the accelerator 12 can identify the lines in the face image through image recognition to determine the emotional state of the user. It can identify whether the user is in an excited, fatigued, anxious, angry, or other more excited emotional state, or whether the user is in a depressed, fatigued, or other more negative emotional state. In these emotional states, the accelerator 12 may further determine the emotional level of the user in that emotion according to the facial temperature change signal and the physiological change signal.

[0023] In step S34, the accelerator 12 evaluates whether the user's emotional state and emotional level have reached the first warning level or the second warning level. Accordingly, it proceeds to step S35 to issue a first warning signal, or proceeds to steps S36 - S37 to issue a second warning signal, or, if neither warning level has been reached, determines whether to continuously monitor the user's emotional state, facial temperature change, and physiological state. Specifically, the accelerator 12 can determine the user's emotional state according to the face image and determine the user's emotional level in the user's emotional state according to the facial temperature change signal. When the accelerator 12 determines that the user's emotional level corresponds to the first range, this may include cases where the emotional level is high (e.g., higher than the first upper limit) or low (e.g., lower than the first lower limit). The accelerator 12 can determine that the first warning level has been reached and proceed to step S35 to instruct to issue a first warning signal.

[0024] Furthermore, the accelerator 12 may further use the physiological change signal as an auxiliary to the facial temperature change signal to further confirm whether the user's physiological state has reached a more urgent second warning state. For example, when the accelerator 12 determines that the user's emotional level corresponds to the second range according to the facial temperature change signal, this may include cases where the emotional level is too high (e.g., higher than the second upper limit, and the second upper limit is greater than the first upper limit), or the emotional level is too low (e.g., lower than the second lower limit, and the second lower limit value is lower than the first lower limit value). The accelerator 12 can further refer to whether the physiological change signal exceeds a predetermined range. When the accelerator 12 determines that the user's emotional level is too high or too low and the physiological change signal exceeds the predetermined range, the accelerator 12 can accordingly determine that the user's emotion has reached the second warning level, which means that the user's physiological state has reached a more urgent situation, and thus proceed to step S36 and attempt to issue a second warning signal.

[0025] In step S36, the electronic system 1 can evaluate whether its network function is on or whether it has the authority to turn on the network function. If the network function is on or if it has the authority to turn on the network function, the electronic system 1 proceeds to step S37 and issues a second warning signal to the user's emergency contacts via the network. If the evaluation result is negative, the accelerator 12 gives up issuing the second warning signal.

[0026] In summary, the electronic system and the physiological monitoring method of the present invention can perform real-time monitoring of the user's physiological state and issue a warning signal when the user's physiological state is critical. In addition, it can effectively warn about the dangerous state of the user caused by fatigue, excessive use of electronic devices, or stress, enabling the user to take emergency measures accordingly.

Industrial Applicability

[0027] The electronic system and the physiological monitoring method of the present invention are applied to image recognition technology and can monitor the user's emotional and physiological states.

Explanation of Reference Numerals

[0028] 1: Electronic system 10: Camera 11: Auxiliary feature sensor 12: Accelerator S20~S22, S30~S37: Steps

Claims

1. a camera configured to capture a facial image of a user; an auxiliary feature sensor configured to sense an auxiliary feature of the user; an accelerator configured to calculate an emotional state of the user according to the face image, determine an emotional level of the user in the emotional state by analyzing the auxiliary features, and determine whether to issue a warning signal accordingly if it is determined that the emotional state satisfies a predetermined emotional state; An electronic system comprising: the auxiliary features include a facial temperature change signal and a physiological change signal; the accelerator instructs a first warning device to issue a first warning signal when the accelerator determines that the emotion level of the user falls within a first range according to the facial temperature change signal; When the accelerator determines that the emotion level of the user falls within a second range according to the facial temperature change signal, the accelerator refers to whether the physiological change signal exceeds a predetermined range, and when the physiological change signal exceeds the predetermined range, the accelerator issues a second warning signal via a network after confirming that a network setting of the electronic system is turned on. Electronic system.

2. the accelerator determines the emotional state of the user by recognizing the facial image through image recognition techniques; The electronic system of claim 1 .

3. the accelerator determines the emotional state of the user by performing the image recognition techniques on the facial image through execution of an artificial intelligence model; The electronic system of claim 2.

4. the physiological change signal includes at least one of blood pressure, blood oxygen, pulse rate, and body temperature, and the accelerator determines the emotion level of the user in the emotional state by analyzing the physiological change signal; The electronic system of claim 1 .

5. The accelerator is disposed on a motherboard or a connector, or on an edge device connected via a connection terminal or a transmission line. The electronic system of claim 1 .

6. Taking a facial image of a user; Sensing an auxiliary characteristic of the user; and calculating an emotional state of the user according to the facial image, and when it is determined that the emotional state satisfies a predetermined emotional state, determining an emotional level of the user in the emotional state by analyzing the auxiliary features including a facial temperature change signal and a physiological change signal, and determining whether to issue a warning signal accordingly; instructing to issue a first warning signal when it is determined that the emotion level of the user falls within a first range according to the facial temperature change signal; when it is determined that the emotion level of the user falls within a second range according to the facial temperature change signal, checking whether the physiological change signal exceeds a predetermined range, and when the physiological change signal exceeds the predetermined range, issuing a second warning signal via a network after confirming that a network setting is on; A physiological monitoring method comprising:

7. determining an emotional state of the user by recognizing the facial image through image recognition techniques; 7. The physiological monitoring method of claim 6, comprising:

8. determining the emotional state of the user by performing the image recognition techniques on the facial image through execution of an artificial intelligence model; 8. The physiological monitoring method of claim 7, comprising:

9. The physiological change signal includes at least one of blood pressure, blood oxygen, pulse rate, and temperature, and the physiological monitoring method includes: determining the emotional level of the user in the emotional state by analyzing the physiological change signal; 7. The physiological monitoring method of claim 6, comprising:

Images