Electronic system and physiological monitoring method
The electronic system with a camera, sensor, and accelerometer analyzes facial images and physiological signals to monitor emotional and physiological states, issuing warnings for critical conditions, addressing the lack of effective monitoring in high-pressure environments.
Patent Information
- Application Number
- JP2024065456
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-12-22
- Filing Date
- 2024-04-15
- Publication Date
- 2025-07-03
- Estimated Expiration
- 2044-04-15
AI Technical Summary
Existing systems fail to effectively monitor and respond to users' physiological states, particularly emotional and health-related changes, under high-pressure and long-hour working conditions.
An electronic system comprising a camera, auxiliary feature sensor, and accelerometer to capture and analyze facial images and physiological signals, determining emotional states and issuing warnings when necessary.
The system provides continuous monitoring and timely warnings for critical emotional and physiological states, enabling users to take emergency measures.
Smart Images
Figure 2025100284000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an electronic system and a physiological monitoring method, and more particularly to an electronic system and a physiological monitoring method configured to monitor a health state.
Background Art
[0002] In modern life, people are often exposed to high-pressure and long-hour working conditions. In such a situation, how to appropriately remind users so that they can respond or improve immediately has become one of the most important issues to be addressed.
Summary of the Invention
Problems to be Solved by the Invention
[0003] The present invention provides an electronic system and a physiological monitoring method that can monitor a user's physiological state and issue a warning if necessary.
Means for Solving the Problems
[0004] The electronic system of the present invention includes a camera, an auxiliary feature sensor, and an accelerometer. The camera is configured to capture a face image of the user. The auxiliary feature sensor is configured to sense auxiliary features of the user. The accelerometer is configured to calculate an emotional state of the user according to the face image. When it is determined that the emotional state satisfies a predetermined emotional state, the accelerometer is configured to determine an emotional level of the user by analyzing the auxiliary features and determine whether to issue a warning signal accordingly.
[0005] The physiological monitoring method of the present invention includes the following operations. A face image of the user is captured. Auxiliary features of the user are sensed. The emotional state of the user is calculated according to the face image. When it is determined that the emotional state satisfies a predetermined emotional state, by analyzing the auxiliary features, the emotional level of the user is determined, and it is determined whether to issue a warning signal accordingly.
Effect of the Invention
[0006] Based on the above, the electronic system and the physiological monitoring method of the present invention can continuously monitor the emotional and physiological states of the user and determine whether to issue a warning signal accordingly.
Brief Description of the Drawings
[0007]
Figure 1
Figure 2
Figure 3
Mode for Carrying Out the Invention
[0008] FIG. 1 shows a schematic block diagram of an electronic system 1 according to an embodiment of the present invention. As shown in FIG. 1, the electronic system 1 includes a camera 10, an auxiliary feature sensor 11, and an accelerator 12. The camera 10 may be configured to capture a face image of the user. The auxiliary feature sensor 11 may be configured to sense auxiliary features of the same user. The accelerator 12 is coupled to the camera 10 and the auxiliary feature sensor 11. The accelerator 12 may be configured to calculate the emotional state of the user according to the face image. When it is determined that the emotional state satisfies a predetermined emotional state, by analyzing the auxiliary features, the emotional level of the user is determined, and it is determined whether to issue a warning signal accordingly.
[0009] Briefly speaking, in some embodiments, the accelerator 12 of the electronic system 1 can first determine the emotional state of the user according to the face image, then assist in determining the emotional level of the user according to the auxiliary features, and determine whether to issue a warning signal according to the emotional level in the emotional state.
[0010] FIG. 2 shows a flowchart of a physiological monitoring method according to an embodiment of the present invention. The physiological monitoring method shown in FIG. 2 can be applied to and executed on the electronic system 1 of FIG. 1. Specifically, the physiological monitoring method in FIG. 2 includes steps S20 to S22.
[0011] In step S20, the electronic system 1 can capture the face image of the user through the camera 10 and provide it to the accelerator 12. In step S21, the auxiliary feature sensor 11 can sense the auxiliary features of the user. Finally, in step S22, the accelerator 12 of the electronic system 1 can calculate the emotional state of the user according to the face image. When it is determined that the emotional state satisfies a predetermined emotional state, the emotional level of the user is determined by analyzing the auxiliary features, and it is determined whether to issue a warning signal accordingly.
[0012] Specifically, in step S22, after receiving the face image, the accelerator 12 can determine the emotional state of the user through image recognition. In some embodiments, the accelerator 12 pre-stores at least one image feature of a predetermined emotional state, and determines whether the face image of the user belongs to a specific predetermined emotional state by identifying whether the face image satisfies any of the image features. For example, for monitoring the health state of the user, the image features stored by the accelerator 12 can be, for example, excitement, fatigue, anxiety, anger, etc., or other more excited emotional states. The accelerator 12 can determine whether the emotion of the user corresponds to one of the more excited emotional states by recognizing the face image.
[0013] Furthermore, when the accelerator 12 determines that the user corresponds to a predetermined emotional state, the accelerator 12 can further obtain the auxiliary features sensed by the auxiliary feature sensor 11 and determine the current emotional level of the user in that emotional state. In some embodiments, the auxiliary feature sensor 11 may be an infrared camera configured to sense auxiliary features such as, for example, a facial temperature change signal. Alternatively, the auxiliary feature sensor 11 may be configured to sense auxiliary features such as, for example, at least one of blood pressure, blood oxygen, pulse, body temperature, or other similar parameters, and may be other wearable or non-wearable sensing devices.
[0014] Taking the facial temperature change as an example, when the accelerator 12 determines that the user is likely to be in an excited state according to the face image, the accelerator 12 can further refer to the facial temperature change signal to determine whether the user's facial temperature change is too high, and further determine whether the emotional level of the user in the excited emotion is too high, that is, whether it is an overly excited emotional state. Specifically, the accelerator 12 can sense the temperature change of a part or all of the user's face, such as the forehead and cheeks, or sense the average temperature change of the entire face. In some embodiments, when the accelerator 12 determines that the user's emotional state has changed to a more excited emotional state such as excitement or anger according to the face image, the user's facial temperature generally increases. Therefore, the accelerator 12 can determine whether the increasing change in the user's facial temperature is still within the normal range through the facial temperature change signal. In contrast, when the accelerator 12 determines that the user's emotional state has changed to a more negative emotional state such as depression or fatigue, the user's facial temperature generally decreases, and the accelerator 12 can determine whether the decreasing change in the user's facial temperature is within the normal range through the facial temperature change signal.
[0015] In some embodiments, the normal range of the facial temperature change signal may have a fixed or variable temperature threshold. For example, the variable temperature threshold can take into account changes in the reference room temperature or other variables. For example, the accelerometer 12 may refer to an increase or decrease in the change of the room temperature. After subtracting the change in the facial temperature change caused by the change in the room temperature, by determining the emotional level according to the subtracted facial temperature change, the determination accuracy of the emotional level is improved. In this way, the accelerometer 12 can determine the emotional state of the user according to the facial image and determine the emotional level of the user according to the auxiliary features.
[0016] In some embodiments, the auxiliary feature used by the accelerometer 12 to determine the emotional level may be one of the physiological change signals such as blood pressure, blood oxygen, pulse, body temperature, etc. Specifically, similar to the above process of determining the emotional level according to the facial temperature change, the accelerometer 12 may determine the emotional level of the user according to whether the physiological change signal exceeds a predetermined range. For example, the normal range of the user's pulse rate is between 60 and 100 beats per minute. The accelerometer 12 may determine the emotional level of the user by determining whether the user's pulse exceeds the normal range or whether the amount of pulse change increases or decreases excessively.
[0017] And when the accelerometer 12 determines that the user's emotional level is too high or too low according to the auxiliary features, the accelerometer 12 may make a corresponding determination and issue a warning signal. In some embodiments, the accelerometer 12 can remind the user in the form of, for example, image display, sound, vibration, etc. Alternatively, the warning signal may be sent by the electronic system 1 to the user's emergency contact via the network. In this way, the electronic system 1 can determine whether the user's physiological state is harmful to health according to the calculated emotional state and the corresponding emotional level, and issue a corresponding warning signal according to the urgency.
[0018] In some embodiments, the accelerator 12 may be integrated on the motherboard, for example, or may be separately disposed on a keyboard, a mouse, a connector, or a peripheral device. In some embodiments, the accelerator 12 may be, for example, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Arithmetic Logic Unit (ALU), a Field Programmable Gate Array (FPGA), any other type of integrated circuit, a state machine, an Advanced RISC Machine (ARM)-based processor, other similar elements, or a combination of the above elements. The accelerator 12 can store a pre-trained artificial intelligence (AI) model. The artificial intelligence model may be trained by a dataset including a plurality of face images having pre-labeled emotional states. As a result, the artificial intelligence model can classify subsequent input face images and determine whether the face image corresponds to any of the predetermined emotional states. The accelerator 12 can determine the emotional state of the user by executing the artificial intelligence model and performing image recognition on the face image. For example, the artificial intelligence model stored in the accelerator 12 may be a machine learning model with or without supervision, or a deep learning model based on the above classification, such as the Region-based Convolutional Neural Networks (RCNN) series, which may be, for example, RCNN, Fast RCNN, Mask RCNN, and Faster RCNN, SSD, and the YOLO (You only look once, YOLO) series (e.g., YOLO, YOLOv2, YOLO 9000, and YOLOv3) and other models. Further, the above-described artificial intelligence model may be constructed based on a decision tree, logistic regression, SVM (support vector machine), naive Bayes, kNN, or other suitable algorithms.Furthermore, for the aforementioned deep learning model, aspects such as image preprocessing, model architecture, model depth, convolutional layer parameters, pooling layer, activation function, loss function, the specific architecture of each model, design concept, accuracy, and execution speed can also be considered. By combining, adding, adjusting, and modifying these aspects, a customized deep learning model can be created.
[0019] In some embodiments, the auxiliary feature sensor 11 may have one or more sensors according to the requirements of the electronic system 1. For example, when the accelerometer 12 needs to evaluate the emotion level based on the facial temperature change signal, the auxiliary feature sensor 11 may include, for example, an infrared camera for capturing and displaying the user's facial temperature change. When the accelerometer 12 determines the emotion level according to, for example, blood pressure, blood oxygen, pulse, body temperature, or other similar physiological change signals, the auxiliary feature sensor 11 may be, for example, a wearable sensing device such as a watch or a bracelet, or a non-wearable sensing device disposed on a mouse, a mouse pad, or earphones.
[0020] FIG. 3 is a flowchart of a physiological monitoring method according to an embodiment of the present invention. The physiological monitoring method shown in FIG. 3 can be applied to and executed on the electronic system 1 of FIG. 1. Specifically, the physiological monitoring method in FIG. 3 includes steps S30 to S37.
[0021] In step S30, the camera 10 can capture the user's face image and provide it to the accelerator 12. In step S31, the auxiliary feature sensor 11 may be an infrared temperature sensor configured to sense, for example, the user's facial temperature and provide a facial temperature change signal to the accelerator 12 accordingly. In step S32, in addition to the above-mentioned infrared temperature sensor, the auxiliary feature sensor 11 may further include other wearable or non-wearable sensing devices for sensing the user's physiological change signals such as at least one of blood pressure, blood oxygen, pulse, body temperature, or other similar parameters, and provide them to the accelerator 12.
[0022] In step S33, the accelerator 12 can determine the user's emotional state according to the face image. For example, the accelerator 12 can identify the lines in the face image through image recognition to determine the user's emotional state. It can identify whether the user is in an excited, fatigued, anxious, angry, or other more excited emotional state, or in a depressed, fatigued, or other more negative emotional state. In these emotional states, the accelerator 12 may further determine the user's emotional level in that emotion according to the facial temperature change signal and the physiological change signal.
[0023] In step S34, the accelerator 12 evaluates whether the user's emotional state and emotional level have reached the first warning level or the second warning level, and accordingly, proceeds to step S35 to issue a first warning signal, or proceeds to steps S36 - S37 to issue a second warning signal, or, if neither warning level has been reached, determines whether to continuously monitor the user's emotional state, facial temperature change, and physiological state. Specifically, the accelerator 12 can determine the user's emotional state according to the face image and determine the user's emotional level in the user's emotional state according to the facial temperature change signal. When the accelerator 12 determines that the user's emotional level corresponds to the first range, this may include cases where the emotional level is high (e.g., higher than the first upper limit) or low (e.g., lower than the first lower limit), and the accelerator 12 can determine that the first warning level has been reached and proceed to step S35 to instruct to issue a first warning signal.
[0024] Furthermore, as an aid to the facial temperature change signal, the accelerator 12 may further use a physiological change signal to further confirm whether the user's physiological state has reached a more urgent second warning state. For example, when the accelerator 12 determines that the user's emotional level corresponds to the second range according to the facial temperature change signal, this may include cases where the emotional level is too high (e.g., higher than the second upper limit, and the second upper limit is greater than the first upper limit), or the emotional level is too low (e.g., lower than the second lower limit, and the second lower limit value is lower than the first lower limit value), and the accelerator 12 can further refer to whether the physiological change signal exceeds a predetermined range. When the accelerator 12 determines that the user's emotional level is too high or too low and the physiological change signal exceeds the predetermined range, the accelerator 12 can accordingly determine that the user's emotion has reached the second warning level, which means that the user's physiological state has reached a more urgent situation, and thus proceed to step S36 to attempt to issue a second warning signal.
[0025] In step S36, the electronic system 1 can evaluate whether its network function is on or whether it has the authority to turn on the network function. If the network function is on or if it has the authority to turn on the network function, the electronic system 1 proceeds to step S37 and issues a second warning signal to the user's emergency contacts via the network. If the evaluation result is negative, the accelerator 12 gives up issuing the second warning signal.
[0026] In summary, the electronic system and the physiological monitoring method of the present invention can perform real-time monitoring of the user's physiological state and issue a warning signal when the user's physiological state is critical. In addition, it can effectively warn about the dangerous state of the user caused by fatigue, excessive use of electronic devices, or stress, enabling the user to take emergency measures accordingly.
Industrial Applicability
[0027] The electronic system and the physiological monitoring method of the present invention are applied to image recognition technology and can monitor the user's emotional and physiological states.
Explanation of Reference Numerals
[0028] 1: Electronic system 10: Camera 11: Auxiliary feature sensor 12: Accelerator S20~S22, S30~S37: Steps
Claims
1. A camera configured to capture a face image of a user, an auxiliary feature sensor configured to sense auxiliary features of the user, an accelerator configured to calculate an emotional state of the user according to the face image, determine an emotional level of the user in the emotional state by analyzing the auxiliary features, and determine whether to issue a warning signal accordingly when it is determined that the emotional state satisfies a predetermined emotional state, An electronic system comprising the above.
2. The accelerator determines the emotional state of the user by recognizing the face image through image recognition technology. The electronic system according to Claim 1.
3. The accelerator determines the emotional state of the user by performing the image recognition technology on the face image through execution of an artificial intelligence model. The electronic system according to Claim 2.
4. The auxiliary features include a facial temperature change signal, and the accelerator determines the emotional level of the user in the emotional state by analyzing the facial temperature change signal of the user. The electronic system according to Claim 1.
5. The auxiliary features include a physiological change signal, the physiological change signal includes at least one of blood pressure, blood oxygen, pulse, and body temperature, and the accelerator determines the emotional level of the user in the emotional state by analyzing the physiological change signal. The electronic system according to Claim 1.
6. The auxiliary features further include a physiological change signal, the accelerator determines the emotional level of the user by analyzing the facial temperature change signal, and then verifies the emotional level by analyzing the physiological change signal. The electronic system according to Claim 4.
7. When the accelerator determines that the emotional level of the user corresponds to a first range, the accelerator instructs a first warning device to issue a first warning signal. When the accelerator determines that the emotional level of the user corresponds to a second range, the accelerator checks that the network setting of the electronic system is on and then issues a second warning signal via the network. The electronic system according to Claim 1.
8. The accelerator is disposed on the motherboard or the connector, or on an edge device connected via a connection terminal or a transmission line. The electronic system according to claim 1.
9. Taking a facial image of the user; Sensing auxiliary features of the user; Calculating the emotional state of the user according to the facial image, and when it is determined that the emotional state satisfies a predetermined emotional state, determining the emotional level of the user in the emotional state by analyzing the auxiliary features, and determining whether to issue a warning signal accordingly; A physiological monitoring method comprising:
10. Determining the emotional state of the user by recognizing the facial image through an image recognition technology; The physiological monitoring method according to claim 9, comprising:
11. Determining the emotional state of the user by executing the image recognition technology on the facial image through the execution of an artificial intelligence model; The physiological monitoring method according to claim 10, comprising:
12. The auxiliary features include a facial temperature change signal, and the physiological monitoring method includes: Determining the emotional level of the user in the emotional state by analyzing the facial temperature change signal of the user; The physiological monitoring method according to claim 9, comprising:
13. The auxiliary features include a physiological change signal, the physiological change signal includes at least one of blood pressure, blood oxygen, pulse, and body temperature, and the physiological monitoring method includes: Determining the emotional level of the user in the emotional state by analyzing the physiological change signal; The physiological monitoring method according to claim 9, comprising:
14. The auxiliary features further include a physiological change signal, and the physiological monitoring method includes: Determining the emotional level of the user by analyzing the facial temperature change signal, and then verifying the emotional level by analyzing the physiological change signal; The physiological monitoring method according to claim 12, comprising:
15. When it is determined that the emotional level of the user corresponds to a first range, instructing to issue a first warning signal; When it is determined that the emotional level of the user corresponds to a second range, after confirming that the network setting is on, issuing a second warning signal via the network; The physiological monitoring method according to claim 9, comprising
Citation Information
Patent Citations
Stress evaluation method and stress evaluation system
JP2017176762A
Feeling improvement device and feeling improvement method
JP2018102705A