Care system and automatic care method

Abstract

A care system and automated care method are provided. The care system is suitable for a cared-for person. Sensors positioned around the cared-for person are used to determine the cared-for person's physiological condition, particularly by monitoring sleep disorders to determine the cared-for person's health. The home care system receives the cared-for person's voice through an audio receiver. After subtracting background sound signals, the audio emitted by the cared-for person is obtained. An image sensor is used to capture the cared-for person's image, from which motion detection is performed to obtain an image of the cared-for person's posture. When combined with other physiological and environmental sensors and after learning various data, an artificial intelligence (AI) technology can be used to establish a care warning prediction model to determine whether the cared-for person's condition has reached a warning threshold.

Description

Technical Field

[0001] The present invention relates to a care system, in particular to a care system for monitoring data of a care recipient's sleep or specific status to achieve early warning and reminder effects, as well as an automatic care method. Background Art

[0002] The existing medical care system mainly operates by installing physiological sensors on or around the person being cared for, and connecting them to a care center through a network. This allows the care center to obtain physiological information transmitted by the terminal's physiological sensors at any time and analyze the physiological status of the person being cared for.

[0003] Whether existing medical care systems can truly achieve their goals of caring for patients depends crucially on the immediate analytical capabilities of the care center and the physiological data provided by the patient's physiological sensors. However, these sensors, commonly worn on patients, provide only limited physiological information and interpret various physiological data independently. Furthermore, accurate physiological status assessment requires a comprehensive assessment based on the patient's past medical records, current environmental factors, and the collected physiological data, a process that current technologies are unable to effectively implement.

[0004] In terms of overall physiological data, professional physicians rely on their experience to make judgments, but this still cannot account for all possibilities. Therefore, they cannot effectively issue warnings or reminders for specific conditions, which undermines the purpose of home care. Specifically for home care purposes, physiological sensors used in home care generally cannot compare to equipment in medical institutions. The physiological data obtained is limited, and they cannot accurately interpret specific conditions, such as the care recipient's sleep quality. This is a key factor affecting home care, affecting not only the health of the care recipient but also the quality of life of the caregiver due to the constant need to pay attention to the care recipient's condition. Summary of the Invention

[0005] The present invention discloses a care system and an automatic care method, which include a machine learning method suitable for a person in need of care at home or in a medical institution. The system achieves the purpose of home care through physiological sensors, image sensors, audio sensors and other devices around the person. One of its main purposes is to enable care providers to perform more efficient care through care programs generated by machine learning mechanisms.

[0006] According to an embodiment of the main architecture of a care system, it includes a data processing unit and multiple sensors. The multiple sensors include a first set of sensors that operate full-time and generate sensory data about a care recipient, and a second set of sensors that can be activated in response to a command from the data processing unit to generate sensory data about the care recipient. When the data processing unit determines that a first set of sensory data generated by the first set of sensors has reached a first threshold, the second set of sensors is activated, and the second set of sensors generates a second set of sensory data. Furthermore, when the data processing unit receives the second set of sensory data, it can determine whether the conditions for issuing an alert are met based on a second threshold.

[0007] According to an embodiment, the main components of the care system include a data processing unit and various sensor components coupled to the data processing unit, such as an audio receiver for receiving the voice of the care recipient and obtaining the audio signal emitted by the care recipient after deducting the background sound signal; an image sensor for obtaining an image of the care recipient and performing motion detection therefrom to obtain an image of the care recipient's posture. In this way, the care system can execute an automatic care method through the data processing unit.

[0008] In this method, an image sensor constantly captures the care recipient's posture and performs motion detection. When the image change value of the care recipient exceeds a threshold, an audio receiver is activated to receive the audio emitted by the care recipient. The audio is used to determine the care recipient's condition and whether a warning threshold has been reached. If so, an alert message is generated.

[0009] Furthermore, the care system may also include one or more physiological sensors for sensing the physiological state of the person being cared for and generating physiological data. In this way, it can be determined based on the physiological data whether to generate a warning message.

[0010] Preferably, in the step of obtaining audio information, the audio comparison sound samples emitted by the care recipient, including volume and / or audio, can be used to determine whether the care recipient is in a situation requiring real-time care, such as respiratory distress such as respiratory arrest, choking sounds, or sputum sounds.

[0011] Furthermore, the care system includes a machine learning unit that generates a care alert prediction model based on the audio and gesture images obtained by the data processing unit and compared with clinical data. This model can incorporate environmental sensor data and information received through the user interface as factors in the machine learning unit's creation of the care alert prediction model.

[0012] According to one embodiment, the automatic care method is applied to the care system, and the automatic care method set includes using an image sensor to capture the posture image of the cared person at all times and perform motion detection. When it is determined that the image change value of the cared person exceeds a threshold, the audio receiver is activated to receive the audio emitted by the cared person, and the condition of the cared person is judged based on the audio to determine whether the warning threshold is reached.

[0013] Furthermore, other data used include physiological data generated by one or more physiological sensors sensing the physiological state of the care recipient, and audio comparison to determine whether the care recipient has specific conditions that require care, such as respiratory arrest, choking sounds, or sputum sounds, etc. A machine learning method can be used to establish a care warning prediction model based on the audio and posture images emitted by the care recipient and compared with clinical data. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 Schematic diagram showing the operational scenario of the care system;

[0015] Figure 2 A schematic diagram showing an embodiment of each functional module in the care system;

[0016] Figure 3 A flow chart showing an embodiment of an automatic care method in a care system;

[0017] Figure 4 A flow chart showing an embodiment of using an image sensor to drive surrounding devices;

[0018] Figure 5 A flowchart showing an embodiment of establishing a prediction model by a machine learning method in an automated care method;

[0019] Figure 6 A schematic diagram showing an embodiment of a care system using artificial intelligence technology to establish a care alert prediction model;

[0020] Figure 7 A flowchart showing an embodiment of an automatic care method for determining the status of a care recipient by using audio information; and

[0021] Figure 8 A flowchart of an embodiment of an automatic care method for predicting abnormal conditions of a care recipient using a care alert prediction model is shown. DETAILED DESCRIPTION

[0022] Given that the current care system relies on manpower (care providers) for care and may need to provide 24-hour care services under certain circumstances, the demand for all manpower in medical institutions places a huge burden on home care providers. Even with various sensor technologies to assist in care, because there is no care alert mechanism based on the individual physiological and environmental conditions of the care recipient, human attention is still required to avoid the system's inability to accurately alert real emergencies or generating false alarms that cause care providers to frequently resolve erroneous information. Thus, the present invention discloses a care system and automatic care method that utilizes multiple sensor technologies, wherein machine learning technology is also introduced to establish a care alert prediction model to provide automated and personalized care alert services. One of the main technical purposes is to alleviate the burden on traditional care providers and avoid the problems caused by traditional manpower or limited sensing technology.

[0023] Examples of care systems can be found in Figure 1 The operating scenario diagram shown in FIG.

[0024] The figure shows a care recipient 10 lying in bed, either at home or in a designated nursing home. Homes and designated nursing homes do not have the same advanced monitoring equipment as hospitals. Therefore, the care system primarily senses the care recipient's 10 physiological data, providing preliminary care functions. However, further, with the addition of artificial intelligence and machine learning methods, it is still possible to determine the care recipient's physiological condition from limited data. In this example, the care recipient 10 is connected to a physiological sensor, such as a sensing bracelet 101, which can obtain physiological data such as body temperature, pulse, respiration, blood pressure, and heart rate. This data can be displayed on a corresponding physiological data display 103. Generally, warning thresholds for various data levels are also set to perform preliminary care functions. Furthermore, the system can be equipped with a blood oxygen detector 102 connected to the care recipient 10, such as a finger, to obtain blood oxygen data, which can be monitored on an oximeter 104.

[0025] In addition, the care system may also include various other physiological sensing devices, such as a chest strap that can obtain the chest rise and fall, chest movement frequency, chest depth difference, etc. of the care recipient 10 as a basis for judging the physiological status of the care recipient 10.

[0026] In this embodiment, an image sensor, such as camera 105, is located next to the bed. This sensor can capture images of the care receiver 10 at all times and perform motion detection during the capture process. To perform motion detection, a background image is first created from continuous video, including stationary objects such as the bed, wardrobe, and fixtures. The image is then compared to the previous and next images to determine the images of movement. The changes detected can be used to determine the posture of the care receiver 10. These posture images can then be used to determine the care receiver's behavior and condition. For example, the posture images can be used to determine whether the care receiver 10 has turned over during sleep, made rapid movements of their hands and feet, fallen out of bed, or gotten out of bed. A warning threshold for posture changes can be set, and an alert message can be generated when an abnormal change occurs.

[0027] Furthermore, an audio receiver, such as microphone 107 shown in the figure, can be installed at the bedside to receive sounds from the care receiver 10, particularly breathing-related sounds. When using microphone 107 to receive ambient sounds, background audio can be first established. Background audio refers to ambient sounds such as equipment operation, cooling fans, and air conditioning. Once the background audio is established, the audio recorded by microphone 107 can be deducted from this background audio to identify the sounds originating from the care receiver 10.

[0028] For example, the care system can be used to monitor the sleep quality of the care receiver 10. Using microphone 107, it can record the sounds made by the care receiver 10 while sleeping to identify conditions requiring immediate care, such as coughing, spitting, and breathing sounds. It can also identify apnea based on continuous sounds. Thus, audio information becomes crucial information for assessing the care receiver 10, particularly regarding physiological conditions related to sleep.

[0029] The physiological data of the care recipient obtained by various sensors as recorded in the above embodiments can be combined with feedback information actively triggered by the care provider or the care recipient to obtain key data within a critical time through the system annotation mechanism. Unlike the big data required to build models in traditional artificial intelligence, the data generated by this type of care system is personalized data, and a care warning prediction model can be established based on the individual's situation. It can provide personalized care warning services and effectively alleviate the burden on care providers.

[0030] It is worth mentioning here that the automatic care method applied in the above-mentioned care scenario can apply various physiological sensors that can sense the care recipient 10, such as sensing the pulse of the care recipient 10 and the heart rate status reflected by the sensing bracelet 102, and sensing the blood oxygen status through the blood oxygen detector 102, so that data correlation can be established between different sensing data, and the situation of the care recipient 10 can be judged individually or comprehensively, so that the care recipient 10 can receive more complete care.

[0031] Figure 2 Next, a schematic diagram of an embodiment of each functional module in the care system is shown, wherein the functional modules can be implemented by software, or by software combined with hardware devices.

[0032] The main components of the care system are various sensing devices around the care recipient, which are equipped with a data processing unit 201 for processing the data generated by each sensing device, such as the central processing unit of a computer system. It can have powerful data processing capabilities. In addition to storing the data that has already been generated, it can also effectively process the data generated in real time and execute machine learning algorithms. After performing big data analysis, a model for predicting the physiological condition of the care recipient is established.

[0033] According to an embodiment of the care system, it includes a variety of sensors electrically connected to a data processing unit 201, which may include various physiological sensors 202, an audio receiver 203, an image sensor 204, and an environmental sensor 205. It may also include a machine learning unit 207 coupled to this data processing unit 201, and can receive information input by the care recipient or the care provider after being connected to the user device 210 through a user interface 209.

[0034] The care system can also connect to an external system 21 or a medical cloud 23 via a network 20 using a specific communication protocol. In addition to providing data on the person being cared for, it can also obtain various clinical data, individual or group big data analysis results, etc., which serve as data for the care system to implement artificial intelligence.

[0035] In this care system, the data processing unit 201, such as the system's central processor, integrates the data generated by various peripheral devices, performs warning judgments after processing, and connects with external systems 21 (such as medical institutions, care centers, etc.), medical cloud 23, etc., and also includes a care warning prediction model established using artificial intelligence to predict the physiological condition of the care recipient.

[0036] The audio receiver 203 is used to receive the voice of the care recipient. As described in the above embodiment, after deducting the background sound signal, the audio signal emitted by the care recipient can be obtained. Therefore, in the automatic care method, the system can obtain the audio emitted by the care recipient based on the sound signal obtained by the audio receiver 203, and then compare the sound samples (including volume and / or audio) to determine whether the care recipient needs care, such as (but not limited to) respiratory arrest, choking sounds, or sputum sounds.

[0037] Image sensor 204 is used to capture images of the care recipient and perform motion detection. According to the methods described in the above embodiments, the care recipient's posture image is determined at any time during full-time photography, and motion detection is performed. According to one embodiment, the motion detection results provided by image sensor 204 can be used as a basis for initiating audio signal reception and determination. For example, if the image determines that the care recipient has fallen asleep, full-time photography is continued and posture changes during sleep are determined. If the image change value of the care recipient exceeds a threshold during this time, it may indicate a sleep disorder, including respiratory distress, or a condition requiring or not requiring intervention by a caregiver. The audio receiver 203 is then activated to receive audio signals from the care recipient. The system can then determine the care recipient's condition based on the audio and whether a warning threshold has been reached. If the warning threshold is reached, indicating that the audio indicates the care recipient is in a critical and distressed condition, such as severe respiratory arrest or shortness of breath, an alert message is generated.

[0038] In this way, the care system can perform care tasks at home or in specific settings based on the audio receiver 203 and image sensor 204 installed on the care recipient. Furthermore, the care system may also include one or more physiological sensors 202 to sense the care recipient's physiological state and generate physiological data, such as body temperature, pulse, respiration, blood pressure, heart rate, etc. In actual operation, the care system is not limited to these physiological sensors and the data they obtain. In the automatic care method executed by the data processing unit 201, after receiving this physiological data, it can determine individually or comprehensively whether to generate an alert message.

[0039] The environmental sensor 205 may be a temperature and humidity sensor, an air quality sensor, an air pressure sensor, etc. These environmental data may be used to correct the threshold for judging the physiological condition of the care recipient, because both the care recipient and the instrument may be affected by these environmental factors. For example, the care recipient may easily feel physically uncomfortable in an environment with poor air quality or abnormal indoor temperature and humidity.

[0040] In one embodiment, the care system may further include a machine learning unit 207 coupled to the data processing unit 201. The machine learning unit 207 can obtain the audio and posture images emitted by the care recipient based on the data processing unit 201, compare them with clinical data, and establish a care warning prediction model through big data analysis methods.

[0041] Furthermore, when implementing the care alert prediction model through the machine learning unit 207, the data used may also include environmental data of the care recipient sensed by the environmental sensor 205 and physiological data generated by the various physiological sensors 202, which become factors in the machine learning unit 207 establishing the care alert prediction model. When the machine learning unit 207 executes machine learning, the care system receives alert information generated by the care recipient or care provider using the user device 210 through the user interface 209. This provides the care recipient, care provider, or others with guidance regarding situations that the system has not yet identified, and also becomes a factor in the machine learning unit 207 establishing the care alert prediction model.

[0042] For example, the care system can connect to the user device 210 via the user interface 209 and receive information input from the care recipient or the care provider. That is, when the system uses the machine learning unit 207 to perform big data analysis to establish a care alert prediction model, the care recipient's actual physiological sensations can be input into the care system through the user device 210 via the user interface 209, providing feedback to the machine learning unit 207. Similarly, the care provider can also generate feedback based on their own judgment and input it into the care system via the user device 210 via the user interface 209. This allows the system to obtain information beyond the data generated by the sensors, which can be used to adjust the care alert prediction model established by the system using machine learning.

[0043] Furthermore, the care system can connect to an external system 21 or a medical cloud 23 via a network 20 using a specific communication protocol. In addition to providing data on the person being cared for, it can also obtain various clinical data, individual or group big data analysis results, etc., which serve as data for the care system to implement artificial intelligence.

[0044] According to an embodiment of the care system, various image, sound, physiological and environmental sensors are installed on or around the person being cared for, such as Figure 2 The physiological sensors (202), audio receivers (203), image sensors (204) and various environmental sensors (205) mentioned in the embodiment, when executing the automatic care method, one or two of these sensors can be in operation at any time, for example, the physiological sensors (which can be called the first group of sensors) can be processed by the data processing unit (201, Figure 2 ) processes and determines that when physiological data exceeds a threshold, a warning message is generated, that is, other sensors in sleep state (power saving mode) (called the second group of sensors) are activated, so that the system can receive a variety of sensing signals at the moment. According to the data collected before and after, including information that is expected to be abnormal but the caregiver is indeed in an emergency state, as well as information that is unexpected but the caregiver has an emergency, it becomes very important data for machine learning.

[0045] In one embodiment, among the multiple sensors in the care system, two or more sensors (a first group of sensors) may be used for full-time operation. When the system comprehensively determines that the warning conditions are met based on the signals generated by one or more of the sensors, in addition to generating a warning message, the system also activates other sensors that are not running full-time (a second group of sensors), so that the care system can obtain more comprehensive information about the person being cared for.

[0046] For the purpose of machine learning, the care system can obtain the conditions for the first group of sensors to generate warning information based on the model obtained by the machine learning algorithm modeling, and the machine learning unit 207 ( Figure 2 ) further acquires data from a second set of sensors. This can be used to learn and update feedback from caregivers and care providers to establish criteria for determining whether the care recipient is in an emergency state and to build a predictive model. The predictive model can be used to predict the care recipient's physiological condition based on limited or complete sensor data.

[0047] Figure 3 A flowchart illustrating an embodiment of an automatic care method in a care system is provided. The care system is provided with a plurality of sensors electrically connected to a data processing unit in the system, including a first group of sensors that operate full-time and generate sensing data about the care recipient, and a second group of sensors that generate sensing data about the care recipient after being activated according to instructions from the data processing unit.

[0048] When the care system is in operation, the first set of sensors (which may include one or more sensors) may operate full-time and continuously generate a first set of sensing data about the care receiver (step S301). The data processing unit then determines, based on a first threshold, whether the first set of sensing data generated by the first set of sensors reaches the threshold, thereby determining whether an alert condition is met (step S303).

[0049] In one embodiment, in addition to the system automatically determining whether the situation meets the threshold for generating an alert during this process, the system can also use Figure 2 The display provides the user device (210) with an external instruction input through the user interface (209) (step S315), which assists in determining whether the current situation meets or does not meet the warning condition, so as to modify the original judgment and allow the system to execute or not continue to execute the following steps. Under this measure, the defects in the system operation can be supplemented, the quality of care can be improved, and both can be reflected in the subsequent machine learning.

[0050] When the sensing data has not reached the warning condition (No), step S301 is repeated and the first set of sensors continues to operate. It should be noted that at this time, the care provider (or a specific person) can still intervene in the care process. That is, even if the sensing data has not reached the warning condition established by the system, if the care provider finds that there may still be a need for an alert, he or she can do so by Figure 2 The user device (210) displays the alarm at this stage through the user interface (209), allowing the process to continue, such as step S305, to start the second set of sensors. The data generated by the intervention of the care provider is still important information for subsequent analysis and machine learning.

[0051] If the first set of sensing data reaches the warning condition (yes), the data processing unit generates an instruction, i.e., starts the second set of sensors (which may include one or more sensors), and generates the second set of sensing data through sensing (step S305). At this time, the data processing unit receives the second set of sensing data and determines whether the conditions for issuing a warning message are met. If the second set of sensing data also reaches the second threshold set by another system, a warning message is issued. In a specific embodiment, since starting the second set of sensors means that the system judges that it is at a critical moment, when the second set of sensors is started, the first set of sensing data and the second set of sensing data can be noted and retained for a period of time to be fed back to the machine learning unit (such as Figure 2 , 207).

[0052] Furthermore, an alert can be issued based on the prediction model's judgment. In this example, the care system can store the data generated by the first set of sensors around the time the alert message was generated, and also store the sensory data generated by the second set of sensors that were activated at that time (step S307). The stored sensory data is then input into the prediction model established by the machine learning algorithm (step S309).

[0053] According to another embodiment, in addition to inputting the sensor data into the prediction model, external data can also be added (step S317), for example, Figure 2 The external data such as the displayed external system (21) or health cloud (23) can make the entire judgment basis more complete through the external data obtained in real time.

[0054] The prediction model is built by learning from data from medical institutions, health clouds, and the care recipient's personal historical data and clinical data, and then deriving a model based on feature correlation. The model can also optimize its parameters based on feedback from actual conditions, effectively utilizing limited sensory data. For example, the data generated by the first and second sets of sensors can be used to predict the care recipient's physiological condition and determine whether to issue an alert (step S311).

[0055] If the prediction model determines that the situation is not an emergency (no), the process returns to step S301. At this point, the first set of sensors can continue to operate and generate continuous sensing data, while the second set of sensors can return to power saving mode without full-time operation. Conversely, if it is determined that an alert is necessary, the system will generate an alert message, as in step S313.

[0056] Likewise, in step S311, even if it is determined that there is no emergency, the system still provides care providers with the following options: Figure 2 The displayed user device (210) sets an alert via the user interface (209), including noting and retaining data for a period of time to feed back to the system.

[0057] In another embodiment, in addition to the second group of sensors being activated only after determining that the warning conditions are met, the first and second groups of sensors can run in the background all the time and continuously collect data. When it is determined that the warning conditions are met based on the data generated by the first group of sensors, the system will actively mark and store the sensing data of the first and second groups of sensors over a period of time (such as 1 minute before and after). The data during this period should be more critical data and can be used for subsequent data analysis, including providing machine analysis.

[0058] In yet another embodiment, the image sensor (204, Figure 2 ) can also be used as the first sensor of the full-time operation. When it is determined that there is any abnormality or the image exceeds the action threshold, an alarm is generated and other sensors are activated. For the embodiment, please refer to Figure 4 .

[0059] Figure 4 A flow chart showing an embodiment of using an image sensor to drive surrounding devices.

[0060] When the care system utilizes an image sensor and an audio receiver as sensing devices for the care recipient, the image sensor, such as the first set of sensors described in the aforementioned embodiment, can be used for full-time recording (step S401), capturing images of the care recipient at every moment. However, all images do not need to be stored until specific system conditions are met. For example, during full-time recording, motion detection can also be performed (step S403). The changes between consecutive images can be determined and compared against a threshold set by the system (such as the first threshold described in the aforementioned embodiment). Image recording begins only when the change exceeds this threshold (step S405). This action can conserve storage space for images.

[0061] When image movement is detected (a certain degree of movement after comparison with a threshold value), the degree and manner of the image movement can be determined (step S407). At this time, the care system compares the degree and manner of the movement before and after to the conditions for issuing an alert. If the degree and manner of the change meet the conditions for generating an alert, an alert message is generated, as in step S409. At the same time, in one embodiment, the system will actively mark the image, time, and related data of a critical period of time (for example, 3 minutes before and after). For example, the posture image of the care recipient can be obtained from the image movement detection, and the mode of movement can be determined from it. For example, rolling over, falling, violent body movement, etc., may trigger an alert. The data of the critical moments before and after the triggering of the alert will be retained for other purposes, such as this key data will become data for subsequent analysis and machine learning.

[0062] In this example, when an alert message is generated, that is, when it is determined that the change in the care recipient's image exceeds a first threshold set by the system, an audio receiver is activated to receive audio signals from the care recipient (step S411). The audio receiver, such as the second set of sensors described in the above embodiment, activates the microphone and other audio receivers to begin recording breathing audio. Once background audio is subtracted, a relatively clean sound of the care recipient's own voice can be obtained (step S413). The sensor data is obtained and stored in the system's storage device before further analysis. Similarly, the audio signals recorded in response to the alert message will also be noted and retained, serving as reference data for subsequent analysis and machine learning.

[0063] According to the automatic care method performed by the care system, the condition of the person being cared for can be judged from the audio, especially for the problem of sleep disorders, wherein the audio emitted by the person being cared for can be compared with the sound samples stored in the system, including frequency and volume samples, or personalized sound samples can be established through past data, or sound samples generated by group data, so as to judge whether the person being cared for needs real-time care, such as respiratory arrest, choking sounds, or sputum sounds and other respiratory distress conditions. When judging whether the warning threshold is reached (such as the second threshold in the above embodiment), if the warning threshold has been reached, such as judging that the respiratory arrest time is too long (exceeding a certain time threshold), the choking sounds or sputum sounds indicate a dangerous situation, etc., a warning message is generated.

[0064] In particular, the first and second thresholds can be set by the system, and the threshold for generating an alert can also be set by the care provider according to actual conditions.

[0065] Furthermore, the care system can also use physiological data generated by one or more physiological sensors (which can be classified as the second group of sensors) to determine whether to generate an alert. Similarly, various physiological and environmental data generated within a key moment set by the system will be noted and retained for subsequent analysis and machine learning purposes.

[0066] Furthermore, the care system also uses a machine learning method to establish a care warning prediction model based on the audio and posture images emitted by the care recipient and clinical data.

[0067] It is worth mentioning that, as described in the above embodiment, the automatic care method can also include the audio receiver in the first group of sensors to receive the audio of the person being cared for full-time. When the characteristics of the sound (frequency, volume) reach a threshold, the image sensor listed as the second group of sensors is activated.

[0068] In another embodiment, the recording of the care recipient's voice messages can be turned on full-time. Similarly, the system can set alarm conditions for valid audio after excluding background audio. When an alarm condition occurs, in addition to retaining the data of the critical moment, the video recording program is further activated to record, annotate and retain the video of the care recipient generated during this critical moment, which can also become data for subsequent analysis and machine learning.

[0069] Related implementation examples Figure 5 A flow chart of an embodiment of establishing a prediction model by machine learning method in the automated care method is shown.

[0070] Initially, as in step S501, the system uses various sensors to collect the care receiver's breathing audio, posture images, physiological data, and environmental data. During data processing, as in step S503, background data must be established to obtain clean data. In step S505, the system also sets alert conditions based on the care receiver's individual circumstances. This is essential for implementing care in specific settings, such as at home and in medical institutions.

[0071] In addition to continuously acquiring the care receiver's breathing audio, posture images, physiological data, and environmental data, actual records are then obtained, as in step S507. These can come from clinical data provided by medical institutions and the Health Cloud, or from personal actual event data generated by the care receiver through the system's feedback mechanism. Thus, as in step S509, machine learning methods are implemented to perform big data analysis. Generally speaking, after big data analysis, a care alert prediction model can be established to predict the care receiver's physiological condition. The care alert prediction model is primarily based on data collected from sensors around the care receiver, combined with actual data provided by the Health Cloud, and information actively fed back to the system by the care receiver and their care provider. Software algorithms learn the characteristics of the data and establish correlations between the data.

[0072] As shown in step S511, after big data analysis and algorithm development, a care alert prediction model is established. In specific applications, the model can determine whether the care receiver is experiencing a sleep disorder such as respiratory distress, or whether intervention by a care provider is required or not, based on the audio, posture images, and / or physiological data emitted by the care receiver during sleep. In this way, verification can be repeatedly performed based on facts (step S513), and the parameters of the care alert prediction model can be adjusted based on actual data.

[0073] For further details, please refer to Figure 6 The illustrated embodiment is a schematic diagram of an embodiment of a care system using artificial intelligence technology to establish a care alert prediction model.

[0074] The figure shows an artificial intelligence module 60 implemented by combining software and hardware computing capabilities in the care system. It includes a specific machine learning method 601, which is trained based on various sensory data provided by the system. It can identify the correlation between the data and achieve predictive goals.

[0075] When running the machine learning method 601, data is first obtained, such as the database 603 in the figure, and image data 61 generated by various sensors are obtained. The image includes the posture of the caregiver, for example, the posture is such as turning over, hand and foot movements; physiological data can be obtained from the chest rise and fall, frequency, depth, etc. After training, the ability to recognize posture can be established, and the correlation between the image of the caregiver's posture and other data can be established; audio data 62, such as the sounds made by the caregiver during normal work and sleep, can identify the sounds generated by specific physiological reactions, such as sputum sounds, choking sounds, and sounds caused by respiratory arrest, etc., and various recognition can be established. The correlation between the audio and other data; physiological data 63, such as the body temperature, heart rate, respiration, pulse, blood oxygen, and other physiological data of the care recipient, in addition to determining the normal and abnormal conditions of physiological data, it is also necessary to establish the correlation with other data at any time; and environmental data 64, the environmental data of the care recipient sensed by various environmental sensors (which can be listed in the first group of sensors or the second group of sensors), becomes one of the factors used by the machine learning method to establish the care warning prediction model 605. The data can include information such as temperature and humidity, air quality, and climate of the care recipient's daily life, and can also be correlated with other data through learning. Then analyze the data to find the overall correlation between the various data.

[0076] In particular, when the artificial intelligence employed in the care system proposed in the present invention utilizes personalized data such as the aforementioned image data 61 , audio data 62 , physiological data 63 , and environmental data 64 , a personalized care alert prediction model can be established.

[0077] Afterwards, the machine learning method 601, in conjunction with the historical and real-time data collected by the database 603 and the actual recorded physiological reactions, can derive personalized rules, and then establish a care warning prediction model 605 for predicting the physiological condition of the care recipient. Even before any dangerous events occur, it is possible to predict what may happen based on various data, thereby achieving the purpose of advance prevention and warning.

[0078] Furthermore, the artificial intelligence module 60 can also be trained and learned from data provided by specific sources through machine learning methods 601. For example, de-identified clinical data of various groups can be obtained from the medical cloud 611. In addition to providing the data required for training, it can also be used to verify the care alert prediction model 605.

[0079] The artificial intelligence module 60 receives feedback information from the care recipient, care provider, or other means through the feedback system 612. As described in the above embodiment, the care system is provided with a user interface that can be used to receive warning information generated by the care recipient or other persons through a user device, which becomes one of the factors used by the machine learning method 601 to establish the care warning prediction model 605. This type of data will be used to verify the prediction model and to adjust the parameters of the care warning prediction model 605 in the artificial intelligence module 60.

[0080] In this way, the machine learning method 601 set in the artificial intelligence module 60 will integrate various data obtained by the care system through the data processing unit, and can set different weights for different data according to the needs of the machine learning method. In addition to personalized data, as well as data from the medical cloud 611 and the feedback system 612, big data analysis is performed to establish a care warning prediction model 605, which is used to determine whether the condition of the care recipient has reached a specific warning threshold, which is also whether the condition for issuing a warning message is met.

[0081] It is worth mentioning that the medical cloud 611 can also obtain terminal data from the care system. In this way, the calculation technology in the medical cloud 611 can obtain information with de-identified annotations, including time information and identity information, and at the same time collect relevant physiological data and environmental data. These data with annotations are obviously relevant to the medical cloud 611 and the artificial intelligence module 60 of the care system.

[0082] These annotated physiological data and various other data are the effective data that artificial intelligence technology needs to learn, enabling the constructed model to identify and predict physiological information. Machine learning and verification continue during the operation process to build a more complete medical cloud 611.

[0083] In various embodiments of the care system, the physiological condition of the care recipient can be determined by audio information, such as Figure 7 Flowchart of an embodiment of the automated care method is shown.

[0084] Initially, as in step S701, the audio receiver in the care system receives the audio information around the person being cared for, as in step S703. During implementation, background audio can be established in the initial state to form a background sample, so that when the system is introduced during actual operation, clean audio information can be obtained.

[0085] In step S705, through software methods, the data processing unit in the care system compares the obtained clean audio with the pre-established sound samples. After comparing the sound characteristics (such as frequency, volume characteristics, etc.), as in step S707, the sounds made by the care recipient, such as choking sounds, sputum sounds, changes in audio, etc., can be identified. These audio signals can be used to judge the physiological condition of the care recipient, as in step S709, to determine whether there is a situation that requires real-time care, one of which is to determine whether there is a situation of respiratory distress (such as respiratory arrest, choking). Then in step S711, the threshold provided by the comparison system is determined to determine whether the warning conditions are met, and subsequent medical measures can be performed accordingly.

[0086] exist Figure 8 , a flow chart of an embodiment of applying a care alert prediction model to predict abnormal conditions of a care receiver in an automatic care method is described.

[0087] In step S801, the care system acquires real-time audio, video, and sensory data from various sensors. In step S803, the system performs preliminary data processing and filtering before feeding the data into a care alert prediction model. The model then determines the care receiver's current physiological condition based on the preceding and subsequent data and the correlations between the various data types, and then performs a prediction. In step S805, the system determines whether to generate an alert message. If the alert conditions are met, the system proceeds to step S807, where an alert message is issued. Otherwise, the process returns to step S801.

[0088] It is worth mentioning that during the step of issuing the warning information, feedback can be provided for the relevant data, and information on the actual situation can be obtained through the feedback system 80, which can then be used by the machine learning unit 82 to optimize the care warning prediction model.

[0089] In summary, the care system and automatic care method described in the above embodiments are applicable to the care environment where people who need care are located. In this automatic care method, in addition to using the data generated by various sensors in the care system (such as various contact or non-contact sensors) to integrate various data to determine physiological conditions, it can also use machine learning methods to train and learn from various data to derive the correlation between data, especially using personalized physiological and environmental data to establish a personalized care warning prediction model for predicting the physiological condition of the care recipient, so as to achieve the purpose of care at home or in specific occasions. Through automatic care methods, or the introduction of artificial intelligence technology, care providers can be helped to effectively care for the care recipients.

[0090] The contents disclosed above are only preferred feasible embodiments of the present invention and do not limit the scope of the patent application of the present invention. Therefore, all equivalent technical changes made using the contents of the description and drawings of the present invention are included in the scope of the patent application of the present invention.

Claims

1. A care system, characterized in that: The system comprises: a data processing unit; a plurality of sensors electrically connected to the data processing unit, including a first set of sensors that operates full-time and generates sensing data about a care recipient, and a second set of sensors that generates sensing data about the care recipient after being activated in response to an instruction from the data processing unit; When the data processing unit determines that a first threshold has been reached based on the first set of sensing data generated by the first set of sensors, the instruction is generated and the second set of sensors is activated. The second set of sensors generates a second set of sensing data and records and retains the first set of sensing data and the second set of sensing data for a period of time. When the data processing unit receives the second set of sensing data, the data processing unit determines whether a condition for issuing a warning message is met based on a second threshold. a machine learning unit coupled to the data processing unit, the machine learning unit integrating various data obtained by the care system through the data processing unit using a machine learning method, including obtaining the first set of sensory data and the second set of sensory data sent by the care receiver, analyzing the various data using big data, comparing the data with clinical data, and using the machine learning method to learn the first set of sensory data and the second set of sensory data that have been annotated and retained for a period of time to establish a care alert prediction model. The trained care alert prediction model is used to determine whether the care receiver's condition meets the conditions for issuing an alert message; In which, the care system also includes a user interface, wherein a user device is connected to the data processing unit through the user interface, so that the care system receives the warning information generated by the care recipient or a care provider through the user device through the user interface, which becomes one of the factors for the machine learning unit to establish the care warning prediction model, and the data associated with the warning information participates in verifying the care warning prediction model and is used to adjust the parameters of the care warning prediction model; wherein, when the data processing unit determines that the first threshold is reached based on the first set of sensing data and starts the second set of sensors, the second set of sensors generates the second set of sensing data, and the first set of sensing data and the second set of sensing data that are annotated and retained for a period of time are fed back to the machine learning unit.

2. The care system according to claim 1, wherein: The plurality of sensors include: an audio receiver electrically connected to the data processing unit for receiving a voice of a care recipient and obtaining an audio message from the care recipient by subtracting background sound signals; and an image sensor electrically connected to the data processing unit for acquiring an image of the care recipient and performing motion detection therefrom to acquire a posture image of the care recipient; The image sensor is the first group of sensors, and the audio receiver is the second group of sensors. The data processing unit executes an automatic care method, including: Using the image sensor to capture the posture image of the care recipient at all times and perform a movement detection; and When it is determined that the image change value of the care recipient exceeds the first threshold, the audio receiver is activated to receive the audio emitted by the care recipient, and the condition of the care recipient is determined based on the audio to determine whether the second threshold is reached. If the second threshold is reached, the warning message is generated.

3. The care system according to claim 2, wherein: The second set of sensors also includes one or more physiological sensors electrically connected to the data processing unit for sensing the physiological state of the care recipient and generating physiological data.

4. The care system according to claim 2, wherein: In the automatic care method, the audio sample, including the volume and / or the audio frequency, is compared with the audio emitted by the care recipient to determine whether the care recipient needs real-time care.

5. The care system according to claim 1, wherein: The multiple sensors also include an environmental sensor for sensing the environmental data of the care recipient, which becomes one of the factors used by the machine learning unit to establish the care alert prediction model.

6. An automatic care method, applied to a care system, the care system comprising a data processing unit and a plurality of sensors electrically connected to the data processing unit, wherein the plurality of sensors comprises a first group of sensors that operate full-time and generate sensing data about a care recipient, and a second group of sensors that generate sensing data about the care recipient when activated in response to an instruction from the data processing unit; characterized in that: In this automated care method: The data processing unit determines that a first threshold is reached based on the first set of sensing data generated by the first set of sensors, generates the instruction, activates the second set of sensors, and enables the second set of sensors to generate a second set of sensing data; When the data processing unit receives the second set of sensing data, it determines whether a condition for issuing an alert message is met according to a second threshold; and determines whether the condition of the care receiver meets the condition for issuing an alert message through a care alert prediction model; as well as When the data processing unit determines that the first threshold is reached based on the first set of sensing data and activates the second set of sensors, the second set of sensors generates a second set of sensing data, and records and retains the first set of sensing data and the second set of sensing data for a period of time; A machine learning method is used to integrate various data obtained by the care system through the data processing unit, including obtaining the first set of sensory data and the second set of sensory data generated by the care recipient, analyzing the various data using big data, comparing them with clinical data, adding the first set of sensory data and the second set of sensory data that have been annotated and retained for a period of time, and referencing warning information generated by the care recipient or a care provider through a user interface of the care system. The machine learning method learns the first set of sensory data and the second set of sensory data that have been annotated and retained for a period of time to establish the care warning prediction model. The trained care warning prediction model is used to determine whether the condition of the care recipient meets the conditions for issuing a warning message. In addition, the care system receives warning information generated by the care recipient or the care provider through a user device through the user interface, which also becomes a factor in establishing the care warning prediction model. Data associated with the warning information participates in validating the care warning prediction model and is used to adjust the parameters of the care warning prediction model.

7. The automatic care method according to claim 6, wherein: The second set of sensors includes an audio receiver for receiving the voice of a care recipient and obtaining the audio signal of the care recipient after deducting the background sound signal. The first set of sensors includes an image sensor for obtaining an image of the care recipient and performing motion detection therefrom to obtain an image of the care recipient's posture. The automatic care method includes: Using the image sensor to capture the posture image of the care recipient at all times and perform a movement detection; and When it is determined that the image change value of the care recipient exceeds the first threshold, the audio receiver is activated to receive the audio emitted by the care recipient, and the condition of the care recipient is determined based on the audio to determine whether the second threshold is reached. If the second threshold is reached, the warning message is generated.

8. The automatic care method according to claim 7, wherein: The second group of sensors also includes one or more physiological sensors. The method receives physiological data generated by the one or more physiological sensors sensing the physiological state of the care recipient, and further determines whether to generate the warning information.

9. The automatic care method according to claim 7, wherein: The audio sample, including the volume and / or the audio frequency, is compared with the audio emitted by the care recipient to determine whether the care recipient needs real-time care.

10. The automatic care method according to claim 8, wherein: The method uses the machine learning method to determine whether the care recipient is in respiratory distress, or requires intervention by a care provider or not, based on the audio, posture images and / or physiological data emitted by the care recipient during sleep.

11. The automatic care method according to claim 6, wherein: The multiple sensors also include an environmental sensor for sensing the environmental data of the care recipient, which becomes one of the factors used by the machine learning method to establish the care alert prediction model.

Images