Ai-based lonely death management system and control box included therein

Abstract

Disclosed are an artificial intelligence (AI)-based lonely death management system and a control box included therein. The control box, which is linked with a plurality of sensors arranged inside furniture of a subject to be cared for so as to detect, on the basis of artificial intelligence (AI), whether the subject being cared for is in danger, comprises: at least one processor; and a memory storing instructions that instruct the at least one processor to perform at least one operation. The at least one operation comprises the steps of: determining, on the basis of a counter sensor among the plurality of sensors, whether a subject to be cared for is present in a room; if a "present state" is determined as a result of determining whether the subject being cared for is present in the room, primarily determining whether the subject being cared for is in danger by using a pre-trained deep learning-based risk detection model that has undergone supervision in advance; and if a "dangerous state" is determined as a result of the primary determination, outputting stimulus data after a pre-specified waiting time.

Description

AI-based lonely death management system and its control box

[0001] The present invention relates to a technology related to a lonely death management system, and more specifically, to an AI-based lonely death management system and a control box included therein.

[0002] The elderly population is steadily increasing due to increased life expectancy, driven by improvements in living standards across society, including medical technology. Furthermore, as individualistic lifestyles become more commonplace, the number of single-person households, particularly among young adults, is steadily increasing, not just among the elderly population.

[0003] Due to the increase in the elderly population and single-person households, the number of deaths due to loneliness is also steadily increasing. However, the number of residents per social welfare worker is approximately 500, which is more than seven times the OECD average, making it difficult to prevent and manage loneliness through human resources.

[0004] In the past, the use of public services such as gas, electricity, and water was mainly checked as a means of predicting the survival response of those who died alone or in single-person households. However, this method is difficult to use as a means of preventing lonely deaths because signs of life disappear after the death, and there is the problem of difficulty in making a precise interpretation.

[0005] Another means of solving this problem is to install occupancy detectors inside the home and utilize solutions that determine survival responses based on whether the occupancy detectors detect the person. However, problems such as malfunctions and detection performance of the occupancy detectors result in excessive unnecessary risk alerts, which in turn increases the workload for managing lonely deaths.

[0006] Furthermore, existing solutions have limitations in their detection systems, which prevent them from providing a proper solution for detecting behavioral risks, such as falls, or security risks among the elderly within the household, beyond determining whether they are alive.

[0007] The purpose of the present invention to solve the above problems is to provide an AI (artificial intelligence)-based lonely death management system and a control box included therein.

[0008] One aspect of the present invention for achieving the above purpose provides a control box that detects whether the care recipient is in danger by linking with a plurality of sensors placed inside the care recipient's furniture based on AI (Artificial Intelligence).

[0009] The control box includes at least one processor; and a memory storing instructions that direct the at least one processor to perform at least one operation.

[0010] The at least one operation includes: a step of determining whether a care recipient is present based on a counter sensor among the plurality of sensors; a step of first determining whether the care recipient is in danger using a pre-trained deep learning-based danger detection model if the determination result of the presence is determined as 'present'; a step of outputting stimulus data after a pre-designated waiting time if the determination result of the first determination is determined as 'in danger'; a step of determining whether the care recipient is in danger i-th (i is a natural number greater than or equal to 2 and less than or equal to N, and N is a pre-designated maximum number of attempts for danger determination) using the danger detection model; and a step of transmitting a notification message to at least one of a guardian terminal and a public institution server if the i-th danger determination result is determined as 'in danger'.

[0011] The above risk detection model classifies the state of the care recipient into one of ‘sleeping state’, ‘active state’, and ‘danger state’, and provides output data indicating the classified class.

[0012] The step of first determining whether the care recipient is at risk includes a step of determining the current state of the care recipient as a 'normal state' if the output data indicates one of a 'sleep state' and an 'active state', and a step of determining the current state of the care recipient as a 'danger state' if the output data indicates a 'danger state'.

[0013] The step of determining whether the above-mentioned care recipient is at risk for the i-th time includes the step of generating input data of the risk detection model and inputting it into the risk detection model.

[0014] The above input data includes the most recently received sensing data of the PIR sensor and the state detection radar sensor installed in each room of the household of the care recipient; the location identification symbol of the sensor corresponding to each of the sensing data; the detection time corresponding to each of the sensing data; the home appliance installation information corresponding to the location identification symbol; the activity data of the care recipient; and the sleeping time of the care recipient.

[0015] The above at least one operation is performed by repeating the step of outputting the stimulus data and the step of determining whether the care recipient is at risk for the i-th time, and the waiting time is set to become gradually shorter according to the following mathematical formula as the number of repetitions increases.

[0016] [Mathematical formula]

[0017]

[0018] In the above mathematical formula, j is the number of repetitions, Tdelay is the waiting time (in minutes), and k is a preset coefficient set to an integer between 1 and 4.

[0019] Another aspect of the present invention for achieving the above purpose provides an AI-based lonely death management system.

[0020] The AI-based lonely death management system includes a control box that detects whether the care recipient is in danger by linking with a plurality of sensors placed inside the care recipient's furniture; a cloud server that is connected to the control box via wireless communication and generates a warning message about a dangerous situation based on whether the care recipient is in danger provided by the control box; and a guardian terminal that receives a warning message from the cloud server and displays to the guardian whether the care recipient is in danger based on the received warning message.

[0021] The control box includes at least one processor; and a memory storing instructions that direct the at least one processor to perform at least one operation.

[0022] The at least one operation comprises: a step of determining whether a care recipient is present based on a counter sensor among the plurality of sensors; a step of first determining whether the care recipient is in danger using a pre-trained deep learning-based danger detection model if the determination result of the presence is determined as 'present'; a step of outputting stimulus data after a pre-designated waiting time if the determination result of the first determination is determined as 'danger'; a step of determining whether the care recipient is in danger i-th (i is a natural number greater than or equal to 2 and less than or equal to N, and N is a pre-designated maximum number of attempts for determining danger) using the danger detection model; and a step of transmitting a notification message to at least one of a guardian terminal and a public institution server if the i-th danger determination result is determined as 'danger'.

[0023] When using the AI ​​(artificial intelligence)-based lonely death management system and the control box included therein according to the present invention as described above, the current status of the care recipient can be checked through multiple stages and immediate response can be taken, thereby preventing lonely death or responding quickly.

[0024] Additionally, because the risk status is ultimately determined through multiple stage verification, the reliability of the risk status is high when a risk warning message is delivered, which has the advantage of reducing unnecessary administrative effort.

[0025] In addition, the deep learning-based risk detection model is advantageous in comprehensively managing and coping with the lives of single-person households, as it uses highly relevant input data to conduct pre-training so that it can predict one of three states: sleep state, activity state, and risk state, rather than simply determining the risk state.

[0026] In addition, the deep learning-based risk detection model can adjust the performance of computing power by selectively adjusting the degree of deactivation of some nodes in the hidden layer according to the risk judgment stage, which is advantageous for constructing and distributing a control box using edge computing with low computing power.

[0027] Figure 1 is a schematic diagram of an AI-based lonely death management system according to one embodiment.

[0028] Figure 2 is a conceptual diagram illustrating sensors and control boxes placed within furniture in an AI-based lonely death management system according to one embodiment.

[0029] Figure 3 is a flowchart exemplarily showing the operation of a control box according to one embodiment.

[0030] FIG. 4 is a diagram illustrating a deep learning-based risk detection model installed in a control box according to one embodiment.

[0031] Figure 5 is a diagram regarding node activation of the deep learning-based risk detection model according to Figure 4.

[0032] Fig. 6 is a drawing exemplarily showing the hardware configuration of a control box according to one embodiment.

[0033] The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention. Throughout the description of each drawing, similar reference numerals have been used to designate similar components.

[0034] Terms such as first, second, A, and B may be used to describe various components, but these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. The term "and / or" includes a combination of multiple related items described herein or any of multiple related items described herein.

[0035] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0036] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0037] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and shall not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0038] Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the attached drawings.

[0039]

[0040] Figure 1 is a schematic diagram of an AI-based lonely death management system according to one embodiment.

[0041] Referring to FIG. 1, an AI-based lonely death management system (1000) may include a control box (100) that detects whether the care recipient is in danger by interlocking with a plurality of sensors (PIR, CNT, LS) placed inside the care recipient's furniture, a cloud server (200) that is connected to a plurality of control boxes installed inside each of a plurality of care recipient's furniture through wireless communication and generates a warning message for a danger situation based on a danger signal provided from at least one of the control boxes; and a guardian terminal (300) that receives a warning message from the cloud server (200) and displays to the guardian whether the care recipient is in danger based on the received warning message.

[0042] The plurality of sensors (PIR, CNT, LS) may include a passive infrared sensor (PIR) that uses infrared to identify objects within the sensor's field of view, an occupancy counter sensor (CNT) that counts the entry and exit of users through the door by identifying the opening and closing of the door, and a vital radar sensor (LS) that detects vital signs from a long distance.

[0043] A radar sensor is a sensor that detects whether a person or animal is breathing or moving using radar signals. It emits electromagnetic waves through a specific frequency band (e.g., UWB, ultra wide band) and measures the distance, direction, altitude, etc. to a specific object (e.g., a person) by receiving electromagnetic waves reflected from the object. Among such radar sensors, a state-detecting radar sensor emits a single pulse electromagnetic wave with a very short pulse width of less than a few nanoseconds (nsec) in the time domain, and detects minute body displacement differences of a person (e.g., up and down movement of the stomach or chest due to breathing) through the size, time, and power information of the signal reflected from a specific object, thereby detecting whether the person is breathing or moving.

[0044] State detection radar sensors, such as respiration detection sensors using radar, are distributed and sold on the market, and the remaining sensors are also already distributed and sold on the market, so they can be easily obtained by ordinary technicians.

[0045] The control box (100) can receive sensing data from multiple sensors and determine whether the care recipient is present based on the received sensing data. Here, the presence can indicate either the presence or absence of the care recipient.

[0046] If it is determined that the care recipient is present, the control box (100) inputs the information into a supervised learning-based deep learning risk detection model (10) and determines whether the care recipient in the household is at risk based on the output of the risk detection model (10).

[0047] The control box (100) can generate a warning message notifying the risk state of the care recipient when the care recipient is determined to be in a risk state and transmit the warning message to the cloud server (200).

[0048] At this time, the control box (100) can generate a warning message notifying the risk status of the care recipient and transmit it to the cloud server (200) even when a user input is received through an input interface (e.g., an emergency button) installed in the control box (100). Through this, an immediate response can be supported by allowing the care recipient or a third party next to the care recipient to directly transmit the warning message to the cloud server (200) using the emergency button.

[0049] In addition, when the control box (100) determines that the care recipient is in an occupied state and not in a dangerous state, it may generate an occupied state notification message indicating the occupied state of the care recipient (sleeping or active) and transmit it to the cloud server (200). Here, the occupied state may indicate either a sleeping state or an active state when the care recipient is in an occupied state.

[0050] When the cloud server (200) receives a warning message from the control box (100), it can generate a danger alarm message including a unique identification symbol of the care recipient through the identification symbol of the control box (100) that transmitted the warning message and transmit the message to the guardian terminal (300). Here, the unique identification symbol of the care recipient may include the care recipient's phone number, resident registration number, address, etc. In this case, the identification symbol of the control box (100) and the unique identification symbol of the care recipient may be matched and stored in advance in the cloud server (200).

[0051] The guardian terminal (300) can generate a request message requesting status confirmation and transmit it to the cloud server (200) in order to check the current status of the care recipient. In response to receiving the request message, the cloud server (200) can select at least one message most recently transmitted from the care recipient's control box (100) and provide the care recipient's risk status or presence status corresponding to the selected message to the guardian terminal (300). Through this, the care recipient's guardian can check and confirm the current status of the care recipient in real time at any time.

[0052] In addition, the guardian terminal (300) can generate a voice message to be delivered to the care recipient and transmit it to the cloud server (200), and the cloud server (200) can generate a first voice output request message including the received voice message and transmit it to the control box (100). When the control box (100) receives the first voice output request message, it can play the voice message included in the voice output request message through a speaker built in therein. At this time, the control box (100) receives the voice message of the care recipient through a built-in microphone (MIC), transmits the received voice message to the cloud server (200), and the cloud server (200) determines a communication identification symbol (IP address or MAC address) of the guardian terminal (300) that is designated in advance so as to match the unique identification symbol of the control box (100) from which the voice message is received, and transmits the voice message to the guardian terminal (300) using the determined communication identification symbol.

[0053] Meanwhile, the AI-based lonely death management system (1000) may further include a public institution server (400) that receives a warning message from the cloud server (200) and provides the welfare manager with information on whether the care recipient is at risk based on the received warning message.

[0054] The public institution server (400) may be an operation server linked to a welfare management server of a local government such as a district office or city hall within the jurisdiction that governs the household of the care recipient, or a situation room of 119 that governs the household.

[0055] When a cloud server (200) receives a warning message from the control box (100), it may generate a danger alarm message and transmit it to the public institution server (400).

[0056] In response to a danger alarm message, the public institution server (400) may transmit an emergency rescue request, such as 119, to a communication-capable device of the department in charge of the relevant work, or to a communication-capable device of a welfare management officer who has been registered in advance as a person in charge of the care recipient of the local government.

[0057] Like the guardian terminal (300), the public institution server (400) can generate a voice message to be delivered to the care recipient and transmit it to the cloud server (200), and the cloud server (200) can generate a second voice output request message including the received voice message and transmit it to the control box (100). When the control box (100) receives the second voice output request message, it can play the voice message included in the second voice output request message through a speaker built in therein. At this time, the control box (100) receives the voice message of the care recipient through a built-in microphone (MIC), and transmits the received voice message to the cloud server (200), and the cloud server (200) can determine a communication identification symbol (IP address or MAC address) of the public institution server (400) that is designated in advance so as to match the unique identification symbol of the control box (100) from which the voice message is received, and transmit the voice message to the public institution server (400) using the determined communication identification symbol.

[0058] The control box (100) may be equipped with a long-distance communication module for wireless communication with a cloud server (200), and may also be equipped with a sensor communication module for short-distance communication with a plurality of sensors. In addition (or optionally), the control box (100) may further be equipped with a short-distance wireless communication module for wirelessly communicating with another control box (100) located in a short distance using short-distance wireless communication. For example, the short-distance wireless communication module may be a communication module that supports amateur radio (HAM). Through this, the control box (100) functions as a type of walkie-talkie, enabling wireless voice conversation between the household in which the control box (100) is installed and the control boxes (100) of other care recipients located in a short distance, thereby supporting remote communication between single-person households or elderly households, thereby enhancing social activities.

[0059]

[0060] Figure 2 is a conceptual diagram illustrating sensors and control boxes placed within furniture in an AI-based lonely death management system according to one embodiment.

[0061] The control box (100) is configured to enable short-range communication with multiple sensors installed within the care recipient's furniture, thereby receiving sensing data from the multiple sensors. To this end, the control box (100) is preferably installed in a location within the care recipient's furniture where smooth short-range communication with all sensors is possible. For example, the control box (100) may be installed in the living room.

[0062] The control box (100) may be implemented in the form of an integrated control box including at least one of a PIR sensor (PIR) and a state detection radar sensor (LS).

[0063] The counter sensor (CNT) provides sensing data to the control box (100) that can identify the presence of the care recipient by counting the opening / closing of the door. The control box (100) can determine the presence (present / absent) of the care recipient based on the sensing data received from the counter sensor (CNT).

[0064] For this purpose, it may be desirable to install a counter sensor (CNT) on the front door to count the opening / closing of the front door within the care recipient's household.

[0065] PIR sensors (PIR) and state detection radar sensors (LS) can be installed in each room and bathroom within the care recipient's home.

[0066] At this time, the sensing data transmitted from the PIR sensor (PIR) and the status detection radar sensor (LS) to the control box (100) may include a location identification symbol corresponding to the location installed in the furniture (bedroom, each room, bathroom, etc.) and one of the unique identification symbols of the corresponding sensor.

[0067] At this time, if a unique identification symbol of the sensor is included, the location identification symbol is matched with the unique identification symbol of the corresponding sensor in advance and stored in the control box (100) as a matching table, and the control box (100) searches for the unique identification symbol included in the received sensing data in the matching table, and can identify the location corresponding to the corresponding sensor data through the location identification symbol matched with the searched unique identification symbol.

[0068] Here, the sensing data provided by the PIR sensor (PIR) may be information indicating whether there is movement or the degree of movement at the installation location, or information indicating whether there is a person determined based on whether there is movement, and the radar sensor (LS) may be information indicating at least one of whether there is movement or the degree of movement, whether there is breathing, and the number of breathing times at the installation location, or information indicating whether there is a person determined based on the information.

[0069] Here, the form of sensing data may vary depending on the product type and specifications of the commercialized sensor, and the presence of a person at the corresponding location may need to be determined primarily by the control box (200). In the following, for convenience of explanation, the sensing data provided by the PIR sensor (PIR) and the state detection radar sensor (LS) is described as binary data indicating the presence of a person at the corresponding location, but is not limited thereto, and may also be data indicating the degree of movement, number of breathing, etc. actually measured.

[0070]

[0071] Figure 3 is a flowchart exemplarily showing the operation of a control box according to one embodiment.

[0072] Referring to FIG. 3, the operating method of the control box (100) may include a step (S100) of determining whether a care recipient is present based on a counter sensor, a step (S200) of determining whether the care recipient is in danger using a pre-trained deep learning-based danger detection model (10) if the first determination result is determined to be in danger, a step (S300) of outputting stimulus data after a pre-designated waiting time, a step (S400) of determining whether the care recipient is in danger using the danger detection model (10) for the i-th time (i is a natural number between 2 and N, and N may be a pre-designated value (for example, 3) as the maximum number of attempts for determining danger), and a step (S500) of transmitting a notification message to at least one of a guardian terminal (300) and a public institution server (400) if the i-th time is the N-th time and the i-th determination result of whether the care recipient is in danger is determined to be in danger.

[0073] In step S100, the operation method of the control box (100) can check whether the patient is present as indicated by sensing data received through the counter sensor, and determine whether the patient is present based on the confirmed presence.

[0074] The operating method of the control box (100) after step S100 may further include a step (S110) of entering an operation standby mode when the care recipient is determined to be 'absent'. In the operation standby mode, the control box (100) may wait until updated sensing data is received through the counter sensor, and may enter step S100 when the updated sensing data is received.

[0075] In step S200, the operating method of the control box (100) may include a step of generating input data using sensing data collected from a plurality of sensors within a preset time, and first determining whether there is a risk based on an output when the generated input data is input to the risk detection model (10). For example, the risk detection model (10) may classify the state of the care recipient into one of a 'sleeping state', an 'active state', and a 'dangerous state' according to the input sensing data, and provide output data indicating the classified class, and the operating method of the control box (100) may include a step of determining the current state of the care recipient as a 'normal state' when the output data indicates one of a 'sleeping state' and an 'active state', and determining the current state of the care recipient as a 'dangerous state' when the output data indicates a 'dangerous state'.

[0076] In the same way, the operating method of the control box (100) in step S400 may include a step of generating input data using sensing data collected from a plurality of sensors within a preset time, and determining whether there is a risk based on an output when the generated input data is input to the risk detection model (10). For example, the risk detection model (10) may classify the state of the care recipient into one of a 'sleeping state', an 'active state', and a 'dangerous state' according to the input sensing data, and provide output data indicating the classified class, and the operating method of the control box (100) may include a step of determining the current state of the care recipient as a 'normal state' when the output data indicates one of a 'sleeping state' and an 'active state', and determining the current state of the care recipient as a 'dangerous state' when the output data indicates a 'dangerous state'.

[0077] If the care recipient is determined to be in a ‘normal state’ in step S200, the control box (100) can perform step S110.

[0078] Here, the stimulus data may include one or more of stimulus voice data and stimulus light data. For example, the operating method of the control box (100) in step S300 may include a step of playing stimulus voice data pre-stored within the control box using a built-in speaker. In one embodiment, the stimulus voice data may include a voice message prompting the care recipient to move or respond.

[0079] In another embodiment, the operating method of the control box (100) in step S300 may include a step of controlling the linked lighting using stimulus lighting data in conjunction with pre-paired lighting within the furniture. For example, the operating method of the control box (100) may include a step of controlling a light installed at a specific location within the furniture to blink or controlling the light to emit a specific light color.

[0080] In step S400, if i is less than Nth and the result of determining whether i-th risk is ‘dangerous’, step S300 can be re-executed.

[0081] That is, steps S300 and S400 are repeatedly performed until i becomes N times, thereby ensuring that stimulus data is output multiple times, thereby ensuring that the risk is not detected incorrectly, thereby reducing the monitoring efforts of local governments or welfare personnel such as 119 due to frequent output of warning messages.

[0082] Meanwhile, the waiting time in step S300 may be set to become progressively shorter as the number of times step S300 is repeated increases. For example, the waiting time may be set to be considerably long, between 1 and 4 hours, when step S300 is performed for the first time, and the waiting time may be shortened to less than half of the waiting time when step S300 is performed for the second time.

[0083] In one embodiment, the waiting time may be set as follows depending on the number of times step S300 is repeated.

[0084]

[0085] In mathematical expression 1, j is the number of times step S300 is repeated, Tdelay is the waiting time (in minutes), and k is a preset coefficient that can be set to an integer between 1 and 4. When the waiting time is set as in mathematical expression 1, the waiting time decreases rapidly in a logarithmic unit at the beginning of the repetition, so that the output interval of the stimulus data is greatly reduced, and after a certain point, the rate of decrease in the output interval of the stimulus data becomes slower, so that inconvenience caused by the continuous output of stimulus data too quickly before the care recipient responds can be prevented.

[0086] In step S400, if the result of the determination of whether there is an i-th risk is 'normal', a response confirmation process (S410) can be performed. In the response confirmation process (S410), the control box (100) can directly confirm the safety of the care recipient by linking with the cloud server (200). For example, an ARS-based phone call can be made to the mobile terminal of the care recipient registered in advance through the cloud server (200), and if the care recipient's safety confirmation input is confirmed through the phone call, the operation standby state (S110) can be entered.

[0087] As another example, in the response confirmation process (S410), the control box (100) may transmit an ARS execution request to a public institution server (400) via a cloud server (200), and an ARS-based phone call may be performed to a mobile terminal of a care recipient registered in advance via the public institution server (400).

[0088] In step S200 and / or step S400, the control box (100) may further include a step of generating the input data of the risk detection model (10). The input data may include the most recently received sensing data of the PIR sensor (PIR) and the state detection radar sensor (LS) installed in each room of the furniture, the position identification symbol of the sensor corresponding to each of the sensing data (for example, a unique identification symbol assigned according to each room, kitchen, bathroom, living room, etc.), the detection time corresponding to each of the sensing data, the home appliance installation information corresponding to the position identification symbol of the sensor (for example, whether a TV, computer, etc. are installed), the user activity data (TV play time, computer operation time), and the user sleep time (wake-up time and bedtime).

[0089]

[0090] FIG. 4 is a diagram illustrating a deep learning-based risk detection model installed in a control box according to one embodiment.

[0091] Referring to Fig. 4, a deep learning-based risk detection model (10) can be pre-supervised learning using a dataset of training input values ​​and training output values.

[0092] At this time, the training input values ​​may include sensing data of PIR sensors (PIR) and state detection radar sensors (LS) installed in each room of the care recipient's household, location identifiers of the sensors corresponding to each sensing data (e.g., unique identifiers assigned according to each room, kitchen, bathroom, living room, etc.), detection times corresponding to each sensing data, installation information of home appliances actually installed in the care recipient's household corresponding to the location identifiers of the sensors (e.g., whether a TV, computer, etc. are installed in a specific room / kitchen / living room, etc.), user activity data (TV play time, computer operation time), and user sleep time (wake-up time and bedtime). It may be desirable to collect sleep time directly by observing the care recipient or by having the care recipient wear a wearable device. It is desirable to determine the user activity data statistically from the usual lifestyle habits of the care recipient, but it can also be obtained through a questionnaire targeting the care recipient for quick construction.

[0093] Here, the sensing data is information that directly indicates whether the care recipient is alive and serves as an indicator for determining presence and danger, and the detection time serves as an indicator for learning the differences in whether the care recipient is sleeping / active depending on the time of day and night, and the sensor location identifier serves as an indicator for learning the importance of the sensor data measured at each sensor installation location differently, and the home appliance installation information serves as an indicator for not mistaking the care recipient as being in a dangerous state even if only a slight movement is detected in one place by learning when the care recipient stays in one place for a long time (such as watching TV or using a computer). In addition, the user's sleep time can be considered along with the detection time to determine whether the care recipient is in a sleeping state rather than a dangerous state even if the care recipient stays in one place for a long time.

[0094] As a specific example, the training input value may be a training input matrix (X) in which each of the sensing data (e.g., the size of movement, the respiration rate, or the value indicating whether or not a room is occupied based on the size of movement or the respiration rate, etc.) is set as the first row, the location identification symbol of the sensor corresponding to each of the sensing data is set as the second row, the detection time corresponding to each of the sensing data (e.g., the current time in seconds based on a day) is set as the third row, and among the sensing data, the identification information of a specific home appliance (a home appliance that maintains the activity of a person receiving care in one place, such as a TV or a computer) is assigned to the column corresponding to the sensing data, and the data for which the remaining columns are set to '0' are set as the fourth row, and among the sensing data, the wake-up time is assigned to the column corresponding to the sensing data installed in the bedroom, and the data for which the remaining columns are set to '0' are set as the fifth row, and the bedtime is assigned to the column corresponding to the sensing data installed in the bedroom, and the data for which the remaining columns are set to '0' are set as the sixth row. There is. At this time, it may be possible to convert the fifth and sixth rows into a single row by converting them into the interval between bedtime and wake-up time (i.e., sleep time).

[0095] The training output value may be data indicating one of the following: "sleep state," "active state," or "danger state," representing the current state of the care recipient, obtained when the care recipient is actually observed using CCTV or other means at the time the corresponding sensing data was collected. In other words, the training output value is utilized as data that labels the actual current state of the care recipient corresponding to the training input value, and a training output vector uniquely assigned to each current state may be defined in advance.

[0096] The risk detection model (10) is supervised learning by sequentially receiving each column of the training input values ​​as input vectors, comparing the current status of the care recipient obtained as the output value of the risk detection model (10) with the training output value, and adjusting the parameters that constitute the risk detection model (10) based on the comparison result.

[0097] Specifically, the risk detection model (10) can calculate a predefined loss function using an output vector (Y`) representing the current status of the care recipient obtained as an output value of the risk detection model (10) and a training output vector (Y) corresponding to the training output value, and can adjust the parameters constituting the risk detection model (10) so that the result value of the calculated loss function is minimized.

[0098] At this time, the loss function may be a cross entropy function.

[0099] Referring to FIG. 4, the risk detection model (10) may include an input layer (11) that receives a training input vector (x) and is composed of input nodes having a number (N) equal to the number of component values ​​of the received training input vector (x), a hidden layer (12) that transmits an output vector (Y`) calculated using output values ​​received from the input layer (11) to an output layer (13), and an output layer (13) that applies an activation function to the output vector (Y`) to determine a probability (p) corresponding to the output vector (Y`) and outputs the output vector (Y`) with the highest determined probability (p). Each of the nodes constituting the artificial neural network engine (102b) in the present invention may also be referred to interchangeably by the term “neuron,” which is a commonly used expression in the technical field to which the present invention belongs.

[0100] Specifically, when the artificial neural network engine (102b) receives a training input vector (x) provided as a training input value, it calculates a loss function using the output vector (Y`) obtained as an output of the hidden layer (12) and the training output vector (Y) indicating the current state of the care recipient corresponding to the training output value, and supervised learning is performed so that the result value of the calculated loss function is minimized.

[0101] For example, the loss function (H(Y,Y`)) can be a cross entropy function. The cross entropy (H(Y,Y`)) between the output vector (Y`) and the training output vector (Y) can be defined as in the following mathematical expression 3.

[0102]

[0103] In mathematical expression 2, Ym may be the mth component (m is a natural number greater than or equal to 1) of the training output vector (Y), and Y`m may be the mth component of the output vector (Y`).

[0104] The input layer (11) receives a training input vector (x) and, for each component of the received training input vector (x), can apply one or more connection strength values ​​corresponding to input nodes and transmit them to the hidden layer (12).

[0105] For example, one or more connection strength values ​​corresponding to each of the input nodes are formed into a first connection strength matrix (W) having a size of N×M. N×M ) can be expressed as. At this time, N can be the same number of input nodes, and M is set to be sufficiently small, less than 1 / 10 times smaller than N. The first connection strength matrix (W N×M ) can be a parameter that is set to an arbitrary initial value and then continuously updated through supervised learning.

[0106] In summary, the input layer (11) adds the first connection strength matrix (W) to the input training input vector (x). N×M) can be transmitted to the hidden layer (12) by performing matrix multiplication operation on the intermediate operation vector (x).

[0107] The hidden layer (12) can generate an output vector (Y`) by applying one or more connection strengths corresponding to each of the hidden nodes to the feature vector (F) obtained from the intermediate operation vector (x) received from the input layer (11), and can transfer the generated output vector (Y`) to the output layer (13).

[0108] For example, the hidden layer (12) can obtain a feature vector (F), which is a diagonal matrix satisfying the following mathematical expression 3, from the intermediate operation vector (x).

[0109]

[0110] In mathematical expression 3, R is a matrix that reversibly satisfies the relationship according to mathematical expression 3 between the feature vector (F), which is a diagonal matrix, and the intermediate operation vector (x), and corresponds to a coordinate transformation matrix. The operation according to mathematical expression 3 can be referred to as one of the diagonalization operations, and since it is easily understandable to those skilled in the art, a detailed explanation is omitted.

[0111] At this time, one or more connection strength values ​​corresponding to each of the hidden nodes are included in a second connection strength matrix (U) having a size of M×Q. M×Q ) can be expressed as, that is, the second connection strength matrix (U M×Q ) increases the feature vector (F) mapped to M dimensions again to Q dimensions. Q is set to a value sufficiently larger than M by at least 10 times.

[0112] Meanwhile, the second link strength matrix (U M×Q ) is set to an arbitrary value, and then the feature vector (F) and the second connection strength matrix (U M×Q ) can be continuously updated so that the output vector (Y`) generated by the matrix multiplication operation between them becomes the training output vector (Y), which is the training output value. That is, the second connection strength matrix (U M×Q) can also be a parameter that is updated as training data is continuously supervised.

[0113] That is, the hidden layer (12) converts the intermediate operation vector (x) received from the input layer (11) into a feature vector and applies a connection strength to the feature vector to generate an output vector (Y`), so that the hidden layer (12) can be configured to perform a function similar to a convolutional layer responsible for feature extraction of a CNN (convolutional neural network).

[0114] The output layer (13) can determine the probability (p) corresponding to the output vector (Y`) by applying an activation function to the output vector (Y`) received from the hidden layer (12), and output the output vector (Y`) with the highest determined probability (p). The activation function has the effect of converting values ​​in various ranges into probabilities by expanding or reducing them to values ​​between 0 and 1. For example, the activation function may be, but is not limited to, a ReLU function or a Softmax function.

[0115]

[0116] Figure 5 is a diagram regarding node activation of the deep learning-based risk detection model according to Figure 4.

[0117] In one embodiment of the present invention, the hidden layer (12) included in the risk detection model (10) may be composed of a plurality of hidden layers having the same number of neurons and being different from each other and sequentially connected. For example, the hidden layer (12) may be composed of the first to m-th hidden layers that are different from each other and sequentially connected. Here, the number m of hidden layers may be determined according to the type of data or the number of rows constituting the training input matrix (X), and in one embodiment of the present invention, since it is composed of sensing data, a sensor location identifier, detection time, home appliance installation information, activity data, and sleep time (wake-up time and bedtime), m may be 6.

[0118] At this time, the first to m-th hidden layers may be linear layers, and at least one of the first to m-th hidden layers may be a dense layer. Here, the designation of a linear layer or a dense layer can be simply set through a predefined function in a known library such as Tensorflow, so an explanation is omitted.

[0119] Meanwhile, the control box (100) according to one embodiment of the present invention is a small computing device placed in each household. Therefore, to prevent excessively high unit prices and ensure widespread distribution to many households, the hardware must be configured to consume limited computing power. Conversely, to install and configure a deep learning-based risk detection model (10), a certain level of computing power must be guaranteed to ensure stable operation.

[0120] Therefore, in order for the deep learning-based risk detection model (10) to operate stably even with small computing power, the risk detection model (10) according to one embodiment of the present invention is configured to adjust the number of nodes in which the hidden layer is activated each time the risk determination order increases.

[0121] That is, in one embodiment of the present invention, at least some of the hidden layers (12) are determined by a probability among the neurons (or hidden nodes) constituting the hidden layer. p ) can be used to disable some of the neurons (hidden nodes) randomly selected by setting their weights to 0 or a predefined constant, thereby preventing excessive computing power consumption even if the prediction performance is somewhat lowered.

[0122] For example, the third hidden layer consists of 768 neurons and has a probability p) is 0.2, the weights for 20% of the neurons randomly selected from the 768 neurons (i.e., 768 / 5 neurons or hidden nodes) can be set to 0 or a predefined constant.

[0123] In one embodiment of the present invention, the deactivation probability (deact p ) can be defined as in the following mathematical equation 4.

[0124]

[0125] In mathematical expression 4, N is the maximum degree preset to determine whether there is a risk, i is the degree for determining whether there is a current risk, cp is the value of the computing power of the control box (100) quantified through a benchmark test, and cp avg is the average value of the computing power of each distributed control box (100) quantified through a benchmark test, and cp div is the standard deviation of the numerical value of the computing power of each distributed control box (100) through a benchmark test.

[0126] As described above, as the number of steps for determining risk increases, the probability of deactivation decreases, allowing for a higher degree of accuracy in determining risk. Therefore, there is an advantage in that a stable risk determination can be made according to the number of steps through a trade-off with the computing power of the computing device. In addition, even if the hardware specifications of the control box (100) are sold in various ways, the computing power of the control box (100) is compared with the computing power of the distributed control boxes (100) to adjust the probability of deactivation. Therefore, there is an advantage in that the computing power can be sold in a variety of forms with higher sales competitiveness.

[0127]

[0128] Fig. 6 is a drawing exemplarily showing the hardware configuration of a control box according to one embodiment.

[0129] The control box (100) may include at least one processor (110) and a memory (120) that stores instructions that instruct the at least one processor (110) to perform at least one operation.

[0130] Here, at least one operation is interpreted as including at least some of the operations or functions of the aforementioned control box (100), and a specific description is omitted to avoid redundant description.

[0131] Here, at least one processor (110) may mean a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed. The memory (120) may be composed of at least one of a volatile storage medium and a non-volatile storage medium.

[0132] The control box (100) may further include a storage device (160) for non-temporarily storing input data, intermediate processing data, temporary data, output data, etc. according to performing at least one operation.

[0133] For example, the memory (120) may be one of read-only memory (ROM) and random access memory (RAM), and the storage device (160) may be a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or various memory cards (e.g., a micro SD card).

[0134] In addition, the control box (100) may further include a transceiver (130) that performs communication via a wireless network. In addition, the control box (100) may further include an input interface device (140), an output interface device (150), etc. Each component included in the control box (100) may be connected by a bus (170) and communicate with each other.

[0135] Examples of the control box (100) include a desktop computer, a laptop computer, a notebook, a smart phone, a tablet PC, a mobile phone, a smart watch, smart glasses, an e-book reader, a portable multimedia player (PMP), a portable game console, a navigation device, a digital camera, a digital multimedia broadcasting (DMB) player, a digital audio recorder, a digital audio player, a digital video recorder, a digital video player, a PDA (Personal Digital Assistant), etc.

[0136]

[0137] The methods according to the present invention may be implemented in the form of program instructions that can be executed by various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either singly or in combination. The program instructions recorded on the computer-readable medium may be those specifically designed and constructed for the present invention, or may be known and available to those skilled in the computer software art.

[0138] Examples of computer-readable media may include hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions may include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate with at least one software module to perform the operations of the present invention, and vice versa.

[0139] Additionally, the above-described method or device may be implemented by combining all or part of its configuration or function, or may be implemented separately.

[0140] Although the present invention has been described above with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various modifications and changes may be made to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below.

Claims

1. A control box that detects whether the care recipient is in danger by linking with multiple sensors placed inside the care recipient's furniture based on AI (Artificial Intelligence). The above control box, at least one processor; and A memory that stores instructions that instruct at least one processor to perform at least one operation, At least one of the above actions, A step of determining whether a care recipient is present based on a counter sensor among the plurality of sensors; If the result of the determination of whether the above person is present is determined as ‘present’, a step of first determining whether the person receiving care is at risk using a pre-trained deep learning-based risk detection model; A step of outputting stimulus data after a predetermined waiting time when the above first decision result is determined to be a ‘dangerous state’; A step of determining whether the care recipient is at risk by using the risk detection model (i is a natural number greater than or equal to 2 and less than or equal to N, and N is the maximum number of attempts specified in advance for risk determination); and A control box comprising: a step of transmitting a notification message to at least one of a guardian terminal and a public institution server when the i is the N and the result of determining whether the i-th risk is a 'dangerous state'; 2. In claim 1, The above risk detection model is, Classify the status of the above care recipient into one of 'sleeping state', 'active state', and 'at-risk state', and provide output data indicating the classified class. The first step in determining whether the above care recipient is at risk is: A control box comprising a step of determining the current state of the care recipient as a 'normal state' when the output data indicates one of a 'sleep state' and an 'active state', and determining the current state of the care recipient as a 'danger state' when the output data indicates a 'danger state'.

3. In claim 1, The step of determining whether the above care recipient is at risk is as follows: Including a step of generating input data of the above risk detection model and inputting it into the above risk detection model, The above input data is, The most recently received sensing data of the PIR sensor and status detection radar sensor installed in each room of the above-mentioned care recipient's household; A location identification symbol of a sensor corresponding to each of the above sensing data; Detection time corresponding to each of the above sensing data; Home appliance installation information corresponding to the above location identification symbol; Activity data of the above care recipient; and including the sleeping time of the above care recipient; Control box.

4. In claim 1, At least one of the above actions, The step of outputting the above stimulus data and the step of determining whether the care recipient is at risk are performed repeatedly, and the waiting time is set to become gradually shorter according to the following mathematical formula as the number of repetitions increases. [Mathematical formula] In the above mathematical formula, j is the number of repetitions, Tdelay is the waiting time (in minutes), and k is a preset coefficient, which is set to an integer between 1 and 4. Control box.

5. As an AI-based lonely death management system, A control box that detects whether the care recipient is in danger by linking with multiple sensors placed inside the care recipient's furniture; A cloud server that is connected to the control box via wireless communication and generates a warning message about a dangerous situation based on whether the care recipient is in danger or not, as provided by the control box; and A guardian terminal that receives a warning message from the cloud server and indicates to the guardian whether the care recipient is in danger based on the received warning message; The above control box, at least one processor; and A memory that stores instructions that instruct at least one processor to perform at least one operation, At least one of the above actions, A step of determining whether a care recipient is present based on a counter sensor among the plurality of sensors; If the result of the determination of whether the above person is present is determined as ‘present’, a step of first determining whether the person receiving care is at risk using a pre-trained deep learning-based risk detection model; A step of outputting stimulus data after a predetermined waiting time when the above first decision result is determined to be a ‘dangerous state’; A step of determining whether there is a risk i (i is a natural number greater than or equal to 2 and less than or equal to N, where N is the maximum number of attempts specified in advance for risk determination) using the above risk detection model; and A step of transmitting a notification message to at least one of a guardian terminal and a public institution server when the i is the N and the result of determining whether the i-th risk is a 'dangerous state'; AI-based lonely death management system.

Images