Residence, System, Device

Abstract

Provide at least a dwelling, a method for providing a dwelling, and an apparatus that enable a resident to promptly receive appropriate assistance or medical treatment when the resident has an accident. 【Solution means】In a house, a video acquisition device acquires video information of the entire room, and the device requests an AI to determine the presence or absence of an accident of a resident based on the video information. When it is determined that the resident has had an accident, information is transmitted to a predetermined external device.

Description

Technical Field

[0001] The present invention relates to a dwelling or a method and apparatus for providing a dwelling.

Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] Patent Document 1 discloses a house.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the inventor has recognized that at least in the above embodiment, there is a drawback in that there is no configuration for transmitting information to a predetermined external device when an accident of an occupant is determined.

Means for Solving the Problems

[0006] At least one aspect of the present disclosure is a dwelling, wherein an image acquisition device acquires video information of the entire room, the device requests an AI to determine the presence or absence of an accident of an occupant based on the video information, and when an accident of an occupant is determined, information is transmitted to a predetermined external device. a house is provided.

Effects of the Invention

[0007] In this configuration, at least when a resident has an accident, there is a usefulness in being able to receive prompt and appropriate assistance or medical treatment.

[0008] These and other aspects, features, and advantages of the present disclosure will become apparent from the following detailed written description of the preferred embodiments and aspects taken in conjunction with the following drawings, but modifications and variations thereof may be made without departing from the spirit and scope of the novel concepts of the present disclosure. Aspects in one embodiment of the present disclosure can be combined with, or replaced by, one or more of the aspects in another embodiment of the present disclosure, as long as they do not conflict.

Mode for Carrying Out the Invention

[0009] In the following disclosure, many different embodiments and examples are provided for implementing different features of the presented subject matter. To simplify the present disclosure, specific examples of components and arrangements are disclosed below. Of course, these are merely examples and are not intended to be limiting. For example, a structure in which a first feature is covered by, or in contact with, a second feature disclosed subsequently may include embodiments in which the first feature and the second feature are formed so as to be in direct contact, as well as embodiments in which additional features are formed between the first feature and the second feature so that the first feature and the second feature are not in direct contact. Further, in the present disclosure, reference numerals and / or letters may be repeated in various examples. Such repetition is for the purpose of simplicity and clarity and does not itself require a relationship between the various embodiments and / or the configurations being described. Further, when a first element is described as being "connected" or "coupled" to a second element, such description includes embodiments in which the first element and the second element are directly connected or coupled to each other, as well as embodiments in which the first element and the second element are indirectly connected or coupled to each other with one or more other elements intervening therebetween.

[0010] As used herein, the recitation "at least one of" includes all variations exemplified. For example, the recitation "comprises at least one of A, B, or C" is synonymous with "consisting of A, B, C and combinations thereof", and encompasses all possible variations of A, B, C, A + B, A + C, B + C, and A + B + C.

[0011] In this disclosure, the disclosure of using a machine, an electronic operator, or a computer can include embodiments of a method, a recording medium, an apparatus, or a program. The description "A is B" as used herein can be replaced with "A includes B" as long as there is no contradiction or unless otherwise stated in this specification.

[0012] The terms in this disclosure, including those recited in the claims, can be interpreted in consideration of the descriptions and drawings described in the specification, and further, as long as there is no contradiction with the suggestions in this disclosure, can be interpreted based on matters that one or more members of the public have so named, indicated, understood, or practiced, or are likely to do so, in the past, present, or future. Regarding the operating method used in at least one or more embodiments, the following embodiments can be adopted. The description of JP6456303, which well explains at least one or more embodiments, is cited for explanation (hereinafter, citation begins).

[0013] As used herein, the term "computer", as known in the art, generally includes a processor, a memory, such as a hard drive, disk drive, flash drive, or memory stick, or other non-transitory computer-readable medium or non-transitory storage device, at least one information storage / search device, such as a keyboard, mouse, point and touch device, touch screen, or microphone, at least one input device, and a display structure such as a well-known computer screen. Additionally, a computer may include one or more network connections, such as a wired or wireless connection. As known in the art, such a computer or computer system may include more or less of the items listed above and is not limited to, for example, tablet computers or smart devices, but includes other electronic media and electronic devices.

[0014] As used herein, the term "cloud" or "cloud computing" refers to a centralized and virtualized computing facility where all computing resources are shared. For application systems and subsystems, since they are all "in the cloud", it is no longer possible to refer to a specific machine.

[0015] As used herein, the term "Distributed Internet Service System" refers to a distributed Internet service platform that transforms Internet applications for execution in various computing environments. The DIS system distributes Internet applications, including content, data, and logic, via a Component Distribution Server / Asset Distribution Server to any number and any type of device, to whatever extent appropriate, and along the network. Through DIS, Internet applications can be hosted and centrally managed as services based on each user's needs, locally cached and executed at the user's device or nearby locations while maintaining their integrity. Web-enabled computing devices can be upgraded with DIS software to become DIS-compliant to enjoy and execute distributed Internet services. The distributed Internet service system is fully described in any one of the patent families of U.S. Patent Nos. 7,136,857, 7,150,015, 7,181,731, 7,209,921, 7,430,610, 7,685,183, 7,685,577, 7,752,214, 8,326,883, 8,386,525, 8,443,035, 8,458,142, 8,458,222, 8,473,468, 8,527,545, and 8,650,226, and U.S. Patent Publications Nos. 2012 / 0005205 and 2013 / 0091252, all of which are jointly owned by OPI40, Holdings, Inc. and are hereby incorporated by reference. (End of citation)

[0016] Regarding the operation method used in at least one or more embodiments, the following embodiments can be taken for the conventional Internet method that does not use a distributed Internet. The description of JP7113047, which well explains at least one or more embodiments, will be cited for explanation (hereinafter, citation starts).

[0017] Embodiments including the matters specifically disclosed in this specification can provide an automatic response system realized in a form that actually converses with a human based on artificial intelligence, thereby enabling a more natural conversation with a user while quickly and conveniently processing inquiries, reservations, delivery orders, and the like.

[0018] The plurality of electronic devices 110, 120, 130, 140 may be fixed terminals or mobile terminals realized by a computer system. Examples of the plurality of electronic devices 110, 120, 130, 140 include AI speakers, smartphones, mobile phones, navigation devices, PCs (personal computers), notebook PCs, digital broadcast terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), tablets, game consoles, wearable devices, IoT (internet of things) devices, VR (virtual reality) devices, AR (augmented reality) devices, and the like. As an example, in FIG. 1, an AI speaker is shown as the electronic device 110. However, in the embodiments of the present invention, the electronic device 110 may mean one of various physical computer systems that can communicate with other electronic devices 120, 130, 140 and / or servers 150, 160 via the network 170 using substantially wireless or wired communication methods.

[0019] The communication method is not limited, and it may include not only a communication method using a communication network that the network 170 can include (for example, a mobile communication network, a wired Internet, a wireless Internet, a broadcast network, a satellite network, etc.), but also short-range wireless communication between devices. For example, the network 170 may include any one or more of networks such as a PAN (personal area network), a LAN (local area network), a CAN (campus area network), a MAN (metropolitan area network), a WAN (wide area network), a BBN (broadband network), and the Internet. Further, the network 170 may include any one or more of network topologies including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or hierarchical network, etc., but is not limited thereto.

[0020] Servers 150 and 160 may each be implemented by one or more computer devices that communicate with a plurality of electronic devices 110, 120, 130, and 140 via a network 170 to provide instructions, code, files, content, services, etc. For example, server 150 may be a system that provides a first service to a plurality of electronic devices 110, 120, 130, and 140 connected via network 170, and server 160 may also be a system that provides a second service to a plurality of electronic devices 110, 120, 130, and 140 connected via network 170. As a more specific example, server 150 may provide, as the first service, a service (such as an automatic response service, for example) targeted by a corresponding application to a plurality of electronic devices 110, 120, 130, and 140 through an application that is a computer program installed and executed in the plurality of electronic devices 110, 120, 130, and 140. As another example, server 160 may provide, as the second service, a service that distributes files for installation and execution of the above-described application to a plurality of electronic devices 110, 120, 130, and 140.

[0021] FIG. 2 is a block diagram for explaining the internal configurations of an electronic device and a server in an embodiment of the present invention. In FIG. 2, the internal configuration of electronic device 110 and the internal configuration of server 150 are described as examples for an electronic device. Also, other electronic devices 120, 130, 140 and server 160 may have the same or similar internal configurations as the above-described electronic device 110 or server 150.

[0022] The electronic device 110 and the server 150 may include memories 211 and 221, processors 212 and 222, communication modules 213 and 223, and input / output interfaces 214 and 224. The memories 211 and 221 may be non-transitory computer-readable recording media, and may include non-transitory mass storage devices such as RAM (random access memory), ROM (read only memory), disk drives, SSDs (solid state drives), flash memories, etc. Here, non-transitory mass storage devices such as ROM, SSD, flash memory, and disk drive may be included in the electronic device 110 or the server 150 as separate non-transitory recording devices distinct from the memories 211 and 221. Also, the memories 211 and 221 may record an operating system and at least one program code (for example, code for a browser installed and executed in the electronic device 110, an application installed in the electronic device 110 for providing a specific service, etc.). Such software components may be loaded from a computer-readable recording medium different from the memories 211 and 221. Such another computer-readable recording medium may include computer-readable recording media such as a floppy (registered trademark) drive, disk, tape, DVD / CD-ROM drive, memory card, etc. In other embodiments, the software components may be loaded into the memories 211 and 221 through the communication modules 213 and 223 which are not computer-readable recording media. For example, at least one program may be loaded into the memories 211 and 221 based on a computer program (for example, the above-described application) installed by a file distributed by a file distribution system (for example, the above-described server 160) that distributes developer or application installation files via the network 170.

[0023] The processors 212 and 222 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. The instructions may be provided to the processors 212 and 222 by the memories 211 and 221 or the communication modules 213 and 223. For example, the processors 212 and 222 may be configured to execute instructions received according to program codes recorded in a recording device such as the memories 211 and 221.

[0024] The communication modules 213 and 223 may provide functions for the electronic device 110 and the server 150 to communicate with each other via the network 170, or may provide functions for the electronic device 110 and / or the server 150 to communicate with other electronic devices (e.g., the electronic device 120) or other servers (e.g., the server 160). As an example, a request generated by the processor 212 of the electronic device 110 according to program codes recorded in a recording device such as the memory 211 may be transmitted to the server 150 via the network 170 under the control of the communication module 213. Conversely, control signals, instructions, contents, files, etc. provided under the control of the processor 222 of the server 150 may be received by the electronic device 110 through the communication module 213 of the electronic device 110 via the communication module 223 and the network 170. For example, control signals, instructions, contents, files, etc. received through the communication module 213 may be transmitted to the processor 212 and the memory 211, and the contents and files, etc. may be recorded in a recording medium (the non-transitory recording device described above) that the electronic device 110 may further include.

[0025] The input / output interface 214 may be means for interfacing with the input / output device 215. For example, the input device may include devices such as a keyboard, a mouse, a microphone, a camera, etc., and the output device may include devices such as a display, a speaker, a tactile feedback device, etc. As another example, the input / output interface 214 may be means for interfacing with a device in which functions for input and output are integrated into one, such as a touch screen. The input / output device 215 may be composed of the electronic device 110 and one device. Also, the input / output interface 224 of the server 150 may be means for interfacing with a device (not shown) for input or output that can be connected to the server 150 or included in the server 150. As a more specific example, when the processor 212 of the electronic device 110 processes the instructions of the computer program loaded in the memory 211, a service screen or content configured using the data provided by the server 150 or the electronic device 120 may be displayed on the display through the input / output interface 214.

[0026] Also, in other embodiments, the electronic device 110 and the server 150 may include more components than the components shown in FIG. 2. However, it is not necessary to clearly show most of the conventional components in the figure. For example, the electronic device 110 may be realized to include at least a part of the above-described input / output device 215, or may further include other components such as a transceiver, a camera, various sensors, a database, etc. As a more specific example, when the electronic device 110 is an AI speaker, various components such as various sensors, a camera module, physical buttons, buttons using a touch panel, an input / output port, a vibrator for vibration, etc., which are generally included in the AI speaker, may be realized to be further included in the electronic device 110. (End of citation)

[0027] A machine is disclosed. According to at least one embodiment, the user terminal comprises a control unit, a RAM, a storage unit, a graphics processing unit, a communication interface, and an interface unit, which are respectively connected by an internal bus.

[0028] According to at least one embodiment, the control unit is composed of a CPU and a ROM. The control unit executes the program stored in the storage unit to control the user terminal. The RAM is the work area of the control unit. The storage unit is a storage area for storing programs and data. The control unit reads the program and data from the RAM for processing. The control unit processes the program and data loaded into the RAM and outputs a drawing command to the graphics processing unit.

[0029] According to at least one embodiment, the graphics processing unit is connected to a display unit. The display unit has a display screen. When the control unit outputs a drawing command to the graphics processing unit, the graphics processing unit outputs a video signal for displaying an image on the display screen. Here, the display unit may be a touch panel provided with a touch sensor. The touch panel of this display unit functions as an input unit.

[0030] According to at least one embodiment, the communication interface can be connected to a communication network wirelessly or by wire, and can transmit and receive data with a server device via the communication network. The data received via the communication interface is loaded into the RAM and processed by the control unit. An external memory (e.g., an SD card, etc.) is connected to the interface unit.

[0031] According to at least one embodiment, the user terminal is not particularly limited as long as it is a computer device having a display screen and an input unit. Examples of the user terminal include a conventional mobile phone, a tablet terminal, a smartphone, a desktop or notebook personal computer, etc. It may also be composed of a VR goggle, that is, a screen (or two display panels, one for each eye) attached to a frame (or headset) fixed or attached to the head with a strap. The user terminal has an audio output unit.

[0032] According to at least one embodiment, the user terminal can be communicatively connected to the server device via a communication network. It can communicate via the communication network to send or receive information.

[0033] According to at least one embodiment, the server device includes at least a control unit, a RAM, a storage unit, and a communication interface, which are connected by an internal bus respectively.

[0034] According to at least one embodiment, the control unit is composed of a CPU and a ROM, executes a program stored in the storage unit, and controls the server device. Also, the control unit has an internal timer for timing. The RAM is a work area of the control unit. The storage unit is a storage area for storing programs and data. The control unit reads the program and data from the RAM, and performs program execution processing based on information received from the user terminal, etc.

[0035] Disclosed is AI. According to at least one embodiment, artificial intelligence includes machine learning, deep learning, generative AI, large language models, LLMs, foundation models, generative AI. Generative AI uses transformers and employs a number of mechanisms called attention. It uses self-supervised learning, Extract Prediction. In this case, the AI can predict the next word. Given a sentence, it predicts the next word from the text up to that point. It creates a large number of supervised learning problems. As a result, an AI that can predict the next word can be created. Generative AI can predict grammar structures, topic connections, and the likelihood that a person with such a style will write such a sentence. Furthermore, generative AI can learn the structure, causal relationships, and knowledge behind just predicting the next sentence. Generative AI has high scalability, and the larger the number of parameters, the higher the accuracy. Ordinary statistics and machine learning will overfit if the model parameters are made too large compared to the data sample size. LLMs increase in accuracy as the number of parameters is increased. One generative AI has 175 billion parameters. Generative AI is trained with supervised learning to have smooth conversations. It is trained not to say strange things. It writes reviews or acts as a call center operator.

[0036] According to at least one embodiment, a large language model (LLM) is, non-exclusively, a natural language processing model of machine learning constructed using a large amount of dataset and deep learning technology. Generally, it is adapted to various natural language processing (NLP) tasks such as text classification / generation, sentiment analysis, text summarization, and question answering by using a method called "fine-tuning" for training on specific tasks. According to at least one embodiment, self-supervised learning is close to human essential intelligence. When humans act, they always predict the next event and the next input. In the process, they can learn the structure of the external world. Predicting the next word is an essential intelligence and is considered to be similar to what the cerebral cortex does. According to at least one embodiment, a large language model memorizes all the input information but generalizes to the extent necessary to predict the next word. It does not generalize all the information from the beginning. A large language model requires capacity to memorize information. Also, parameters are required for that purpose. According to at least one embodiment, a large language model is equipped with 175 billion parameters or eight models with 220 billion parameters.

[0037] According to at least one embodiment, videos and images are represented as a set of visual patches, which are small data units similar to the text tokens of an LLM. Patches can effectively represent the model of visual data and are used as a very scalable and effective representation for training generative models with various types of videos and images. First, a video is compressed into a low-dimensional latent space, and then the representation is decomposed into spatio-temporal patches to convert the video into patches.

[0038] According to at least one embodiment, a Video compression network is a network that reduces the dimension of visual data. It receives raw videos as input and outputs a temporally and spatially compressed latent representation. AI is trained in this compressed latent space and then generates videos within this compressed latent space.

[0039] According to at least one embodiment, given a compressed input video, Spacetime Latent Patches extracts a series of spatio-temporal patches that function as transformer tokens. With the patch-based representation, Sora can be trained on videos and images of various resolutions, lengths, and aspect ratios, and at inference time, by placing randomly initialized patches into a grid of appropriate size, it controls the size of the generated video.

[0040] According to at least one embodiment, the AI is a diffusion model and is trained to predict the original "clean" patches when noisy patches (and conditional information such as text prompts) are input. The AI is a diffusion transformer, which exhibits remarkable scaling properties in various areas such as language modeling, computer vision, and image generation. The diffusion transformer is also effective as a video generation model. As the computational cost of training increases, the quality of the samples improves significantly for the AI.

[0041] According to at least one embodiment, the AI applies caption regeneration technology to train a very explanatory caption model and then uses it to generate text captions for all videos in the training set. Training highly explanatory captions improves not only the overall quality of the generated videos but also the faithfulness of the text. Utilize GPT to convert short user prompts into long detailed captions and send them to the model. Thereby, the AI can generate high-quality videos that exactly follow the user's prompts.

[0042] According to at least one embodiment, in natural language processing, vectorization can be performed by AI along the following process. First, cleaning processing of the text given as preprocessing is performed. In the cleaning processing, unnecessary words such as JavaScript code and HTML tags included in the text are deleted. Since these codes are codes used for display on the Internet, they are generally information not used in natural language processing. Subsequently, the text is split into word levels by morphological analysis. Morphological analysis is to classify into the smallest language units with meaning in a sentence of natural language written in characters. As morphological analysis tools, "MeCab", "JUMAN", and "JANOME" can be used. In normalization, words with the same meaning such as writing variations are unified into one word. Stop words are words that are excluded from processing for reasons such as being unusable in natural language processing. Examples of stop words include those that do not have meaning alone, such as particles and auxiliary verbs among words. When calculating vectors, these may be removed and only meaningful words may be targeted. Vectorization may also be performed without removing these stop words. Vectorization is a process of converting a word, which is a character string, into a vector. By vectorization, word data is converted into numerical data. When converting a word into a vector, it is performed by a method called Bag of Words or distributed representation. Bag of Words is a method of vectorizing a text by using the number of occurrences of words that appear in the given text. Since it focuses on how many words appear in the text, the order of words and text is not considered. Distributed representation is a method of vectorizing by focusing on the meaning of a word. By vectorizing the meaning of a word, it is possible to give vectors close to words with similar meanings and usage, and the relationship between words can also be expressed by vectors. By expressing in vectors, addition and subtraction of the meanings of words are possible. The application process can utilize the natural language converted into numerical data as the input of machine learning. Specifically, the vectorized natural language is input into a classifier to perform text classification.Examples of tools used here include "TensorFlow", "scikit-learn", "PyTorch", etc.

[0043] Disclose a dwelling. In at least one embodiment, a dwelling is a place where a person uses it as a living place. Non-limiting examples include a single-family house (an independent house built on one lot), an apartment building (a house where multiple households gather in one building, such as a condominium, apartment, or housing complex), a row house (a house where multiple households are connected in a row house format), a villa, a dormitory, a share house, etc.

[0044] Disclose an image acquisition device. In at least one embodiment, the image acquisition device can be realized by any of the devices described in this disclosure. Non-limiting examples include an image acquisition device including a device that captures an image and converts it into digital data. A digital camera (captures still images and videos and stores them on a recording medium), a video camera (mainly captures videos), a web camera (connected to a personal computer and used for video conferencing, live streaming, etc.), a surveillance camera (installed for security purposes and constantly records images), and a smartphone camera.

[0045] Disclosed is a step of a video acquisition device acquiring video information of all rooms in a dwelling. In at least one embodiment, all rooms in a dwelling include all rooms where a resident can live. Thus, rooms that do not meet the purpose of a resident's living can be excluded from the definition of all rooms. As an example, a room or space solely for storage purposes can be cited. In at least one embodiment, "all rooms" does not necessarily cover all spaces in a dwelling. "All rooms" is a collection of predetermined spaces in a dwelling where a resident is considered to mainly live. To be able to acquire video information of all rooms, a video acquisition device is arranged in the dwelling. As an example, a video acquisition device is installed in a room. It is not necessary to install one or more video acquisition devices in each room. The minimum number of installed video acquisition devices can be less than the number of rooms as long as video information of all rooms can be acquired. As an example, a resident can also wear a wearable terminal as a video acquisition device to acquire video information of all rooms. The time when the video acquisition device acquires video does not have to be all the time. The time for acquiring video is arbitrarily determined. Video can be acquired every five minutes. If it is known that a resident will be away for a long time, video does not have to be acquired during the absence period. Of course, video can be acquired all the time.

[0046] Disclosed is a step of requiring an AI to determine the presence or absence of an accident of an occupant based on video information. In the present disclosure, "accident" is not limited to an accident related to an object, but is a concept including the illness of an occupant. Non-exhaustive examples include fires (accidents caused by fire), natural disasters (disasters caused by natural phenomena such as earthquakes, typhoons, floods, etc.), and product accidents. Further, it includes the occupant's fall, fainting, seizures that never occur, convulsions, strange behavior, cerebrovascular disorders, aortic dissection, and cardiovascular disorders. In at least one embodiment, the AI performs supervised learning about the presence or absence of an accident of an occupant. That is, the AI receives supervised learning with non-accident videos and accident videos, and fine-tuning as necessary. By this, for example, it is possible to determine whether the occupant is simply taking a nap or writhing in pain due to an injury. Examples of fine-tuning include learning the state of a healthy person going to bed, suddenly collapsing due to a cardiovascular disorder, and becoming immobile due to a cerebral hemorrhage. Learning the state of a healthy person doing yoga and the state of a seizure. Learning the state of a healthy person accidentally falling and getting up, and the state of falling and hurting bones and muscles and being unable to get up. In addition, learning is performed with the states of fires, natural disasters, and product accidents. The device requires the AI to determine the presence or absence of an accident of an occupant based on the video information.

[0047] Disclosed is a step of transmitting information to a predetermined external device when an accident of an occupant is determined. The predetermined external device includes, as non-exhaustive examples, a medical institution or a medical worker, a communication terminal of a caregiver or a relative. The occupant can register the external device in advance, and any external device can be registered. When the AI determines an accident of the occupant, the device transmits information to the registered external device. Examples of the information include the determination that an accident of the occupant has occurred, the video of the occupant, the reason why the AI determines an accident, and combinations thereof.

[0048] According to this embodiment, it is possible to reduce the mortality rate of residents due to accidents. For example, cerebrovascular disorders have a high incidence rate in the early morning. There are cases where a person collapses while trying to go to the toilet early in the morning. However, residents who have developed these disorders die without being found by their family members while lying collapsed in their living rooms or in front of the toilet. According to this embodiment, the device identifies a resident who has been in such an accident and promptly transmits information to that effect to an external device. Therefore, the resident can receive appropriate medical treatment and assistance.

[0049] In at least one embodiment, the request to the AI is to determine whether or not a resident has fallen. As already explained, the AI performs supervised learning or fine-tuning based on the behavior of a healthy person who accidentally falls and gets up, and the behavior of a person who falls and cannot get up due to pain in the bones and muscles. Elderly residents may fall and fracture their bones at home because their bone strength has decreased. However, a single elderly person may not be able to call for help while having a fracture. According to this embodiment, the device identifies a resident who has been in such an accident and promptly transmits information to that effect to an external device. Therefore, the resident can receive appropriate medical treatment and assistance.

[0050] In at least one embodiment, the video information relates to the bathroom, and the request to the AI is to determine whether the occupant has drowned in the bathroom. If it is determined that the occupant has drowned, a request is made to the bathtub drain to drain the bathtub. Drowning is among the leading causes of death in the residence. The cause of drowning includes sudden death in the bathroom. The video acquisition device acquires video information of the bathroom. As a non-exhaustive example, a video acquisition device is provided in the bathroom. The device is provided on the ceiling or wall of the bathroom. "Video" in the present disclosure is a term with a very broad meaning. Generally, it refers to an image created by light, that is, visual information that can be captured by the eye. That is, "video" includes thermography (a technology for visualizing invisible heat). Thermography uses an infrared camera to detect infrared rays emitted from an object and captures the temperature distribution as an image by displaying its intensity in different colors. Naturally, the "video acquisition device" includes a device that acquires thermography. The device requests the AI to determine whether there is an accident of the occupant based on the video information. The AI performs supervised learning or fine-tuning on the state of drowning in the bathroom and the state of simply being in the bathtub. The video acquisition device provided on the wall captures the occupant from the side CopyThis is possible. Therefore, this device can detect whether the face of the occupant is above the water surface or the bathtub. Therefore, without using AI, the device can determine whether the occupant is drowning by determining whether the position of the occupant's face is above a predetermined position such as the water surface, the bathtub, or the like. These determinations can be made that the occupant is drowning if the time that the occupant's face is below the predetermined position is longer than the predetermined time. On the other hand, AI performs supervised learning or fine-tuning when the position of the occupant's face is above a predetermined position such as the water surface, the bathtub, or the like and when it is below. These learnings include fine-tuning between the case where the face is submerged in the hot water for only an instant and the case of drowning. The device requests this learned AI to determine whether the occupant is drowning. The bathtub drainer has a function of mechanically draining the bathtub. As an example, the drain plug or valve opens and closes by mechanical movement. The bathroom drainer opens and closes the drain plug or valve according to the instruction by receiving an electrical signal or an electromagnetic signal. The device requests the bathtub drainer to drain the bathtub when it is determined that the occupant is drowning. According to this embodiment, it is possible to prevent drowning when the occupant drowns in the bathtub. This is because when drowning in the bathtub, it is impossible for the occupant to call for help on their own, so basically they will drown. In at least one embodiment, further, when it is determined that the occupant is drowning, an embodiment of transmitting a signal to a predetermined external device is combined. Examples of the predetermined external device include a terminal where an emergency responder is present in addition to the examples shown in the present disclosure. According to this embodiment, it is possible to fundamentally prevent drowning by draining the hot water in the bathtub and to perform emergency treatment promptly.

[0051] Disclosed is a step in which an image acquisition device acquires image information of at least one room, and the device requests an AI to identify the food and drink of the occupant based on the image information. In at least one embodiment, the image acquisition device acquires image information of at least one room. The expression "at least one" does not mean that only the image information of one room should be acquired. It is also possible to acquire images of all rooms. The image acquisition device acquires images of food and drink. Therefore, the device is provided on the ceiling. The device acquires an image in the vertically downward direction from the ceiling. Therefore, the device is favored with an opportunity to accurately capture the panorama of the food and drink. However, in the present disclosure, it is not a necessary condition for the device to be on the ceiling. The device may be provided on a wall or other furniture. The device may even be the occupant's wearable terminal. The AI performs supervised learning or fine-tuning with the images of food and drink and the names of the food and drink. As a non-exhaustive example, the AI identifies a sandwich from an image of a sandwich and a steak from an image of a steak.

[0052] The device discloses the step of estimating the nutritional components of a specified food or beverage from a database. The database holds data on the nutritional components of foods and beverages. This data includes, for example, the nutrition facts table of foods. The nutrition facts table is a list of the types and amounts of nutrients contained in that food. In addition to major nutrients such as calories, protein, fat, carbohydrates, and sodium, information such as vitamins and minerals is also described for some foods. The device estimates the nutritional components of the food or beverage identified by the AI from the database. The device may identify the size of the resident's food or beverage from the video. The provided video acquisition device and the dining table have a fixed distance between the dining table and the video acquisition device. Therefore, the device can convert the size of the captured video into the actual size of the food or beverage using a predetermined conversion value. The device can estimate the size of the food or beverage by comparing the video of the tableware reflected beside the food or beverage in the video acquired by the video acquisition device of the wearable terminal. The database holds data on the amounts of nutritional components corresponding to the size of the food or beverage. The device estimates the nutritional components from the database based not only on the name of the specified food or beverage but also on the size of the food or beverage.

[0053] The device discloses a step of transmitting a signal to a predetermined external device when the nutrient component is below a predetermined value. Humans have determined nutritional requirements. The "nutritional requirements" in the present disclosure calculate the daily required energy, protein, fat, and carbohydrate amounts, and further determine the vitamin, trace element, and water amounts. Nutritional requirements vary slightly depending on human characteristics (gender, nationality, age). The device obtains this information from a database of nutritional requirements corresponding to the characteristics according to the characteristics of the resident. The device totals the nutritional components of the specified foods and beverages over a predetermined period (for example, one day, two days, or one week). If the predetermined period is set to one week, the nutritional components of the specified foods and beverages for one week are added up. For example, among the images obtained by the image acquisition device, if it is curry, 700 kcal of energy is added, and if it is a large curry, 1000 kcal is added to calculate the nutritional components of the foods and beverages for one week. The device calculates the nutritional requirements over a predetermined period. The "predetermined value" in the present disclosure is the nutritional requirement over the calculated predetermined period. The predetermined value is calculated for each nutritional item such as the required energy fee and vitamin amount. If the predetermined period is set to one week, the value obtained by multiplying the daily nutritional requirement of zinc registered in the database by seven is the nutritional requirement of zinc. The device transmits information to a predetermined external device when the nutritional components of the specified foods and beverages are below the predetermined value. The "predetermined external device" is as described in other embodiments. Additionally, the external device includes the terminal of a dietitian, the terminal of a delivery service provider that delivers foods and beverages to the residence, or a restaurant. The "information" includes that the nutritional components of the specified foods and beverages are below the predetermined value, the images of the foods and beverages, the insufficient difference between the nutritional components of the specified foods and beverages and the predetermined value, the nutritional components below the predetermined value, the foods and beverages having nutritional components greater than the difference of the nutritional components below the predetermined value, and combinations of these pieces of information. According to the present disclosure, a person or service institution outside can take measures to correct the nutritional imbalance of the resident. As a result, the health of the resident can be improved, nutritional disorders can be prevented, and diseases can be prevented. This is because residents are mostly unaware of the lack of their essential nutrients and may overeat.

[0054] When the nutritional component is below a predetermined value, a step of identifying one or more food and drink items having a nutritional component greater than the difference of the nutritional component below the predetermined value from a database is disclosed. In at least one embodiment, the difference between the predetermined value and the total value of the nutritional components of the identified food and drink item is calculated. The database stores information regarding the nutritional components of each food and drink item. For example, in a certain database, the nutritional components of oysters are 3 mg of vitamin C, 88 mg of calcium, 1.9 mg of iron, 0 μg of vitamin D, 0.1 mg of vitamin B6, 28.1 μg of cobalamin, and 74 mg of magnesium per 100 g serving. When the difference in the nutritional component below the predetermined value is 70 mg of magnesium, 100 g of oysters is identified. Of course, 100 g of oysters is just an example. The device may also identify raw materials that are easier to cook or the dishes themselves for which the nutritional content is registered.

[0055] A step of the device presenting information regarding the identified food and drink item to the resident is disclosed. In at least one embodiment, the device transmits information regarding the identified food and drink item to the resident's terminal. The terminal is one of the machines described in the present disclosure. As a non-exhaustive example, it includes the resident's smartphone, tablet terminal, wearable terminal, computer or tablet terminal provided in the residence. The terminal presents the transmitted information. The method of "presenting" includes methods such as outputting and displaying an image on a display and using a sound output unit to speak. According to the present disclosure, the resident can obtain guidelines for supplementing their lacking nutrients. This is because the resident can simply eat the food and drink items they prefer from among the presented candidates.

[0056] Disclosed is a step of a voice acquisition device acquiring voice information of at least one room. In at least one embodiment, the voice acquisition device acquires voice information of at least one room. The expression "at least one" does not mean that only the voice information of one room should be acquired. It may acquire the voice of all rooms. The voice acquisition device may be provided in a room or may be the resident's terminal itself. The voice acquisition device acquires the speech content of the resident in the room.

[0057] Disclosed is a step of the device requesting the AI to talk to the resident. A series of flows of the AI conversation is to correctly recognize the words written or spoken by humans, interpret the meaning and intention of the words, and give a response that matches the intention. The natural language processing model of machine learning described in this disclosure is used. The AI is fine-tuned in the way of a psychiatrist's interview. By doing so, the AI can analogize and generate questions for determining whether the resident has a mental illness or mental abnormality. The device presents the conversation content generated by the AI to the resident. The method of "presentation" is as described above. The voice acquisition device acquires the speech content of the resident in the room and transmits the information to the AI. By this method, the AI can talk to the resident.

[0058] Disclosed is a step of requiring AI to determine whether a resident has a mental illness or mental abnormality based on voice information. In at least one embodiment, the AI is trained with teacher learning using information on psychiatric interviews and diagnoses and is fine-tuned. Further, it is trained with teacher learning using information on neurological interviews and diagnoses and is fine-tuned. By doing so, the AI can determine the possibility of a mental illness or mental abnormality corresponding to the content of the resident's speech. As an example, mental illnesses include major depressive disorder, bipolar disorder, anxiety disorder, and schizophrenia. "Mental abnormality" is a broad concept and includes dementia and cognitive impairment in the present disclosure. Dementia refers to a state in which cognitive functions such as memory, judgment, and thinking ability decline due to a brain disease, causing problems in daily life. If the flow of the resident's conversation contains contradictions or there are memory impairments, it can be determined as dementia.

[0059] Disclosed is a step of transmitting a signal to a predetermined external device when it is determined that the resident has a mental illness or mental abnormality. In at least one embodiment, the "external device" is any of the embodiments described in the present disclosure. Further, the external device includes the terminals of medical staff in neurology, psychiatry, and psychosomatic medicine. According to the present disclosure, it is possible to quickly detect the possibility that a resident living alone has a mental illness or mental abnormality and perform necessary treatment and intervention. This is because mental illnesses or mental abnormalities of residents living alone are often left unnoticed by the individuals themselves.

[0060] Disclosed is a step in which the device stores the conversation content and receives an inquiry from the resident. In at least one embodiment, the device stores the conversation content between the resident and the AI. The resident can make an inquiry to the device from the resident's terminal. As a non-exclusive example, a question such as "What did I eat the day before yesterday?" is input as voice or text to the terminal or the device. Of course, the inquiry about what was eaten is just an example. "Inquiry" includes all inquiries that can be answered with past conversation content.

[0061] Disclosed is a step in which the device presents conversation content corresponding to an inquiry to the resident. In at least one embodiment, the device searches whether the answer corresponding to the inquiry is in the stored conversation content. When the device finds the answer or conversation content related to the answer as a result of the search, it presents that information to the resident. "Presentation" is any of the embodiments described in the present disclosure. According to this embodiment, the resident can utilize the conversation with the AI as a diary or a memo. Things that the resident has forgotten can also be easily recalled with the assistance of the device.

[0062] At least one embodiment is as follows. The residence in the present disclosure can be connected even if they are away as long as there is a device, whether it is an ordinary family or a family living alone. An AI intervenes among these people. The merits in the present disclosure are not limited to devices for the elderly, but also exist for young people and children living alone. "Device" includes dedicated machines, mobile phones (including smartphones), external inputs of televisions, and smart devices (Google Home (registered trademark)).

[0063] It has a health monitoring and alert function. The device measures vital signs. The device receives information on heart rate, blood pressure, body temperature, and oxygen concentration measured by the measuring device and monitors these vital signs. When an abnormal value of the vital sign is detected, the device transmits information to an external terminal. "Information" includes the fact that an abnormality has occurred, the abnormal value, and combinations of these information. "External terminal" is any of the embodiments described in the present disclosure. According to this embodiment, when there is a danger to the life of the resident, medical intervention can be carried out promptly.

[0064] The device has the function of a medication reminder. The resident may need to take medicine three times a day. If such medicine-taking is neglected, there is a possibility of causing intracerebral hemorrhage again. The device stores the resident's medicine-taking time. "Remind" includes the device verbally confirming with the resident whether the medicine has been taken or sending information indicating that the resident has not taken the medicine to an external device. The device can determine from the video that the resident is not performing the behavior of taking medicine. According to this embodiment, it is possible to prevent the situation where the resident does not take the necessary medicine. An external person can intervene with a resident who absolutely refuses to take medicine.

[0065] The device identifies the resident's food and drink based on video information. The food and drink include water. As a non-exhaustive example, from the video of a cup of water, the fact of 200 ml of water intake is inferred. Water is, of course, an essential nutrient. The device has a function of tracking food intake and water intake. The device records the food intake and water intake amount and warns the resident or an external terminal if there is an insufficient intake. "Warning" includes transmitting or displaying to the terminal the fact of the shortage, the amount of the shortage, a combination of these information. As a non-exhaustive example, display on the resident's terminal that there is a possibility of dehydration. According to this embodiment, the number of incidents where the resident is taken to the emergency due to heatstroke can be reduced. This is because the resident is not aware that their water intake is insufficient.

[0066] The device has the function of daily life support. The device has a contact function for proxy shopping. When food and daily necessities are insufficient, the device sends a request to the terminal of the local government or the family. "Request" includes the fact that something is insufficient, the amount of shortage compared to a predetermined value, the amount to be purchased, and a combination of these information. When the device determines the amount to be purchased, it automatically settles the purchase and the item is delivered to the residence. As a non-exhaustive example, the device determines based on the video that daily necessities are insufficient. In this case, the video acquisition device is in the space where daily necessities are stocked. As an example, a storage room, inside a closet, or inside a refrigerator can be mentioned. According to this embodiment, daily necessities can be automatically stocked.

[0067] The device has a simple household appliance operation support function. It is possible to operate lighting, air conditioners, TVs, etc. through the resident's terminal by voice or touch. According to this embodiment, the living convenience of the resident is improved. This is because there is no need to physically operate different switches for each device.

[0068] The device has a reminder function. The device stores information regarding schedules such as the day of trash collection and hospital appointments. The resident terminal or an external terminal can transmit the information of these schedules to the device and have it stored in the device. Based on the schedule information, the device "presents" the fact and the schedule to the resident on that day, at that time, and at a prior time when they should be alerted. "Presentation" is any of the embodiments described in this disclosure. According to this embodiment, the resident can utilize the device as a reminder. Things that the resident has forgotten can also be easily recalled with the assistance of the device.

[0069] The device has a psychological and emotional care function. The AI chats with the resident and listens to the resident's consultations. The device in this disclosure becomes an AI counselor. The device provides cognitive decline prevention games (simple puzzles and memory training games) to the resident through the terminal. The device determines the resident's mood based on the conversation information between the AI and the resident. The device regularly checks the user's mood and notifies family members or caregivers if a drop in mood is observed. According to this embodiment, the sense of loneliness of the resident can be reduced.

[0070] The device can respond to falls and emergencies. The fall detection function is a function that automatically contacts local governments or family members in case the device detects a fall. The emergency button has a function that can be used to make an emergency call with one touch. The call notifies the user's guardian or medical institution of the fact that the emergency button has been pressed and the location. The button is a physical button and multiple buttons are provided in the residence. In the present disclosure, the voice acquisition device analyzes the voice of the resident by language processing and can determine whether the resident has the intention to make a call. When the device makes such a determination, it transmits information such as voice and combinations thereof to an external terminal to that effect. According to this embodiment, assistance can be promptly started for the accident of the resident.

[0071] The device has a function to promote social participation. The guidance of virtual events is a mechanism that enables easy participation in local online events and hobby gatherings (referred to as "events, etc." in the present disclosure). The external terminal transmits information about events, etc. to the device. The device presents information about events, etc. to the resident's terminal. The video call function with family and friends is a function that supports regular communication with family members with easy operation. The device communicates with the terminals of the resident's family and friends as an external terminal. "Communication" includes video calls. The device realizes video calls using a video acquisition device and a voice acquisition device. According to this embodiment, social isolation of the resident can be prevented. This is because social isolation of the single-person household is a social problem.

[0072] The device has a cooperation function with local governments. The guidance of local government services is a function that appropriately notifies users of available welfare services and subsidy information. The data sharing function is a mechanism for securely sharing health data and consultation histories with local governments. In the case of data sharing, the privacy of residents is taken into consideration. The device communicates with the terminals of local government staff as external terminals. The said terminals transmit information regarding the guidance of local government services to the device. The device presents the said information to the residents' terminals. The device receives information regarding the health data and consultation histories of residents from the residents' terminals or voice acquisition devices. The device transmits the received information to external terminals. According to this embodiment, the cooperation between residents and local governments can be made more efficient.

[0073] The device has a personalized AI function. Based on conversations with residents, the AI learns the preferences and habits of the residents (users). Based on the learning, the AI provides more appropriate advice and services. As a non-exhaustive example, the AI can give advice on diet and exercise tailored to specific health problems (such as diabetes and hypertension). According to this embodiment, individual and specific problems of residents can be solved.

[0074] The device has a health management AI function. As a non-exhaustive example, it analyzes words such as "having no appetite recently" of residents determined by a voice acquisition device or the slowdown of the movements of residents determined by a video acquisition device, and transmits information to the terminals of medical institutions or family members as necessary. According to the present disclosure, by combining natural language processing and behavior analysis, abnormalities can be detected in advance and appropriate interventions can be made.

[0075] The device can analyze the changes in the resident's emotions based on the daily conversations and behaviors between the resident and the AI, and detect signs of their loneliness and depressive symptoms. The AI uses information regarding the tone of the resident's voice, word usage, and conversation frequency for these analyses and judgments. As a non-exhaustive example, if the resident continues to utter negative language, the AI provides the resident with encouraging conversations. Furthermore, the device transmits information regarding the fact that the resident continues to utter negative language, the content of such utterances, and combinations thereof to the family's terminal. According to the present disclosure, more natural and effective conversations can be provided to the resident through advanced emotion analysis. Furthermore, when the resident is mentally unwell, external parties can intervene promptly.

[0076] The device has an individually optimized healthcare AI function. The AI learns the user's medical history, lifestyle habits, and likes and dislikes based on information input by the resident or external terminals and the content of conversations between the AI and the resident. The AI generates an individually optimized health management plan. Non-exhaustive examples include dietary suggestions for the resident, rehabilitation exercise programs, and sleep management. The device daily modifies or updates these plans based on the daily conversation content. The device communicates these plans with an external terminal. Doctors and dietitians may view the received plans and modify these plans. The device modifies the plan based on the plan transmitted from the external terminal. According to the present disclosure, precise health management incorporating external advice can be achieved.

[0077] The device has a memory assistance AI function. The device revives the "forgotten memories" of the elderly. The device records the content of conversations between the AI and the resident. The device records or organizes the necessary information to assist the resident's memory. As a non-exhaustive example, in response to a resident's question such as "I don't remember what I ate yesterday," the device answers based on the recorded information, "I ate udon for lunch yesterday." According to the present disclosure, the device assists the resident's memory and makes the resident's life more convenient.

[0078] The following discloses an overview of the embodiments described above.

[0079] A dwelling, wherein an image acquisition device acquires video information of the entire room, the device requests an AI to determine whether there is an accident of an occupant based on the video information, and when it is determined that there is an accident of an occupant, a signal is transmitted to a predetermined external device. A dwelling.

[0080] The above-mentioned dwelling, wherein the request to the AI is a request to determine whether the occupant has fallen. A dwelling.

[0081] A dwelling, wherein an image acquisition device acquires video information of a bathroom, the device requests an AI to determine whether the occupant has drowned in the bathroom based on the video information, and when it is determined that the occupant has drowned, the bathtub drain is requested to drain the bathtub. A dwelling.

[0082] A dwelling, wherein an image acquisition device acquires video information of at least one room, the device requests an AI to identify the food and drink of the occupant based on the video information, estimates the nutritional components of the identified food and drink from a database, and when the nutritional components are below a predetermined value, information is transmitted to a predetermined external device. A dwelling.

[0083] A dwelling, wherein an image acquisition device acquires video information of at least one room, the device requests an AI to identify the food and drink of the occupant based on the video information, estimates the nutritional components of the identified food and drink from a database, when the nutritional components are below a predetermined value, one or more food and drink having nutritional components greater than the difference of the nutritional components below the predetermined value are identified from the database, and information regarding the identified food and drink is presented to the occupant. Residence

[0084] A residence, wherein at least one voice acquisition device acquires voice information of at least one room, the device requests the AI to converse with the resident, the device requests the AI to determine whether the resident has a mental illness or mental abnormality based on the voice information, and when it is determined that the resident has a mental illness or mental abnormality, a signal is transmitted to a predetermined external device. Residence

[0085] The above-mentioned residence, wherein the device stores the conversation content, receives an inquiry from the resident, and presents the conversation content corresponding to the inquiry to the resident. Residence

[0086] In at least one embodiment, the residence described in any of the present embodiments can be replaced with an embodiment of a system, program, or device.

[0087] A system, comprising a first terminal that acquires vital data of a user, and a second terminal that is held by at least a guardian of the user and displays that the vital data has deviated from a predetermined value. A system having the above

[0088] In at least one embodiment, the vital data includes data obtained by digitizing vital signs that are indicators of a person being alive. Specifically, it includes body temperature, pulse, blood pressure, respiration, etc. The means for acquiring vital data includes thermometers (electronic thermometers, tympanic thermometers, non-contact thermometers), sphygmomanometers (upper arm sphygmomanometers, wrist sphygmomanometers), pulse oximeters, electrocardiographs, and respiratory rate meters. Embodiments include incorporating vital data measuring devices into wearable devices such as smartwatches and wristbands. Non-contact vital sensing is included. The first terminal includes a terminal used or possessed by the user. The device determines whether the acquired vital data deviates from a predetermined value. For example, in the case of blood pressure, it is when the systolic blood pressure exceeds 140 and the diastolic blood pressure exceeds 90. Of course, these are merely examples. The provider or user of the residence, system, device, program according to this embodiment, and the user's guardian can set any predetermined value for each vital data. When the vital data deviates from the predetermined value, the device presents to at least the second terminal possessed by the user's guardian that the vital data has deviated from the predetermined value. The presentation methods include displaying on a display and notifying by voice or vibration. The presentation includes information such as the fact that it has deviated from the predetermined value, the vital data, the predetermined value, and combinations of one or more of these. According to this embodiment, at least the user's guardian can quickly detect the user's physical crisis and provide necessary assistance, which has convenience and industrial applicability.

[0089] The above system, The first terminal presents, on the first terminal, medication reminder information based on the expiration date information of the user's medication, analyzes the voice of the user to determine whether the user has taken the medication, system.

[0090] In at least one embodiment, the expiration information of the user's medication includes the timing of taking the medication. As an example, it could be taking one tablet three times a day. The medication reminder information includes embodiments where, for each medication, it is presented based on the expiration information of the medication. As an example, the reminder "Have you taken Tablet A?" is presented three times a day. The device presents the medication reminder information on the first terminal. The user vocalizes the medication situation to the device. The device analyzes the user's voice and determines whether the user has taken the medication. The device requests the AI to determine whether the user has taken the medication as needed. The device converts the user's voice into character information, analyzes the synonyms of the words and sentences, and determines whether the user has taken the medication. According to this embodiment, at least the user will not forget to take the medication, so the convenience of the user's medication management is improved and it has industrial applicability.

[0091] The above system, The vital data includes the nutrient intake or water intake, The predetermined value is determined based on the registered required amount of nutrients or required amount of water, System.

[0092] The method for obtaining data on the user's nutrient intake or water intake and the method for obtaining the predetermined value based on the required amount of nutrients are as described in any embodiment of the present disclosure. Regarding the required amount of water, since there are reference values for the required amount of water for humans by age and gender, the required amount of water can be obtained with reference to those values. According to this embodiment, there is at least the convenience and industrial applicability of preventing the user from developing malnutrition or dehydration symptoms.

[0093] A system, Obtains the consumption period information, which is the consumption period of the consumable, When the consumption period has elapsed, information regarding the consumable is displayed on the first terminal held by the user and the second terminal held by the user's caregiver. System.

[0094] In at least one embodiment, consumables include items that are consumed and worn out as they are used and ultimately used up. As an example, it includes office supplies, daily necessities (tissue paper, toilet paper, detergent, garbage bags), packaging supplies, cleaning supplies, tools and instruments, fuels (such as gasoline, kerosene), batteries, dry batteries, light bulbs, etc. The user or the user's guardian arbitrarily determines the consumption period, which is the consumption period of the consumable, for each consumable. As an example, a two-month consumption period is set for tissue paper. The user or the user's guardian inputs this consumption period information into the device. The device displays information regarding the consumable on the first terminal possessed by the user and the second terminal possessed by the user's guardian when the consumption period has elapsed since the start date of the period measurement. Information regarding the consumable includes information such as the consumable name, consumption period, price of the consumable, URL where the consumable can be purchased, and information of one or more combinations of these. According to this embodiment, since there is an opportunity to make a decision to purchase the consumable at least before the consumable is used up, there is the convenience and industrial applicability that the consumable does not run out.

[0095] A system, wherein a first detection means detects the current stock of the consumable, and the device displays information regarding the consumable on the first terminal possessed by the user and the second terminal possessed by the user's guardian when the current stock of the consumable falls below a predetermined amount based on the data of the predetermined amount of the consumable. System.

[0096] In at least one embodiment, the first detection means includes a camera and a scale. The device detects the current stock of consumables by means of the camera. The camera may be installed in a storage area. The device acquires an image of the consumables by means of the camera. The device analyzes the image and detects how many consumables or what volume there is in the image. The device identifies the current stock of consumables based on the weight of the consumables placed on the scale by means of the scale. When the consumable is a liquid, the device identifies the amount of the consumable by means of a level sensor (including a liquid level gauge, a float type, a capacitance type, an ultrasonic type, a radar type, a guided pulse type) attached to the liquid container. The user arbitrarily determines a predetermined amount for each consumable and inputs the information into the device. For example, for toilet paper, five pieces are the predetermined amount. Based on the data of the predetermined amount of the consumable, when the current stock of the consumable falls below the predetermined amount, the device displays information regarding the consumable on a first terminal possessed by the user and a second terminal possessed by the user's caregiver. According to this embodiment, there is convenience and industrial applicability in that the consumables do not run out because an opportunity for a purchasing decision can be obtained at least before the consumables are used up.

[0097] The above system, An image acquisition device that acquires video information acquires the user's video, The device requests the AI to determine whether the user's actions in the video are similar to predetermined actions, When it is determined that they are similar, information regarding ordering consumables is generated, System

[0098] In at least one embodiment, for the video acquisition device, the embodiments already described are incorporated. The user or the user's guardian determines a predetermined action. For example, drawing a circle with both hands above the head, making a peace sign with the hand, etc. The device requests the AI to determine whether the action of the user in the video is similar to the predetermined action. The AI includes an AI that has undergone supervised learning based on videos with the predetermined action and videos without the predetermined action. When the AI determines that the user's video includes the predetermined action or an action similar to the predetermined action, the device generates information regarding ordering consumables. The information regarding ordering consumables includes ordering the consumables. The user registers the payment means with the device in advance. The device makes the purchase payment on behalf of the user for the consumables. According to this embodiment, at least what the user should do when ordering consumables is only to perform the predetermined action. This improves convenience and has industrial applicability.

[0099] The above system, Based on reminder data regarding a specific action and its execution date, the device displays data regarding the specific action on the first terminal on the execution date. System.

[0100] In at least one embodiment, the specific actions include taking out the trash in the residential jurisdiction area and visiting the hospital. The execution date includes information on the date and time when the specific action should be performed. For example, the specific action is taking out the combustible trash, and the execution date is every Tuesday. In the case of visiting the hospital, the execution date is 2:00 p.m. on May 3rd. The reminder data includes the specific action, its execution date, information related to these, and information on one or more combinations of these. The user or the user's guardian inputs the reminder data into the device. The device presents data regarding the specific action on the first terminal on the execution date. It may also be presented on the second terminal. According to this embodiment, since at least reminders regarding actions necessary for life can be received, there is the convenience and industrial applicability that the user can surely perform the actions.

[0101] The above-mentioned system, wherein a first terminal communicates with a household appliance by an electromagnetic method, and is capable of starting or controlling the household appliance, is a system.

[0102] In at least one embodiment, the first terminal can communicate with the household appliance by an electromagnetic method. The device has a simple household appliance operation support function. Lighting, air conditioners, televisions, etc. can be operated by voice or touch through the resident's first terminal. The household appliance has a device for receiving electromagnetic signals. The first terminal communicates with the household appliance by an electromagnetic method and starts or controls the household appliance. Starting includes sending a signal for the household appliance to perform its function. Controlling includes controlling the performance of the specific functions of the household appliance. For example, changing the TV channels from channel 2 to channel 6. According to this embodiment, at least the convenience of the user's residence is improved, and there is industrial applicability.

[0103] A system, wherein a first terminal owned by a user acquires the user's voice data and video data, AI interacts with the user, and AI determines whether the user has a mental illness or mental abnormality based on the voice data and video data, and when it is determined that the user has or may have a mental illness or mental abnormality, data regarding the determination is transmitted to at least a second terminal held by the user's guardian, is a system.

[0104] In at least one embodiment, the above embodiment incorporates the methods described in the previously described embodiments.

[0105] The above-mentioned system, wherein a game is displayed on the first terminal, is a system.

[0106] In at least one embodiment, a game is displayed on a first terminal. The game is played according to certain rules and includes activities for participants to enjoy, have a sense of achievement, or compete for victory or defeat. For example, games that maintain or improve cognitive functions, Othello, and card-matching games can be mentioned. The device transmits the result of the game to a second terminal. According to this embodiment, there are advantages and industrial applicability in at least examining the user's mental illness and improving the user's cognitive function. By transmitting the result of the game to the second terminal, the user's caregiver can know the status of the user's cognitive function and thus feel at ease.

[0107] A system, wherein a first detection means detects a user's fall, and when the fall is detected, information regarding the fall is transmitted to at least a terminal held by the user's caregiver. The system.

[0108] In at least one embodiment, the first detection means includes a video acquisition device, a vibration detection device, and an electromagnetic sensor (including sensors using infrared rays or ultrasonic waves). The video acquisition device refers to the embodiments already described. The vibration detection device is provided to the user. This includes the method of making it a wearable terminal of the user. It is provided at any location in the dwelling (such as the floor). This includes combinations of one or more of these embodiments. The vibration detection device detects vibrations exceeding a predetermined value. The predetermined value is determined by the provider of the business of this embodiment. As an example, analyze the frequency of the vibration and set an unusual frequency as the predetermined value. For example, when the vibration intensity exceeds a predetermined value, or when including the waveform of the wavelength of the vibration. The electromagnetic sensor can detect that a person has fallen and is curled up on the floor. When the first detection means detects the user's fall, it transmits information regarding the fall to at least the terminal possessed by the user's guardian. This includes the embodiment of transmitting to the second terminal. The information regarding the fall includes the fact of the fall and the information used for analysis (including the waveform of the vibration, infrared data, ultrasonic data, and video). When the first detection means is a video acquisition device or an electromagnetic sensor, after the user has fallen, it measures the time the user does not get up and includes the data related to that time. According to this embodiment, there is the convenience and industrial applicability that at least when the user falls, the user's guardian can promptly start assistance.

[0109] The above system, further comprising a first terminal for the user to send a request for rescue, System.

[0110] In at least one embodiment, the first terminal can send a rescue request. The emergency button has a function that can be reported with one touch in an emergency. The report notifies the user's guardian or medical institution of the fact that the emergency button has been pressed and its location. The button is a physical button provided on the first terminal. Embodiments include where the first terminal is a tablet terminal or a smartphone and a predetermined location on the screen is tapped. In the present disclosure, the voice acquisition device can analyze the voice of the resident by language processing and determine that the resident has the intention to report. When the device makes such a determination, it transmits information such as the voice, the user's address, and combinations thereof to the terminal owned by the user's guardian or medical institution. According to this embodiment, there are convenience and industrial applicability in that assistance can be promptly started for the accident of the resident.

[0111] The above system, wherein the first terminal owned by the user receives information on event information, welfare service information, or subsidy information and the second terminal owned by the user's guardian or organization transmits information on event information, welfare service information, or subsidy information, and the first terminal can acquire the user's voice data and video data and transmit them to the second terminal, a system.

[0112] In at least one embodiment, the event information refers to the embodiments already described. Welfare services include various types of support and services provided to enable people in need of support, such as the elderly, disabled, children, and people living in poverty, to lead an independent life or to support their lives. For example, long-term care insurance services (home care, day care, institutional care, etc.), life support services, community comprehensive care systems, services based on the Act on Comprehensive Support for Persons with Disabilities (home care, employment support, life training, etc.), provision of prosthetics, payment of allowances, nurseries, children's welfare facilities, child guidance centers, child-rearing support services, the public assistance system, and the system for supporting the self-reliance of people living in poverty. Further, the first terminal can acquire the user's voice data and video data and transmit them to the second terminal. This embodiment refers to the embodiments already described. According to this embodiment, there are advantages and industrial applicability in that at least the user can obtain information on appropriate social services and the user can be provided with an opportunity to interact with society. As a result of the inventor's earnest consideration, the user can receive event information, information on welfare services, or information on subsidies, and the effect of reducing the feeling of loneliness is synergistically enhanced by conversing with an external person. This effect was not known at all in the prior art and has novelty and remarkable effects.

[0113] A system, wherein a device acquires the voice or video of a user, requests an AI to determine whether the user has a feeling of loneliness based on the voice or video of the user, and when it is determined that the user has or may have a feeling of loneliness, transmits data regarding the determination to at least a second terminal held by the user's guardian. The system.

[0114] In at least one embodiment, the device analyzes changes in the resident's emotions based on daily conversations and actions between the resident and the AI, and detects signs of their loneliness and depressive symptoms. The AI includes an AI that has learned with instruction using the voice or video of a person with a feeling of loneliness and the voice or video of a person without a feeling of loneliness. As these learning data, voice or video data based on the cooperation of a person determined to have a feeling of loneliness and a person not determined as a result of a questionnaire can be used. When the device determines that the user has or may have a feeling of loneliness, the device transmits data regarding the determination to at least a second terminal possessed by the user's guardian. The data regarding the determination includes that there is a feeling of loneliness and the facial expression or linguistic elements of the conversation that are the reasons therefor. The simplest linguistic element is the confession of being lonely. Of course, this is only an example, and the AI understands the context behind the conversation by natural language processing and determines the presence or absence of a feeling of loneliness. According to the present embodiment, there are convenience and industrial applicability in that at least the user's feeling of loneliness can be noticed early by the user's guardian and necessary human intervention can be made.

[0115] The above system, The voice of the user serving as a basis for determination includes any one or more pieces of information among the tone of voice, diction, frequency of conversation, and frequency of making negative statements, The AI learns with voice data of a person having a feeling of loneliness including any one or more pieces of information as described above. System.

[0116] In at least one embodiment, as a result of the inventor's sincere consideration, in the learning with instruction of the AI, learning based on the tone of voice, diction, frequency of conversation, and frequency of making negative statements is effective. In a person with a strong feeling of loneliness, the secretion of serotonin decreases. This is because the frequency of making negative statements increases. This effect has not been known at all in the conventional technology and has novelty and remarkable effects. According to the present embodiment, there are convenience and industrial applicability in that at least the user's feeling of loneliness can be detected by the user's indirect speech and behavior (speech and behavior other than the direct speech and behavior of saying "lonely").

[0117] The above system, when it is determined that the user has or may have a feeling of loneliness, the device requests the AI to interact with the user. System.

[0118] In at least one embodiment, the device stores the initial conversation sentence of the conversation. The device presents the initial conversation sentence to the user terminal. When the user answers, the device verbalizes the content and provides it to the AI. The AI performs natural language processing and generates a sentence corresponding to the content. By repeating this, the AI interacts with the user. As a result of the inventor's earnest consideration, it has been found that humans can relieve loneliness by having conversations. This effect is exhibited not only when the conversation partner is a human but also when the conversation partner is an AI. This effect was not known at all in the prior art and has novelty and remarkable effects. According to this embodiment, there is convenience and industrial applicability in that at least when the user has a feeling of loneliness, the feeling of loneliness can be eliminated.

[0119] A system, wherein the device acquires the user's voice or video over a period of two or more days, generates feature data regarding the user's lifestyle from the information, and creates proposal information on the user's diet content based on the feature data. System.

[0120] In at least one embodiment, the method of obtaining the user's voice or video employs one of the methods described herein. "Lifestyle" refers to the actions and habits that people repeat in their daily lives, including various elements across all aspects of life such as diet, sleep, exercise, work, hobbies, etc. For example, it includes eating habits (contents, time, frequency, balance of meals, etc.), sleep (sleep duration, quality of sleep, bedtime and wake-up time, etc.), exercise (frequency, intensity, type of exercise, etc.), rest (way of taking breaks, relaxation methods, etc.), smoking and drinking (presence or absence of smoking habits, frequency and amount of drinking, etc.), work (working hours, work content, workplace environment, etc.), hobbies and entertainment (hobby and entertainment activities, ways of spending time, etc.). The feature quantity is a numerical value indicating these frequencies, contents, and amounts that the device can extract from the voice or video. For example, having meals twice a day. The device creates proposal information on the user's meal content based on the feature quantity data. The device allows medical personnel to input proposal information based on the feature quantity data into the device. It is desirable to have more of these proposal information. The device creates proposal information on the user's meal content based on the feature quantity data. It is also possible to create two or more of these. For example, propose having meals three times a day. According to this embodiment, there are at least the convenience and industrial applicability of improving at least the user's lifestyle and preventing lifestyle diseases.

[0121] A system, wherein the device obtains the user's voice or video over a period of two or more days, generates feature quantity data regarding the user's lifestyle from the said information, and creates proposal information on the user's sleep method based on the feature quantity data. The system.

[0122] In at least one embodiment, the device creates proposal information on how the user sleeps. As an example, the feature amount includes sleep data. The sleep data includes bedtime and wake-up time, brain waves, eye movements, electromyogram, electrocardiogram, respiratory state, and blood oxygen saturation. The proposal information is, for example, to refrain from alcohol, go to bed at 22:00 at night, etc. According to this embodiment, there are convenience and industrial applicability in that at least the user's lifestyle can be improved and lifestyle-related diseases can be prevented.

[0123] A system, wherein the device acquires the user's voice or video over a period of two or more days, generates feature amount data related to the user's lifestyle from the information, and creates proposal information on the content of the user's rehabilitation based on the feature amount data. The system.

[0124] In at least one embodiment, the device creates proposal information on the content of rehabilitation. As an example, a rehabilitation medical staff inputs proposal information based on the feature amount data into the device. It is desirable to have more of these. The device creates proposal information on the content of the user's rehabilitation based on the feature amount data. Two or more of these may be created. For example, when the feature amount is that the user has few daily hobbies and entertainment activities, as an occupational therapy, content that supports participation in social activities such as working, going to school, and participating in community activities is proposed. According to this embodiment, there are convenience and industrial applicability in that at least the quality of the user's rehabilitation can be improved and the QOL can be improved.

[0125] The above system, wherein the device receives a question from the user, judges the answer to the question based on the acquired voice or video information, and presents information regarding the judgment to the user. The system.

[0126] In at least one embodiment, the device receives questions from users through devices or terminals, including receiving questions by voice and receiving text inputs. The device stores the acquired voice or video information. The device determines the answer to the question based on the acquired voice or video information. Embodiments include those that require AI to make such determinations. For example, a user asks the device, "What did I eat the day before yesterday?" Since the device has acquired the user's eating video with a video acquisition device, it refers to the data from the day before yesterday and answers, "It seems that yesterday morning you had curry, at noon you had udon, and you didn't eat anything at night." According to this embodiment, there are at least the convenience and industrial applicability of supporting the memory of at least users and residents.

[0127] In at least one embodiment, the description of "entire room" can be replaced with the description of "at least one room".

[0128] A dwelling, wherein a video acquisition device acquires video information of at least one room, and the device requires AI to determine the presence or absence of an accident of a resident based on the video information, and when it is determined that there is an accident of the resident, a signal is sent to a predetermined external device. The dwelling.

[0129] According to this embodiment, there are at least the convenience and industrial applicability of being able to provide immediate assistance when at least a resident has an accident.

[0130] A dwelling, wherein a voice acquisition device acquires voice information of at least one room, and the device requires AI to talk to the resident, and requires AI to determine whether the resident has a feeling of loneliness based on the voice information, and when it is determined that the resident has a mental illness or mental abnormality, a signal is sent to a predetermined external device. The dwelling.

[0131] A dwelling, wherein an audio acquisition device acquires audio information of at least one room, the device requests the AI to converse with the occupant, requests the AI to determine, based on the audio information, whether the occupant has or may have a mental illness, mental abnormality, or feelings of loneliness, and requests the AI to converse with the occupant when it is determined that the occupant has or may have a mental illness, mental abnormality, or feelings of loneliness, the dwelling.

[0132] According to this embodiment, at least a lonely resident can activate their mental activities through interaction with an AI or a device when they have a mental illness, mental abnormality, or feelings of loneliness. This offers the convenience and industrial applicability of delaying the progression of mental illness or mental abnormality, or reducing feelings of loneliness. In at least one embodiment, the AI can determine, based on voice information or video information, whether a person with a mental illness, mental abnormality, or feelings of loneliness has such conditions or the possibility thereof through supervised learning using the voice information or video information of such a person. In this case, the AI can not only determine that a resident has a mental illness, mental abnormality, or feelings of loneliness, but also determine the possibility thereof. As an example, the possibility can be evaluated with a statistical probability. That is, the AI analyzes elements of the voice information or video information of an individual specific resident and calculates the probability that such information is included in the set of voice information or video information of a person with a mental illness, mental abnormality, or feelings of loneliness. In one example, if it is included in the set of voice information or video information of a person with a mental illness, mental abnormality, or feelings of loneliness with a probability of 0.3, it is determined that the resident may have a mental illness, mental abnormality, or feelings of loneliness. Of course, 0.3 is just an example, and any value can be set. By this method, the AI can determine, based on voice information or video information, whether a resident may have a mental illness, mental abnormality, or feelings of loneliness.

[0133] A dwelling, wherein at least one voice acquisition device acquires voice information of a room, and based on the expiration date information of the resident's medication, presents medication reminder information on a first terminal, and the device analyzes the voice of the resident to determine whether the resident has taken the medication. The dwelling.

[0134] The above-mentioned dwelling, When it is determined that the resident has taken medicine, the device stores information on the medicine intake, and when the number of times of medicine intake stored reaches a predetermined value, information regarding a visit to a medical institution is presented to the first terminal. Residence.

[0135] According to the present embodiment, there is convenience and industrial applicability in that at least a situation where a resident forgets to take medicine can be prevented. Further, there is convenience and industrial applicability in that at least a situation where a resident forgets to visit a hospital can be prevented. In at least one embodiment, when it is determined that the resident has taken medicine, the device stores information on the medicine intake. The information on the medicine intake includes the name of the medicine, the date or time of taking the medicine, the amount or number of tablets taken, the amount or number of tablets taken in the past, and one or more combinations of these pieces of information. The device can collect this information based on a declaration from the resident through a video acquisition device or an audio acquisition device. The device stores information regarding a predetermined value of the number of times of medicine intake. The resident, the caregiver of the resident, a medical institution, etc. can input information regarding such a predetermined value into the device and store it. The predetermined value can be determined for each medicine. For example, if the prescribed number of tablets is 30 and the medicine is to be taken three times a day, theoretically, the resident must obtain the next medicine in 10 days. In this case, 30 becomes the predetermined value. When the number of times of medicine intake reaches the predetermined value, the device presents information regarding a visit to a medical institution to the first terminal. The information regarding a visit to a medical institution includes the available date and time for reservation at the registered medical institution, the name or address or opening hours of one or more medical institutions that can prescribe medicine identified based on the location information of the residence, information regarding the method of visiting these medical institutions or temporary closure, items necessary at the time of visit, and information for transmitting a confirmation of the reservation to the terminal of the medical institution when presenting the available date and time for reservation at the medical institution, and one or more combinations of these pieces of information.

[0136] A residence, wherein the video acquisition device acquires video information of at least one room, and the audio acquisition device acquires audio information of at least one room. The device requests the AI to determine whether an abnormality has occurred to the resident based on video information and audio information, and when it is determined that an abnormality has occurred, transmits a signal to a predetermined external device. Residence.

[0137] A residence, wherein a video acquisition device acquires video information of at least one room, an audio acquisition device acquires audio information of at least one room, the device requests the AI to determine whether an abnormality has occurred to the resident based on video information and audio information, and when it is determined that an abnormality has occurred, transmits a signal to a predetermined external device. Residence.

[0138] According to the present embodiment, there is convenience and industrial applicability in that at least when an abnormality occurs to the resident and the resident cannot cope with the abnormal situation by himself / herself, necessary assistance can be promptly provided to the resident. In at least one embodiment, the abnormality refers to the situations described in the present disclosure. Non-exclusively, it includes accidents, falls, drowning in the bathroom, malnutrition, mental illness, mental abnormalities, and feelings of loneliness.

[0139] The invention according to the present disclosure only needs to be able to exhibit at least one of the above-described effects.

Claims

1. A residence, The image capture device captures image information of the bathroom, The device asks the AI ​​to determine whether a resident has drowned in the bathroom based on the video information, If a resident is determined to have drowned, it will request the bathtub drainer to drain the bathtub; Sending the determination and the resident's address to the terminal of the medical institution, medical staff, or emergency personnel; Residence.

2. A residence, The image capture device captures image information of the bathroom, The device asks the AI ​​to determine whether a resident has drowned in the bathroom based on the video information, If a resident is determined to have drowned, it will request the bathtub drainer to drain the bathtub; The medical institution, medical staff, or emergency personnel is informed of the result and the address of the resident. Send a notification of the determination to the device of the resident's caregiver, relative, or guardian; Residence.

3. A residence according to claim 1 or 2, The image acquisition device includes a device for acquiring thermography. Housing

4. A residence according to claim 1 or 2, The image capture device captures the resident from the side, Housing

5. A dwelling as claimed in claim 1 or 2, the device determines whether the resident's face is above the water surface or the bathtub based on the video information; Housing

6. A residence according to claim 1 or 2, The image capture device is located on the ceiling or wall of the bathroom. Housing

7. A dwelling according to any one of claims 1 or 2, The AI ​​is supervised or fine-tuned to recognize drowning in the bathroom versus simply taking a bath. Housing

8. A dwelling as claimed in any one of claims 1 or 2, The AI ​​is supervised or fine-tuned to recognize when the resident's face is above or below the water surface, bathtub, or a predefined position. Housing

9. A dwelling according to any one of claims 1 or 2, The AI ​​is fine-tuned to detect when a resident simply submerges their face in the water for a moment, versus when they are drowning. Housing

10. A residence according to claim 1 or 2, The device determines that the resident is drowning if the resident's face is below a predetermined position for a period of time greater than a predetermined period of time. Housing

11. A system comprising: The image capture device captures image information of the bathroom, The device asks the AI ​​to determine whether a resident has drowned in the bathroom based on the video information, If a resident is determined to have drowned, it will request the bathtub drainer to drain the bathtub; Sending the determination and the resident's address to the terminal of the medical institution, medical staff, or emergency personnel; system.

12. A system comprising: The image capture device captures image information of the bathroom, The device asks the AI ​​to determine whether a resident has drowned in the bathroom based on the video information, If a resident is determined to have drowned, it will request the bathtub drainer to drain the bathtub; The medical institution, medical staff, or emergency personnel is informed of the result and the address of the resident. Send a notification of the determination to the device of the resident's caregiver, relative, or guardian; system.

13. An apparatus comprising: The image capture device captures image information of the bathroom, The device asks the AI ​​to determine whether a resident has drowned in the bathroom based on the video information, If a resident is determined to have drowned, it will request the bathtub drainer to drain the bathtub; Sending the determination and the resident's address to the terminal of the medical institution, medical staff, or emergency personnel; Device.

14. An apparatus comprising: The image capture device captures image information of the bathroom, The device asks the AI ​​to determine whether a resident has drowned in the bathroom based on the video information, If a resident is determined to have drowned, it will request the bathtub drainer to drain the bathtub; The medical institution, medical staff, or emergency personnel is informed of the result and the address of the resident. Send a notification of the determination to the device of the resident's caregiver, relative, or guardian; Device.