Video-based remote home care system

Abstract

The present invention discloses a homecare system. More specifically, the present invention relates to a video-based remote homecare system that monitors the occurrence of various unexpected events such as emergency situations in real time based on the video of the person being observed captured by using an IP-CAM in a remote location. According to an embodiment of the present invention, one or more IP-CAMs or smart terminals are installed in a residence where a vulnerable class such as an elderly person living alone lives, and the current appearance of the protected person is captured and collected in real time, and the presence or absence of an emergency situation is determined through analysis of the collected video, and the protected person is protected by reporting it to a linked emergency situation center.

Description

[Technical Field]

[0001] The present invention relates to a home care system, and more particularly to a video-based remote home care system that monitors in real time the occurrence of various unexpected events, such as emergency situations, based on video of a person being observed captured using an IP-CAM in a remote location. [Background technology]

[0002] Recently, the trend towards nuclear families and an aging society has become more prominent, and as a result, various types of vulnerable social classes that are exposed to dangers even in everyday life are increasing. In particular, the number of elderly people living alone without a caregiver who can help them in an emergency is a trend that is increasing every year due to the aging population.

[0003] When such elderly people living alone suddenly experience a change in their physical condition, they are unable to receive immediate help from family members or others due to restrictions on their movements, even though they need help from others for emergency medical treatment.

[0004] Social and economic measures to address illness, loneliness, poverty, unemployment, and other issues associated with aging are urgent issues facing an aging society, and various monitoring systems are needed to respond to emergencies such as sudden health abnormalities in elderly people living alone who have difficulty receiving help.

[0005] As a system proposed for such elderly people living alone, Patent Publication No. 10-1927373 discloses a monitoring method in which a resident monitoring device attempts to capture the resident's movements using data received from a camera or motion detector, and if no movement is detected within a specified time, the resident monitoring device outputs an alarm signal. Following the alarm signal, the resident monitoring device attempts to capture the resident's movements, and if no movement is detected within the specified time, the resident monitoring device transmits an alarm message to a management server via a broadband communication network, and a microphone connected to the resident monitoring device via a short-range personal communication network senses biometric data through a biometric signal sensor built into the microphone grill.

[0006] According to such prior art documents, it is expected that various conditions of a resident can be remotely monitored, and the resident can be managed based on the monitoring results. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Republic of Korea Patent Registration No. 10-1927373 (Publication Date: December 12, 2018) Summary of the Invention [Problem to be solved by the invention]

[0008] The present invention aims to provide a remote home care service that remotely manages and supervises vulnerable social classes, such as the elderly living alone, and enables immediate action to be taken in the event of an emergency without missing the golden time.

[0009] Another objective of the present invention is to provide a system that acquires different types of images from multiple IP-CAMs installed in the home and one or more smart devices capable of video capture, and standardizes the images of the protected person collected through video correction to derive accurate analysis results. [Means for solving the problem]

[0010] In order to solve the above-mentioned problems, a video-based remote home care system according to a preferred embodiment of the present invention includes one or more IP-CAMs installed at the residence where the protected person resides and capturing the motion of the protected person in real time, a gateway connected to the IP-CAMs via a wired or wireless connection and transmitting the video of the protected person captured by the IP-CAMs to an external information and communication network, and a main server connected to the gateway of each residence via the information and communication network and collecting video of the protected person in real time, determining the current status of the protected person through motion analysis for each protected person, and requesting emergency response from an emergency situation center in the event of an emergency. The main server can analyze the collected video of the protected person using a learning model that has undergone machine learning of a large number of learning videos captured in various environments for more than several months.

[0011] The gateway is further connected to one or more smart terminals located at the residence and having installed thereon an application that interfaces with the main server, and the application can capture images of the protected person in real time from different positions and angles than the IP-CAM through a camera installed in the smart terminal, and transmit the images of the protected person to the main server through the gateway.

[0012] The main server may include a video collection unit that receives one or more videos of protected persons in real time from the gateway of each residence; a member management unit that provides membership registration procedures for protected persons and guardians, stores information on protected persons and guardians who have registered as members in a member database, and identifies protected persons who are the parties to the collected videos; a video classification unit that classifies videos of protected persons according to the identified protected persons and stores them in a video archive; a video analysis unit that extracts protected persons appearing in the classified videos of protected persons and analyzes the motion of the extracted protected persons to determine whether an emergency situation has occurred; and a learning unit that performs machine learning using a large number of learning videos stored in a learning database and provides a learning model learned when the video analysis unit analyzes the motion of protected persons.

[0013] The main server may include a notification generation unit that, when the video analysis unit determines that an emergency situation has occurred for the protected person, identifies the guardian of the protected person by referring to information about the guardian and the protected person, and provides a warning notification of the emergency situation to the guardian terminal.

[0014] The protected person's video is divided into a plurality of videos having different characteristics that were taken at the same time by a plurality of IP-CAMs or smart devices, and the video analysis unit may include an attribute extraction unit that extracts attributes including the scale, resolution, and frame of each video by referring to meta information contained in the plurality of videos, a video processing unit that converts the plurality of videos into video data of a consistent format based on the extracted attributes and tracks the protected person appearing in the video, and a motion determination unit that recognizes the motion of the protected person appearing in the video using a learning model and determines whether the motion corresponds to an emergency situation.

[0015] The image processing unit may include a conversion unit that converts the scale and resolution of each image to correspond to the attributes of a reference image, an interpolation unit that generates and adds interpolated frames for images that are below the reference frame among the multiple images, and a matching unit that defines the area captured by each image based on the shooting direction and angle, and matches the areas of the images if the protected person appears in two or more images at the same time.

[0016] The main server is further connected to one or more guardian terminals owned by the guardian or social worker of the protected person through an information and communication network, and the guardian terminal receives a warning notification from the main server when an emergency situation occurs around the related protected person, and can request an emergency dispatch from the emergency situation center by the guardian's operation. [Effects of the Invention]

[0017] According to an embodiment of the present invention, one or more IP-CAMs or smart terminals are installed in residential areas where vulnerable groups such as elderly people living alone reside, and the current appearance of the protected person is photographed and collected in real time. The collected images are analyzed to determine whether an emergency situation has occurred and report it to the linked emergency situation center, thereby protecting the protected person.

[0018] In addition, according to an embodiment of the present invention, the images collected from multiple IP-CAMs and smart devices installed in the residential area to be managed are standardized according to certain standards, and then analyzed in conjunction with each other, thereby providing more accurate analysis results. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a diagram illustrating an overall structure of a video-based remote home care system according to an embodiment of the present invention. [Figure 2] 2 is a diagram illustrating a configuration of a main server of a video-based remote home care system according to an embodiment of the present invention. [Figure 3] 1 is a diagram illustrating the structure of a video analysis unit installed in a main server according to an embodiment of the present invention. [Figure 4] 1 is a diagram illustrating a method for providing home care to a protected person in a video-based remote home care system according to an embodiment of the present invention. [Figure 5] 10 is a diagram illustrating an example of a screen provided by an application linked to a video-based remote home care system according to an embodiment of the present invention. [Figure 6] 10 is a diagram illustrating an example of a screen provided by an application linked to a video-based remote home care system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0020] The present invention will now be described in detail with reference to the accompanying drawings and examples.

[0021] It should be noted that the technical terms used in the present invention are merely used to describe specific embodiments and are not intended to limit the present invention. Furthermore, unless specifically defined differently in the present invention, the technical terms used in the present invention should be interpreted in a way that is commonly understood by a person of ordinary skill in the art to which the present invention belongs, and should not be interpreted in an overly comprehensive or overly narrow sense. Furthermore, when a technical term used in the present invention is an incorrect technical term that does not accurately express the idea of ​​the present invention, it should be substituted with a technical term that can be correctly understood by a person skilled in the art. Furthermore, general terms used in the present invention should be interpreted in accordance with dictionary definitions or in accordance with the context, and should not be interpreted in an overly narrow sense.

[0022] Furthermore, singular expressions used in the present invention include plural expressions unless the context clearly indicates otherwise. In the present invention, terms such as "comprise" or "include" should not be interpreted as including all of the various components or steps described in the invention, but should be interpreted as meaning that some of the components or steps may not be included, or that additional components or steps may be further included.

[0023] Furthermore, terms including ordinal numbers such as "first," "second," etc., used in the present invention may be used to describe elements, but elements should not be limited by the terms. Terms are used only to distinguish one element from another. For example, a first element may be called a second element, and similarly, a second element may be called a first element, without departing from the scope of the present invention.

[0024] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Regardless of the drawing numbers, identical or similar components will be designated by the same reference numerals, and redundant descriptions thereof will be omitted.

[0025] In addition, if it is determined that the detailed description of the related prior art may obscure the gist of the present invention, the detailed description will be omitted. Also, please note that the attached drawings are only for facilitating understanding of the concept of the present invention, and should not be construed as limiting the concept of the present invention.

[0026] In the following description, the term "video-based remote home care system" of the present invention may be used interchangeably with "system" for the sake of convenience.

[0027] FIG. 1 is a diagram illustrating the overall structure of a video-based remote home care system according to an embodiment of the present invention.

[0028] Referring to Fig. 1, a video-based remote home care system according to an embodiment of the present invention installs one or more IP-CAMs 10 at residences 1 where guardians who require monitoring and supervision reside, records the protected person's appearance 24 hours a day, and provides the protected person's video to a main server 100 at a remote central management center 3 via a gateway 30. The main server 100 is connected to gateways 30 at residences 1 in each region to collect protected person's video in real time, classify the collected video by protected person, and perform motion analysis through a machine learning model to determine whether the protected person is in a normal state or an emergency situation. If it is determined that an emergency situation has occurred, the main server 100 notifies an emergency situation center, such as a connected fire station or police station, so that emergency medical assistance can be provided to the protected person promptly.

[0029] Here, the IP-CAM 10 may be a camera device that performs known IP-based data communication, and can operate 24 hours a day to capture images of the protected person in their residence.

[0030] In addition to the fixed-installed IP-CAM 10, smart devices 20 such as smartphones and tablet PCs equipped with a camera module that allows the protected person to photograph themselves at a position of their choice may also be installed in the residence 1 and wirelessly connected to the gateway 30, and together with the IP-CAM 10, they may photograph the protected person and provide additional images of the protected person.

[0031] In addition, the main server 100 can register guardians 5, such as family members or social workers, related to the protected person, and in the event of an emergency, the relevant guardian can be identified and a warning notification can be provided to the guardian terminal 50 in the form of a push message or the like.

[0032] The information communication network connecting the gateway 30 and the main server 100 may be a wired telephone network such as the Public Switched Telephone Network (PSTN), a closed network such as a Local Area Network (LAN) or a Wide Area Network (WAN), or an open network such as the Internet. The Internet may refer to a global open computer network structure that provides the TCP / IP protocol and various services present at its upper layer, namely, HyperText Transfer Protocol (HTTP), Telnet, File Transfer Protocol (FTP), Domain Name System (DNS), Simple Mail Transfer Protocol (SMTP), Simple Network Management Protocol (SNMP), Network File Service (NFS), and Network Information Service (NIS). In addition, as an example of the wireless communication network, any one of technical standards or communication methods for mobile communication (e.g., GSM (Global System for Mobile communication), LTE (long-term evolution) (registered trademark), LTE-A (LTE Advance), CDMA (code division multiple access), WCDMA (wideband CDMA), UMTS (universal mobile telecommunications system), WiBro (Wireless Broadband), GSM (Global System for Mobile Communications), 5G, etc. may be used, but is not limited thereto.

[0033] In particular, the main server 100 according to an embodiment of the present invention determines whether an emergency situation has occurred by analyzing the motion of the protected person appearing in the collected protected person video, and for this purpose, may be equipped with a machine learning module. A detailed description of such machine learning will be provided later.

[0034] With the above-described structure, the video-based remote home care system according to an embodiment of the present invention uses IP-CAM to capture real-time images of the protected person, who is the monitoring target, for socially vulnerable groups such as elderly people living alone in their local residences, and analyzes the protected person's video in a remote location to determine whether an emergency situation has occurred and take appropriate measures.

[0035] Hereinafter, a main server of a video-based remote home care system according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

[0036] 2 is a diagram showing the structure of a main server of a video-based remote home care system according to an embodiment of the present invention. In the following description, the main server 100 of the present invention and its components can be installed and operated on a known cloud platform. Operating a server based on a cloud platform has the advantage of being convenient for the free expansion of computer resources required for storing and analyzing large amounts of video.

[0037] 2, the main server 100 of the video-based remote home care system according to an embodiment of the present invention may include a video collection unit 110 that receives one or more videos of protected persons in real time from a gateway 30 of each residence; a member management unit 120 that provides a membership registration procedure for protected persons and their guardians, stores information of the protected persons and guardians who have registered as members in a member database 125, and identifies protected persons who are involved in the collected videos; a video classification unit 130 that classifies videos of protected persons according to the identified protected persons and stores them in a video archive 135; a video analysis unit 140 that extracts protected persons appearing in the classified videos of protected persons and analyzes the motion of the extracted protected persons to determine whether an emergency situation has occurred; a learning unit 150 that performs machine learning using a number of learning videos stored in a learning database 155 and provides a learning model learned when the video analysis unit 140 analyzes the motion of the protected person; and a notification generation unit 160 that, when the video analysis unit 140 determines that an emergency situation has occurred for the protected person, identifies the guardian of the corresponding protected person by referring to information about the guardian and the protected person and provides a warning notification of the emergency situation to the guardian terminal 50.

[0038] The video collection unit 110 can receive protected video captured by one or more IP-CAMs installed in residential areas located in each region from the gateway 30. The video collection unit 110 can identify the source of each protected video through the gateway's IP information, and the IP information can be used for subsequent video classification.

[0039] The member management unit 120 can process member registration procedures and login procedures for guardians and guardians who wish to use the remote home care service according to an embodiment of the present invention, and can store information about guardians and guardians in the member database 125.

[0040] In order to use the service of the present invention, membership is required, and the guardian and ward information acquired through the membership registration process can be used to implement an emergency emergency notification function.

[0041] The member database 125 can store information on guardians and wards who have registered as members. This member database 125 may include descriptions of guardians and wards, contact information, addresses, IP information, and the use of IP-CAMs installed at each residence, and this information can be used to classify real-time collected images of wards by ward and, in the event of an emergency, to confirm the residential addresses of wards and the contact information of the guardians for the wards.

[0042] The video classification unit 130 can classify the videos of protected persons collected in real time from the gateways 30 by protected person. Since the main server 100 collects the videos of protected persons in real time from a large number of gateways 30 installed in a large number of residential areas in each region, classification is required for analysis. The video classification unit 130 can identify the protected persons using IP information of the gateways 30 to which each video was transmitted, classify the collected videos, and store them in the video archive 135.

[0043] The video archive 135 can store videos of protected persons classified by protected person in the form of files. The video archive 135 can be allocated storage space for each account, can store videos of protected persons collected in real time by the video collection unit 110, and can automatically delete the videos after a certain period of time, thereby continuously maintaining a certain amount of free space for storing videos of additional protected persons.

[0044] In addition, according to one embodiment of the present invention, the video archive 135 can provide specific stored videos, particularly videos of times when an emergency situation is determined based on motion analysis of the protected person, to the learning database 155, thereby being used to update the learning videos.

[0045] The video analysis unit 140 processes the classified video of the protected person to extract and track the protected person in the video, and analyzes the motion of the extracted protected person to determine whether there is any abnormal motion. The gist of the present invention is to analyze the video collected from IP-CAMs installed in residential areas in each region to determine the current status of the protected person, and in the event of an emergency, to take immediate measures remotely. Therefore, the video analysis unit 140 can analyze the video of the protected person for each protected person using a prepared machine learning model, extract and track objects corresponding to the protected person in the video, and analyze the motion of the protected person to determine whether there is any abnormal motion.

[0046] The learning unit 150 provides the video analysis unit 140 with a machine learning model for extracting protected persons and analyzing motions, and performs machine learning on the learning model through a large number of training videos. Here, the learning model can identify abnormal motions based on the results of learning a large number of training videos containing human motions appearing in various backgrounds for more than six months.

[0047] The learning database 155 can provide a variety of types of learning videos to the learning unit 150. As described above, the learning video is a video for determining whether the motion of a person appearing in the video is an abnormal motion corresponding to an emergency situation, and such learning video can be prepared in advance using big data. In addition, the learning database 155 can add videos of protected persons stored in the video archive 135 corresponding to emergency situations collected by the main server 100 to the learning video and provide the video to the learning model.

[0048] When it is determined that an emergency situation has occurred due to an abnormal motion of a specific protected person as a result of the analysis by the video analysis unit 140, the notification generation unit 160 can identify the guardian associated with the protected person and transmit a warning notification message to the guardian terminal 50. In addition, the notification generation unit 160 can request emergency first aid for the protected person from the center terminal 60 of the associated emergency situation center.

[0049] In addition, the notification generating unit 160 can provide a function that allows the guardian to immediately assess the current situation by transmitting a video of the protected person in an emergency situation to the guardian terminal 50 and the center terminal 60 along with a warning notification message.

[0050] Hereinafter, a video analysis unit installed in a main server for video analysis according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

[0051] FIG. 3 is a diagram showing the structure of a video analysis unit installed in a main server according to an embodiment of the present invention.

[0052] Referring to FIG. 3, the video analysis unit 140 installed in the main server of the video-based remote home care system according to an embodiment of the present invention includes an attribute extraction unit 141 that refers to meta information included in a plurality of videos and extracts attributes including the scale, resolution, and frame of each video, a video processing unit 143 that converts the plurality of videos into video data of a consistent format based on the extracted attributes and tracks the protected person appearing in the video, and a motion determination unit 145 that recognizes the motion of the protected person appearing in the video using a learning model and determines whether the motion corresponds to an emergency situation.

[0053] The attribute extraction unit 141 can extract attributes of the protected person's video in various formats to be transmitted. The system according to an embodiment of the present invention captures videos from multiple different directions and angles in multiple residential locations, and collects and analyzes the protected person's videos from IP-CAMs with different specifications as well as smart devices added by the protected person or the guardian. This can result in differences in the scale and resolution of the collected videos, which can cause a decrease in accuracy when analyzing videos from the same residential location.

[0054] In order to alleviate this problem and improve the accuracy of the analysis results, the video analysis unit 140 according to one embodiment of the present invention can extract meta information corresponding to attributes contained in the video from the first to nth (n is a natural number) videos of different formats collected from one residence.

[0055] The image processor 143 can process the formats of the first to nth images from which the attributes have been extracted as an image having one consistent format by converting, interpolating, and aligning the formats according to a standard.

[0056] In detail, the image processing unit 143 is configured to implement the above-mentioned functions and may include a conversion unit 1431 that converts the scale and resolution of each image to correspond to the attributes of the reference image, an interpolation unit 1433 that generates and adds interpolated frames for images below the reference frame among the multiple images, and an alignment unit 1435 that defines the area captured by each image based on the shooting direction and angle, and aligns the areas of the images if the protected person appears in two or more images at the same time.

[0057] The converter 1431 determines the image scale, resolution, etc. of each image by referring to meta information for the first to nth images, and can convert the remaining images (e.g., the second to nth images) based on one of the reference images (e.g., the first image) or a separate fixed reference (e.g., image scale=2.5 arcsecs / pixel, resolution=FHD (1920×1080), etc.).

[0058] The interpolation unit 1433 can add interpolation frames to the images so that the frames of the first to nth images match. For example, if the frames of the first and third images are each 60 fps and the second image is 30 fps, the interpolation unit 1433 can process the images so that they are close to 60 fps by generating and inserting additional frames having an inter-frame median between each frame of the second image.

[0059] The matching unit 1435 can identify an object that appears in the first to nth images and process it so that its position and motion are shared. Since the first to nth images are images taken by IP-CAMs and smart devices installed in different directions and positions, the protected person may appear in some or all of them. In order to accurately determine whether the object appearing in each image represents the same protected person or another object (e.g., an outside person, furniture, etc.), although the coordinates in which the object appears in the images are different, if it is the same object, it indicates substantially the same area. Therefore, if an object appearing in two or more images is a protected person, it indicates the same area of ​​the same object, so that the areas of the two images can be matched to track the position of the protected person when it appears.

[0060] The motion determination unit 145 can use the prepared learning model to determine whether or not there is any abnormal motion in the motion of the protected person appearing in the video after the conversion, interpolation, and matching processes have been completed, and output the result.

[0061] Due to the above-described structure, the video-based remote home care system according to an embodiment of the present invention can detect abnormal motions of protected persons with high accuracy through video processing processes such as conversion, interpolation, and matching of collected images of protected persons.

[0062] Hereinafter, a home care method for a protected person using a video-based remote home care system according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

[0063] 4 is a diagram illustrating a method for providing home care to a protected person in a video-based remote home care system according to an embodiment of the present invention. In the following description, unless otherwise specified, the subjects of execution of each step are the main server and each component constituting the video-based remote home care system of the present invention.

[0064] Referring to FIG. 4, a method for monitoring protected persons in a video-based remote home care system according to an embodiment of the present invention may include a step (S100) in which an IP-CAM installed at a residence where the protected person resides captures the motion of the protected person in real time; a step (S110) in which a gateway connected to the IP-CAM via a wired or wireless connection transmits the video of the protected person captured by the IP-CAM to a main server through an external information and communication network; a step (S120) in which the main server connected to the gateway of each residence via an information and communication network collects the video of the protected person in real time; a step (S130) in which the main server classifies the collected video of the protected person by protected person; and a step (S140) in which the main server determines the current status of the protected person through motion analysis for each protected person and requests an emergency response from an emergency situation center when an emergency occurs.

[0065] First, in the step (S100) where an IP-CAM installed at the residence where the protected person resides captures the protected person's motion in real time, the guardian and protected person who wish to use the home care service of the present invention complete the membership registration procedure through the main server, and then one or more IP-CAMs and gateways are installed at the residence of the protected person, and the IP-CAMs capture the protected person's motion in real time.

[0066] Next, in step S110, a gateway connected to the IP-CAM via a wired or wireless connection transmits the video of the protected person captured by the IP-CAM to a main server via an external information and communication network. The IP-CAM captures the inside of the residence in real time, and the connected gateway transmits the captured video of the protected person to the main server via an information and communication network. Here, the IP-CAM can be connected to the gateway via a wired LAN, Bluetooth, or WiFi protocol, and can perform data communication with the main server via TCP / IP protocol. Such a gateway can be implemented as a router, broadband router, etc. installed in the residence.

[0067] Next, in the step S120 in which the main server connected to the gateway of each residence through the information and communication network collects the protected person's video in real time, the main server receives and collects the protected person's video in real time from the gateway installed in one or more residences of the protected person who have registered for the service in step S110.

[0068] Next, in the step (S130) where the main server classifies the collected protected person videos by protected person, protected person videos for various protected persons are transmitted in real time from multiple gateways, and the main server classifies the collected protected person videos by protected person and stores them in a database.

[0069] Then, in the step (S140) where the main server determines the current status of each protected person through motion analysis for each protected person and requests emergency dispatch from the emergency situation center when an emergency situation occurs, the main server extracts the protected person from the video classified by protected person through a machine learning model installed in the main server and determines whether the current motion is abnormal motion. If abnormal motion is detected, it is considered that an emergency situation has occurred and a warning notification can be sent to the guardian's terminal. The main server can also request emergency first aid for the protected person from the emergency situation center.

[0070] Here, the machine learning model may apply one or more deep learning techniques and may be used in the analysis process after undergoing a learning process of at least six months through a large number of learning videos.

[0071] Hereinafter, the technical concept of the present invention will be described in detail with reference to an example of a screen provided by an application of a video-based remote home care system according to an embodiment of the present invention.

[0072] 5 and 6 are diagrams illustrating examples of screens provided by an application linked to a video-based remote home care system according to an embodiment of the present invention.

[0073] Referring to FIG. 5, an application of the video-based remote home care system according to an embodiment of the present invention can be installed and executed on a parental terminal 50, such as a parent's smartphone, associated with the protected person, and can provide functions related to parental monitoring and receive warning notification messages when an emergency situation occurs.

[0074] In detail, when the parental terminal 50 executes the application, it connects to the main server through the information and communication network and can provide monitoring-related functions such as viewing real-time video 51, connecting to the protected person 52, and connecting to the emergency situation center 53. These applications usually run in the background of the parental terminal 50 and can receive warning notification messages from the main server in the form of push messages.

[0075] First, the function to view real-time video 51 is a function that receives and displays the video of the protected person so that the guardian can check the current status of the protected person. When the guardian selects this function, the application will request the current video of the protected person from the main server. In response, the main server will check the identification means of the guardian terminal 50, such as MAC or phone number, to identify the person in question, check the protected person linked to the identified guardian, and transmit the currently received video of the protected person to the guardian terminal 50, thereby allowing the protected person's video to be viewed in real time on the screen of the guardian terminal 50.

[0076] The Protected Person Connection 52 function activates the normal telephone call or video call function. When this function is selected, the application attempts to connect to the protected person's smart device by telephone using the registered phone number, or requests the main server to make a video call with the protected person's smart device, and can perform a voice or video call with the protected person through the smart device.

[0077] The emergency situation center connection 53 function is a function that requests a connection to an emergency situation center located in an area adjacent to the protected person, such as a fire station or police station, when the protected person becomes aware of an emergency situation with the guardian through a video check of the protected person or a phone call. If the guardian who has confirmed an emergency situation regarding the protected person selects this function, the function will be connected to a terminal operated by the emergency situation center and a request will be made to dispatch the guardian.

[0078] Referring to Figure 6, the video-based remote home care system according to an embodiment of the present invention illustrates a function of providing the current situation to the guardian terminal 50 when an abnormal motion is detected during video analysis of the protected person using a machine learning model while monitoring the protected person.

[0079] When it is determined that an abnormal motion has occurred in the protected person, the main server can transmit a total of 20 seconds of video of the protected person, approximately 10 seconds before and after the point in time when the abnormal motion was determined, along with a warning notification to the guardian terminal 50 of the guardian.In response, the application can display the aforementioned amount of video 55 of the protected person where the abnormal motion was determined, along with a warning notification message, on the guardian terminal 50.

[0080] This allows the guardian to directly check the current status of the protected person and whether or not an emergency situation has occurred, and if it is determined that an emergency situation has occurred, the guardian can select the emergency situation center connection function 53 to attempt to connect to the emergency situation center and request a dispatch for the protected person.

[0081] Although many specific details are set forth in the above description, these should not be construed as limiting the scope of the invention but as exemplification of preferred embodiments. Therefore, the invention should not be defined by the described embodiments but by the claims and their equivalents. [Explanation of symbols]

[0082] 1: Residence 3: Central Management Center 5: Parents 7: Emergency Situation Center 10: IP-CAM 20: Smart terminal 30: Gateway 50: Parent terminal 100: Main server 110: Video collection unit 120: Membership Management Department 125: Membership Database 130: Video Classification Department 135: Video Archive 140: Video analysis unit 141: Attribute extraction unit 143: Video processing unit 1431: Conversion unit 1433: Interpolation section 1435: Matching section 145: Motion discrimination unit 150: Learning unit 155: Learning database 160: Notification generation unit

Claims

1. one or more IP-CAMs installed in the residence where the protected person resides to capture the protected person's movements in real time; a gateway connected to the IP-CAM via wired or wireless means and transmitting the protected person's video captured by the IP-CAM to an external information and communication network; and It includes a main server that is connected to the gateway of each residence through the information and communication network to collect the protected person's video in real time, determines the current status of each protected person through motion analysis for each protected person, and requests emergency dispatch from the emergency situation center in the event of an emergency. The main server The protected person's video is analyzed using a learning model that has undergone machine learning for more than six months on multiple learning videos taken in multiple environments. the gateway is further connected to one or more smart terminals located at the residence and having installed thereon an application that operates in conjunction with the main server; The application A video-based remote home care system that takes real-time images of the protected person from different positions and angles from the IP-CAM using a camera installed in the smart terminal and transmits the images of the protected person to the main server through the gateway.

2. one or more IP-CAMs installed in the residence where the protected person resides to capture the protected person's movements in real time; A gateway connected to the IP-CAM via wired or wireless connection and transmitting the images of the protected person captured by the IP-CAM to an external information and communication network; and It includes a main server that is connected to the gateway of each residence through the information and communication network to collect the protected person's video in real time, determines the current status of each protected person through motion analysis for each protected person, and requests emergency dispatch from the emergency situation center in the event of an emergency. The main server The protected person's video is analyzed using a learning model that has undergone machine learning for more than six months on multiple learning videos taken in multiple environments. The main server a video collection unit that receives one or more protected person videos in real time from each residential gateway; A member management department that provides membership registration procedures for wards and guardians, stores information on wards and guardians who have registered as members in a member database, and identifies wards who are involved in collected videos; a video classification unit that classifies the videos of protected persons according to the identified protected persons and stores them in a video archive; A video analysis unit that extracts the protected person appearing in the classified protected person video and analyzes the motion of the extracted protected person to determine whether an emergency situation has occurred; and A video-based remote home care system including a learning unit that performs machine learning using a large number of learning videos stored in a learning database and provides a learned learning model when the video analysis unit analyzes the protected person's motion.

3. The main server 3. The video-based remote home care system of claim 2, further comprising a notification generation unit that, when the video analysis unit determines that an emergency situation has occurred for the protected person, identifies the guardian of the protected person by referring to information about the guardian and the protected person, and provides a warning notification about the emergency situation to the guardian terminal.

4. The protected images are divided into a plurality of images having different attributes, The video analysis unit an attribute extraction unit that refers to meta information included in the plurality of videos and extracts attributes including a scale, a resolution, and a frame of each video; a video processing unit that converts the plurality of videos into video data of a consistent format according to the extracted attributes and tracks the protected person appearing in the videos; and The video-based remote home care system of claim 2, further comprising a motion determination unit that uses a learning model to recognize the motion of the protected person appearing in the video and determines whether the motion corresponds to an emergency situation.

5. The video processing unit includes: a conversion unit that converts the scale and resolution of each image to correspond to the attributes of the reference image; an interpolation unit that generates and adds interpolated frames to images that are equal to or smaller than the reference frame among the plurality of images; and 5. The video-based remote home care system of claim 4, further comprising a matching unit that defines the area captured by each video according to the direction and angle of the video, and matches the areas of the corresponding video if the protected person appears in two or more videos at the same time.

6. The main server is further connected to one or more guardian terminals owned by the guardian or social worker of the protected person through an information and communication network, The parent terminal includes:

2. The video-based remote home care system of claim 1, wherein when an emergency situation occurs around a related protected person, a warning notification is received from the main server and an emergency situation center is requested to be dispatched by the guardian.

Images