System

Abstract

A system is provided.SOLUTION: What is claimed is: A system comprising: user interface means for a user to enter data and review an elderly care plan or medical record; communication means for transmitting the data to a server; database means for the server to store and manage the data; and communication means for the server to transmit the data to the user interface means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The "Problem that the invention aims to solve" and "Means for solving the problem" of the document are shown below.

[0005] In elderly care, traditional management methods based on paper or electronic medical records can lead to a decline in the quality of care due to the inability to share information promptly. Furthermore, staff training and new staff support are not carried out effectively, resulting in inconsistent quality of care. Furthermore, it is difficult to respond quickly in emergencies, making it difficult to ensure the safety of elderly people. Because traditional systems make it difficult to share information in real time and provide effective care support, a new system is needed to improve the quality of care and staff efficiency. [Means for solving the problem]

[0006] The present invention provides a system that includes a user interface means for allowing a user to input data and check an elderly person's care plan and medical records in real time, a database means for transmitting the input data to a server, and a communication means for transmitting the data stored by the server to the user interface means.

[0007] This system not only allows users to provide high-quality care in real time, but also effectively trains new staff and provides remote support. Furthermore, it enables rapid notification and response in the event of an emergency, ensuring the safety of the elderly. Through these measures, the aim is to solve existing problems and improve the quality of life for the elderly and the work efficiency of staff.

[0008] "User interface means" means including input and display devices for a user to input data into or obtain data from a system.

[0009] "Communication means" refers to means including networks and their protocols for transmitting and receiving data.

[0010] The term "server means" refers to means including a computer and its software for storing and managing data and providing data in response to requests from other devices.

[0011] A "database means" is a means including a data structure for storing and managing data in an organized manner and software for manipulating the data.

[0012] "Video stream" means the continuous transmission and reception of video and audio data in real time.

[0013] A "reminder notification" is a method of displaying an alert message at a specified time to prompt a specific action.

[0014] "Emergency Response Protocol" means a set of procedures to be executed immediately in the event of an emergency.

[0015] A "Training Module" is a means including educational content and its implementation environment for new staff to acquire the necessary knowledge and skills. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention relates to a smart glasses-based care assistant system for improving elderly care, providing a system for inputting user data, managing elderly care plans and medical records, real-time video calling, reminder notifications, training new staff, and responding to emergency care. Specific embodiments for implementing the present invention are described in detail below.

[0038] User Data Input

[0039] Users input the elderly person's care plan and medical records through the smart glasses interface. The device then sends the input data to a server, which analyzes the received data and stores and manages it in a database.

[0040] For example, a user inputs the care plan for elderly person Tanaka, such as "daily exercise" and "medicine at 8 am," and also inputs medical records such as "high blood pressure" and "arthritis." The device sends the input data to the server, which then stores the data in a database.

[0041] Real-time video calling

[0042] The user uses the smart glasses interface to make real-time video calls with remote family members or medical staff. The device activates the camera and microphone and initiates the video call session. The server processes the video call data stream and manages real-time communication.

[0043] For example, if a user wants to start a video call with Tanaka's daughter, the smart glasses will turn on the camera and microphone and transmit the video stream. The server processes this data and starts the call.

[0044] Reminders

[0045] Users can set reminders to prompt the elderly to perform an action at a specific time. The device sends the reminder information to the server, which stores and manages the reminder information in a database. When the reminder time arrives, the server sends a reminder notification to the device, which then displays the notification on the user interface.

[0046] For example, if Tanaka sets a reminder to take her morning medicine, the server stores that information in a database and sends a notification to the device when the specified time arrives. The device then displays a notification saying, "It's time to take your morning medicine."

[0047] Medical record review

[0048] The user can check the elderly person's medical records using the interface of the smart glasses. The device sends the user's request to the server, which retrieves the medical records from the database and sends them to the device. The device then displays the retrieved medical records on the user interface.

[0049] For example, if a user checks Mr. Tanaka's medical records, the server retrieves information about "high blood pressure" and "arthritis" from the database and sends it to the terminal, which then displays that information.

[0050] Training new staff

[0051] New staff can receive training using smart glasses. The terminal sends a request for a training module to the server, and the server sends the training content to the terminal, which then plays it and provides it to the user.

[0052] For example, when a new staff member, Mr. Yamada, starts training, the terminal requests a training module from the server, and the server sends the training video to the terminal, which then plays the video.

[0053] Rapid response to emergency care

[0054] In the event of an emergency, the user can press the emergency button on the smart glasses to send a notification. The device will then send the emergency notification to the server, which will trigger the emergency response protocol and send an alert to the relevant care staff. The device will receive the alert and display it immediately.

[0055] For example, if Tanaka presses the emergency button, the device sends an emergency notification to the server, which then alerts all staff. The device displays, "Emergency! Please respond immediately."

[0056] As described above, the present invention improves the quality of care for the elderly, while enabling efficient support and emergency response by staff, thereby improving the quality of life for the elderly.

[0057] The processing flow will be explained below.

[0058] OK, so let's go into more detail and explain what the program does, step by step.

[0059] User Data Input

[0060] Step 1:

[0061] Users enter the elderly person's care plan and medical records through the smart glasses interface.

[0062] Specifically, data is input using a keyboard, voice input, a touch panel, etc.

[0063] Step 2:

[0064] The terminal transmits the input data to the server.

[0065] Specifically, the input data is converted into packets and sent to the server using a communication protocol.

[0066] Step 3:

[0067] The server analyzes the received data and stores it in a database.

[0068] Specifically, the received packet is analyzed and the data is stored in the appropriate database field.

[0069] Real-time video calling

[0070] Step 1:

[0071] The user selects the person with whom they want to start a video call using the interface of the smart glasses.

[0072] Specifically, you select the ID of the person you want to call using the touch panel or voice commands.

[0073] Step 2:

[0074] The terminal sends a request for a call session to the server.

[0075] Specifically, the server sends a request packet containing the ID of the selected call partner to the server.

[0076] Step 3:

[0077] The server processes the call request and establishes the call session.

[0078] Specifically, a new session ID is generated and notified to both terminals.

[0079] Step 4:

[0080] The device will enable the camera and microphone and start the video stream.

[0081] Specifically, the hardware camera and microphone are turned on, and video and audio data are sent and received as streams.

[0082] Reminders

[0083] Step 1:

[0084] The user inputs the reminder content and time through the smart glasses interface.

[0085] Specifically, the reminder content and time can be set using the touch panel or voice input.

[0086] Step 2:

[0087] The terminal transmits the reminder information to the server.

[0088] Specifically, the set reminder information is packetized and sent to the server.

[0089] Step 3:

[0090] The server stores the reminder information and monitors the reminder time.

[0091] Specifically, the system stores the reminder information in a database and waits for the specified time.

[0092] Step 4:

[0093] The server sends a notification to the terminal when the reminder time arrives.

[0094] Specifically, the system refers to the saved reminder information and sends a notification packet to the terminal when the reminder time is reached.

[0095] Step 5:

[0096] The terminal receives the reminder notification and displays it to the user.

[0097] Specifically, the received notification packet is analyzed and the reminder content is displayed on the user interface.

[0098] Medical record review

[0099] Step 1:

[0100] The user submits a request to review medical records through the smart glasses interface.

[0101] Specifically, the user selects the medical record item they wish to check using the touch panel or voice.

[0102] Step 2:

[0103] The terminal sends a confirmation request to the server.

[0104] Specifically, the operation involves sending a request packet containing the selected medical record items to the server.

[0105] Step 3:

[0106] The server retrieves the relevant medical records from the database.

[0107] Specifically, the system analyzes the received request and extracts the relevant medical record data from the database.

[0108] Step 4:

[0109] The server transmits the acquired medical record data to the terminal.

[0110] Specifically, the extracted medical record data is converted into packets and sent to the terminal.

[0111] Step 5:

[0112] The terminal displays the received medical record data on a user interface.

[0113] Specifically, the received data packets are analyzed and the medical records are displayed on the user interface.

[0114] Training new staff

[0115] Step 1:

[0116] The user selects a training module from the smart glasses interface.

[0117] Specifically, the training menu is selected using the touch panel or voice.

[0118] Step 2:

[0119] The terminal sends a request for a training module to the server.

[0120] Specifically, the ID of the selected training module is sent to the server.

[0121] Step 3:

[0122] The server obtains the training content and transmits it to the terminal.

[0123] Specifically, the relevant training content is retrieved from the database and sent to the terminal.

[0124] Step 4:

[0125] The terminal plays back the received training content.

[0126] Specifically, the received training video or manual is played on the user interface.

[0127] Rapid response to emergency care

[0128] Step 1:

[0129] The user presses the emergency button on the smart glasses to notify them of an emergency.

[0130] Specific actions include physically pressing an emergency button or issuing a specific voice command.

[0131] Step 2:

[0132] The terminal sends an emergency notification to the server.

[0133] Specifically, it detects that the emergency button has been pressed and sends an emergency notification packet to the server.

[0134] Step 3:

[0135] The server receives the emergency notification and executes the emergency response protocol.

[0136] Specifically, it analyzes the received emergency notification and initiates a response according to the protocol.

[0137] Step 4:

[0138] The server sends an alert notification to the relevant care staff.

[0139] A specific operation is to send an alert notification packet to the device of the necessary care staff.

[0140] Step 5:

[0141] The terminal receives the warning notification and displays it to the user.

[0142] Specifically, the device analyzes the warning notification received and displays "Emergency situation! Please respond immediately" on the user interface.

[0143] The above is the specific processing flow of each function detailed for each process.

[0144] Example 1

[0145] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0146] To improve elderly care, a system for efficient data management and emergency response is needed. However, the current system does not centrally manage care plans, medical record entry and review, real-time video calls, reminder notifications, new staff training, and emergency care responses, making it difficult to provide comprehensive support.

[0147] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0148] In this invention, the server includes a user interface means for a user to input data and check care plans and medical records, a communication means for transmitting the data to the server, a database means for the server to store and manage the data, a communication means for the server to transmit the data to the user interface means, a sensor means for transmitting an emergency notification to the server when a user presses an emergency button, and a server means for the server to execute emergency response protocols and send alerts to relevant staff. This allows management of elderly care plans and medical records, real-time video calls, reminder notifications, training of new staff, and emergency care responses to be performed in a unified manner, thereby improving the quality and efficiency of care.

[0149] "User interface means" refers to hardware and software for displaying data entered by the user and notifications from the system.

[0150] "Communication means" refers to the network interface and communication protocol for sending and receiving data.

[0151] "Database means" refers to a storage system or database management system for storing and managing data.

[0152] "Sensor means" refers to a device or module that detects user input or changes in the environment and acquires data.

[0153] "Server means" refers to a server system for processing, storing, managing, and communicating data.

[0154] "Emergency response protocols" are response procedures and steps that the system automatically executes in the event of an emergency.

[0155] A "video stream" is a flow of video data that is sent and received in real time.

[0156] A "reminder" is an alert or notification that notifies a user of a specific action or time.

[0157] These definitions provide a clearer understanding of the role and function of each element included in the claims.

[0158] The present invention relates to a smart glasses-based care assistant system for improving elderly care, providing a system for user data entry, managing elderly care plans and medical records, real-time video calling, reminder notifications, training new staff, and emergency care response. Specific embodiments for implementing the present invention are described in detail below.

[0159] User Data Input

[0160] Users can input care plans and medical records through the smart glasses interface. The data is then sent to a server via the device. The server analyzes the received data and stores and manages it in a database. This allows for centralized management of elderly care plans and medical records.

[0161] Specifically, users can use the smart glasses' voice recognition system or touchpad to input instructions such as "Take today's medicine at 8 o'clock." The device converts this voice data into text and sends it to the server. The server then converts this data into JSON format and stores it in a database such as MySQL.

[0162] Real-time video calling

[0163] Users can use the smart glasses interface to make real-time video calls with remote family members or medical staff. The device activates the camera and microphone and initiates the video call session. The server processes the video data stream and manages the communication in real time.

[0164] For example, if you say "start a video call" to your smart glasses, the device will turn on the camera and microphone and start a call session using WebRTC. The server manages this and ensures the call proceeds smoothly.

[0165] Reminders

[0166] Users can set reminders to prompt the elderly to perform specific actions at specific times. The device sends the reminder information to the server, which stores and manages the reminder information in a database. At the set time, the server sends a reminder notification to the device, which then displays the notification on the user interface.

[0167] Specifically, when someone types "Set a reminder to take my medicine at 8 a.m.", the device converts this into text and sends it to the server. The server stores the reminder information in a database and sends a notification to the device at the specified time. The device then displays "It's time to take your morning medicine."

[0168] Medical record review

[0169] The user can check their medical records through the interface of the smart glasses. A request is sent from the device to the server, which retrieves the medical records from the database and sends them to the device. The device then displays the medical records on the user interface.

[0170] For example, if you say "Check Tanaka-san's medical records," the server will retrieve information on "high blood pressure" and "arthritis" from the database and send it to the device, which will then display it on its screen.

[0171] Training new staff

[0172] New staff can receive training using smart glasses. The device sends a request for a training module to the server, and the server sends the training content to the device. The device can then play the provided content and display it to the user.

[0173] Specifically, when a new staff member says "Start training," the terminal sends this request to the server, which then sends the training video to the terminal, which then plays the video.

[0174] Rapid response to emergency care

[0175] In the event of an emergency, users can press the emergency button on the smart glasses to send an emergency notification. The device will then send the emergency notification to the server, which will trigger the emergency response protocol to alert relevant staff. The device will then display the alert.

[0176] For example, if Tanaka presses the emergency button, the device sends an emergency notification to the server, which then executes emergency response protocols and alerts all staff. The device displays, "Emergency! Please respond immediately."

[0177] As described above, the present invention improves the quality of care for the elderly, while enabling efficient support and emergency response by staff, thereby improving the quality of life for the elderly.

[0178] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0179] Step 1:

[0180] The user inputs data through the smart glasses interface. For example, they can enter instructions such as "Take today's medicine at 8 o'clock" using a voice recognition system or touchpad. The input in this step is voice or text data, and the device receives this input.

[0181] Step 2:

[0182] The device converts the received voice data into text data. It uses voice recognition technology to analyze the user's voice input and outputs it as text. The converted text data is obtained. This specific operation is performed using the voice recognition library installed on the device.

[0183] Step 3:

[0184] The terminal transmits text data to the server. The data is transmitted to the server via the internet via the communication means. The input of this step is text data, and the output is data transmission to the server.

[0185] Step 4:

[0186] The server parses the received text data, converts it into the required data format (for example, JSON format), and saves the converted data in a database. The input is text data, and the output is saving the converted data in a database.

[0187] Step 5:

[0188] The server sets reminder information and stores it until the specified time. When the specified time arrives, it sends a reminder notification to the device. The input is the saved reminder information, and the output is to send a reminder notification to the device when the time arrives.

[0189] Step 6:

[0190] The device displays the reminder notification received from the server on the user interface and informs the user of the reminder content. Specifically, the device displays "It's time to take your morning medicine" on the display and also notifies the user by voice. The input is the reminder notification from the server, and the output is the display on the user interface and the voice notification.

[0191] Step 7:

[0192] When a user presses the emergency button, the terminal is instructed to notify the user of an emergency. The input is the user's operation of the emergency button, and the terminal captures this operation.

[0193] Step 8:

[0194] The terminal sends an emergency notification to the server. Emergency status data is sent to the server using a communication method. The input is the emergency button operation event, and the output is the transmission of emergency notification data to the server.

[0195] Step 9:

[0196] When the server receives an emergency notification, it activates the emergency response protocol and sends an alert to the relevant staff. Specifically, the server sends an emergency email or SMS to all registered staff. The input is the emergency notification data, and the output is sending an alert notification to the staff.

[0197] Step 10:

[0198] The relevant staff receives the alert and initiates a response based on the information. The input is an emergency alert notification from the server, and the output is the execution of emergency response actions.

[0199] Through these steps, the system can efficiently support the various needs of elderly care.

[0200] (Application example 1)

[0201] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0202] Conventional elderly care systems have had problems such as the time-consuming task of managing elderly care plans and medical records, and the difficulty of sharing care information in real time. Furthermore, while rapid response is required in the event of an emergency, conventional systems do not adequately address this need. Furthermore, security staff lacked the efficiency and immediacy required for on-site monitoring and emergency response, making it difficult to ensure smooth work progress through reminder notifications.

[0203] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0204] In this invention, the server includes a user interface means for a user to input data and check the elderly person's care plan and medical records, a communication means for transmitting the data to the server, a database means for storing and managing the data, a communication means for transmitting the data to the user interface means, a communication means for transmitting and receiving video streams, a communication means for issuing emergency notifications, and a display means for displaying reminder notifications, thereby enabling more efficient information management in elderly care, real-time information sharing, faster emergency response, and improved work efficiency of security staff.

[0205] "User interface means" refers to an interface through which a user inputs data and confirms and manipulates information.

[0206] "Communication means" refers to the technical means used to send and receive data and information between servers.

[0207] The "database means" refers to a data storage system for saving and managing input data and its management function.

[0208] The "server means" is a server system for processing and managing data.

[0209] A "video stream" is a continuous flow of video data that is sent and received in real time.

[0210] "Emergency notifications" are messages or alerts that are sent to quickly warn relevant parties when an emergency occurs.

[0211] The "display means" refers to a display or monitor for visually presenting information to the user.

[0212] A "reminder notification" is a notification that prompts the user to take action at a specific time or under specific conditions.

[0213] "Real-time video calling" is a call format in which both parties communicate with each other in real time.

[0214] "Communication means for issuing emergency notifications" means the technical means for sending immediate warning messages in the event of an emergency.

[0215] The present invention relates to a smart glasses-based assistant system for security and elderly care, which includes a user interface means, a communication means, a database means, a server means, and a display means.

[0216] System Overview

[0217] The system consists of the following main components:

[0218] 1. User interface means: The smart glasses' display and microphone allow users to input data, view information, and manipulate it.

[0219] 2. Communication Method: The technical means for transmitting data and video streams between the smart glasses and the server, including Wi-Fi and cellular communication.

[0220] 3. Database Means: A cloud storage system for storing and managing user-entered data. Specifically, this includes Google Cloud Firestore.

[0221] 4. Server infrastructure: Server infrastructure that processes and manages data. For example, Google Cloud or AWS is used.

[0222] 5. Display means: A means for visually presenting information to the user using the smart glasses display.

[0223] Program Overview

[0224] The process of a user wearing smart glasses and entering data, making video calls, setting reminders, and sending emergency notifications through the user interface is described.

[0225] Entering and verifying data

[0226] The user inputs data using the smart glasses' display and voice input. For example, in elderly care, the user can input care plans and medical records. The input data is sent to a server via a communication means and stored in a database means. The stored data can be viewed at any time via a user interface means.

[0227] Real-time video calling

[0228] A user can use the interface of the smart glasses to make a real-time video call with another user in a remote location (e.g., a security center or a family member). The device captures the video stream using a camera and microphone and transmits it to a server through a communication means. The server means processes the video stream in real time and manages the call function.

[0229] Reminders

[0230] A user can set a reminder for a task to be performed at a specific time. The reminder information is sent to the server via the communication means and stored in the database means. When the specified time arrives, the server means sends a notification to the terminal, and the reminder is displayed to the user via the display means.

[0231] Sending emergency notifications

[0232] When a user faces an emergency, they can press the emergency button on the smart glasses to issue an emergency notification. The emergency notification is sent to the server via the communication means, and the server means activates an emergency response protocol and sends the emergency notification to all related terminals. A prompt response is then made via the display means.

[0233] Specific examples

[0234] When a security staff member discovers a suspicious individual while on patrol, an emergency notification is sent from the smart glasses. The server immediately sends an alert to multiple staff members, enabling a swift response on-site.

[0235] At elderly care facilities, care staff input care plans into smart glasses, and the data is stored on a server, allowing family members to remotely check the progress of care in real time.

[0236] Prompt Sentence Examples

[0237] "Please outline an application that uses smart glasses to enable security personnel to initiate video calls, send emergency notifications, and display reminder notifications in real time. Please explain the specific features."

[0238] Overall, the system will streamline care and security management and improve real-time response capabilities.

[0239] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0240] Step 1:

[0241] The user wears the smart glasses and inputs data through the user interface, such as entering care plans and medical records. The input data is sent to a server via a communication means. Input is performed using the smart glasses' display and voice input function, and the data is sent to the server as output.

[0242] Step 2:

[0243] The server stores and manages the received data using database means. When the input data reaches the server, it is stored in the database. The server uses Google Cloud Firestore to store the data. After the data is stored, the database is updated.

[0244] Step 3:

[0245] The user checks the entered care plan and medical records through the interface of the smart glasses. A request is sent to access the data stored on the server. The server receives this request, retrieves the relevant data from the database, and sends it back to the smart glasses via a communication method. As a result, the user can view the data on the display.

[0246] Step 4:

[0247] Users use the smart glasses interface to make real-time video calls with other users in remote locations. The smart glasses' camera and microphone capture video and audio and send them to a server via a communication method. The server uses Twilio's API to manage the video call session and send the video stream to the other user's device. This enables two-way communication of video and audio in real time.

[0248] Step 5:

[0249] The user sets a reminder for a specific time, for example, inputting a reminder for rounds or taking medicine. The reminder information is sent to the server via the interface of the smart glasses using a communication means. The server receives this information, stores it in a database, and then triggers a reminder notification based on the specified time. The notification is sent back to the smart glasses via the communication means and displayed to the user via the display means. As a result, the user receives the notification at the specified time.

[0250] Step 6:

[0251] When a user faces an emergency, they press the emergency button on the smart glasses to send an emergency notification. Input is made by pressing the button, and the notification is sent to the server using a communication method. The server receives the emergency notification, activates a response protocol, and sends the emergency notification to all related devices. This allows a notification of an "emergency" to be displayed on multiple devices, enabling a quick response.

[0252] This allows the system to enable users to effectively manage data, communicate and respond quickly to emergencies.

[0253] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0254] The present invention relates to a smart glasses-based care assistant system for improving elderly care, which provides a system for inputting user data, managing elderly care plans and medical records, real-time video calling, reminder notifications, training new staff, responding to emergency care, and emotion recognition using an emotion engine. The following describes in detail the embodiments of the present invention.

[0255] User Data Input

[0256] Users input the elderly person's care plan and medical records through the smart glasses interface. The device then sends the input data to a server, which analyzes the data and stores it in a database for management.

[0257] For example, a user inputs an elderly person's care plan, "Daily Exercises" and "Medicine at 8 AM," and inputs medical records "High Blood Pressure" and "Arthritis." The terminal sends the input data to the server, which stores the data in a database.

[0258] Real-time video calling

[0259] The user uses the smart glasses interface to make real-time video calls with remote family members or medical staff. The device activates the camera and microphone and initiates the video call session. The server processes the video call data stream and manages real-time communication.

[0260] For example, if a user wants to start a video call with the elderly person's daughter, the smart glasses will turn on the camera and microphone and transmit the video stream, and the server will process this data and start the call.

[0261] Reminders

[0262] Users can set reminders to prompt the elderly to take action at specific times. The device sends the reminder information to the server, which stores the information in a database and sends a notification to the device when the reminder time arrives. The device then displays the notification on the user interface.

[0263] For example, when a user sets a reminder for an elderly person to take their morning medication, the server stores that information in a database and sends a notification to the device at the specified time, which displays a message saying, "It's time to take your morning medication."

[0264] Medical record review

[0265] The user can check the elderly person's medical records using the smart glasses interface. The device sends the user's request to the server, which retrieves the medical records from the database and sends them to the device. The device then displays the retrieved medical records on the user interface.

[0266] For example, if a user checks the medical records of an elderly person, the server retrieves information about "high blood pressure" and "arthritis" from the database and sends it to the terminal, which then displays the information.

[0267] Training new staff

[0268] New staff can receive training using smart glasses. The device sends a request for a training module to the server, and the server sends the training content to the device, which then plays it and provides it to the user.

[0269] For example, when a new staff member starts training, the terminal requests a training module from the server, and the server sends a training video to the terminal, which then plays the video.

[0270] Rapid response to emergency care

[0271] In the event of an emergency, the user can press the emergency button on the smart glasses to send a notification. The device will then send the emergency notification to the server, which will trigger the emergency response protocol and send an alert notification to the relevant care staff. The device will then receive the alert and display it to the user.

[0272] For example, if an elderly person presses the emergency button, the device will send an emergency notification to the server, which will then alert all staff. The device will display "Emergency! Please respond immediately."

[0273] Emotion recognition by emotion engine

[0274] The emotion engine means recognizes the user's emotions and transmits them to the server, which stores and analyzes the emotion data, and provides feedback to the user based on the emotion data.

[0275] For example, if the emotion engine detects that an elderly person is in a stressful state, the device will send this emotion data to the server, which will analyze the data and provide feedback to the user such as "listen to some relaxing music."

[0276] Care plan coordination

[0277] Based on the emotion data recognized by the emotion engine, the server adjusts the care plan, which is then notified to the user and implemented.

[0278] For example, if the emotion engine analyzes the emotional data of an elderly person and determines that they are under high stress, the server will adjust the care plan to a "relaxation exercise" and notify the device to carry it out.

[0279] Applying emotion recognition to video calls

[0280] The emotion engine means recognizes the user's emotions during a real-time video call and notifies the video call partner of the emotion data. Based on the emotion data, a warning or recommended action can be displayed to the video call partner.

[0281] For example, if the emotion engine recognizes that an elderly person is becoming increasingly stressed during a video call, the caller will see a message saying, "You are feeling stressed. Please change the topic to help you relax."

[0282] As described above, the present invention improves the quality of care for the elderly while providing efficient support and emergency response from staff, and by combining it with an emotion engine, more detailed care and support becomes possible.

[0283] The processing flow will be explained below.

[0284] User Data Input

[0285] Step 1:

[0286] Users enter the elderly person's care plan and medical records through the smart glasses interface.

[0287] Example: A user enters an elderly person's care plan, "Daily Exercises" and "8 AM Medication," and medical records, including "High Blood Pressure" and "Arthritis."

[0288] Step 2:

[0289] The terminal converts the input data into packets and transmits them to the server.

[0290] Step 3:

[0291] The server analyzes the received data and stores it in a database.

[0292] A database stores care plans and medical records.

[0293] Real-time video calling

[0294] Step 1:

[0295] The user uses the interface of the smart glasses to select who to start a video call with.

[0296] Example: A user selects the ID of an elderly person's daughter.

[0297] Step 2:

[0298] The terminal sends a call request packet to the server.

[0299] The request containing the submitted ID arrives at the server.

[0300] Step 3:

[0301] The server processes the call request and generates a new session ID.

[0302] The generated session ID is notified to both terminals.

[0303] Step 4:

[0304] The device will enable the camera and microphone and start the video stream.

[0305] A video stream is sent from the elderly person's smart glasses and displayed on his daughter's device.

[0306] Reminders

[0307] Step 1:

[0308] The user inputs the reminder content and time through the smart glasses interface.

[0309] Example: A user sets a reminder for an elderly person to take their morning medication at 08:00.

[0310] Step 2:

[0311] The terminal converts the reminder information into packets and transmits them to the server.

[0312] Step 3:

[0313] The server stores the received reminder information in a database and monitors the specified time.

[0314] Step 4:

[0315] When the reminder time arrives, the server sends a notification packet to the terminal.

[0316] Step 5:

[0317] The terminal analyzes the received notification and displays it on the user interface.

[0318] Example: The device displays "Time to take your morning medicine."

[0319] Medical record review

[0320] Step 1:

[0321] The user enters a request to review medical records through the smart glasses interface.

[0322] Example: A user sends a request to review an elderly person's medical records.

[0323] Step 2:

[0324] The terminal sends a request packet to the server.

[0325] Step 3:

[0326] The server retrieves the relevant medical records from the database.

[0327] The server extracts information on "high blood pressure" and "arthritis" from the database.

[0328] Step 4:

[0329] The server converts the acquired medical record data into packets and transmits them to the terminal.

[0330] Step 5:

[0331] The terminal analyzes the received data and displays it on a user interface.

[0332] Example: Medical records for "high blood pressure" and "arthritis" are displayed.

[0333] Training new staff

[0334] Step 1:

[0335] The user selects a training module from the smart glasses interface.

[0336] Example: New staff members select a training menu.

[0337] Step 2:

[0338] The terminal sends a request packet for the training module to the server.

[0339] Step 3:

[0340] The server retrieves training content from the database, converts it into packets, and transmits them to the terminal.

[0341] Step 4:

[0342] The terminal plays back the received training content.

[0343] Example: A device plays a training video that new staff members watch.

[0344] Rapid response to emergency care

[0345] Step 1:

[0346] The user presses the emergency button on the smart glasses to send an emergency notification.

[0347] Example: An elderly person presses the emergency button.

[0348] Step 2:

[0349] The terminal transmits an emergency notification packet to the server.

[0350] Step 3:

[0351] The server executes emergency response protocols and sends emergency notifications to all staff.

[0352] Step 4:

[0353] The terminal analyzes the received emergency notification and displays it on the user interface.

[0354] Example: "Emergency! Please respond immediately."

[0355] Emotion recognition by emotion engine

[0356] Step 1:

[0357] The terminal analyzes the user's emotions using an emotion engine and generates emotion data.

[0358] Example: The stress state of the elderly is recognized by the emotion engine.

[0359] Step 2:

[0360] The terminal converts the generated emotion data into packets and transmits them to the server.

[0361] Step 3:

[0362] The server stores the received emotion data in a database and analyzes it.

[0363] Step 4:

[0364] The server generates feedback based on the emotion data and transmits it to the terminal.

[0365] Step 5:

[0366] The terminal analyzes the received feedback and displays it on the user interface.

[0367] Example: "Listen to some relaxing music" is displayed to the user.

[0368] Care plan coordination

[0369] Step 1:

[0370] The server analyzes the emotion data recognized by the emotion engine and adjusts the care plan.

[0371] Example: If stress levels are high, change the care plan to "relaxation exercises."

[0372] Step 2:

[0373] The server converts the adjusted care plan into packets and transmits them to the terminal.

[0374] Step 3:

[0375] The terminal displays the received coordinated care plan on a user interface and executes it.

[0376] Applying emotion recognition to video calls

[0377] Step 1:

[0378] The device analyzes the user's emotions using an emotion engine during a video call.

[0379] Example: The emotion engine recognizes the stress state of elderly people during video calls.

[0380] Step 2:

[0381] The terminal converts the recognized emotion data into packets and transmits them to the terminal of the other party.

[0382] Step 3:

[0383] The other party's device analyzes the received emotion data and displays warnings and recommended actions on the user interface.

[0384] Example: Displaying "You are feeling stressed, please change the topic to relax."

[0385] The above is the specific processing flow of each function, including the emotion engine.

[0386] Example 2

[0387] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0388] In elderly care, managing care plans and medical records is cumbersome, and remote status checks and communication are difficult. Furthermore, reminder functions and emergency responses are insufficient, and personalized care based on emotion recognition is not provided. Therefore, there is a need to improve the quality of care and provide efficient support for staff.

[0389] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an interface means for a user to input data and check the elderly person's care plan and medical records, a communication means for transmitting the data to the server, an electronic database means for the server to store and manage the data, a communication means for the server to transmit the data to the interface means, an interface means for conducting video calls in real time, a communication means for transmitting and receiving the video stream, a server means for processing and managing the video stream, an emotion recognition means for recognizing the elderly person's emotion, and an emergency notification means for notifying the elderly person of an emergency. This enables efficient management of the elderly person's care plan and medical records, remote care via real-time video calls, appropriate reminder notifications and emergency responses, and detailed care through emotion recognition.

[0390] "User" refers to a person who uses the system to input and check elderly care plans and medical records.

[0391] "Interface means" refers to devices and screens that allow users to input data and view care plans and medical records.

[0392] "Communication means" refers to a function or device for transmitting data to a server and receiving data from the server.

[0393] "Electronic database means" refers to a database system in which the server stores and manages data.

[0394] The "server means" is a server device that manages, stores, and transmits data, and processes and manages video streams.

[0395] A "video stream" is a continuous stream of video data sent and received during a video call.

[0396] The "emotion recognition means" is a system for analyzing and recognizing the emotions of elderly people in real time.

[0397] An "emergency notification means" is a function or device for promptly notifying relevant parties when an emergency occurs.

[0398] "Reminder" is a notification function that prompts elderly people to take specific actions based on times and conditions set by the user.

[0399] "Training tools" are support functions and devices that enable new staff to learn the knowledge and skills necessary to perform their jobs properly.

[0400] The present invention relates to a smart glasses-based care assistant system for supporting elderly care, which provides user data input, real-time video calling, reminder notifications, medical record review, new staff training, emergency care response, and emotion recognition with an emotion engine.

[0401] Hardware and software used

[0402] 1. Smart Glasses:

[0403] Hardware: camera, microphone, display, emergency button

[0404] Software: Interface app, emotion engine

[0405] 2. Server:

[0406] Hardware: High-performance server machine

[0407] Software: Database management system (MySQL), communication management system (WebRTC server), emotion recognition analysis system

[0408] 3. Means of communication:

[0409] Hardware: Communication modules (Wi-Fi, 4G / 5G modules)

[0410] Software: HTTP / HTTPS protocol, JSON format data transmission and reception

[0411] Program processing explanation

[0412] User Data Input

[0413] Users input care plans and medical records through the smart glasses interface, such as "daily exercise," "8 AM medication," "high blood pressure," and "arthritis." The device then sends this data to a server, which analyzes it and stores it in a database.

[0414] Real-time video calling

[0415] Users use smart glasses to make real-time video calls with remote family members or medical staff. The smart glasses are equipped with a camera and microphone, and a video call is initiated by activating these devices. The server processes the data stream of the video call and manages real-time communication. For example, when a user initiates a video call with an elderly person's daughter, the smart glasses turn on the camera and microphone and send the video stream to the server.

[0416] Reminders

[0417] Users can set reminders to prompt them to take action at specific times. The set reminder information is sent to the server via the device, and the server stores this information in a database. When the specified time arrives, the server sends a reminder notification to the device, and the device displays this notification on its user interface. For example, if a user sets a reminder to "take their morning medicine," a notification will be displayed on the device when the specified time arrives.

[0418] Medical record review

[0419] The user uses the smart glasses interface to send a request to check the elderly person's medical records. The device then sends this request to the server, which retrieves the medical records from the database and sends them to the device. The retrieved medical records are then displayed on the smart glasses' display. For example, if you want to check the medical records for "high blood pressure" or "arthritis," the server retrieves the data and the device displays it.

[0420] Training new staff

[0421] The new staff member uses the smart glasses to request a training module, and the server responds by sending the training content to the terminal. The terminal plays the training video and provides it to the new staff member. For example, when the new staff member starts training, the video content is streamed to the smart glasses.

[0422] Rapid response to emergency care

[0423] When an emergency occurs, the user presses the emergency button on the smart glasses to send a notification. The device sends the emergency notification to the server, which then activates the emergency response protocol and sends an alert notification to the relevant care staff. The device receives the alert and displays it to the user. For example, if an elderly person presses the emergency button, a notification saying "Emergency has occurred!" is immediately displayed.

[0424] Emotion engine recognizes emotions and adjusts care plans

[0425] The emotion engine recognizes the elderly person's emotions in real time and sends the emotional data to the server. The server analyzes this data and adjusts the care plan as needed. For example, if high stress is recognized, the server will change the care plan to "relaxation exercises" and send a notification to the device. Emotional data during a video call is also notified to the other party, allowing appropriate action to be taken.

[0426] Prompt Sentence Examples

[0427] Below is an example of a prompt sentence to input to the generative AI model.

[0428] "Enter your care plan for today."

[0429] "Start a video call with the elderly person's daughter."

[0430] "Set a reminder to take your medicine at 8am."

[0431] "Check the elderly person's medical records."

[0432] "Play the training video for new staff."

[0433] "Please press the emergency button."

[0434] "Please send us the emotional data of the elderly."

[0435] Using the procedures and prompts provided above, a specific system for improving care for seniors can be implemented. Embodiments of this system can efficiently and effectively manage seniors' health and adjust their care plans.

[0436] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0437] Step 1:

[0438] User Data Entry

[0439] How it works: Users enter care plans and medical records through the smart glasses interface.

[0440] Example: Enter care plan items such as "Daily exercise" and "8am medication" and medical records such as "High blood pressure" and "Arthritis."

[0441] Input: Care plans and medical records entered by the user through the smart glasses interface.

[0442] Data processing: The terminal organizes this data and converts it into JSON format.

[0443] Output: JSON data for sending organized data to the server.

[0444] Step 2:

[0445] Data transmission from the device

[0446] Operation: The terminal sends the entered data to the server.

[0447] Specific operation: Sends JSON format data obtained from the interface as an HTTP POST request.

[0448] Input: Organized care plan and medical record JSON data.

[0449] Data computation: Generates an HTTP request to send data.

[0450] Output: The data is sent to the server as an HTTP POST request.

[0451] Step 3:

[0452] Data analysis and storage on the server

[0453] How it works: The server parses the data sent and stores it in a database.

[0454] Specific operation: Parses the received JSON data and stores it in a MySQL database as a care plan and medical record.

[0455] Input: JSON data sent to the server as an HTTP POST request.

[0456] Data Computing: Running SQL queries to analyze data and store it in a database.

[0457] Output: Care plans and medical records stored in a database on the server.

[0458] Step 4:

[0459] Start a real-time video call

[0460] How it works: A user initiates a video call through the smart glasses interface.

[0461] Example: Pressing the "Start video call" button from the smart glasses interface.

[0462] Input: User's request to start a video call.

[0463] Data calculation: Session creation to start a video call.

[0464] Output: Start of video call session.

[0465] Step 5:

[0466] Enabling the device's camera and microphone

[0467] What it does: The device enables the camera and microphone.

[0468] Specific Action: Turns on the built-in camera and microphone device and starts capturing.

[0469] Input: Instruction to start a video call session.

[0470] Data Computing: Enabling camera and microphone devices.

[0471] Output: Start video and audio capture.

[0472] Step 6:

[0473] Server video call data processing

[0474] How it works: The server processes the data stream of the video call and manages the real-time communication.

[0475] Specific operation: The video stream is processed in real time using a WebRTC server and the stream is sent to the other device.

[0476] Input: Captured video and audio data.

[0477] Data Computing: Compression and transmission of data streams via WebRTC.

[0478] Output: Real-time video and audio communication.

[0479] Step 7:

[0480] Reminder settings

[0481] How it works: The user sets a reminder to take action at a specific time.

[0482] Example: Set a reminder to "take your morning medicine."

[0483] Input: Reminder information set by the user.

[0484] Data Calculation: Reminder information is organized and converted to JSON format.

[0485] Output: JSON data to send organized reminder information to the server.

[0486] Step 8:

[0487] Server reminder storage and notifications

[0488] How it works: The server stores the reminder information in a database and sends notifications to the device at the specified time.

[0489] Specific behavior: Saves reminder information in the database and sends a push notification at the specified time.

[0490] Input: JSON data of organized reminder information.

[0491] Data calculation: Save reminder information and trigger notifications at specified times.

[0492] Output: Send reminder notification.

[0493] Step 9:

[0494] Display reminders on your device

[0495] Action: The device displays the notification in its user interface.

[0496] Specific behavior: Display a reminder notification on the smart glasses display.

[0497] Input: Reminder notification sent from the server.

[0498] Data Calculation: Parsing notification data and converting it into a display format.

[0499] Output: Display reminder notification on the smart glasses display.

[0500] Step 10:

[0501] Submit a medical records request

[0502] How it works: A user uses the smart glasses interface to submit a request to view their medical records.

[0503] Example: Checking an elderly person's medical records for "high blood pressure" and "arthritis."

[0504] Input: User confirmation request.

[0505] Data operation: Generates an HTTP GET request to send the request.

[0506] Output: Sends to the server as an HTTP GET request.

[0507] Step 11:

[0508] Server data acquisition and transmission

[0509] Operation: The server retrieves medical records from the database and sends them to the device.

[0510] Specific behavior: Retrieves medical records from the database and sends them to the device in JSON format.

[0511] Input: User's medical records review request.

[0512] Data operations: Retrieving medical records and converting them to JSON format.

[0513] Output: Sends the medical record JSON data to the device.

[0514] Step 12:

[0515] Displaying medical records on the device

[0516] Action: The terminal displays the medical record in a user interface.

[0517] Specific operation: Displaying medical records on the smart glasses display.

[0518] Input: JSON data of medical records sent from the server.

[0519] Data Calculation: Analyzing data and converting it into a display format.

[0520] Output: Display medical records on the smart glasses display.

[0521] Step 13:

[0522] New staff training requests

[0523] How it works: A new employee requests training through the smart glasses interface.

[0524] Example: Submit a request for a training module.

[0525] Input: New staff training request.

[0526] Data operation: Generates an HTTP GET request to send the request.

[0527] Output: Sends to the server as an HTTP GET request.

[0528] Step 14:

[0529] Server training content transmission

[0530] Operation: The server sends training content to the device.

[0531] Specific operation: Sends training videos to the device in streaming format.

[0532] Input: New staff training request.

[0533] Data Computing: Acquiring training videos and converting them into a streaming format.

[0534] Output: Sends the training video to the device.

[0535] Step 15:

[0536] Training video playback on device

[0537] Action: The device plays a training video.

[0538] Specific operation: Playing a training video on the smart glasses display.

[0539] Input: The training video stream sent from the server.

[0540] Data operations: decoding and playing video streams.

[0541] Output: Playback of training videos to the smart glasses display.

[0542] Step 16:

[0543] Emergency button pressed

[0544] How it works: The user presses the emergency button on their smart glasses to send a notification.

[0545] Example: An elderly person presses an emergency button to send a notification.

[0546] Input: An input signal that the emergency button has been pressed.

[0547] Data Computation: Data construction for generating and sending emergency notifications.

[0548] Output: Sends an emergency notification to the server.

[0549] Step 17:

[0550] Server emergency response protocol activation

[0551] Actions: The server initiates emergency response protocols and sends alert notifications to relevant care staff.

[0552] What it does: Send alerts to all staff using a push notification system.

[0553] Input: Emergency notification sent from the device.

[0554] Data operations: Executing emergency response protocols and generating warning notifications.

[0555] Output: Send warning notification to all staff.

[0556] Step 18:

[0557] Emergency warning display on device

[0558] Action: The terminal receives the alert and displays it to the user.

[0559] Specific operation: A notification "Emergency has occurred!" is displayed on the smart glasses display.

[0560] Input: The warning notification sent by the server.

[0561] Data Calculation: Parsing alert data and converting it into a display format.

[0562] Output: Show a warning notification on the smart glasses display.

[0563] (Application example 2)

[0564] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0565] In elderly care, it is extremely important to monitor the elderly's emotions and location information in real time while ensuring their safety and responding quickly in emergencies. However, current systems lack the means to quickly notify and check the situation when an elderly person faces an emergency, making it difficult to respond effectively. In addition, there is a lack of systems that can monitor the elderly's emotional state and location information in real time and take action based on that information. Technological solutions to these problems are needed.

[0566] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a user interface means for a user to input data and check the elderly person's care plan and medical records, a communication means for transmitting the data to the server, a database means for the server to store and manage the data, a communication means for the server to transmit the data to the user interface means, a means having an emergency button for transmitting an emergency notification, a GPS means for acquiring and transmitting location information of the elderly person, an emotion recognition means for recognizing emotions, and a communication means for transmitting a notification based on data from the GPS means and the emotion recognition means. This allows an emergency notification to be transmitted quickly when the elderly person faces an emergency, and enables an effective response while monitoring the location information and emotional state in real time.

[0567] "User interface means" refers to the interactive screen display and operation means through which a user can input data and view the senior's care plan and medical records.

[0568] "Communication means" refers to means having a communication function for transmitting data to a server and receiving data from a server.

[0569] "Database means" refers to a database system for storing and managing data on a server.

[0570] An "emergency button" is a button-type operating device that an elderly person can press to send an emergency notification when they encounter an emergency.

[0571] "GPS Means" means a means utilizing the Global Positioning System (GPS) for obtaining and transmitting current location information.

[0572] The "emotion recognition means" is a means that uses emotion recognition technology to analyze the emotional state of the elderly and record and transmit the analysis results.

[0573] An "emergency notification" is notification information that is sent promptly to designated contacts when an emergency occurs.

[0574] "Real-time video call means" refers to means having the function of transmitting and receiving video and audio for conducting real-time video calls.

[0575] The present invention relates to a smart glasses-based care assistant system and security service for improving care for the elderly. Detailed embodiments of the present invention are described below.

[0576] System Configuration

[0577] The system consists of the following major components:

[0578] User Interface Means

[0579] communication means

[0580] Database Means

[0581] Emergency button

[0582] GPS means

[0583] emotion recognition means

[0584] Notification means

[0585] Server Means

[0586] Specific implementation methods

[0587] 1. User Interface Methods

[0588] Users input and view data through smart glasses or smartphones, including elderly care plans and medical records, with an intuitive and easy-to-use user interface.

[0589] 2. Means of communication

[0590] Data is sent from the device to the server and received from the server using a secure protocol.

[0591] 3. Database Means

[0592] The server stores the received data in a database and manages it appropriately, using a common database management system such as SQL.

[0593] 4. Emergency Button

[0594] If an elderly person faces an emergency, they can use the emergency button on their smart glasses or smartphone to send an emergency notification to designated contacts.

[0595] 5. GPS means

[0596] The system acquires the location information of the elderly person in real time and sends it to a server, allowing family members and caregivers to know the elderly person's location.

[0597] 6. Emotion recognition means

[0598] It analyzes the user's emotional state and detects stress, anxiety, etc. The emotion recognition means analyzes facial expressions using the camera in smart glasses or smartphones.

[0599] 7. Means of notification

[0600] Based on information obtained from emotion recognition and GPS means, necessary notifications are sent, including emergency notifications and notifications about emotional state.

[0601] 8. Server Means

[0602] The server is responsible for overall management and data processing, and also includes a video call module for processing real-time video calls.

[0603] Example of operation

[0604] For example, a user can enter an elderly person's care plan ("Daily Exercise" and "8 AM Medications") and check their medical records. If the elderly person encounters an emergency and presses the emergency button, their real-time location and current emotional state will be automatically sent to a designated contact. The contact who receives the emergency notification will then initiate a video call via smartphone or smart glasses to check on the elderly person's condition and provide assistance.

[0605] Examples of using generative AI models

[0606] When analyzing an elderly person's emotional state, generative AI models can be used to suggest "relaxing music" or "stress-reducing activities," with example prompts such as:

[0607] Prompt Sentence Examples

[0608] "The senior's panic button is pressed, which notifies family members of their real-time location and current emotional state (stress or anxiety). A video call is then automatically initiated, and the closest family member in their contact list receives the call."

[0609] From the above, a specific embodiment of the present invention has been presented and it can be seen how this system improves the care and security of the elderly.

[0610] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0611] Step 1:

[0612] Users input the elderly person's care plan and medical records through smart glasses or a smartphone. The input data is sent from the device to a server via a communication means. The server receives the data and stores it in a database for management. This ensures that the input information is securely stored on the server.

[0613] Step 2:

[0614] When needed, a user or caregiver sends a request to check the data stored on the server. The server retrieves the necessary information from the database and sends it to the terminal via a communication means. The terminal displays the received data on a user interface, allowing the user to check the elderly person's care plan and medical records.

[0615] Step 3:

[0616] When an elderly person faces an emergency, they press the emergency button on their smart glasses or smartphone. The device generates an emergency notification signal and sends it to the server via communication means. The server immediately receives the emergency notification and sends it to the registered emergency contacts.

[0617] Step 4:

[0618] At the same time, the GPS unit of the device acquires the current location information and transmits the location data to the server via the communication unit. The server processes the location information and transmits the location data together with the emergency notification. This allows the location of the elderly person to be monitored in real time and necessary measures to be taken promptly.

[0619] Step 5:

[0620] The device's emotion recognition means analyzes the elderly person's emotional state and acquires emotion data. For example, facial expressions are analyzed to detect signs of stress or anxiety. The acquired emotion data is then sent to a server via a communication means. The server analyzes the emotion data and generates an appropriate notification based on it.

[0621] Step 6:

[0622] The server then sends emergency notifications to emergency contacts along with warnings and advice based on the emotion data, such as "An elderly person is in a state of stress, please take immediate action."

[0623] Step 7:

[0624] The emergency contact who receives the emergency notification can use the video call function to contact the elderly person in real time. The server manages the video call session, activates the device's camera and microphone, and starts the video call. This allows the emergency contact to directly check the elderly person's current situation and ensure their safety.

[0625] Examples of prompt statements

[0626] "The senior's panic button is pressed, which notifies family members of their real-time location and current emotional state (stress or anxiety). A video call is then automatically initiated, and the closest family member in their contact list receives the call."

[0627] Through these steps, the system is able to comprehensively improve the care and security of the elderly.

[0628] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0629] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0630] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0631] [Second embodiment]

[0632] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0633] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0634] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0635] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0636] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0637] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0638] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0639] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0640] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0641] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0642] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0643] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0644] The present invention relates to a smart glasses-based care assistant system for improving elderly care, providing a system for inputting user data, managing elderly care plans and medical records, real-time video calling, reminder notifications, training new staff, and responding to emergency care. Specific embodiments for implementing the present invention are described in detail below.

[0645] User Data Input

[0646] Users input the elderly person's care plan and medical records through the smart glasses interface. The device then sends the input data to a server, which analyzes the received data and stores and manages it in a database.

[0647] For example, a user inputs the care plan for elderly person Tanaka, such as "daily exercise" and "medicine at 8 am," and also inputs medical records such as "high blood pressure" and "arthritis." The device sends the input data to the server, which then stores the data in a database.

[0648] Real-time video calling

[0649] The user uses the smart glasses interface to make real-time video calls with remote family members or medical staff. The device activates the camera and microphone and initiates the video call session. The server processes the video call data stream and manages real-time communication.

[0650] For example, if a user wants to start a video call with Tanaka's daughter, the smart glasses will turn on the camera and microphone and transmit the video stream. The server processes this data and starts the call.

[0651] Reminders

[0652] Users can set reminders to prompt the elderly to perform an action at a specific time. The device sends the reminder information to the server, which stores and manages the reminder information in a database. When the reminder time arrives, the server sends a reminder notification to the device, which then displays the notification on the user interface.

[0653] For example, if Tanaka sets a reminder to take her morning medicine, the server stores that information in a database and sends a notification to the device when the specified time arrives. The device then displays a notification saying, "It's time to take your morning medicine."

[0654] Medical record review

[0655] The user can check the elderly person's medical records using the interface of the smart glasses. The device sends the user's request to the server, which retrieves the medical records from the database and sends them to the device. The device then displays the retrieved medical records on the user interface.

[0656] For example, if a user checks Mr. Tanaka's medical records, the server retrieves information about "high blood pressure" and "arthritis" from the database and sends it to the terminal, which then displays that information.

[0657] Training new staff

[0658] New staff can receive training using smart glasses. The terminal sends a request for a training module to the server, and the server sends the training content to the terminal, which then plays it and provides it to the user.

[0659] For example, when a new staff member, Mr. Yamada, starts training, the terminal requests a training module from the server, and the server sends the training video to the terminal, which then plays the video.

[0660] Rapid response to emergency care

[0661] In the event of an emergency, the user can press the emergency button on the smart glasses to send a notification. The device will then send the emergency notification to the server, which will trigger the emergency response protocol and send an alert to the relevant care staff. The device will receive the alert and display it immediately.

[0662] For example, if Tanaka presses the emergency button, the device sends an emergency notification to the server, which then alerts all staff. The device displays, "Emergency! Please respond immediately."

[0663] As described above, the present invention improves the quality of care for the elderly, while enabling efficient support and emergency response by staff, thereby improving the quality of life for the elderly.

[0664] The processing flow will be explained below.

[0665] OK, so let's go into more detail and explain what the program does, step by step.

[0666] User Data Input

[0667] Step 1:

[0668] Users enter the elderly person's care plan and medical records through the smart glasses interface.

[0669] Specifically, data is input using a keyboard, voice input, a touch panel, etc.

[0670] Step 2:

[0671] The terminal transmits the input data to the server.

[0672] Specifically, the input data is converted into packets and sent to the server using a communication protocol.

[0673] Step 3:

[0674] The server analyzes the received data and stores it in a database.

[0675] Specifically, the received packet is analyzed and the data is stored in the appropriate database field.

[0676] Real-time video calling

[0677] Step 1:

[0678] The user selects the person with whom they want to start a video call using the interface of the smart glasses.

[0679] Specifically, you select the ID of the person you want to call using the touch panel or voice commands.

[0680] Step 2:

[0681] The terminal sends a request for a call session to the server.

[0682] Specifically, the server sends a request packet containing the ID of the selected call partner to the server.

[0683] Step 3:

[0684] The server processes the call request and establishes the call session.

[0685] Specifically, a new session ID is generated and notified to both terminals.

[0686] Step 4:

[0687] The device will enable the camera and microphone and start the video stream.

[0688] Specifically, the hardware camera and microphone are turned on, and video and audio data are sent and received as streams.

[0689] Reminders

[0690] Step 1:

[0691] The user inputs the reminder content and time through the smart glasses interface.

[0692] Specifically, the reminder content and time can be set using the touch panel or voice input.

[0693] Step 2:

[0694] The terminal transmits the reminder information to the server.

[0695] Specifically, the set reminder information is packetized and sent to the server.

[0696] Step 3:

[0697] The server stores the reminder information and monitors the reminder time.

[0698] Specifically, the system stores the reminder information in a database and waits for the specified time.

[0699] Step 4:

[0700] The server sends a notification to the terminal when the reminder time arrives.

[0701] Specifically, the system refers to the saved reminder information and sends a notification packet to the terminal when the reminder time is reached.

[0702] Step 5:

[0703] The terminal receives the reminder notification and displays it to the user.

[0704] Specifically, the received notification packet is analyzed and the reminder content is displayed on the user interface.

[0705] Medical record review

[0706] Step 1:

[0707] The user submits a request to review medical records through the smart glasses interface.

[0708] Specifically, the user selects the medical record item they wish to check using the touch panel or voice.

[0709] Step 2:

[0710] The terminal sends a confirmation request to the server.

[0711] Specifically, the operation involves sending a request packet containing the selected medical record items to the server.

[0712] Step 3:

[0713] The server retrieves the relevant medical records from the database.

[0714] Specifically, the system analyzes the received request and extracts the relevant medical record data from the database.

[0715] Step 4:

[0716] The server transmits the acquired medical record data to the terminal.

[0717] Specifically, the extracted medical record data is converted into packets and sent to the terminal.

[0718] Step 5:

[0719] The terminal displays the received medical record data on a user interface.

[0720] Specifically, the received data packets are analyzed and the medical records are displayed on the user interface.

[0721] Training new staff

[0722] Step 1:

[0723] The user selects a training module from the smart glasses interface.

[0724] Specifically, the training menu is selected using the touch panel or voice.

[0725] Step 2:

[0726] The terminal sends a request for a training module to the server.

[0727] Specifically, the ID of the selected training module is sent to the server.

[0728] Step 3:

[0729] The server obtains the training content and transmits it to the terminal.

[0730] Specifically, the relevant training content is retrieved from the database and sent to the terminal.

[0731] Step 4:

[0732] The terminal plays back the received training content.

[0733] Specifically, the received training video or manual is played on the user interface.

[0734] Rapid response to emergency care

[0735] Step 1:

[0736] The user presses the emergency button on the smart glasses to notify them of an emergency.

[0737] Specific actions include physically pressing an emergency button or issuing a specific voice command.

[0738] Step 2:

[0739] The terminal sends an emergency notification to the server.

[0740] Specifically, it detects that the emergency button has been pressed and sends an emergency notification packet to the server.

[0741] Step 3:

[0742] The server receives the emergency notification and executes the emergency response protocol.

[0743] Specifically, it analyzes the received emergency notification and initiates a response according to the protocol.

[0744] Step 4:

[0745] The server sends an alert notification to the relevant care staff.

[0746] A specific operation is to send an alert notification packet to the device of the necessary care staff.

[0747] Step 5:

[0748] The terminal receives the warning notification and displays it to the user.

[0749] Specifically, the device analyzes the warning notification received and displays "Emergency situation! Please respond immediately" on the user interface.

[0750] The above is the specific processing flow of each function detailed for each process.

[0751] Example 1

[0752] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0753] To improve elderly care, a system for efficient data management and emergency response is needed. However, the current system does not centrally manage care plans, medical record entry and review, real-time video calls, reminder notifications, new staff training, and emergency care responses, making it difficult to provide comprehensive support.

[0754] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0755] In this invention, the server includes a user interface means for a user to input data and check care plans and medical records, a communication means for transmitting the data to the server, a database means for the server to store and manage the data, a communication means for the server to transmit the data to the user interface means, a sensor means for transmitting an emergency notification to the server when a user presses an emergency button, and a server means for the server to execute emergency response protocols and send alerts to relevant staff. This allows management of elderly care plans and medical records, real-time video calls, reminder notifications, training of new staff, and emergency care responses to be performed in a unified manner, thereby improving the quality and efficiency of care.

[0756] "User interface means" refers to hardware and software for displaying data entered by the user and notifications from the system.

[0757] "Communication means" refers to the network interface and communication protocol for sending and receiving data.

[0758] "Database means" refers to a storage system or database management system for storing and managing data.

[0759] "Sensor means" refers to a device or module that detects user input or changes in the environment and acquires data.

[0760] "Server means" refers to a server system for processing, storing, managing, and communicating data.

[0761] "Emergency response protocols" are response procedures and steps that the system automatically executes in the event of an emergency.

[0762] A "video stream" is a flow of video data that is sent and received in real time.

[0763] A "reminder" is an alert or notification that notifies a user of a specific action or time.

[0764] These definitions provide a clearer understanding of the role and function of each element included in the claims.

[0765] The present invention relates to a smart glasses-based care assistant system for improving elderly care, providing a system for user data entry, managing elderly care plans and medical records, real-time video calling, reminder notifications, training new staff, and emergency care response. Specific embodiments for implementing the present invention are described in detail below.

[0766] User Data Input

[0767] Users can input care plans and medical records through the smart glasses interface. The data is then sent to a server via the device. The server analyzes the received data and stores and manages it in a database. This allows for centralized management of elderly care plans and medical records.

[0768] Specifically, users can use the smart glasses' voice recognition system or touchpad to input instructions such as "Take today's medicine at 8 o'clock." The device converts this voice data into text and sends it to the server. The server then converts this data into JSON format and stores it in a database such as MySQL.

[0769] Real-time video calling

[0770] Users can use the smart glasses interface to make real-time video calls with remote family members or medical staff. The device activates the camera and microphone and initiates the video call session. The server processes the video data stream and manages the communication in real time.

[0771] For example, if you say "start a video call" to your smart glasses, the device will turn on the camera and microphone and start a call session using WebRTC. The server manages this and ensures the call proceeds smoothly.

[0772] Reminders

[0773] Users can set reminders to prompt the elderly to perform specific actions at specific times. The device sends the reminder information to the server, which stores and manages the reminder information in a database. At the set time, the server sends a reminder notification to the device, which then displays the notification on the user interface.

[0774] Specifically, when someone types "Set a reminder to take my medicine at 8 a.m.", the device converts this into text and sends it to the server. The server stores the reminder information in a database and sends a notification to the device at the specified time. The device then displays "It's time to take your morning medicine."

[0775] Medical record review

[0776] The user can check their medical records through the interface of the smart glasses. A request is sent from the device to the server, which retrieves the medical records from the database and sends them to the device. The device then displays the medical records on the user interface.

[0777] For example, if you say "Check Tanaka-san's medical records," the server will retrieve information on "high blood pressure" and "arthritis" from the database and send it to the device, which will then display it on its screen.

[0778] Training new staff

[0779] New staff can receive training using smart glasses. The device sends a request for a training module to the server, and the server sends the training content to the device. The device can then play the provided content and display it to the user.

[0780] Specifically, when a new staff member says "Start training," the terminal sends this request to the server, which then sends the training video to the terminal, which then plays the video.

[0781] Rapid response to emergency care

[0782] In the event of an emergency, users can press the emergency button on the smart glasses to send an emergency notification. The device will then send the emergency notification to the server, which will trigger the emergency response protocol to alert relevant staff. The device will then display the alert.

[0783] For example, if Tanaka presses the emergency button, the device sends an emergency notification to the server, which then executes emergency response protocols and alerts all staff. The device displays, "Emergency! Please respond immediately."

[0784] As described above, the present invention improves the quality of care for the elderly, while enabling efficient support and emergency response by staff, thereby improving the quality of life for the elderly.

[0785] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0786] Step 1:

[0787] The user inputs data through the smart glasses interface. For example, they can enter instructions such as "Take today's medicine at 8 o'clock" using a voice recognition system or touchpad. The input in this step is voice or text data, and the device receives this input.

[0788] Step 2:

[0789] The device converts the received voice data into text data. It uses voice recognition technology to analyze the user's voice input and outputs it as text. The converted text data is obtained. This specific operation is performed using the voice recognition library installed on the device.

[0790] Step 3:

[0791] The terminal transmits text data to the server. The data is transmitted to the server via the internet via the communication means. The input of this step is text data, and the output is data transmission to the server.

[0792] Step 4:

[0793] The server parses the received text data, converts it into the required data format (for example, JSON format), and saves the converted data in a database. The input is text data, and the output is saving the converted data in a database.

[0794] Step 5:

[0795] The server sets reminder information and stores it until the specified time. When the specified time arrives, it sends a reminder notification to the device. The input is the saved reminder information, and the output is to send a reminder notification to the device when the time arrives.

[0796] Step 6:

[0797] The device displays the reminder notification received from the server on the user interface and informs the user of the reminder content. Specifically, the device displays "It's time to take your morning medicine" on the display and also notifies the user by voice. The input is the reminder notification from the server, and the output is the display on the user interface and the voice notification.

[0798] Step 7:

[0799] When a user presses the emergency button, the terminal is instructed to notify the user of an emergency. The input is the user's operation of the emergency button, and the terminal captures this operation.

[0800] Step 8:

[0801] The terminal sends an emergency notification to the server. Emergency status data is sent to the server using a communication method. The input is the emergency button operation event, and the output is the transmission of emergency notification data to the server.

[0802] Step 9:

[0803] When the server receives an emergency notification, it activates the emergency response protocol and sends an alert to the relevant staff. Specifically, the server sends an emergency email or SMS to all registered staff. The input is the emergency notification data, and the output is sending an alert notification to the staff.

[0804] Step 10:

[0805] The relevant staff receives the alert and initiates a response based on the information. The input is an emergency alert notification from the server, and the output is the execution of emergency response actions.

[0806] Through these steps, the system can efficiently support the various needs of elderly care.

[0807] (Application example 1)

[0808] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0809] Conventional elderly care systems have had problems such as the time-consuming task of managing elderly care plans and medical records, and the difficulty of sharing care information in real time. Furthermore, while rapid response is required in the event of an emergency, conventional systems do not adequately address this need. Furthermore, security staff lacked the efficiency and immediacy required for on-site monitoring and emergency response, making it difficult to ensure smooth work progress through reminder notifications.

[0810] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0811] In this invention, the server includes a user interface means for a user to input data and check the elderly person's care plan and medical records, a communication means for transmitting the data to the server, a database means for storing and managing the data, a communication means for transmitting the data to the user interface means, a communication means for transmitting and receiving video streams, a communication means for issuing emergency notifications, and a display means for displaying reminder notifications, thereby enabling more efficient information management in elderly care, real-time information sharing, faster emergency response, and improved work efficiency of security staff.

[0812] "User interface means" refers to an interface through which a user inputs data and confirms and manipulates information.

[0813] "Communication means" refers to the technical means used to send and receive data and information between servers.

[0814] The "database means" refers to a data storage system for saving and managing input data and its management function.

[0815] The "server means" is a server system for processing and managing data.

[0816] A "video stream" is a continuous flow of video data that is sent and received in real time.

[0817] "Emergency notifications" are messages or alerts that are sent to quickly warn relevant parties when an emergency occurs.

[0818] The "display means" refers to a display or monitor for visually presenting information to the user.

[0819] A "reminder notification" is a notification that prompts the user to take action at a specific time or under specific conditions.

[0820] "Real-time video calling" is a call format in which both parties communicate with each other in real time.

[0821] "Communication means for issuing emergency notifications" means the technical means for sending immediate warning messages in the event of an emergency.

[0822] The present invention relates to a smart glasses-based assistant system for security and elderly care, which includes a user interface means, a communication means, a database means, a server means, and a display means.

[0823] System Overview

[0824] The system consists of the following main components:

[0825] 1. User interface means: The smart glasses' display and microphone allow users to input data, view information, and manipulate it.

[0826] 2. Communication Method: The technical means for transmitting data and video streams between the smart glasses and the server, including Wi-Fi and cellular communication.

[0827] 3. Database Means: A cloud storage system for storing and managing user-entered data. Specifically, this includes Google Cloud Firestore.

[0828] 4. Server infrastructure: Server infrastructure that processes and manages data. For example, Google Cloud or AWS is used.

[0829] 5. Display means: A means for visually presenting information to the user using the smart glasses display.

[0830] Program Overview

[0831] The process of a user wearing smart glasses and entering data, making video calls, setting reminders, and sending emergency notifications through the user interface is described.

[0832] Entering and verifying data

[0833] The user inputs data using the smart glasses' display and voice input. For example, in elderly care, the user can input care plans and medical records. The input data is sent to a server via a communication means and stored in a database means. The stored data can be viewed at any time via a user interface means.

[0834] Real-time video calling

[0835] A user can use the interface of the smart glasses to make a real-time video call with another user in a remote location (e.g., a security center or a family member). The device captures the video stream using a camera and microphone and transmits it to a server through a communication means. The server means processes the video stream in real time and manages the call function.

[0836] Reminders

[0837] A user can set a reminder for a task to be performed at a specific time. The reminder information is sent to the server via the communication means and stored in the database means. When the specified time arrives, the server means sends a notification to the terminal, and the reminder is displayed to the user via the display means.

[0838] Sending emergency notifications

[0839] When a user faces an emergency, they can press the emergency button on the smart glasses to issue an emergency notification. The emergency notification is sent to the server via the communication means, and the server means activates an emergency response protocol and sends the emergency notification to all related terminals. A prompt response is then made via the display means.

[0840] Specific examples

[0841] When a security staff member discovers a suspicious individual while on patrol, an emergency notification is sent from the smart glasses. The server immediately sends an alert to multiple staff members, enabling a swift response on-site.

[0842] At elderly care facilities, care staff input care plans into smart glasses, and the data is stored on a server, allowing family members to remotely check the progress of care in real time.

[0843] Prompt Sentence Examples

[0844] "Please outline an application that uses smart glasses to enable security personnel to initiate video calls, send emergency notifications, and display reminder notifications in real time. Please explain the specific features."

[0845] Overall, the system will streamline care and security management and improve real-time response capabilities.

[0846] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0847] Step 1:

[0848] The user wears the smart glasses and inputs data through the user interface, such as entering care plans and medical records. The input data is sent to a server via a communication means. Input is performed using the smart glasses' display and voice input function, and the data is sent to the server as output.

[0849] Step 2:

[0850] The server stores and manages the received data using database means. When the input data reaches the server, it is stored in the database. The server uses Google Cloud Firestore to store the data. After the data is stored, the database is updated.

[0851] Step 3:

[0852] The user checks the entered care plan and medical records through the interface of the smart glasses. A request is sent to access the data stored on the server. The server receives this request, retrieves the relevant data from the database, and sends it back to the smart glasses via a communication method. As a result, the user can view the data on the display.

[0853] Step 4:

[0854] Users use the smart glasses interface to make real-time video calls with other users in remote locations. The smart glasses' camera and microphone capture video and audio and send them to a server via a communication method. The server uses Twilio's API to manage the video call session and send the video stream to the other user's device. This enables two-way communication of video and audio in real time.

[0855] Step 5:

[0856] The user sets a reminder for a specific time, for example, inputting a reminder for rounds or taking medicine. The reminder information is sent to the server via the interface of the smart glasses using a communication means. The server receives this information, stores it in a database, and then triggers a reminder notification based on the specified time. The notification is sent back to the smart glasses via the communication means and displayed to the user via the display means. As a result, the user receives the notification at the specified time.

[0857] Step 6:

[0858] When a user faces an emergency, they press the emergency button on the smart glasses to send an emergency notification. Input is made by pressing the button, and the notification is sent to the server using a communication method. The server receives the emergency notification, activates a response protocol, and sends the emergency notification to all related devices. This allows a notification of an "emergency" to be displayed on multiple devices, enabling a quick response.

[0859] This allows the system to enable users to effectively manage data, communicate and respond quickly to emergencies.

[0860] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0861] The present invention relates to a smart glasses-based care assistant system for improving elderly care, which provides a system for inputting user data, managing elderly care plans and medical records, real-time video calling, reminder notifications, training new staff, responding to emergency care, and emotion recognition using an emotion engine. The following describes in detail the embodiments of the present invention.

[0862] User Data Input

[0863] Users input the elderly person's care plan and medical records through the smart glasses interface. The device then sends the input data to a server, which analyzes the data and stores it in a database for management.

[0864] For example, a user inputs an elderly person's care plan, "Daily Exercises" and "Medicine at 8 AM," and inputs medical records "High Blood Pressure" and "Arthritis." The terminal sends the input data to the server, which stores the data in a database.

[0865] Real-time video calling

[0866] The user uses the smart glasses interface to make real-time video calls with remote family members or medical staff. The device activates the camera and microphone and initiates the video call session. The server processes the video call data stream and manages real-time communication.

[0867] For example, if a user wants to start a video call with the elderly person's daughter, the smart glasses will turn on the camera and microphone and transmit the video stream, and the server will process this data and start the call.

[0868] Reminders

[0869] Users can set reminders to prompt the elderly to take action at specific times. The device sends the reminder information to the server, which stores the information in a database and sends a notification to the device when the reminder time arrives. The device then displays the notification on the user interface.

[0870] For example, when a user sets a reminder for an elderly person to take their morning medication, the server stores that information in a database and sends a notification to the device at the specified time, which displays a message saying, "It's time to take your morning medication."

[0871] Medical record review

[0872] The user can check the elderly person's medical records using the smart glasses interface. The device sends the user's request to the server, which retrieves the medical records from the database and sends them to the device. The device then displays the retrieved medical records on the user interface.

[0873] For example, if a user checks the medical records of an elderly person, the server retrieves information about "high blood pressure" and "arthritis" from the database and sends it to the terminal, which then displays the information.

[0874] Training new staff

[0875] New staff can receive training using smart glasses. The device sends a request for a training module to the server, and the server sends the training content to the device, which then plays it and provides it to the user.

[0876] For example, when a new staff member starts training, the terminal requests a training module from the server, and the server sends a training video to the terminal, which then plays the video.

[0877] Rapid response to emergency care

[0878] In the event of an emergency, the user can press the emergency button on the smart glasses to send a notification. The device will then send the emergency notification to the server, which will trigger the emergency response protocol and send an alert notification to the relevant care staff. The device will then receive the alert and display it to the user.

[0879] For example, if an elderly person presses the emergency button, the device will send an emergency notification to the server, which will then alert all staff. The device will display "Emergency! Please respond immediately."

[0880] Emotion recognition by emotion engine

[0881] The emotion engine means recognizes the user's emotions and transmits them to the server, which stores and analyzes the emotion data, and provides feedback to the user based on the emotion data.

[0882] For example, if the emotion engine detects that an elderly person is in a stressful state, the device will send this emotion data to the server, which will analyze the data and provide feedback to the user such as "listen to some relaxing music."

[0883] Care plan coordination

[0884] Based on the emotion data recognized by the emotion engine, the server adjusts the care plan, which is then notified to the user and implemented.

[0885] For example, if the emotion engine analyzes the emotional data of an elderly person and determines that they are under high stress, the server will adjust the care plan to a "relaxation exercise" and notify the device to carry it out.

[0886] Applying emotion recognition to video calls

[0887] The emotion engine means recognizes the user's emotions during a real-time video call and notifies the video call partner of the emotion data. Based on the emotion data, a warning or recommended action can be displayed to the video call partner.

[0888] For example, if the emotion engine recognizes that an elderly person is becoming increasingly stressed during a video call, the caller will see a message saying, "You are feeling stressed. Please change the topic to help you relax."

[0889] As described above, the present invention improves the quality of care for the elderly while providing efficient support and emergency response from staff, and by combining it with an emotion engine, more detailed care and support becomes possible.

[0890] The processing flow will be explained below.

[0891] User Data Input

[0892] Step 1:

[0893] Users enter the elderly person's care plan and medical records through the smart glasses interface.

[0894] Example: A user enters an elderly person's care plan, "Daily Exercises" and "8 AM Medication," and medical records, including "High Blood Pressure" and "Arthritis."

[0895] Step 2:

[0896] The terminal converts the input data into packets and transmits them to the server.

[0897] Step 3:

[0898] The server analyzes the received data and stores it in a database.

[0899] A database stores care plans and medical records.

[0900] Real-time video calling

[0901] Step 1:

[0902] The user uses the interface of the smart glasses to select who to start a video call with.

[0903] Example: A user selects the ID of an elderly person's daughter.

[0904] Step 2:

[0905] The terminal sends a call request packet to the server.

[0906] The request containing the submitted ID arrives at the server.

[0907] Step 3:

[0908] The server processes the call request and generates a new session ID.

[0909] The generated session ID is notified to both terminals.

[0910] Step 4:

[0911] The device will enable the camera and microphone and start the video stream.

[0912] A video stream is sent from the elderly person's smart glasses and displayed on his daughter's device.

[0913] Reminders

[0914] Step 1:

[0915] The user inputs the reminder content and time through the smart glasses interface.

[0916] Example: A user sets a reminder for an elderly person to take their morning medication at 08:00.

[0917] Step 2:

[0918] The terminal converts the reminder information into packets and transmits them to the server.

[0919] Step 3:

[0920] The server stores the received reminder information in a database and monitors the specified time.

[0921] Step 4:

[0922] When the reminder time arrives, the server sends a notification packet to the terminal.

[0923] Step 5:

[0924] The terminal analyzes the received notification and displays it on the user interface.

[0925] Example: The device displays "Time to take your morning medicine."

[0926] Medical record review

[0927] Step 1:

[0928] The user enters a request to review medical records through the smart glasses interface.

[0929] Example: A user sends a request to review an elderly person's medical records.

[0930] Step 2:

[0931] The terminal sends a request packet to the server.

[0932] Step 3:

[0933] The server retrieves the relevant medical records from the database.

[0934] The server extracts information on "high blood pressure" and "arthritis" from the database.

[0935] Step 4:

[0936] The server converts the acquired medical record data into packets and transmits them to the terminal.

[0937] Step 5:

[0938] The terminal analyzes the received data and displays it on a user interface.

[0939] Example: Medical records for "high blood pressure" and "arthritis" are displayed.

[0940] Training new staff

[0941] Step 1:

[0942] The user selects a training module from the smart glasses interface.

[0943] Example: New staff members select a training menu.

[0944] Step 2:

[0945] The terminal sends a request packet for the training module to the server.

[0946] Step 3:

[0947] The server retrieves training content from the database, converts it into packets, and transmits them to the terminal.

[0948] Step 4:

[0949] The terminal plays back the received training content.

[0950] Example: A device plays a training video that new staff members watch.

[0951] Rapid response to emergency care

[0952] Step 1:

[0953] The user presses the emergency button on the smart glasses to send an emergency notification.

[0954] Example: An elderly person presses the emergency button.

[0955] Step 2:

[0956] The terminal transmits an emergency notification packet to the server.

[0957] Step 3:

[0958] The server executes emergency response protocols and sends emergency notifications to all staff.

[0959] Step 4:

[0960] The terminal analyzes the received emergency notification and displays it on the user interface.

[0961] Example: "Emergency! Please respond immediately."

[0962] Emotion recognition by emotion engine

[0963] Step 1:

[0964] The terminal analyzes the user's emotions using an emotion engine and generates emotion data.

[0965] Example: The stress state of the elderly is recognized by the emotion engine.

[0966] Step 2:

[0967] The terminal converts the generated emotion data into packets and transmits them to the server.

[0968] Step 3:

[0969] The server stores the received emotion data in a database and analyzes it.

[0970] Step 4:

[0971] The server generates feedback based on the emotion data and transmits it to the terminal.

[0972] Step 5:

[0973] The terminal analyzes the received feedback and displays it on the user interface.

[0974] Example: "Listen to some relaxing music" is displayed to the user.

[0975] Care plan coordination

[0976] Step 1:

[0977] The server analyzes the emotion data recognized by the emotion engine and adjusts the care plan.

[0978] Example: If stress levels are high, change the care plan to "relaxation exercises."

[0979] Step 2:

[0980] The server converts the adjusted care plan into packets and transmits them to the terminal.

[0981] Step 3:

[0982] The terminal displays the received coordinated care plan on a user interface and executes it.

[0983] Applying emotion recognition to video calls

[0984] Step 1:

[0985] The device analyzes the user's emotions using an emotion engine during a video call.

[0986] Example: The emotion engine recognizes the stress state of elderly people during video calls.

[0987] Step 2:

[0988] The terminal converts the recognized emotion data into packets and transmits them to the terminal of the other party.

[0989] Step 3:

[0990] The other party's device analyzes the received emotion data and displays warnings and recommended actions on the user interface.

[0991] Example: Displaying "You are feeling stressed, please change the topic to relax."

[0992] The above is the specific processing flow of each function, including the emotion engine.

[0993] Example 2

[0994] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0995] In elderly care, managing care plans and medical records is cumbersome, and remote status checks and communication are difficult. Furthermore, reminder functions and emergency responses are insufficient, and personalized care based on emotion recognition is not provided. Therefore, there is a need to improve the quality of care and provide efficient support for staff.

[0996] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an interface means for a user to input data and check the elderly person's care plan and medical records, a communication means for transmitting the data to the server, an electronic database means for the server to store and manage the data, a communication means for the server to transmit the data to the interface means, an interface means for conducting video calls in real time, a communication means for transmitting and receiving the video stream, a server means for processing and managing the video stream, an emotion recognition means for recognizing the elderly person's emotion, and an emergency notification means for notifying the elderly person of an emergency. This enables efficient management of the elderly person's care plan and medical records, remote care via real-time video calls, appropriate reminder notifications and emergency responses, and detailed care through emotion recognition.

[0997] "User" refers to a person who uses the system to input and check elderly care plans and medical records.

[0998] "Interface means" refers to devices and screens that allow users to input data and view care plans and medical records.

[0999] "Communication means" refers to a function or device for transmitting data to a server and receiving data from the server.

[1000] "Electronic database means" refers to a database system in which the server stores and manages data.

[1001] The "server means" is a server device that manages, stores, and transmits data, and processes and manages video streams.

[1002] A "video stream" is a continuous stream of video data sent and received during a video call.

[1003] The "emotion recognition means" is a system for analyzing and recognizing the emotions of elderly people in real time.

[1004] An "emergency notification means" is a function or device for promptly notifying relevant parties when an emergency occurs.

[1005] "Reminder" is a notification function that prompts elderly people to take specific actions based on times and conditions set by the user.

[1006] "Training tools" are support functions and devices that enable new staff to learn the knowledge and skills necessary to perform their jobs properly.

[1007] The present invention relates to a smart glasses-based care assistant system for supporting elderly care, which provides user data input, real-time video calling, reminder notifications, medical record review, new staff training, emergency care response, and emotion recognition with an emotion engine.

[1008] Hardware and software used

[1009] 1. Smart Glasses:

[1010] Hardware: camera, microphone, display, emergency button

[1011] Software: Interface app, emotion engine

[1012] 2. Server:

[1013] Hardware: High-performance server machine

[1014] Software: Database management system (MySQL), communication management system (WebRTC server), emotion recognition analysis system

[1015] 3. Means of communication:

[1016] Hardware: Communication modules (Wi-Fi, 4G / 5G modules)

[1017] Software: HTTP / HTTPS protocol, JSON format data transmission and reception

[1018] Program processing explanation

[1019] User Data Input

[1020] Users input care plans and medical records through the smart glasses interface, such as "daily exercise," "8 AM medication," "high blood pressure," and "arthritis." The device then sends this data to a server, which analyzes it and stores it in a database.

[1021] Real-time video calling

[1022] Users use smart glasses to make real-time video calls with remote family members or medical staff. The smart glasses are equipped with a camera and microphone, and a video call is initiated by activating these devices. The server processes the data stream of the video call and manages real-time communication. For example, when a user initiates a video call with an elderly person's daughter, the smart glasses turn on the camera and microphone and send the video stream to the server.

[1023] Reminders

[1024] Users can set reminders to prompt them to take action at specific times. The set reminder information is sent to the server via the device, and the server stores this information in a database. When the specified time arrives, the server sends a reminder notification to the device, and the device displays this notification on its user interface. For example, if a user sets a reminder to "take their morning medicine," a notification will be displayed on the device when the specified time arrives.

[1025] Medical record review

[1026] The user uses the smart glasses interface to send a request to check the elderly person's medical records. The device then sends this request to the server, which retrieves the medical records from the database and sends them to the device. The retrieved medical records are then displayed on the smart glasses' display. For example, if you want to check the medical records for "high blood pressure" or "arthritis," the server retrieves the data and the device displays it.

[1027] Training new staff

[1028] The new staff member uses the smart glasses to request a training module, and the server responds by sending the training content to the terminal. The terminal plays the training video and provides it to the new staff member. For example, when the new staff member starts training, the video content is streamed to the smart glasses.

[1029] Rapid response to emergency care

[1030] When an emergency occurs, the user presses the emergency button on the smart glasses to send a notification. The device sends the emergency notification to the server, which then activates the emergency response protocol and sends an alert notification to the relevant care staff. The device receives the alert and displays it to the user. For example, if an elderly person presses the emergency button, a notification saying "Emergency has occurred!" is immediately displayed.

[1031] Emotion engine recognizes emotions and adjusts care plans

[1032] The emotion engine recognizes the elderly person's emotions in real time and sends the emotional data to the server. The server analyzes this data and adjusts the care plan as needed. For example, if high stress is recognized, the server will change the care plan to "relaxation exercises" and send a notification to the device. Emotional data during a video call is also notified to the other party, allowing appropriate action to be taken.

[1033] Prompt Sentence Examples

[1034] Below is an example of a prompt sentence to input to the generative AI model.

[1035] "Enter your care plan for today."

[1036] "Start a video call with the elderly person's daughter."

[1037] "Set a reminder to take your medicine at 8am."

[1038] "Check the elderly person's medical records."

[1039] "Play the training video for new staff."

[1040] "Please press the emergency button."

[1041] "Please send us the emotional data of the elderly."

[1042] Using the procedures and prompts provided above, a specific system for improving care for seniors can be implemented. Embodiments of this system can efficiently and effectively manage seniors' health and adjust their care plans.

[1043] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1044] Step 1:

[1045] User Data Entry

[1046] How it works: Users enter care plans and medical records through the smart glasses interface.

[1047] Example: Enter care plan items such as "Daily exercise" and "8am medication" and medical records such as "High blood pressure" and "Arthritis."

[1048] Input: Care plans and medical records entered by the user through the smart glasses interface.

[1049] Data processing: The terminal organizes this data and converts it into JSON format.

[1050] Output: JSON data for sending organized data to the server.

[1051] Step 2:

[1052] Data transmission from the device

[1053] Operation: The terminal sends the entered data to the server.

[1054] Specific operation: Sends JSON format data obtained from the interface as an HTTP POST request.

[1055] Input: Organized care plan and medical record JSON data.

[1056] Data computation: Generates an HTTP request to send data.

[1057] Output: The data is sent to the server as an HTTP POST request.

[1058] Step 3:

[1059] Data analysis and storage on the server

[1060] How it works: The server parses the data sent and stores it in a database.

[1061] Specific operation: Parses the received JSON data and stores it in a MySQL database as a care plan and medical record.

[1062] Input: JSON data sent to the server as an HTTP POST request.

[1063] Data Computing: Running SQL queries to analyze data and store it in a database.

[1064] Output: Care plans and medical records stored in a database on the server.

[1065] Step 4:

[1066] Start a real-time video call

[1067] How it works: A user initiates a video call through the smart glasses interface.

[1068] Example: Pressing the "Start video call" button from the smart glasses interface.

[1069] Input: User's request to start a video call.

[1070] Data calculation: Session creation to start a video call.

[1071] Output: Start of video call session.

[1072] Step 5:

[1073] Enabling the device's camera and microphone

[1074] What it does: The device enables the camera and microphone.

[1075] Specific Action: Turns on the built-in camera and microphone device and starts capturing.

[1076] Input: Instruction to start a video call session.

[1077] Data Computing: Enabling camera and microphone devices.

[1078] Output: Start video and audio capture.

[1079] Step 6:

[1080] Server video call data processing

[1081] How it works: The server processes the data stream of the video call and manages the real-time communication.

[1082] Specific operation: The video stream is processed in real time using a WebRTC server and the stream is sent to the other device.

[1083] Input: Captured video and audio data.

[1084] Data Computing: Compression and transmission of data streams via WebRTC.

[1085] Output: Real-time video and audio communication.

[1086] Step 7:

[1087] Reminder settings

[1088] How it works: The user sets a reminder to take action at a specific time.

[1089] Example: Set a reminder to "take your morning medicine."

[1090] Input: Reminder information set by the user.

[1091] Data Calculation: Reminder information is organized and converted to JSON format.

[1092] Output: JSON data to send organized reminder information to the server.

[1093] Step 8:

[1094] Server reminder storage and notifications

[1095] How it works: The server stores the reminder information in a database and sends notifications to the device at the specified time.

[1096] Specific behavior: Saves reminder information in the database and sends a push notification at the specified time.

[1097] Input: JSON data of organized reminder information.

[1098] Data calculation: Save reminder information and trigger notifications at specified times.

[1099] Output: Send reminder notification.

[1100] Step 9:

[1101] Display reminders on your device

[1102] Action: The device displays the notification in its user interface.

[1103] Specific behavior: Display a reminder notification on the smart glasses display.

[1104] Input: Reminder notification sent from the server.

[1105] Data Calculation: Parsing notification data and converting it into a display format.

[1106] Output: Display reminder notification on the smart glasses display.

[1107] Step 10:

[1108] Submit a medical records request

[1109] How it works: A user uses the smart glasses interface to submit a request to view their medical records.

[1110] Example: Checking an elderly person's medical records for "high blood pressure" and "arthritis."

[1111] Input: User confirmation request.

[1112] Data operation: Generates an HTTP GET request to send the request.

[1113] Output: Sends to the server as an HTTP GET request.

[1114] Step 11:

[1115] Server data acquisition and transmission

[1116] Operation: The server retrieves medical records from the database and sends them to the device.

[1117] Specific behavior: Retrieves medical records from the database and sends them to the device in JSON format.

[1118] Input: User's medical records review request.

[1119] Data operations: Retrieving medical records and converting them to JSON format.

[1120] Output: Sends the medical record JSON data to the device.

[1121] Step 12:

[1122] Displaying medical records on the device

[1123] Action: The terminal displays the medical record in a user interface.

[1124] Specific operation: Displaying medical records on the smart glasses display.

[1125] Input: JSON data of medical records sent from the server.

[1126] Data Calculation: Analyzing data and converting it into a display format.

[1127] Output: Display medical records on the smart glasses display.

[1128] Step 13:

[1129] New staff training requests

[1130] How it works: A new employee requests training through the smart glasses interface.

[1131] Example: Submit a request for a training module.

[1132] Input: New staff training request.

[1133] Data operation: Generates an HTTP GET request to send the request.

[1134] Output: Sends to the server as an HTTP GET request.

[1135] Step 14:

[1136] Server training content transmission

[1137] Operation: The server sends training content to the device.

[1138] Specific operation: Sends training videos to the device in streaming format.

[1139] Input: New staff training request.

[1140] Data Computing: Acquiring training videos and converting them into a streaming format.

[1141] Output: Sends the training video to the device.

[1142] Step 15:

[1143] Training video playback on device

[1144] Action: The device plays a training video.

[1145] Specific operation: Playing a training video on the smart glasses display.

[1146] Input: The training video stream sent from the server.

[1147] Data operations: decoding and playing video streams.

[1148] Output: Playback of training videos to the smart glasses display.

[1149] Step 16:

[1150] Emergency button pressed

[1151] How it works: The user presses the emergency button on their smart glasses to send a notification.

[1152] Example: An elderly person presses an emergency button to send a notification.

[1153] Input: An input signal that the emergency button has been pressed.

[1154] Data Computation: Data construction for generating and sending emergency notifications.

[1155] Output: Sends an emergency notification to the server.

[1156] Step 17:

[1157] Server emergency response protocol activation

[1158] Actions: The server initiates emergency response protocols and sends alert notifications to relevant care staff.

[1159] What it does: Send alerts to all staff using a push notification system.

[1160] Input: Emergency notification sent from the device.

[1161] Data operations: Executing emergency response protocols and generating warning notifications.

[1162] Output: Send warning notification to all staff.

[1163] Step 18:

[1164] Emergency warning display on device

[1165] Action: The terminal receives the alert and displays it to the user.

[1166] Specific operation: A notification "Emergency has occurred!" is displayed on the smart glasses display.

[1167] Input: The warning notification sent by the server.

[1168] Data Calculation: Parsing alert data and converting it into a display format.

[1169] Output: Show a warning notification on the smart glasses display.

[1170] (Application example 2)

[1171] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1172] In elderly care, it is extremely important to monitor the elderly's emotions and location information in real time while ensuring their safety and responding quickly in emergencies. However, current systems lack the means to quickly notify and check the situation when an elderly person faces an emergency, making it difficult to respond effectively. In addition, there is a lack of systems that can monitor the elderly's emotional state and location information in real time and take action based on that information. Technological solutions to these problems are needed.

[1173] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a user interface means for a user to input data and check the elderly person's care plan and medical records, a communication means for transmitting the data to the server, a database means for the server to store and manage the data, a communication means for the server to transmit the data to the user interface means, a means having an emergency button for transmitting an emergency notification, a GPS means for acquiring and transmitting location information of the elderly person, an emotion recognition means for recognizing emotions, and a communication means for transmitting a notification based on data from the GPS means and the emotion recognition means. This allows an emergency notification to be transmitted quickly when the elderly person faces an emergency, and enables an effective response while monitoring the location information and emotional state in real time.

[1174] "User interface means" refers to the interactive screen display and operation means through which a user can input data and view the senior's care plan and medical records.

[1175] "Communication means" refers to means having a communication function for transmitting data to a server and receiving data from a server.

[1176] "Database means" refers to a database system for storing and managing data on a server.

[1177] An "emergency button" is a button-type operating device that an elderly person can press to send an emergency notification when they encounter an emergency.

[1178] "GPS Means" means a means utilizing the Global Positioning System (GPS) for obtaining and transmitting current location information.

[1179] The "emotion recognition means" is a means that uses emotion recognition technology to analyze the emotional state of the elderly and record and transmit the analysis results.

[1180] An "emergency notification" is notification information that is sent promptly to designated contacts when an emergency occurs.

[1181] "Real-time video call means" refers to means having the function of transmitting and receiving video and audio for conducting real-time video calls.

[1182] The present invention relates to a smart glasses-based care assistant system and security service for improving care for the elderly. Detailed embodiments of the present invention are described below.

[1183] System Configuration

[1184] The system consists of the following major components:

[1185] User Interface Means

[1186] communication means

[1187] Database Means

[1188] Emergency button

[1189] GPS means

[1190] emotion recognition means

[1191] Notification means

[1192] Server Means

[1193] Specific implementation methods

[1194] 1. User Interface Methods

[1195] Users input and view data through smart glasses or smartphones, including elderly care plans and medical records, with an intuitive and easy-to-use user interface.

[1196] 2. Means of communication

[1197] Data is sent from the device to the server and received from the server using a secure protocol.

[1198] 3. Database Means

[1199] The server stores the received data in a database and manages it appropriately, using a common database management system such as SQL.

[1200] 4. Emergency Button

[1201] If an elderly person faces an emergency, they can use the emergency button on their smart glasses or smartphone to send an emergency notification to designated contacts.

[1202] 5. GPS means

[1203] The system acquires the location information of the elderly person in real time and sends it to a server, allowing family members and caregivers to know the elderly person's location.

[1204] 6. Emotion recognition means

[1205] It analyzes the user's emotional state and detects stress, anxiety, etc. The emotion recognition means analyzes facial expressions using the camera in smart glasses or smartphones.

[1206] 7. Means of notification

[1207] Based on information obtained from emotion recognition and GPS means, necessary notifications are sent, including emergency notifications and notifications about emotional state.

[1208] 8. Server Means

[1209] The server is responsible for overall management and data processing, and also includes a video call module for processing real-time video calls.

[1210] Example of operation

[1211] For example, a user can enter an elderly person's care plan ("Daily Exercise" and "8 AM Medications") and check their medical records. If the elderly person encounters an emergency and presses the emergency button, their real-time location and current emotional state will be automatically sent to a designated contact. The contact who receives the emergency notification will then initiate a video call via smartphone or smart glasses to check on the elderly person's condition and provide assistance.

[1212] Examples of using generative AI models

[1213] When analyzing an elderly person's emotional state, generative AI models can be used to suggest "relaxing music" or "stress-reducing activities," with example prompts such as:

[1214] Prompt Sentence Examples

[1215] "The senior's panic button is pressed, which notifies family members of their real-time location and current emotional state (stress or anxiety). A video call is then automatically initiated, and the closest family member in their contact list receives the call."

[1216] From the above, a specific embodiment of the present invention has been presented and it can be seen how this system improves the care and security of the elderly.

[1217] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1218] Step 1:

[1219] Users input the elderly person's care plan and medical records through smart glasses or a smartphone. The input data is sent from the device to a server via a communication means. The server receives the data and stores it in a database for management. This ensures that the input information is securely stored on the server.

[1220] Step 2:

[1221] When needed, a user or caregiver sends a request to check the data stored on the server. The server retrieves the necessary information from the database and sends it to the terminal via a communication means. The terminal displays the received data on a user interface, allowing the user to check the elderly person's care plan and medical records.

[1222] Step 3:

[1223] When an elderly person faces an emergency, they press the emergency button on their smart glasses or smartphone. The device generates an emergency notification signal and sends it to the server via communication means. The server immediately receives the emergency notification and sends it to the registered emergency contacts.

[1224] Step 4:

[1225] At the same time, the GPS unit of the device acquires the current location information and transmits the location data to the server via the communication unit. The server processes the location information and transmits the location data together with the emergency notification. This allows the location of the elderly person to be monitored in real time and necessary measures to be taken promptly.

[1226] Step 5:

[1227] The device's emotion recognition means analyzes the elderly person's emotional state and acquires emotion data. For example, facial expressions are analyzed to detect signs of stress or anxiety. The acquired emotion data is then sent to a server via a communication means. The server analyzes the emotion data and generates an appropriate notification based on it.

[1228] Step 6:

[1229] The server then sends emergency notifications to emergency contacts along with warnings and advice based on the emotion data, such as "An elderly person is in a state of stress, please take immediate action."

[1230] Step 7:

[1231] The emergency contact who receives the emergency notification can use the video call function to contact the elderly person in real time. The server manages the video call session, activates the device's camera and microphone, and starts the video call. This allows the emergency contact to directly check the elderly person's current situation and ensure their safety.

[1232] Examples of prompt statements

[1233] "The senior's panic button is pressed, which notifies family members of their real-time location and current emotional state (stress or anxiety). A video call is then automatically initiated, and the closest family member in their contact list receives the call."

[1234] Through these steps, the system is able to comprehensively improve the care and security of the elderly.

[1235] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1236] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1237] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1238] [Third embodiment]

[1239] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1240] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1241] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1242] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1243] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1244] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1245] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1246] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1247] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1248] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1249] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1250] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1251] The present invention relates to a smart glasses-based care assistant system for improving elderly care, providing a system for inputting user data, managing elderly care plans and medical records, real-time video calling, reminder notifications, training new staff, and responding to emergency care. Specific embodiments for implementing the present invention are described in detail below.

[1252] User Data Input

[1253] Users input the elderly person's care plan and medical records through the smart glasses interface. The device then sends the input data to a server, which analyzes the received data and stores and manages it in a database.

[1254] For example, a user inputs the care plan for elderly person Tanaka, such as "daily exercise" and "medicine at 8 am," and also inputs medical records such as "high blood pressure" and "arthritis." The device sends the input data to the server, which then stores the data in a database.

[1255] Real-time video calling

[1256] The user uses the smart glasses interface to make real-time video calls with remote family members or medical staff. The device activates the camera and microphone and initiates the video call session. The server processes the video call data stream and manages real-time communication.

[1257] For example, if a user wants to start a video call with Tanaka's daughter, the smart glasses will turn on the camera and microphone and transmit the video stream. The server processes this data and starts the call.

[1258] Reminders

[1259] Users can set reminders to prompt the elderly to perform an action at a specific time. The device sends the reminder information to the server, which stores and manages the reminder information in a database. When the reminder time arrives, the server sends a reminder notification to the device, which then displays the notification on the user interface.

[1260] For example, if Tanaka sets a reminder to take her morning medicine, the server stores that information in a database and sends a notification to the device when the specified time arrives. The device then displays a notification saying, "It's time to take your morning medicine."

[1261] Medical record review

[1262] The user can check the elderly person's medical records using the interface of the smart glasses. The device sends the user's request to the server, which retrieves the medical records from the database and sends them to the device. The device then displays the retrieved medical records on the user interface.

[1263] For example, if a user checks Mr. Tanaka's medical records, the server retrieves information about "high blood pressure" and "arthritis" from the database and sends it to the terminal, which then displays that information.

[1264] Training new staff

[1265] New staff can receive training using smart glasses. The terminal sends a request for a training module to the server, and the server sends the training content to the terminal, which then plays it and provides it to the user.

[1266] For example, when a new staff member, Mr. Yamada, starts training, the terminal requests a training module from the server, and the server sends the training video to the terminal, which then plays the video.

[1267] Rapid response to emergency care

[1268] In the event of an emergency, the user can press the emergency button on the smart glasses to send a notification. The device will then send the emergency notification to the server, which will trigger the emergency response protocol and send an alert to the relevant care staff. The device will receive the alert and display it immediately.

[1269] For example, if Tanaka presses the emergency button, the device sends an emergency notification to the server, which then alerts all staff. The device displays, "Emergency! Please respond immediately."

[1270] As described above, the present invention improves the quality of care for the elderly, while enabling efficient support and emergency response by staff, thereby improving the quality of life for the elderly.

[1271] The processing flow will be explained below.

[1272] OK, so let's go into more detail and explain what the program does, step by step.

[1273] User Data Input

[1274] Step 1:

[1275] Users enter the elderly person's care plan and medical records through the smart glasses interface.

[1276] Specifically, data is input using a keyboard, voice input, a touch panel, etc.

[1277] Step 2:

[1278] The terminal transmits the input data to the server.

[1279] Specifically, the input data is converted into packets and sent to the server using a communication protocol.

[1280] Step 3:

[1281] The server analyzes the received data and stores it in a database.

[1282] Specifically, the received packet is analyzed and the data is stored in the appropriate database field.

[1283] Real-time video calling

[1284] Step 1:

[1285] The user selects the person with whom they want to start a video call using the interface of the smart glasses.

[1286] Specifically, you select the ID of the person you want to call using the touch panel or voice commands.

[1287] Step 2:

[1288] The terminal sends a request for a call session to the server.

[1289] Specifically, the server sends a request packet containing the ID of the selected call partner to the server.

[1290] Step 3:

[1291] The server processes the call request and establishes the call session.

[1292] Specifically, a new session ID is generated and notified to both terminals.

[1293] Step 4:

[1294] The device will enable the camera and microphone and start the video stream.

[1295] Specifically, the hardware camera and microphone are turned on, and video and audio data are sent and received as streams.

[1296] Reminders

[1297] Step 1:

[1298] The user inputs the reminder content and time through the smart glasses interface.

[1299] Specifically, the reminder content and time can be set using the touch panel or voice input.

[1300] Step 2:

[1301] The terminal transmits the reminder information to the server.

[1302] Specifically, the set reminder information is packetized and sent to the server.

[1303] Step 3:

[1304] The server stores the reminder information and monitors the reminder time.

[1305] Specifically, the system stores the reminder information in a database and waits for the specified time.

[1306] Step 4:

[1307] The server sends a notification to the terminal when the reminder time arrives.

[1308] Specifically, the system refers to the saved reminder information and sends a notification packet to the terminal when the reminder time is reached.

[1309] Step 5:

[1310] The terminal receives the reminder notification and displays it to the user.

[1311] Specifically, the received notification packet is analyzed and the reminder content is displayed on the user interface.

[1312] Medical record review

[1313] Step 1:

[1314] The user submits a request to review medical records through the smart glasses interface.

[1315] Specifically, the user selects the medical record item they wish to check using the touch panel or voice.

[1316] Step 2:

[1317] The terminal sends a confirmation request to the server.

[1318] Specifically, the operation involves sending a request packet containing the selected medical record items to the server.

[1319] Step 3:

[1320] The server retrieves the relevant medical records from the database.

[1321] Specifically, the system analyzes the received request and extracts the relevant medical record data from the database.

[1322] Step 4:

[1323] The server transmits the acquired medical record data to the terminal.

[1324] Specifically, the extracted medical record data is converted into packets and sent to the terminal.

[1325] Step 5:

[1326] The terminal displays the received medical record data on a user interface.

[1327] Specifically, the received data packets are analyzed and the medical records are displayed on the user interface.

[1328] Training new staff

[1329] Step 1:

[1330] The user selects a training module from the smart glasses interface.

[1331] Specifically, the training menu is selected using the touch panel or voice.

[1332] Step 2:

[1333] The terminal sends a request for a training module to the server.

[1334] Specifically, the ID of the selected training module is sent to the server.

[1335] Step 3:

[1336] The server obtains the training content and transmits it to the terminal.

[1337] Specifically, the relevant training content is retrieved from the database and sent to the terminal.

[1338] Step 4:

[1339] The terminal plays back the received training content.

[1340] Specifically, the received training video or manual is played on the user interface.

[1341] Rapid response to emergency care

[1342] Step 1:

[1343] The user presses the emergency button on the smart glasses to notify them of an emergency.

[1344] Specific actions include physically pressing an emergency button or issuing a specific voice command.

[1345] Step 2:

[1346] The terminal sends an emergency notification to the server.

[1347] Specifically, it detects that the emergency button has been pressed and sends an emergency notification packet to the server.

[1348] Step 3:

[1349] The server receives the emergency notification and executes the emergency response protocol.

[1350] Specifically, it analyzes the received emergency notification and initiates a response according to the protocol.

[1351] Step 4:

[1352] The server sends an alert notification to the relevant care staff.

[1353] A specific operation is to send an alert notification packet to the device of the necessary care staff.

[1354] Step 5:

[1355] The terminal receives the warning notification and displays it to the user.

[1356] Specifically, the device analyzes the warning notification received and displays "Emergency situation! Please respond immediately" on the user interface.

[1357] The above is the specific processing flow of each function detailed for each process.

[1358] Example 1

[1359] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1360] To improve elderly care, a system for efficient data management and emergency response is needed. However, the current system does not centrally manage care plans, medical record entry and review, real-time video calls, reminder notifications, new staff training, and emergency care responses, making it difficult to provide comprehensive support.

[1361] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1362] In this invention, the server includes a user interface means for a user to input data and check care plans and medical records, a communication means for transmitting the data to the server, a database means for the server to store and manage the data, a communication means for the server to transmit the data to the user interface means, a sensor means for transmitting an emergency notification to the server when a user presses an emergency button, and a server means for the server to execute emergency response protocols and send alerts to relevant staff. This allows management of elderly care plans and medical records, real-time video calls, reminder notifications, training of new staff, and emergency care responses to be performed in a unified manner, thereby improving the quality and efficiency of care.

[1363] "User interface means" refers to hardware and software for displaying data entered by the user and notifications from the system.

[1364] "Communication means" refers to the network interface and communication protocol for sending and receiving data.

[1365] "Database means" refers to a storage system or database management system for storing and managing data.

[1366] "Sensor means" refers to a device or module that detects user input or changes in the environment and acquires data.

[1367] "Server means" refers to a server system for processing, storing, managing, and communicating data.

[1368] "Emergency response protocols" are response procedures and steps that the system automatically executes in the event of an emergency.

[1369] A "video stream" is a flow of video data that is sent and received in real time.

[1370] A "reminder" is an alert or notification that notifies a user of a specific action or time.

[1371] These definitions provide a clearer understanding of the role and function of each element included in the claims.

[1372] The present invention relates to a smart glasses-based care assistant system for improving elderly care, providing a system for user data entry, managing elderly care plans and medical records, real-time video calling, reminder notifications, training new staff, and emergency care response. Specific embodiments for implementing the present invention are described in detail below.

[1373] User Data Input

[1374] Users can input care plans and medical records through the smart glasses interface. The data is then sent to a server via the device. The server analyzes the received data and stores and manages it in a database. This allows for centralized management of elderly care plans and medical records.

[1375] Specifically, users can use the smart glasses' voice recognition system or touchpad to input instructions such as "Take today's medicine at 8 o'clock." The device converts this voice data into text and sends it to the server. The server then converts this data into JSON format and stores it in a database such as MySQL.

[1376] Real-time video calling

[1377] Users can use the smart glasses interface to make real-time video calls with remote family members or medical staff. The device activates the camera and microphone and initiates the video call session. The server processes the video data stream and manages the communication in real time.

[1378] For example, if you say "start a video call" to your smart glasses, the device will turn on the camera and microphone and start a call session using WebRTC. The server manages this and ensures the call proceeds smoothly.

[1379] Reminders

[1380] Users can set reminders to prompt the elderly to perform specific actions at specific times. The device sends the reminder information to the server, which stores and manages the reminder information in a database. At the set time, the server sends a reminder notification to the device, which then displays the notification on the user interface.

[1381] Specifically, when someone types "Set a reminder to take my medicine at 8 a.m.", the device converts this into text and sends it to the server. The server stores the reminder information in a database and sends a notification to the device at the specified time. The device then displays "It's time to take your morning medicine."

[1382] Medical record review

[1383] The user can check their medical records through the interface of the smart glasses. A request is sent from the device to the server, which retrieves the medical records from the database and sends them to the device. The device then displays the medical records on the user interface.

[1384] For example, if you say "Check Tanaka-san's medical records," the server will retrieve information on "high blood pressure" and "arthritis" from the database and send it to the device, which will then display it on its screen.

[1385] Training new staff

[1386] New staff can receive training using smart glasses. The device sends a request for a training module to the server, and the server sends the training content to the device. The device can then play the provided content and display it to the user.

[1387] Specifically, when a new staff member says "Start training," the terminal sends this request to the server, which then sends the training video to the terminal, which then plays the video.

[1388] Rapid response to emergency care

[1389] In the event of an emergency, users can press the emergency button on the smart glasses to send an emergency notification. The device will then send the emergency notification to the server, which will trigger the emergency response protocol to alert relevant staff. The device will then display the alert.

[1390] For example, if Tanaka presses the emergency button, the device sends an emergency notification to the server, which then executes emergency response protocols and alerts all staff. The device displays, "Emergency! Please respond immediately."

[1391] As described above, the present invention improves the quality of care for the elderly, while enabling efficient support and emergency response by staff, thereby improving the quality of life for the elderly.

[1392] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1393] Step 1:

[1394] The user inputs data through the smart glasses interface. For example, they can enter instructions such as "Take today's medicine at 8 o'clock" using a voice recognition system or touchpad. The input in this step is voice or text data, and the device receives this input.

[1395] Step 2:

[1396] The device converts the received voice data into text data. It uses voice recognition technology to analyze the user's voice input and outputs it as text. The converted text data is obtained. This specific operation is performed using the voice recognition library installed on the device.

[1397] Step 3:

[1398] The terminal transmits text data to the server. The data is transmitted to the server via the internet via the communication means. The input of this step is text data, and the output is data transmission to the server.

[1399] Step 4:

[1400] The server parses the received text data, converts it into the required data format (for example, JSON format), and saves the converted data in a database. The input is text data, and the output is saving the converted data in a database.

[1401] Step 5:

[1402] The server sets reminder information and stores it until the specified time. When the specified time arrives, it sends a reminder notification to the device. The input is the saved reminder information, and the output is to send a reminder notification to the device when the time arrives.

[1403] Step 6:

[1404] The device displays the reminder notification received from the server on the user interface and informs the user of the reminder content. Specifically, the device displays "It's time to take your morning medicine" on the display and also notifies the user by voice. The input is the reminder notification from the server, and the output is the display on the user interface and the voice notification.

[1405] Step 7:

[1406] When a user presses the emergency button, the terminal is instructed to notify the user of an emergency. The input is the user's operation of the emergency button, and the terminal captures this operation.

[1407] Step 8:

[1408] The terminal sends an emergency notification to the server. Emergency status data is sent to the server using a communication method. The input is the emergency button operation event, and the output is the transmission of emergency notification data to the server.

[1409] Step 9:

[1410] When the server receives an emergency notification, it activates the emergency response protocol and sends an alert to the relevant staff. Specifically, the server sends an emergency email or SMS to all registered staff. The input is the emergency notification data, and the output is sending an alert notification to the staff.

[1411] Step 10:

[1412] The relevant staff receives the alert and initiates a response based on the information. The input is an emergency alert notification from the server, and the output is the execution of emergency response actions.

[1413] Through these steps, the system can efficiently support the various needs of elderly care.

[1414] (Application example 1)

[1415] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1416] Conventional elderly care systems have had problems such as the time-consuming task of managing elderly care plans and medical records, and the difficulty of sharing care information in real time. Furthermore, while rapid response is required in the event of an emergency, conventional systems do not adequately address this need. Furthermore, security staff lacked the efficiency and immediacy required for on-site monitoring and emergency response, making it difficult to ensure smooth work progress through reminder notifications.

[1417] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1418] In this invention, the server includes a user interface means for a user to input data and check the elderly person's care plan and medical records, a communication means for transmitting the data to the server, a database means for storing and managing the data, a communication means for transmitting the data to the user interface means, a communication means for transmitting and receiving video streams, a communication means for issuing emergency notifications, and a display means for displaying reminder notifications, thereby enabling more efficient information management in elderly care, real-time information sharing, faster emergency response, and improved work efficiency of security staff.

[1419] "User interface means" refers to an interface through which a user inputs data and confirms and manipulates information.

[1420] "Communication means" refers to the technical means used to send and receive data and information between servers.

[1421] The "database means" refers to a data storage system for saving and managing input data and its management function.

[1422] The "server means" is a server system for processing and managing data.

[1423] A "video stream" is a continuous flow of video data that is sent and received in real time.

[1424] "Emergency notifications" are messages or alerts that are sent to quickly warn relevant parties when an emergency occurs.

[1425] The "display means" refers to a display or monitor for visually presenting information to the user.

[1426] A "reminder notification" is a notification that prompts the user to take action at a specific time or under specific conditions.

[1427] "Real-time video calling" is a call format in which both parties communicate with each other in real time.

[1428] "Communication means for issuing emergency notifications" means the technical means for sending immediate warning messages in the event of an emergency.

[1429] The present invention relates to a smart glasses-based assistant system for security and elderly care, which includes a user interface means, a communication means, a database means, a server means, and a display means.

[1430] System Overview

[1431] The system consists of the following main components:

[1432] 1. User interface means: The smart glasses' display and microphone allow users to input data, view information, and manipulate it.

[1433] 2. Communication Method: The technical means for transmitting data and video streams between the smart glasses and the server, including Wi-Fi and cellular communication.

[1434] 3. Database Means: A cloud storage system for storing and managing user-entered data. Specifically, this includes Google Cloud Firestore.

[1435] 4. Server infrastructure: Server infrastructure that processes and manages data. For example, Google Cloud or AWS is used.

[1436] 5. Display means: A means for visually presenting information to the user using the smart glasses display.

[1437] Program Overview

[1438] The process of a user wearing smart glasses and entering data, making video calls, setting reminders, and sending emergency notifications through the user interface is described.

[1439] Entering and verifying data

[1440] The user inputs data using the smart glasses' display and voice input. For example, in elderly care, the user can input care plans and medical records. The input data is sent to a server via a communication means and stored in a database means. The stored data can be viewed at any time via a user interface means.

[1441] Real-time video calling

[1442] A user can use the interface of the smart glasses to make a real-time video call with another user in a remote location (e.g., a security center or a family member). The device captures the video stream using a camera and microphone and transmits it to a server through a communication means. The server means processes the video stream in real time and manages the call function.

[1443] Reminders

[1444] A user can set a reminder for a task to be performed at a specific time. The reminder information is sent to the server via the communication means and stored in the database means. When the specified time arrives, the server means sends a notification to the terminal, and the reminder is displayed to the user via the display means.

[1445] Sending emergency notifications

[1446] When a user faces an emergency, they can press the emergency button on the smart glasses to issue an emergency notification. The emergency notification is sent to the server via the communication means, and the server means activates an emergency response protocol and sends the emergency notification to all related terminals. A prompt response is then made via the display means.

[1447] Specific examples

[1448] When a security staff member discovers a suspicious individual while on patrol, an emergency notification is sent from the smart glasses. The server immediately sends an alert to multiple staff members, enabling a swift response on-site.

[1449] At elderly care facilities, care staff input care plans into smart glasses, and the data is stored on a server, allowing family members to remotely check the progress of care in real time.

[1450] Prompt Sentence Examples

[1451] "Please outline an application that uses smart glasses to enable security personnel to initiate video calls, send emergency notifications, and display reminder notifications in real time. Please explain the specific features."

[1452] Overall, the system will streamline care and security management and improve real-time response capabilities.

[1453] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1454] Step 1:

[1455] The user wears the smart glasses and inputs data through the user interface, such as entering care plans and medical records. The input data is sent to a server via a communication means. Input is performed using the smart glasses' display and voice input function, and the data is sent to the server as output.

[1456] Step 2:

[1457] The server stores and manages the received data using database means. When the input data reaches the server, it is stored in the database. The server uses Google Cloud Firestore to store the data. After the data is stored, the database is updated.

[1458] Step 3:

[1459] The user checks the entered care plan and medical records through the interface of the smart glasses. A request is sent to access the data stored on the server. The server receives this request, retrieves the relevant data from the database, and sends it back to the smart glasses via a communication method. As a result, the user can view the data on the display.

[1460] Step 4:

[1461] Users use the smart glasses interface to make real-time video calls with other users in remote locations. The smart glasses' camera and microphone capture video and audio and send them to a server via a communication method. The server uses Twilio's API to manage the video call session and send the video stream to the other user's device. This enables two-way communication of video and audio in real time.

[1462] Step 5:

[1463] The user sets a reminder for a specific time, for example, inputting a reminder for rounds or taking medicine. The reminder information is sent to the server via the interface of the smart glasses using a communication means. The server receives this information, stores it in a database, and then triggers a reminder notification based on the specified time. The notification is sent back to the smart glasses via the communication means and displayed to the user via the display means. As a result, the user receives the notification at the specified time.

[1464] Step 6:

[1465] When a user faces an emergency, they press the emergency button on the smart glasses to send an emergency notification. Input is made by pressing the button, and the notification is sent to the server using a communication method. The server receives the emergency notification, activates a response protocol, and sends the emergency notification to all related devices. This allows a notification of an "emergency" to be displayed on multiple devices, enabling a quick response.

[1466] This allows the system to enable users to effectively manage data, communicate and respond quickly to emergencies.

[1467] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1468] The present invention relates to a smart glasses-based care assistant system for improving elderly care, which provides a system for inputting user data, managing elderly care plans and medical records, real-time video calling, reminder notifications, training new staff, responding to emergency care, and emotion recognition using an emotion engine. The following describes in detail the embodiments of the present invention.

[1469] User Data Input

[1470] Users input the elderly person's care plan and medical records through the smart glasses interface. The device then sends the input data to a server, which analyzes the data and stores it in a database for management.

[1471] For example, a user inputs an elderly person's care plan, "Daily Exercises" and "Medicine at 8 AM," and inputs medical records "High Blood Pressure" and "Arthritis." The terminal sends the input data to the server, which stores the data in a database.

[1472] Real-time video calling

[1473] The user uses the smart glasses interface to make real-time video calls with remote family members or medical staff. The device activates the camera and microphone and initiates the video call session. The server processes the video call data stream and manages real-time communication.

[1474] For example, if a user wants to start a video call with the elderly person's daughter, the smart glasses will turn on the camera and microphone and transmit the video stream, and the server will process this data and start the call.

[1475] Reminders

[1476] Users can set reminders to prompt the elderly to take action at specific times. The device sends the reminder information to the server, which stores the information in a database and sends a notification to the device when the reminder time arrives. The device then displays the notification on the user interface.

[1477] For example, when a user sets a reminder for an elderly person to take their morning medication, the server stores that information in a database and sends a notification to the device at the specified time, which displays a message saying, "It's time to take your morning medication."

[1478] Medical record review

[1479] The user can check the elderly person's medical records using the smart glasses interface. The device sends the user's request to the server, which retrieves the medical records from the database and sends them to the device. The device then displays the retrieved medical records on the user interface.

[1480] For example, if a user checks the medical records of an elderly person, the server retrieves information about "high blood pressure" and "arthritis" from the database and sends it to the terminal, which then displays the information.

[1481] Training new staff

[1482] New staff can receive training using smart glasses. The device sends a request for a training module to the server, and the server sends the training content to the device, which then plays it and provides it to the user.

[1483] For example, when a new staff member starts training, the terminal requests a training module from the server, and the server sends a training video to the terminal, which then plays the video.

[1484] Rapid response to emergency care

[1485] In the event of an emergency, the user can press the emergency button on the smart glasses to send a notification. The device will then send the emergency notification to the server, which will trigger the emergency response protocol and send an alert notification to the relevant care staff. The device will then receive the alert and display it to the user.

[1486] For example, if an elderly person presses the emergency button, the device will send an emergency notification to the server, which will then alert all staff. The device will display "Emergency! Please respond immediately."

[1487] Emotion recognition by emotion engine

[1488] The emotion engine means recognizes the user's emotions and transmits them to the server, which stores and analyzes the emotion data, and provides feedback to the user based on the emotion data.

[1489] For example, if the emotion engine detects that an elderly person is in a stressful state, the device will send this emotion data to the server, which will analyze the data and provide feedback to the user such as "listen to some relaxing music."

[1490] Care plan coordination

[1491] Based on the emotion data recognized by the emotion engine, the server adjusts the care plan, which is then notified to the user and implemented.

[1492] For example, if the emotion engine analyzes the emotional data of an elderly person and determines that they are under high stress, the server will adjust the care plan to a "relaxation exercise" and notify the device to carry it out.

[1493] Applying emotion recognition to video calls

[1494] The emotion engine means recognizes the user's emotions during a real-time video call and notifies the video call partner of the emotion data. Based on the emotion data, a warning or recommended action can be displayed to the video call partner.

[1495] For example, if the emotion engine recognizes that an elderly person is becoming increasingly stressed during a video call, the caller will see a message saying, "You are feeling stressed. Please change the topic to help you relax."

[1496] As described above, the present invention improves the quality of care for the elderly while providing efficient support and emergency response from staff, and by combining it with an emotion engine, more detailed care and support becomes possible.

[1497] The processing flow will be explained below.

[1498] User Data Input

[1499] Step 1:

[1500] Users enter the elderly person's care plan and medical records through the smart glasses interface.

[1501] Example: A user enters an elderly person's care plan, "Daily Exercises" and "8 AM Medication," and medical records, including "High Blood Pressure" and "Arthritis."

[1502] Step 2:

[1503] The terminal converts the input data into packets and transmits them to the server.

[1504] Step 3:

[1505] The server analyzes the received data and stores it in a database.

[1506] A database stores care plans and medical records.

[1507] Real-time video calling

[1508] Step 1:

[1509] The user uses the interface of the smart glasses to select who to start a video call with.

[1510] Example: A user selects the ID of an elderly person's daughter.

[1511] Step 2:

[1512] The terminal sends a call request packet to the server.

[1513] The request containing the submitted ID arrives at the server.

[1514] Step 3:

[1515] The server processes the call request and generates a new session ID.

[1516] The generated session ID is notified to both terminals.

[1517] Step 4:

[1518] The device will enable the camera and microphone and start the video stream.

[1519] A video stream is sent from the elderly person's smart glasses and displayed on his daughter's device.

[1520] Reminders

[1521] Step 1:

[1522] The user inputs the reminder content and time through the smart glasses interface.

[1523] Example: A user sets a reminder for an elderly person to take their morning medication at 08:00.

[1524] Step 2:

[1525] The terminal converts the reminder information into packets and transmits them to the server.

[1526] Step 3:

[1527] The server stores the received reminder information in a database and monitors the specified time.

[1528] Step 4:

[1529] When the reminder time arrives, the server sends a notification packet to the terminal.

[1530] Step 5:

[1531] The terminal analyzes the received notification and displays it on the user interface.

[1532] Example: The device displays "Time to take your morning medicine."

[1533] Medical record review

[1534] Step 1:

[1535] The user enters a request to review medical records through the smart glasses interface.

[1536] Example: A user sends a request to review an elderly person's medical records.

[1537] Step 2:

[1538] The terminal sends a request packet to the server.

[1539] Step 3:

[1540] The server retrieves the relevant medical records from the database.

[1541] The server extracts information on "high blood pressure" and "arthritis" from the database.

[1542] Step 4:

[1543] The server converts the acquired medical record data into packets and transmits them to the terminal.

[1544] Step 5:

[1545] The terminal analyzes the received data and displays it on a user interface.

[1546] Example: Medical records for "high blood pressure" and "arthritis" are displayed.

[1547] Training new staff

[1548] Step 1:

[1549] The user selects a training module from the smart glasses interface.

[1550] Example: New staff members select a training menu.

[1551] Step 2:

[1552] The terminal sends a request packet for the training module to the server.

[1553] Step 3:

[1554] The server retrieves training content from the database, converts it into packets, and transmits them to the terminal.

[1555] Step 4:

[1556] The terminal plays back the received training content.

[1557] Example: A device plays a training video that new staff members watch.

[1558] Rapid response to emergency care

[1559] Step 1:

[1560] The user presses the emergency button on the smart glasses to send an emergency notification.

[1561] Example: An elderly person presses the emergency button.

[1562] Step 2:

[1563] The terminal transmits an emergency notification packet to the server.

[1564] Step 3:

[1565] The server executes emergency response protocols and sends emergency notifications to all staff.

[1566] Step 4:

[1567] The terminal analyzes the received emergency notification and displays it on the user interface.

[1568] Example: "Emergency! Please respond immediately."

[1569] Emotion recognition by emotion engine

[1570] Step 1:

[1571] The terminal analyzes the user's emotions using an emotion engine and generates emotion data.

[1572] Example: The stress state of the elderly is recognized by the emotion engine.

[1573] Step 2:

[1574] The terminal converts the generated emotion data into packets and transmits them to the server.

[1575] Step 3:

[1576] The server stores the received emotion data in a database and analyzes it.

[1577] Step 4:

[1578] The server generates feedback based on the emotion data and transmits it to the terminal.

[1579] Step 5:

[1580] The terminal analyzes the received feedback and displays it on the user interface.

[1581] Example: "Listen to some relaxing music" is displayed to the user.

[1582] Care plan coordination

[1583] Step 1:

[1584] The server analyzes the emotion data recognized by the emotion engine and adjusts the care plan.

[1585] Example: If stress levels are high, change the care plan to "relaxation exercises."

[1586] Step 2:

[1587] The server converts the adjusted care plan into packets and transmits them to the terminal.

[1588] Step 3:

[1589] The terminal displays the received coordinated care plan on a user interface and executes it.

[1590] Applying emotion recognition to video calls

[1591] Step 1:

[1592] The device analyzes the user's emotions using an emotion engine during a video call.

[1593] Example: The emotion engine recognizes the stress state of elderly people during video calls.

[1594] Step 2:

[1595] The terminal converts the recognized emotion data into packets and transmits them to the terminal of the other party.

[1596] Step 3:

[1597] The other party's device analyzes the received emotion data and displays warnings and recommended actions on the user interface.

[1598] Example: Displaying "You are feeling stressed, please change the topic to relax."

[1599] The above is the specific processing flow of each function, including the emotion engine.

[1600] Example 2

[1601] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1602] In elderly care, managing care plans and medical records is cumbersome, and remote status checks and communication are difficult. Furthermore, reminder functions and emergency responses are insufficient, and personalized care based on emotion recognition is not provided. Therefore, there is a need to improve the quality of care and provide efficient support for staff.

[1603] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an interface means for a user to input data and check the elderly person's care plan and medical records, a communication means for transmitting the data to the server, an electronic database means for the server to store and manage the data, a communication means for the server to transmit the data to the interface means, an interface means for conducting video calls in real time, a communication means for transmitting and receiving the video stream, a server means for processing and managing the video stream, an emotion recognition means for recognizing the elderly person's emotion, and an emergency notification means for notifying the elderly person of an emergency. This enables efficient management of the elderly person's care plan and medical records, remote care via real-time video calls, appropriate reminder notifications and emergency responses, and detailed care through emotion recognition.

[1604] "User" refers to a person who uses the system to input and check elderly care plans and medical records.

[1605] "Interface means" refers to devices and screens that allow users to input data and view care plans and medical records.

[1606] "Communication means" refers to a function or device for transmitting data to a server and receiving data from the server.

[1607] "Electronic database means" refers to a database system in which the server stores and manages data.

[1608] The "server means" is a server device that manages, stores, and transmits data, and processes and manages video streams.

[1609] A "video stream" is a continuous stream of video data sent and received during a video call.

[1610] The "emotion recognition means" is a system for analyzing and recognizing the emotions of elderly people in real time.

[1611] An "emergency notification means" is a function or device for promptly notifying relevant parties when an emergency occurs.

[1612] "Reminder" is a notification function that prompts elderly people to take specific actions based on times and conditions set by the user.

[1613] "Training tools" are support functions and devices that enable new staff to learn the knowledge and skills necessary to perform their jobs properly.

[1614] The present invention relates to a smart glasses-based care assistant system for supporting elderly care, which provides user data input, real-time video calling, reminder notifications, medical record review, new staff training, emergency care response, and emotion recognition with an emotion engine.

[1615] Hardware and software used

[1616] 1. Smart Glasses:

[1617] Hardware: camera, microphone, display, emergency button

[1618] Software: Interface app, emotion engine

[1619] 2. Server:

[1620] Hardware: High-performance server machine

[1621] Software: Database management system (MySQL), communication management system (WebRTC server), emotion recognition analysis system

[1622] 3. Means of communication:

[1623] Hardware: Communication modules (Wi-Fi, 4G / 5G modules)

[1624] Software: HTTP / HTTPS protocol, JSON format data transmission and reception

[1625] Program processing explanation

[1626] User Data Input

[1627] Users input care plans and medical records through the smart glasses interface, such as "daily exercise," "8 AM medication," "high blood pressure," and "arthritis." The device then sends this data to a server, which analyzes it and stores it in a database.

[1628] Real-time video calling

[1629] Users use smart glasses to make real-time video calls with remote family members or medical staff. The smart glasses are equipped with a camera and microphone, and a video call is initiated by activating these devices. The server processes the data stream of the video call and manages real-time communication. For example, when a user initiates a video call with an elderly person's daughter, the smart glasses turn on the camera and microphone and send the video stream to the server.

[1630] Reminders

[1631] Users can set reminders to prompt them to take action at specific times. The set reminder information is sent to the server via the device, and the server stores this information in a database. When the specified time arrives, the server sends a reminder notification to the device, and the device displays this notification on its user interface. For example, if a user sets a reminder to "take their morning medicine," a notification will be displayed on the device when the specified time arrives.

[1632] Medical record review

[1633] The user uses the smart glasses interface to send a request to check the elderly person's medical records. The device then sends this request to the server, which retrieves the medical records from the database and sends them to the device. The retrieved medical records are then displayed on the smart glasses' display. For example, if you want to check the medical records for "high blood pressure" or "arthritis," the server retrieves the data and the device displays it.

[1634] Training new staff

[1635] The new staff member uses the smart glasses to request a training module, and the server responds by sending the training content to the terminal. The terminal plays the training video and provides it to the new staff member. For example, when the new staff member starts training, the video content is streamed to the smart glasses.

[1636] Rapid response to emergency care

[1637] When an emergency occurs, the user presses the emergency button on the smart glasses to send a notification. The device sends the emergency notification to the server, which then activates the emergency response protocol and sends an alert notification to the relevant care staff. The device receives the alert and displays it to the user. For example, if an elderly person presses the emergency button, a notification saying "Emergency has occurred!" is immediately displayed.

[1638] Emotion engine recognizes emotions and adjusts care plans

[1639] The emotion engine recognizes the elderly person's emotions in real time and sends the emotional data to the server. The server analyzes this data and adjusts the care plan as needed. For example, if high stress is recognized, the server will change the care plan to "relaxation exercises" and send a notification to the device. Emotional data during a video call is also notified to the other party, allowing appropriate action to be taken.

[1640] Prompt Sentence Examples

[1641] Below is an example of a prompt sentence to input to the generative AI model.

[1642] "Enter your care plan for today."

[1643] "Start a video call with the elderly person's daughter."

[1644] "Set a reminder to take your medicine at 8am."

[1645] "Check the elderly person's medical records."

[1646] "Play the training video for new staff."

[1647] "Please press the emergency button."

[1648] "Please send us the emotional data of the elderly."

[1649] Using the procedures and prompts provided above, a specific system for improving care for seniors can be implemented. Embodiments of this system can efficiently and effectively manage seniors' health and adjust their care plans.

[1650] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1651] Step 1:

[1652] User Data Entry

[1653] How it works: Users enter care plans and medical records through the smart glasses interface.

[1654] Example: Enter care plan items such as "Daily exercise" and "8am medication" and medical records such as "High blood pressure" and "Arthritis."

[1655] Input: Care plans and medical records entered by the user through the smart glasses interface.

[1656] Data processing: The terminal organizes this data and converts it into JSON format.

[1657] Output: JSON data for sending organized data to the server.

[1658] Step 2:

[1659] Data transmission from the device

[1660] Operation: The terminal sends the entered data to the server.

[1661] Specific operation: Sends JSON format data obtained from the interface as an HTTP POST request.

[1662] Input: Organized care plan and medical record JSON data.

[1663] Data computation: Generates an HTTP request to send data.

[1664] Output: The data is sent to the server as an HTTP POST request.

[1665] Step 3:

[1666] Data analysis and storage on the server

[1667] How it works: The server parses the data sent and stores it in a database.

[1668] Specific operation: Parses the received JSON data and stores it in a MySQL database as a care plan and medical record.

[1669] Input: JSON data sent to the server as an HTTP POST request.

[1670] Data Computing: Running SQL queries to analyze data and store it in a database.

[1671] Output: Care plans and medical records stored in a database on the server.

[1672] Step 4:

[1673] Start a real-time video call

[1674] How it works: A user initiates a video call through the smart glasses interface.

[1675] Example: Pressing the "Start video call" button from the smart glasses interface.

[1676] Input: User's request to start a video call.

[1677] Data calculation: Session creation to start a video call.

[1678] Output: Start of video call session.

[1679] Step 5:

[1680] Enabling the device's camera and microphone

[1681] What it does: The device enables the camera and microphone.

[1682] Specific Action: Turns on the built-in camera and microphone device and starts capturing.

[1683] Input: Instruction to start a video call session.

[1684] Data Computing: Enabling camera and microphone devices.

[1685] Output: Start video and audio capture.

[1686] Step 6:

[1687] Server video call data processing

[1688] How it works: The server processes the data stream of the video call and manages the real-time communication.

[1689] Specific operation: The video stream is processed in real time using a WebRTC server and the stream is sent to the other device.

[1690] Input: Captured video and audio data.

[1691] Data Computing: Compression and transmission of data streams via WebRTC.

[1692] Output: Real-time video and audio communication.

[1693] Step 7:

[1694] Reminder settings

[1695] How it works: The user sets a reminder to take action at a specific time.

[1696] Example: Set a reminder to "take your morning medicine."

[1697] Input: Reminder information set by the user.

[1698] Data Calculation: Reminder information is organized and converted to JSON format.

[1699] Output: JSON data to send organized reminder information to the server.

[1700] Step 8:

[1701] Server reminder storage and notifications

[1702] How it works: The server stores the reminder information in a database and sends notifications to the device at the specified time.

[1703] Specific behavior: Saves reminder information in the database and sends a push notification at the specified time.

[1704] Input: JSON data of organized reminder information.

[1705] Data calculation: Save reminder information and trigger notifications at specified times.

[1706] Output: Send reminder notification.

[1707] Step 9:

[1708] Display reminders on your device

[1709] Action: The device displays the notification in its user interface.

[1710] Specific behavior: Display a reminder notification on the smart glasses display.

[1711] Input: Reminder notification sent from the server.

[1712] Data Calculation: Parsing notification data and converting it into a display format.

[1713] Output: Display reminder notification on the smart glasses display.

[1714] Step 10:

[1715] Submit a medical records request

[1716] How it works: A user uses the smart glasses interface to submit a request to view their medical records.

[1717] Example: Checking an elderly person's medical records for "high blood pressure" and "arthritis."

[1718] Input: User confirmation request.

[1719] Data operation: Generates an HTTP GET request to send the request.

[1720] Output: Sends to the server as an HTTP GET request.

[1721] Step 11:

[1722] Server data acquisition and transmission

[1723] Operation: The server retrieves medical records from the database and sends them to the device.

[1724] Specific behavior: Retrieves medical records from the database and sends them to the device in JSON format.

[1725] Input: User's medical records review request.

[1726] Data operations: Retrieving medical records and converting them to JSON format.

[1727] Output: Sends the medical record JSON data to the device.

[1728] Step 12:

[1729] Displaying medical records on the device

[1730] Action: The terminal displays the medical record in a user interface.

[1731] Specific operation: Displaying medical records on the smart glasses display.

[1732] Input: JSON data of medical records sent from the server.

[1733] Data Calculation: Analyzing data and converting it into a display format.

[1734] Output: Display medical records on the smart glasses display.

[1735] Step 13:

[1736] New staff training requests

[1737] How it works: A new employee requests training through the smart glasses interface.

[1738] Example: Submit a request for a training module.

[1739] Input: New staff training request.

[1740] Data operation: Generates an HTTP GET request to send the request.

[1741] Output: Sends to the server as an HTTP GET request.

[1742] Step 14:

[1743] Server training content transmission

[1744] Operation: The server sends training content to the device.

[1745] Specific operation: Sends training videos to the device in streaming format.

[1746] Input: New staff training request.

[1747] Data Computing: Acquiring training videos and converting them into a streaming format.

[1748] Output: Sends the training video to the device.

[1749] Step 15:

[1750] Training video playback on device

[1751] Action: The device plays a training video.

[1752] Specific operation: Playing a training video on the smart glasses display.

[1753] Input: The training video stream sent from the server.

[1754] Data operations: decoding and playing video streams.

[1755] Output: Playback of training videos to the smart glasses display.

[1756] Step 16:

[1757] Emergency button pressed

[1758] How it works: The user presses the emergency button on their smart glasses to send a notification.

[1759] Example: An elderly person presses an emergency button to send a notification.

[1760] Input: An input signal that the emergency button has been pressed.

[1761] Data Computation: Data construction for generating and sending emergency notifications.

[1762] Output: Sends an emergency notification to the server.

[1763] Step 17:

[1764] Server emergency response protocol activation

[1765] Actions: The server initiates emergency response protocols and sends alert notifications to relevant care staff.

[1766] What it does: Send alerts to all staff using a push notification system.

[1767] Input: Emergency notification sent from the device.

[1768] Data operations: Executing emergency response protocols and generating warning notifications.

[1769] Output: Send warning notification to all staff.

[1770] Step 18:

[1771] Emergency warning display on device

[1772] Action: The terminal receives the alert and displays it to the user.

[1773] Specific operation: A notification "Emergency has occurred!" is displayed on the smart glasses display.

[1774] Input: The warning notification sent by the server.

[1775] Data Calculation: Parsing alert data and converting it into a display format.

[1776] Output: Show a warning notification on the smart glasses display.

[1777] (Application example 2)

[1778] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1779] In elderly care, it is extremely important to monitor the elderly's emotions and location information in real time while ensuring their safety and responding quickly in emergencies. However, current systems lack the means to quickly notify and check the situation when an elderly person faces an emergency, making it difficult to respond effectively. In addition, there is a lack of systems that can monitor the elderly's emotional state and location information in real time and take action based on that information. Technological solutions to these problems are needed.

[1780] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a user interface means for a user to input data and check the elderly person's care plan and medical records, a communication means for transmitting the data to the server, a database means for the server to store and manage the data, a communication means for the server to transmit the data to the user interface means, a means having an emergency button for transmitting an emergency notification, a GPS means for acquiring and transmitting location information of the elderly person, an emotion recognition means for recognizing emotions, and a communication means for transmitting a notification based on data from the GPS means and the emotion recognition means. This allows an emergency notification to be transmitted quickly when the elderly person faces an emergency, and enables an effective response while monitoring the location information and emotional state in real time.

[1781] "User interface means" refers to the interactive screen display and operation means through which a user can input data and view the senior's care plan and medical records.

[1782] "Communication means" refers to means having a communication function for transmitting data to a server and receiving data from a server.

[1783] "Database means" refers to a database system for storing and managing data on a server.

[1784] An "emergency button" is a button-type operating device that an elderly person can press to send an emergency notification when they encounter an emergency.

[1785] "GPS Means" means a means utilizing the Global Positioning System (GPS) for obtaining and transmitting current location information.

[1786] The "emotion recognition means" is a means that uses emotion recognition technology to analyze the emotional state of the elderly and record and transmit the analysis results.

[1787] An "emergency notification" is notification information that is sent promptly to designated contacts when an emergency occurs.

[1788] "Real-time video call means" refers to means having the function of transmitting and receiving video and audio for conducting real-time video calls.

[1789] The present invention relates to a smart glasses-based care assistant system and security service for improving care for the elderly. Detailed embodiments of the present invention are described below.

[1790] System Configuration

[1791] The system consists of the following major components:

[1792] User Interface Means

[1793] communication means

[1794] Database Means

[1795] Emergency button

[1796] GPS means

[1797] emotion recognition means

[1798] Notification means

[1799] Server Means

[1800] Specific implementation methods

[1801] 1. User Interface Methods

[1802] Users input and view data through smart glasses or smartphones, including elderly care plans and medical records, with an intuitive and easy-to-use user interface.

[1803] 2. Means of communication

[1804] Data is sent from the device to the server and received from the server using a secure protocol.

[1805] 3. Database Means

[1806] The server stores the received data in a database and manages it appropriately, using a common database management system such as SQL.

[1807] 4. Emergency Button

[1808] If an elderly person faces an emergency, they can use the emergency button on their smart glasses or smartphone to send an emergency notification to designated contacts.

[1809] 5. GPS means

[1810] The system acquires the location information of the elderly person in real time and sends it to a server, allowing family members and caregivers to know the elderly person's location.

[1811] 6. Emotion recognition means

[1812] It analyzes the user's emotional state and detects stress, anxiety, etc. The emotion recognition means analyzes facial expressions using the camera in smart glasses or smartphones.

[1813] 7. Means of notification

[1814] Based on information obtained from emotion recognition and GPS means, necessary notifications are sent, including emergency notifications and notifications about emotional state.

[1815] 8. Server Means

[1816] The server is responsible for overall management and data processing, and also includes a video call module for processing real-time video calls.

[1817] Example of operation

[1818] For example, a user can enter an elderly person's care plan ("Daily Exercise" and "8 AM Medications") and check their medical records. If the elderly person encounters an emergency and presses the emergency button, their real-time location and current emotional state will be automatically sent to a designated contact. The contact who receives the emergency notification will then initiate a video call via smartphone or smart glasses to check on the elderly person's condition and provide assistance.

[1819] Examples of using generative AI models

[1820] When analyzing an elderly person's emotional state, generative AI models can be used to suggest "relaxing music" or "stress-reducing activities," with example prompts such as:

[1821] Prompt Sentence Examples

[1822] "The senior's panic button is pressed, which notifies family members of their real-time location and current emotional state (stress or anxiety). A video call is then automatically initiated, and the closest family member in their contact list receives the call."

[1823] From the above, a specific embodiment of the present invention has been presented and it can be seen how this system improves the care and security of the elderly.

[1824] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1825] Step 1:

[1826] Users input the elderly person's care plan and medical records through smart glasses or a smartphone. The input data is sent from the device to a server via a communication means. The server receives the data and stores it in a database for management. This ensures that the input information is securely stored on the server.

[1827] Step 2:

[1828] When needed, a user or caregiver sends a request to check the data stored on the server. The server retrieves the necessary information from the database and sends it to the terminal via a communication means. The terminal displays the received data on a user interface, allowing the user to check the elderly person's care plan and medical records.

[1829] Step 3:

[1830] When an elderly person faces an emergency, they press the emergency button on their smart glasses or smartphone. The device generates an emergency notification signal and sends it to the server via communication means. The server immediately receives the emergency notification and sends it to the registered emergency contacts.

[1831] Step 4:

[1832] At the same time, the GPS unit of the device acquires the current location information and transmits the location data to the server via the communication unit. The server processes the location information and transmits the location data together with the emergency notification. This allows the location of the elderly person to be monitored in real time and necessary measures to be taken promptly.

[1833] Step 5:

[1834] The device's emotion recognition means analyzes the elderly person's emotional state and acquires emotion data. For example, facial expressions are analyzed to detect signs of stress or anxiety. The acquired emotion data is then sent to a server via a communication means. The server analyzes the emotion data and generates an appropriate notification based on it.

[1835] Step 6:

[1836] The server then sends emergency notifications to emergency contacts along with warnings and advice based on the emotion data, such as "An elderly person is in a state of stress, please take immediate action."

[1837] Step 7:

[1838] The emergency contact who receives the emergency notification can use the video call function to contact the elderly person in real time. The server manages the video call session, activates the device's camera and microphone, and starts the video call. This allows the emergency contact to directly check the elderly person's current situation and ensure their safety.

[1839] Examples of prompt statements

[1840] "The senior's panic button is pressed, which notifies family members of their real-time location and current emotional state (stress or anxiety). A video call is then automatically initiated, and the closest family member in their contact list receives the call."

[1841] Through these steps, the system is able to comprehensively improve the care and security of the elderly.

[1842] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1843] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1844] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1845] [Fourth embodiment]

[1846] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1847] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1848] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1849] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1850] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1851] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1852] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1853] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1854] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1855] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1856] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1857] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1858] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1859] The present invention relates to a smart glasses-based care assistant system for improving elderly care, providing a system for inputting user data, managing elderly care plans and medical records, real-time video calling, reminder notifications, training new staff, and responding to emergency care. Specific embodiments for implementing the present invention are described in detail below.

[1860] User Data Input

[1861] Users input the elderly person's care plan and medical records through the smart glasses interface. The device then sends the input data to a server, which analyzes the received data and stores and manages it in a database.

[1862] For example, a user inputs the care plan for elderly person Tanaka, such as "daily exercise" and "medicine at 8 am," and also inputs medical records such as "high blood pressure" and "arthritis." The device sends the input data to the server, which then stores the data in a database.

[1863] Real-time video calling

[1864] The user uses the smart glasses interface to make real-time video calls with remote family members or medical staff. The device activates the camera and microphone and initiates the video call session. The server processes the video call data stream and manages real-time communication.

[1865] For example, if a user wants to start a video call with Tanaka's daughter, the smart glasses will turn on the camera and microphone and transmit the video stream. The server processes this data and starts the call.

[1866] Reminders

[1867] Users can set reminders to prompt the elderly to perform an action at a specific time. The device sends the reminder information to the server, which stores and manages the reminder information in a database. When the reminder time arrives, the server sends a reminder notification to the device, which then displays the notification on the user interface.

[1868] For example, if Tanaka sets a reminder to take her morning medicine, the server stores that information in a database and sends a notification to the device when the specified time arrives. The device then displays a notification saying, "It's time to take your morning medicine."

[1869] Medical record review

[1870] The user can check the elderly person's medical records using the interface of the smart glasses. The device sends the user's request to the server, which retrieves the medical records from the database and sends them to the device. The device then displays the retrieved medical records on the user interface.

[1871] For example, if a user checks Mr. Tanaka's medical records, the server retrieves information about "high blood pressure" and "arthritis" from the database and sends it to the terminal, which then displays that information.

[1872] Training new staff

[1873] New staff can receive training using smart glasses. The terminal sends a request for a training module to the server, and the server sends the training content to the terminal, which then plays it and provides it to the user.

[1874] For example, when a new staff member, Mr. Yamada, starts training, the terminal requests a training module from the server, and the server sends the training video to the terminal, which then plays the video.

[1875] Rapid response to emergency care

[1876] In the event of an emergency, the user can press the emergency button on the smart glasses to send a notification. The device will then send the emergency notification to the server, which will trigger the emergency response protocol and send an alert to the relevant care staff. The device will receive the alert and display it immediately.

[1877] For example, if Tanaka presses the emergency button, the device sends an emergency notification to the server, which then alerts all staff. The device displays, "Emergency! Please respond immediately."

[1878] As described above, the present invention improves the quality of care for the elderly, while enabling efficient support and emergency response by staff, thereby improving the quality of life for the elderly.

[1879] The processing flow will be explained below.

[1880] OK, so let's go into more detail and explain what the program does, step by step.

[1881] User Data Input

[1882] Step 1:

[1883] Users enter the elderly person's care plan and medical records through the smart glasses interface.

[1884] Specifically, data is input using a keyboard, voice input, a touch panel, etc.

[1885] Step 2:

[1886] The terminal transmits the input data to the server.

[1887] Specifically, the input data is converted into packets and sent to the server using a communication protocol.

[1888] Step 3:

[1889] The server analyzes the received data and stores it in a database.

[1890] Specifically, the received packet is analyzed and the data is stored in the appropriate database field.

[1891] Real-time video calling

[1892] Step 1:

[1893] The user selects the person with whom they want to start a video call using the interface of the smart glasses.

[1894] Specifically, you select the ID of the person you want to call using the touch panel or voice commands.

[1895] Step 2:

[1896] The terminal sends a request for a call session to the server.

[1897] Specifically, the server sends a request packet containing the ID of the selected call partner to the server.

[1898] Step 3:

[1899] The server processes the call request and establishes the call session.

[1900] Specifically, a new session ID is generated and notified to both terminals.

[1901] Step 4:

[1902] The device will enable the camera and microphone and start the video stream.

[1903] Specifically, the hardware camera and microphone are turned on, and video and audio data are sent and received as streams.

[1904] Reminders

[1905] Step 1:

[1906] The user inputs the reminder content and time through the smart glasses interface.

[1907] Specifically, the reminder content and time can be set using the touch panel or voice input.

[1908] Step 2:

[1909] The terminal transmits the reminder information to the server.

[1910] Specifically, the set reminder information is packetized and sent to the server.

[1911] Step 3:

[1912] The server stores the reminder information and monitors the reminder time.

[1913] Specifically, the system stores the reminder information in a database and waits for the specified time.

[1914] Step 4:

[1915] The server sends a notification to the terminal when the reminder time arrives.

[1916] Specifically, the system refers to the saved reminder information and sends a notification packet to the terminal when the reminder time is reached.

[1917] Step 5:

[1918] The terminal receives the reminder notification and displays it to the user.

[1919] Specifically, the received notification packet is analyzed and the reminder content is displayed on the user interface.

[1920] Medical record review

[1921] Step 1:

[1922] The user submits a request to review medical records through the smart glasses interface.

[1923] Specifically, the user selects the medical record item they wish to check using the touch panel or voice.

[1924] Step 2:

[1925] The terminal sends a confirmation request to the server.

[1926] Specifically, the operation involves sending a request packet containing the selected medical record items to the server.

[1927] Step 3:

[1928] The server retrieves the relevant medical records from the database.

[1929] Specifically, the system analyzes the received request and extracts the relevant medical record data from the database.

[1930] Step 4:

[1931] The server transmits the acquired medical record data to the terminal.

[1932] Specifically, the extracted medical record data is converted into packets and sent to the terminal.

[1933] Step 5:

[1934] The terminal displays the received medical record data on a user interface.

[1935] Specifically, the received data packets are analyzed and the medical records are displayed on the user interface.

[1936] Training new staff

[1937] Step 1:

[1938] The user selects a training module from the smart glasses interface.

[1939] Specifically, the training menu is selected using the touch panel or voice.

[1940] Step 2:

[1941] The terminal sends a request for a training module to the server.

[1942] Specifically, the ID of the selected training module is sent to the server.

[1943] Step 3:

[1944] The server obtains the training content and transmits it to the terminal.

[1945] Specifically, the relevant training content is retrieved from the database and sent to the terminal.

[1946] Step 4:

[1947] The terminal plays back the received training content.

[1948] Specifically, the received training video or manual is played on the user interface.

[1949] Rapid response to emergency care

[1950] Step 1:

[1951] The user presses the emergency button on the smart glasses to notify them of an emergency.

[1952] Specific actions include physically pressing an emergency button or issuing a specific voice command.

[1953] Step 2:

[1954] The terminal sends an emergency notification to the server.

[1955] Specifically, it detects that the emergency button has been pressed and sends an emergency notification packet to the server.

[1956] Step 3:

[1957] The server receives the emergency notification and executes the emergency response protocol.

[1958] Specifically, it analyzes the received emergency notification and initiates a response according to the protocol.

[1959] Step 4:

[1960] The server sends an alert notification to the relevant care staff.

[1961] A specific operation is to send an alert notification packet to the device of the necessary care staff.

[1962] Step 5:

[1963] The terminal receives the warning notification and displays it to the user.

[1964] Specifically, the device analyzes the warning notification received and displays "Emergency situation! Please respond immediately" on the user interface.

[1965] The above is the specific processing flow of each function detailed for each process.

[1966] Example 1

[1967] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1968] To improve elderly care, a system for efficient data management and emergency response is needed. However, the current system does not centrally manage care plans, medical record entry and review, real-time video calls, reminder notifications, new staff training, and emergency care responses, making it difficult to provide comprehensive support.

[1969] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1970] In this invention, the server includes a user interface means for a user to input data and check care plans and medical records, a communication means for transmitting the data to the server, a database means for the server to store and manage the data, a communication means for the server to transmit the data to the user interface means, a sensor means for transmitting an emergency notification to the server when a user presses an emergency button, and a server means for the server to execute emergency response protocols and send alerts to relevant staff. This allows management of elderly care plans and medical records, real-time video calls, reminder notifications, training of new staff, and emergency care responses to be performed in a unified manner, thereby improving the quality and efficiency of care.

[1971] "User interface means" refers to hardware and software for displaying data entered by the user and notifications from the system.

[1972] "Communication means" refers to the network interface and communication protocol for sending and receiving data.

[1973] "Database means" refers to a storage system or database management system for storing and managing data.

[1974] "Sensor means" refers to a device or module that detects user input or changes in the environment and acquires data.

[1975] "Server means" refers to a server system for processing, storing, managing, and communicating data.

[1976] "Emergency response protocols" are response procedures and steps that the system automatically executes in the event of an emergency.

[1977] A "video stream" is a flow of video data that is sent and received in real time.

[1978] A "reminder" is an alert or notification that notifies a user of a specific action or time.

[1979] These definitions provide a clearer understanding of the role and function of each element included in the claims.

[1980] The present invention relates to a smart glasses-based care assistant system for improving elderly care, providing a system for user data entry, managing elderly care plans and medical records, real-time video calling, reminder notifications, training new staff, and emergency care response. Specific embodiments for implementing the present invention are described in detail below.

[1981] User Data Input

[1982] Users can input care plans and medical records through the smart glasses interface. The data is then sent to a server via the device. The server analyzes the received data and stores and manages it in a database. This allows for centralized management of elderly care plans and medical records.

[1983] Specifically, users can use the smart glasses' voice recognition system or touchpad to input instructions such as "Take today's medicine at 8 o'clock." The device converts this voice data into text and sends it to the server. The server then converts this data into JSON format and stores it in a database such as MySQL.

[1984] Real-time video calling

[1985] Users can use the smart glasses interface to make real-time video calls with remote family members or medical staff. The device activates the camera and microphone and initiates the video call session. The server processes the video data stream and manages the communication in real time.

[1986] For example, if you say "start a video call" to your smart glasses, the device will turn on the camera and microphone and start a call session using WebRTC. The server manages this and ensures the call proceeds smoothly.

[1987] Reminders

[1988] Users can set reminders to prompt the elderly to perform specific actions at specific times. The device sends the reminder information to the server, which stores and manages the reminder information in a database. At the set time, the server sends a reminder notification to the device, which then displays the notification on the user interface.

[1989] Specifically, when someone types "Set a reminder to take my medicine at 8 a.m.", the device converts this into text and sends it to the server. The server stores the reminder information in a database and sends a notification to the device at the specified time. The device then displays "It's time to take your morning medicine."

[1990] Medical record review

[1991] The user can check their medical records through the interface of the smart glasses. A request is sent from the device to the server, which retrieves the medical records from the database and sends them to the device. The device then displays the medical records on the user interface.

[1992] For example, if you say "Check Tanaka-san's medical records," the server will retrieve information on "high blood pressure" and "arthritis" from the database and send it to the device, which will then display it on its screen.

[1993] Training new staff

[1994] New staff can receive training using smart glasses. The device sends a request for a training module to the server, and the server sends the training content to the device. The device can then play the provided content and display it to the user.

[1995] Specifically, when a new staff member says "Start training," the terminal sends this request to the server, which then sends the training video to the terminal, which then plays the video.

[1996] Rapid response to emergency care

[1997] In the event of an emergency, users can press the emergency button on the smart glasses to send an emergency notification. The device will then send the emergency notification to the server, which will trigger the emergency response protocol to alert relevant staff. The device will then display the alert.

[1998] For example, if Tanaka presses the emergency button, the device sends an emergency notification to the server, which then executes emergency response protocols and alerts all staff. The device displays, "Emergency! Please respond immediately."

[1999] As described above, the present invention improves the quality of care for the elderly, while enabling efficient support and emergency response by staff, thereby improving the quality of life for the elderly.

[2000] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2001] Step 1:

[2002] The user inputs data through the smart glasses interface. For example, they can enter instructions such as "Take today's medicine at 8 o'clock" using a voice recognition system or touchpad. The input in this step is voice or text data, and the device receives this input.

[2003] Step 2:

[2004] The device converts the received voice data into text data. It uses voice recognition technology to analyze the user's voice input and outputs it as text. The converted text data is obtained. This specific operation is performed using the voice recognition library installed on the device.

[2005] Step 3:

[2006] The terminal transmits text data to the server. The data is transmitted to the server via the internet via the communication means. The input of this step is text data, and the output is data transmission to the server.

[2007] Step 4:

[2008] The server parses the received text data, converts it into the required data format (for example, JSON format), and saves the converted data in a database. The input is text data, and the output is saving the converted data in a database.

[2009] Step 5:

[2010] The server sets reminder information and stores it until the specified time. When the specified time arrives, it sends a reminder notification to the device. The input is the saved reminder information, and the output is to send a reminder notification to the device when the time arrives.

[2011] Step 6:

[2012] The device displays the reminder notification received from the server on the user interface and informs the user of the reminder content. Specifically, the device displays "It's time to take your morning medicine" on the display and also notifies the user by voice. The input is the reminder notification from the server, and the output is the display on the user interface and the voice notification.

[2013] Step 7:

[2014] When a user presses the emergency button, the terminal is instructed to notify the user of an emergency. The input is the user's operation of the emergency button, and the terminal captures this operation.

[2015] Step 8:

[2016] The terminal sends an emergency notification to the server. Emergency status data is sent to the server using a communication method. The input is the emergency button operation event, and the output is the transmission of emergency notification data to the server.

[2017] Step 9:

[2018] When the server receives an emergency notification, it activates the emergency response protocol and sends an alert to the relevant staff. Specifically, the server sends an emergency email or SMS to all registered staff. The input is the emergency notification data, and the output is sending an alert notification to the staff.

[2019] Step 10:

[2020] The relevant staff receives the alert and initiates a response based on the information. The input is an emergency alert notification from the server, and the output is the execution of emergency response actions.

[2021] Through these steps, the system can efficiently support the various needs of elderly care.

[2022] (Application example 1)

[2023] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2024] Conventional elderly care systems have had problems such as the time-consuming task of managing elderly care plans and medical records, and the difficulty of sharing care information in real time. Furthermore, while rapid response is required in the event of an emergency, conventional systems do not adequately address this need. Furthermore, security staff lacked the efficiency and immediacy required for on-site monitoring and emergency response, making it difficult to ensure smooth work progress through reminder notifications.

[2025] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2026] In this invention, the server includes a user interface means for a user to input data and check the elderly person's care plan and medical records, a communication means for transmitting the data to the server, a database means for storing and managing the data, a communication means for transmitting the data to the user interface means, a communication means for transmitting and receiving video streams, a communication means for issuing emergency notifications, and a display means for displaying reminder notifications, thereby enabling more efficient information management in elderly care, real-time information sharing, faster emergency response, and improved work efficiency of security staff.

[2027] "User interface means" refers to an interface through which a user inputs data and confirms and manipulates information.

[2028] "Communication means" refers to the technical means used to send and receive data and information between servers.

[2029] The "database means" refers to a data storage system for saving and managing input data and its management function.

[2030] The "server means" is a server system for processing and managing data.

[2031] A "video stream" is a continuous flow of video data that is sent and received in real time.

[2032] "Emergency notifications" are messages or alerts that are sent to quickly warn relevant parties when an emergency occurs.

[2033] The "display means" refers to a display or monitor for visually presenting information to the user.

[2034] A "reminder notification" is a notification that prompts the user to take action at a specific time or under specific conditions.

[2035] "Real-time video calling" is a call format in which both parties communicate with each other in real time.

[2036] "Communication means for issuing emergency notifications" means the technical means for sending immediate warning messages in the event of an emergency.

[2037] The present invention relates to a smart glasses-based assistant system for security and elderly care, which includes a user interface means, a communication means, a database means, a server means, and a display means.

[2038] System Overview

[2039] The system consists of the following main components:

[2040] 1. User interface means: The smart glasses' display and microphone allow users to input data, view information, and manipulate it.

[2041] 2. Communication Method: The technical means for transmitting data and video streams between the smart glasses and the server, including Wi-Fi and cellular communication.

[2042] 3. Database Means: A cloud storage system for storing and managing user-entered data. Specifically, this includes Google Cloud Firestore.

[2043] 4. Server infrastructure: Server infrastructure that processes and manages data. For example, Google Cloud or AWS is used.

[2044] 5. Display means: A means for visually presenting information to the user using the smart glasses display.

[2045] Program Overview

[2046] The process of a user wearing smart glasses and entering data, making video calls, setting reminders, and sending emergency notifications through the user interface is described.

[2047] Entering and verifying data

[2048] The user inputs data using the smart glasses' display and voice input. For example, in elderly care, the user can input care plans and medical records. The input data is sent to a server via a communication means and stored in a database means. The stored data can be viewed at any time via a user interface means.

[2049] Real-time video calling

[2050] A user can use the interface of the smart glasses to make a real-time video call with another user in a remote location (e.g., a security center or a family member). The device captures the video stream using a camera and microphone and transmits it to a server through a communication means. The server means processes the video stream in real time and manages the call function.

[2051] Reminders

[2052] A user can set a reminder for a task to be performed at a specific time. The reminder information is sent to the server via the communication means and stored in the database means. When the specified time arrives, the server means sends a notification to the terminal, and the reminder is displayed to the user via the display means.

[2053] Sending emergency notifications

[2054] When a user faces an emergency, they can press the emergency button on the smart glasses to issue an emergency notification. The emergency notification is sent to the server via the communication means, and the server means activates an emergency response protocol and sends the emergency notification to all related terminals. A prompt response is then made via the display means.

[2055] Specific examples

[2056] When a security staff member discovers a suspicious individual while on patrol, an emergency notification is sent from the smart glasses. The server immediately sends an alert to multiple staff members, enabling a swift response on-site.

[2057] At elderly care facilities, care staff input care plans into smart glasses, and the data is stored on a server, allowing family members to remotely check the progress of care in real time.

[2058] Prompt Sentence Examples

[2059] "Please outline an application that uses smart glasses to enable security personnel to initiate video calls, send emergency notifications, and display reminder notifications in real time. Please explain the specific features."

[2060] Overall, the system will streamline care and security management and improve real-time response capabilities.

[2061] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2062] Step 1:

[2063] The user wears the smart glasses and inputs data through the user interface, such as entering care plans and medical records. The input data is sent to a server via a communication means. Input is performed using the smart glasses' display and voice input function, and the data is sent to the server as output.

[2064] Step 2:

[2065] The server stores and manages the received data using database means. When the input data reaches the server, it is stored in the database. The server uses Google Cloud Firestore to store the data. After the data is stored, the database is updated.

[2066] Step 3:

[2067] The user checks the entered care plan and medical records through the interface of the smart glasses. A request is sent to access the data stored on the server. The server receives this request, retrieves the relevant data from the database, and sends it back to the smart glasses via a communication method. As a result, the user can view the data on the display.

[2068] Step 4:

[2069] Users use the smart glasses interface to make real-time video calls with other users in remote locations. The smart glasses' camera and microphone capture video and audio and send them to a server via a communication method. The server uses Twilio's API to manage the video call session and send the video stream to the other user's device. This enables two-way communication of video and audio in real time.

[2070] Step 5:

[2071] The user sets a reminder for a specific time, for example, inputting a reminder for rounds or taking medicine. The reminder information is sent to the server via the interface of the smart glasses using a communication means. The server receives this information, stores it in a database, and then triggers a reminder notification based on the specified time. The notification is sent back to the smart glasses via the communication means and displayed to the user via the display means. As a result, the user receives the notification at the specified time.

[2072] Step 6:

[2073] When a user faces an emergency, they press the emergency button on the smart glasses to send an emergency notification. Input is made by pressing the button, and the notification is sent to the server using a communication method. The server receives the emergency notification, activates a response protocol, and sends the emergency notification to all related devices. This allows a notification of an "emergency" to be displayed on multiple devices, enabling a quick response.

[2074] This allows the system to enable users to effectively manage data, communicate and respond quickly to emergencies.

[2075] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2076] The present invention relates to a smart glasses-based care assistant system for improving elderly care, which provides a system for inputting user data, managing elderly care plans and medical records, real-time video calling, reminder notifications, training new staff, responding to emergency care, and emotion recognition using an emotion engine. The following describes in detail the embodiments of the present invention.

[2077] User Data Input

[2078] Users input the elderly person's care plan and medical records through the smart glasses interface. The device then sends the input data to a server, which analyzes the data and stores it in a database for management.

[2079] For example, a user inputs an elderly person's care plan, "Daily Exercises" and "Medicine at 8 AM," and inputs medical records "High Blood Pressure" and "Arthritis." The terminal sends the input data to the server, which stores the data in a database.

[2080] Real-time video calling

[2081] The user uses the smart glasses interface to make real-time video calls with remote family members or medical staff. The device activates the camera and microphone and initiates the video call session. The server processes the video call data stream and manages real-time communication.

[2082] For example, if a user wants to start a video call with the elderly person's daughter, the smart glasses will turn on the camera and microphone and transmit the video stream, and the server will process this data and start the call.

[2083] Reminders

[2084] Users can set reminders to prompt the elderly to take action at specific times. The device sends the reminder information to the server, which stores the information in a database and sends a notification to the device when the reminder time arrives. The device then displays the notification on the user interface.

[2085] For example, when a user sets a reminder for an elderly person to take their morning medication, the server stores that information in a database and sends a notification to the device at the specified time, which displays a message saying, "It's time to take your morning medication."

[2086] Medical record review

[2087] The user can check the elderly person's medical records using the smart glasses interface. The device sends the user's request to the server, which retrieves the medical records from the database and sends them to the device. The device then displays the retrieved medical records on the user interface.

[2088] For example, if a user checks the medical records of an elderly person, the server retrieves information about "high blood pressure" and "arthritis" from the database and sends it to the terminal, which then displays the information.

[2089] Training new staff

[2090] New staff can receive training using smart glasses. The device sends a request for a training module to the server, and the server sends the training content to the device, which then plays it and provides it to the user.

[2091] For example, when a new staff member starts training, the terminal requests a training module from the server, and the server sends a training video to the terminal, which then plays the video.

[2092] Rapid response to emergency care

[2093] In the event of an emergency, the user can press the emergency button on the smart glasses to send a notification. The device will then send the emergency notification to the server, which will trigger the emergency response protocol and send an alert notification to the relevant care staff. The device will then receive the alert and display it to the user.

[2094] For example, if an elderly person presses the emergency button, the device will send an emergency notification to the server, which will then alert all staff. The device will display "Emergency! Please respond immediately."

[2095] Emotion recognition by emotion engine

[2096] The emotion engine means recognizes the user's emotions and transmits them to the server, which stores and analyzes the emotion data, and provides feedback to the user based on the emotion data.

[2097] For example, if the emotion engine detects that an elderly person is in a stressful state, the device will send this emotion data to the server, which will analyze the data and provide feedback to the user such as "listen to some relaxing music."

[2098] Care plan coordination

[2099] Based on the emotion data recognized by the emotion engine, the server adjusts the care plan, which is then notified to the user and implemented.

[2100] For example, if the emotion engine analyzes the emotional data of an elderly person and determines that they are under high stress, the server will adjust the care plan to a "relaxation exercise" and notify the device to carry it out.

[2101] Applying emotion recognition to video calls

[2102] The emotion engine means recognizes the user's emotions during a real-time video call and notifies the video call partner of the emotion data. Based on the emotion data, a warning or recommended action can be displayed to the video call partner.

[2103] For example, if the emotion engine recognizes that an elderly person is becoming increasingly stressed during a video call, the caller will see a message saying, "You are feeling stressed. Please change the topic to help you relax."

[2104] As described above, the present invention improves the quality of care for the elderly while providing efficient support and emergency response from staff, and by combining it with an emotion engine, more detailed care and support becomes possible.

[2105] The processing flow will be explained below.

[2106] User Data Input

[2107] Step 1:

[2108] Users enter the elderly person's care plan and medical records through the smart glasses interface.

[2109] Example: A user enters an elderly person's care plan, "Daily Exercises" and "8 AM Medication," and medical records, including "High Blood Pressure" and "Arthritis."

[2110] Step 2:

[2111] The terminal converts the input data into packets and transmits them to the server.

[2112] Step 3:

[2113] The server analyzes the received data and stores it in a database.

[2114] A database stores care plans and medical records.

[2115] Real-time video calling

[2116] Step 1:

[2117] The user uses the interface of the smart glasses to select who to start a video call with.

[2118] Example: A user selects the ID of an elderly person's daughter.

[2119] Step 2:

[2120] The terminal sends a call request packet to the server.

[2121] The request containing the submitted ID arrives at the server.

[2122] Step 3:

[2123] The server processes the call request and generates a new session ID.

[2124] The generated session ID is notified to both terminals.

[2125] Step 4:

[2126] The device will enable the camera and microphone and start the video stream.

[2127] A video stream is sent from the elderly person's smart glasses and displayed on his daughter's device.

[2128] Reminders

[2129] Step 1:

[2130] The user inputs the reminder content and time through the smart glasses interface.

[2131] Example: A user sets a reminder for an elderly person to take their morning medication at 08:00.

[2132] Step 2:

[2133] The terminal converts the reminder information into packets and transmits them to the server.

[2134] Step 3:

[2135] The server stores the received reminder information in a database and monitors the specified time.

[2136] Step 4:

[2137] When the reminder time arrives, the server sends a notification packet to the terminal.

[2138] Step 5:

[2139] The terminal analyzes the received notification and displays it on the user interface.

[2140] Example: The device displays "Time to take your morning medicine."

[2141] Medical record review

[2142] Step 1:

[2143] The user enters a request to review medical records through the smart glasses interface.

[2144] Example: A user sends a request to review an elderly person's medical records.

[2145] Step 2:

[2146] The terminal sends a request packet to the server.

[2147] Step 3:

[2148] The server retrieves the relevant medical records from the database.

[2149] The server extracts information on "high blood pressure" and "arthritis" from the database.

[2150] Step 4:

[2151] The server converts the acquired medical record data into packets and transmits them to the terminal.

[2152] Step 5:

[2153] The terminal analyzes the received data and displays it on a user interface.

[2154] Example: Medical records for "high blood pressure" and "arthritis" are displayed.

[2155] Training new staff

[2156] Step 1:

[2157] The user selects a training module from the smart glasses interface.

[2158] Example: New staff members select a training menu.

[2159] Step 2:

[2160] The terminal sends a request packet for the training module to the server.

[2161] Step 3:

[2162] The server retrieves training content from the database, converts it into packets, and transmits them to the terminal.

[2163] Step 4:

[2164] The terminal plays back the received training content.

[2165] Example: A device plays a training video that new staff members watch.

[2166] Rapid response to emergency care

[2167] Step 1:

[2168] The user presses the emergency button on the smart glasses to send an emergency notification.

[2169] Example: An elderly person presses the emergency button.

[2170] Step 2:

[2171] The terminal transmits an emergency notification packet to the server.

[2172] Step 3:

[2173] The server executes emergency response protocols and sends emergency notifications to all staff.

[2174] Step 4:

[2175] The terminal analyzes the received emergency notification and displays it on the user interface.

[2176] Example: "Emergency! Please respond immediately."

[2177] Emotion recognition by emotion engine

[2178] Step 1:

[2179] The terminal analyzes the user's emotions using an emotion engine and generates emotion data.

[2180] Example: The stress state of the elderly is recognized by the emotion engine.

[2181] Step 2:

[2182] The terminal converts the generated emotion data into packets and transmits them to the server.

[2183] Step 3:

[2184] The server stores the received emotion data in a database and analyzes it.

[2185] Step 4:

[2186] The server generates feedback based on the emotion data and transmits it to the terminal.

[2187] Step 5:

[2188] The terminal analyzes the received feedback and displays it on the user interface.

[2189] Example: "Listen to some relaxing music" is displayed to the user.

[2190] Care plan coordination

[2191] Step 1:

[2192] The server analyzes the emotion data recognized by the emotion engine and adjusts the care plan.

[2193] Example: If stress levels are high, change the care plan to "relaxation exercises."

[2194] Step 2:

[2195] The server converts the adjusted care plan into packets and transmits them to the terminal.

[2196] Step 3:

[2197] The terminal displays the received coordinated care plan on a user interface and executes it.

[2198] Applying emotion recognition to video calls

[2199] Step 1:

[2200] The device analyzes the user's emotions using an emotion engine during a video call.

[2201] Example: The emotion engine recognizes the stress state of elderly people during video calls.

[2202] Step 2:

[2203] The terminal converts the recognized emotion data into packets and transmits them to the terminal of the other party.

[2204] Step 3:

[2205] The other party's device analyzes the received emotion data and displays warnings and recommended actions on the user interface.

[2206] Example: Displaying "You are feeling stressed, please change the topic to relax."

[2207] The above is the specific processing flow of each function, including the emotion engine.

[2208] Example 2

[2209] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2210] In elderly care, managing care plans and medical records is cumbersome, and remote status checks and communication are difficult. Furthermore, reminder functions and emergency responses are insufficient, and personalized care based on emotion recognition is not provided. Therefore, there is a need to improve the quality of care and provide efficient support for staff.

[2211] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an interface means for a user to input data and check the elderly person's care plan and medical records, a communication means for transmitting the data to the server, an electronic database means for the server to store and manage the data, a communication means for the server to transmit the data to the interface means, an interface means for conducting video calls in real time, a communication means for transmitting and receiving the video stream, a server means for processing and managing the video stream, an emotion recognition means for recognizing the elderly person's emotion, and an emergency notification means for notifying the elderly person of an emergency. This enables efficient management of the elderly person's care plan and medical records, remote care via real-time video calls, appropriate reminder notifications and emergency responses, and detailed care through emotion recognition.

[2212] "User" refers to a person who uses the system to input and check elderly care plans and medical records.

[2213] "Interface means" refers to devices and screens that allow users to input data and view care plans and medical records.

[2214] "Communication means" refers to a function or device for transmitting data to a server and receiving data from the server.

[2215] "Electronic database means" refers to a database system in which the server stores and manages data.

[2216] The "server means" is a server device that manages, stores, and transmits data, and processes and manages video streams.

[2217] A "video stream" is a continuous stream of video data sent and received during a video call.

[2218] The "emotion recognition means" is a system for analyzing and recognizing the emotions of elderly people in real time.

[2219] An "emergency notification means" is a function or device for promptly notifying relevant parties when an emergency occurs.

[2220] "Reminder" is a notification function that prompts elderly people to take specific actions based on times and conditions set by the user.

[2221] "Training tools" are support functions and devices that enable new staff to learn the knowledge and skills necessary to perform their jobs properly.

[2222] The present invention relates to a smart glasses-based care assistant system for supporting elderly care, which provides user data input, real-time video calling, reminder notifications, medical record review, new staff training, emergency care response, and emotion recognition with an emotion engine.

[2223] Hardware and software used

[2224] 1. Smart Glasses:

[2225] Hardware: camera, microphone, display, emergency button

[2226] Software: Interface app, emotion engine

[2227] 2. Server:

[2228] Hardware: High-performance server machine

[2229] Software: Database management system (MySQL), communication management system (WebRTC server), emotion recognition analysis system

[2230] 3. Means of communication:

[2231] Hardware: Communication modules (Wi-Fi, 4G / 5G modules)

[2232] Software: HTTP / HTTPS protocol, JSON format data transmission and reception

[2233] Program processing explanation

[2234] User Data Input

[2235] Users input care plans and medical records through the smart glasses interface, such as "daily exercise," "8 AM medication," "high blood pressure," and "arthritis." The device then sends this data to a server, which analyzes it and stores it in a database.

[2236] Real-time video calling

[2237] Users use smart glasses to make real-time video calls with remote family members or medical staff. The smart glasses are equipped with a camera and microphone, and a video call is initiated by activating these devices. The server processes the data stream of the video call and manages real-time communication. For example, when a user initiates a video call with an elderly person's daughter, the smart glasses turn on the camera and microphone and send the video stream to the server.

[2238] Reminders

[2239] Users can set reminders to prompt them to take action at specific times. The set reminder information is sent to the server via the device, and the server stores this information in a database. When the specified time arrives, the server sends a reminder notification to the device, and the device displays this notification on its user interface. For example, if a user sets a reminder to "take their morning medicine," a notification will be displayed on the device when the specified time arrives.

[2240] Medical record review

[2241] The user uses the smart glasses interface to send a request to check the elderly person's medical records. The device then sends this request to the server, which retrieves the medical records from the database and sends them to the device. The retrieved medical records are then displayed on the smart glasses' display. For example, if you want to check the medical records for "high blood pressure" or "arthritis," the server retrieves the data and the device displays it.

[2242] Training new staff

[2243] The new staff member uses the smart glasses to request a training module, and the server responds by sending the training content to the terminal. The terminal plays the training video and provides it to the new staff member. For example, when the new staff member starts training, the video content is streamed to the smart glasses.

[2244] Rapid response to emergency care

[2245] When an emergency occurs, the user presses the emergency button on the smart glasses to send a notification. The device sends the emergency notification to the server, which then activates the emergency response protocol and sends an alert notification to the relevant care staff. The device receives the alert and displays it to the user. For example, if an elderly person presses the emergency button, a notification saying "Emergency has occurred!" is immediately displayed.

[2246] Emotion engine recognizes emotions and adjusts care plans

[2247] The emotion engine recognizes the elderly person's emotions in real time and sends the emotional data to the server. The server analyzes this data and adjusts the care plan as needed. For example, if high stress is recognized, the server will change the care plan to "relaxation exercises" and send a notification to the device. Emotional data during a video call is also notified to the other party, allowing appropriate action to be taken.

[2248] Prompt Sentence Examples

[2249] Below is an example of a prompt sentence to input to the generative AI model.

[2250] "Enter your care plan for today."

[2251] "Start a video call with the elderly person's daughter."

[2252] "Set a reminder to take your medicine at 8am."

[2253] "Check the elderly person's medical records."

[2254] "Play the training video for new staff."

[2255] "Please press the emergency button."

[2256] "Please send us the emotional data of the elderly."

[2257] Using the procedures and prompts provided above, a specific system for improving care for seniors can be implemented. Embodiments of this system can efficiently and effectively manage seniors' health and adjust their care plans.

[2258] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2259] Step 1:

[2260] User Data Entry

[2261] How it works: Users enter care plans and medical records through the smart glasses interface.

[2262] Example: Enter care plan items such as "Daily exercise" and "8am medication" and medical records such as "High blood pressure" and "Arthritis."

[2263] Input: Care plans and medical records entered by the user through the smart glasses interface.

[2264] Data processing: The terminal organizes this data and converts it into JSON format.

[2265] Output: JSON data for sending organized data to the server.

[2266] Step 2:

[2267] Data transmission from the device

[2268] Operation: The terminal sends the entered data to the server.

[2269] Specific operation: Sends JSON format data obtained from the interface as an HTTP POST request.

[2270] Input: Organized care plan and medical record JSON data.

[2271] Data computation: Generates an HTTP request to send data.

[2272] Output: The data is sent to the server as an HTTP POST request.

[2273] Step 3:

[2274] Data analysis and storage on the server

[2275] How it works: The server parses the data sent and stores it in a database.

[2276] Specific operation: Parses the received JSON data and stores it in a MySQL database as a care plan and medical record.

[2277] Input: JSON data sent to the server as an HTTP POST request.

[2278] Data Computing: Running SQL queries to analyze data and store it in a database.

[2279] Output: Care plans and medical records stored in a database on the server.

[2280] Step 4:

[2281] Start a real-time video call

[2282] How it works: A user initiates a video call through the smart glasses interface.

[2283] Example: Pressing the "Start video call" button from the smart glasses interface.

[2284] Input: User's request to start a video call.

[2285] Data calculation: Session creation to start a video call.

[2286] Output: Start of video call session.

[2287] Step 5:

[2288] Enabling the device's camera and microphone

[2289] What it does: The device enables the camera and microphone.

[2290] Specific Action: Turns on the built-in camera and microphone device and starts capturing.

[2291] Input: Instruction to start a video call session.

[2292] Data Computing: Enabling camera and microphone devices.

[2293] Output: Start video and audio capture.

[2294] Step 6:

[2295] Server video call data processing

[2296] How it works: The server processes the data stream of the video call and manages the real-time communication.

[2297] Specific operation: The video stream is processed in real time using a WebRTC server and the stream is sent to the other device.

[2298] Input: Captured video and audio data.

[2299] Data Computing: Compression and transmission of data streams via WebRTC.

[2300] Output: Real-time video and audio communication.

[2301] Step 7:

[2302] Reminder settings

[2303] How it works: The user sets a reminder to take action at a specific time.

[2304] Example: Set a reminder to "take your morning medicine."

[2305] Input: Reminder information set by the user.

[2306] Data Calculation: Reminder information is organized and converted to JSON format.

[2307] Output: JSON data to send organized reminder information to the server.

[2308] Step 8:

[2309] Server reminder storage and notifications

[2310] How it works: The server stores the reminder information in a database and sends notifications to the device at the specified time.

[2311] Specific behavior: Saves reminder information in the database and sends a push notification at the specified time.

[2312] Input: JSON data of organized reminder information.

[2313] Data calculation: Save reminder information and trigger notifications at specified times.

[2314] Output: Send reminder notification.

[2315] Step 9:

[2316] Display reminders on your device

[2317] Action: The device displays the notification in its user interface.

[2318] Specific behavior: Display a reminder notification on the smart glasses display.

[2319] Input: Reminder notification sent from the server.

[2320] Data Calculation: Parsing notification data and converting it into a display format.

[2321] Output: Display reminder notification on the smart glasses display.

[2322] Step 10:

[2323] Submit a medical records request

[2324] How it works: A user uses the smart glasses interface to submit a request to view their medical records.

[2325] Example: Checking an elderly person's medical records for "high blood pressure" and "arthritis."

[2326] Input: User confirmation request.

[2327] Data operation: Generates an HTTP GET request to send the request.

[2328] Output: Sends to the server as an HTTP GET request.

[2329] Step 11:

[2330] Server data acquisition and transmission

[2331] Operation: The server retrieves medical records from the database and sends them to the device.

[2332] Specific behavior: Retrieves medical records from the database and sends them to the device in JSON format.

[2333] Input: User's medical records...

Claims

1. user interface means for a user to input data and review the elder's care plan and medical records; communication means for transmitting the data to a server; a database means for the server to store and manage the data; communication means for the server to transmit the data to a user interface means; A system including:

2. a user interface means for conducting a real-time video call; a communication means for transmitting and receiving video streams; server means for processing and managing said video streams; The system of claim 1 , comprising:

3. user interface means for setting and notifying reminders; communication means for transmitting reminder information to a server; a server means for storing the reminder information and issuing a notification at a specified time; communication and display means for receiving and displaying said reminder notice to a user; The system of claim 1 , comprising:

Images