System

Abstract

A system is provided.SOLUTION: A nursing care system, comprising: means for inputting login information; means for transmitting the login information to a server and authenticating the login information received by the server; means for inputting an instruction related to a nursing care service; means for receiving the instruction input by the server and transmitting the instruction to a AI robot; and means for executing the nursing care service based on the instruction received by the AI robot, the system includes a means for photographing the executing state by a camera, a means for transmitting the video photographed by the AI robot to a server in real time, and a means for distributing the video received by the server to terminals and displaying the video to users by the terminals.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Current nursing care work requires knowledge and physical strength, placing a heavy burden on caregivers. In addition to the heavy workload, caregivers are at high risk of exposure to violence and stress from patients. This situation reduces job satisfaction among caregivers and contributes to a high turnover rate. Furthermore, the demand for nursing care is increasing in an aging society, and the labor shortage is becoming more serious. There is a need to improve this situation and provide efficient and safe nursing care services while reducing the burden on caregivers. [Means for solving the problem]

[0005] To address the above-mentioned issues, the present invention provides a system for introducing AI-equipped robots into nursing care settings. Specifically, the system includes: a means for a user to input login information using a terminal; a means for the terminal to transmit the login information to a server and for the server to authenticate the received login information; a means for the user to input instructions related to nursing care tasks from the terminal; a means for the server to receive the input instructions and forward them to an AI robot; a means for the AI ​​robot to perform nursing care tasks based on the received instructions and capture the performance status with a camera; a means for transmitting the video captured by the AI ​​robot in real time to the server; and a means for the server to distribute the received video to a terminal, which then displays the video to the user. This system eliminates the need for caregivers to perform nursing care tasks directly, thereby relieving them of physical and psychological stress. Furthermore, data received by the server is communicated using a secure protocol, ensuring data security and reliability. The system also includes a means for generating an authentication token when a user logs in from a terminal and using the token to verify the integrity of subsequent instructions, enhancing system security.

[0006] A "user" is a person who accesses a terminal to direct or monitor care work using the system.

[0007] A "terminal" is an electronic device used by a user to give instructions for care work and check the status of care.

[0008] "Login information" refers to authentication data such as ID and password required to authenticate a user.

[0009] The "server" is a central system that receives login information and instructions for care work, processes the data, and transfers it to the AI ​​robot.

[0010] An "authentication token" is a digital certificate that is issued upon successful login and is used to verify that the user's operation is legitimate.

[0011] "Care work" refers to specific tasks aimed at caring for, transporting, and protecting patients.

[0012] An "AI robot" is an autonomous mechanical device that performs nursing care tasks based on instructions it receives and records its work with a camera.

[0013] The "camera" is a device installed on the AI ​​robot that captures the status of nursing care work in real time.

[0014] A "secure protocol" is a communication method used to protect the confidentiality and integrity of data communications.

[0015] A "database" is a storage system that allows the server to store and collate data such as login information and nursing care instructions.

[0016] "Real-time" means that processing or communication occurs immediately with little or no delay. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The present invention relates to a nursing robot system equipped with AI for supporting the work of a caregiver. Hereinafter, an embodiment of the present invention will be specifically described.

[0039] In this system, users use a terminal to give instructions on care tasks, and the AI ​​robot carries out the tasks based on those instructions. Furthermore, the progress of the tasks can be captured on a camera and monitored remotely in real time.

[0040] Login and Authentication Process

[0041] Authentication process

[0042] When a user tries to access a system using a terminal, they must first log in. The terminal displays a login screen to the user and asks them to enter their login ID and password. Once the user enters the login information, the terminal sends it to the server. The server compares the received login information with a database and authenticates whether the user is a legitimate user. If this authentication is successful, the server sends an authentication token to the terminal, and the user gains access to the system.

[0043] Directing and carrying out nursing care tasks

[0044] Sending instructions

[0045] After the user logs in from the device, an interface for inputting instructions for nursing care work is displayed. For example, specific nursing care instructions such as "move patient A from the wheelchair to the bed" are entered. The device sends this instruction along with an authentication token to the server. The server analyzes the received instructions and forwards them to the AI ​​robot.

[0046] Robot movement

[0047] The AI ​​robot receives instructions from the server and begins its nursing care tasks based on those instructions. For example, it safely performs tasks such as moving and providing care. While working, the robot captures its actions with a camera and sends the footage to the server in real time.

[0048] Video distribution and monitoring

[0049] Real-time monitoring

[0050] The server receives the video sent by the AI ​​robot. This video is then distributed to the terminal using a secure protocol. The terminal displays the video in real time, allowing the user to monitor it and issue additional instructions as needed.

[0051] Specific examples

[0052] For example, if a user inputs an instruction into a terminal to move patient A from a wheelchair to a bed at 9:00 a.m., the system operates as follows:

[0053] 1. The user logs in to the terminal and enters the instruction "Move patient A from the wheelchair to the bed."

[0054] 2. The terminal sends the entered instructions to the server.

[0055] 3. The server receives the instructions and forwards them to the AI ​​robot.

[0056] 4. The AI ​​robot begins its nursing care duties based on the instructions it receives.

[0057] 5. While the AI ​​robot is working, a camera captures its movements and sends the footage to a server in real time.

[0058] 6. The server receives the video and distributes it to the device using a secure protocol.

[0059] 7. The user checks the real-time video on the terminal and enters additional instructions as necessary.

[0060] This invention realizes automation and efficiency of nursing care work, dramatically reducing the burden on caregivers, making it an extremely useful system in an aging society.

[0061] The processing flow will be explained below.

[0062] Step 1:

[0063] The user accesses the login screen of the device, and the device displays a form for entering the login ID and password.

[0064] Step 2:

[0065] The user enters their login ID and password and clicks the send button.

[0066] Step 3:

[0067] The terminal encrypts the entered login ID and password and sends them to the server.

[0068] Step 4:

[0069] The server decrypts the encrypted login ID and password it receives and checks them against the database.

[0070] Step 5:

[0071] The server checks the authentication result, and if the authentication is successful, generates an authentication token and sends it to the device.

[0072] Step 6:

[0073] The terminal receives a successful authentication message and token and displays to the user that they have been granted access to the system.

[0074] Step 7:

[0075] The user accesses an interface that allows them to input instructions for nursing care tasks into the terminal. For example, they input an instruction to "move patient A from the wheelchair to the bed."

[0076] Step 8:

[0077] The terminal transmits the input instruction data and the authentication token to the server.

[0078] Step 9:

[0079] The server checks the received instruction data and authentication token and analyzes the instruction content.

[0080] Step 10:

[0081] The server then transfers the analyzed instruction data to the AI ​​robot.

[0082] Step 11:

[0083] The AI ​​robot receives and analyzes the instructions, then begins its caregiving tasks, such as moving patient A from a wheelchair to a bed.

[0084] Step 12:

[0085] While the AI ​​robot is performing its nursing care duties, a camera will record what is happening.

[0086] Step 13:

[0087] The AI ​​robot sends the footage it captures in real time to a server.

[0088] Step 14:

[0089] The server delivers the received video data to the terminal using a secure protocol.

[0090] Step 15:

[0091] The terminal displays the video data received from the server to the user.

[0092] Step 16:

[0093] The user monitors the video footage through the device to ensure that care work is being carried out properly.

[0094] Step 17:

[0095] If necessary, the user inputs additional instructions, and the terminal transmits the data to the server again.

[0096] Step 18:

[0097] The server forwards the received additional instructions to the AI ​​robot, which then changes or performs additional nursing care tasks based on the instructions.

[0098] Example 1

[0099] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0100] As the aging society progresses, the workload of caregivers is increasing. In particular, the diversity and heavy workload of caregiving tasks have become issues, and there is a need to improve work efficiency and reduce the burden. In addition, monitoring and giving instructions on caregiving tasks from remote locations is also very important, so a system that can grasp the situation in real time and respond quickly is required.

[0101] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0102] In this invention, the server includes: means for a user to input login information using a device; means for the device to transmit the login information to a computer and for the computer to authenticate the received login information; means for the user to input instructions related to nursing care tasks from the device; means for the computer to receive the input instructions and forward the instructions to an artificial intelligence device; means for the artificial intelligence device to perform the nursing care tasks based on the received instructions and capture the performance status with an imaging device; means for the artificial intelligence device to transmit the captured image to the computer in real time; means for the computer to distribute the received image to the device and for the device to display the image to the user; means for communicating the data received by the computer using a secure protocol; and means for generating an authentication token when a user logs in from the device and using the token to confirm the consistency of subsequent instructions. This automates and streamlines nursing care tasks, reduces the burden on caregivers, and enables remote monitoring and instruction of nursing care tasks in real time.

[0103] "User" refers to people such as caregivers and care staff who use the system.

[0104] "Device" refers to an electronic terminal used for logging in, inputting instructions for care work, displaying images, etc.

[0105] "Login information" refers to information such as a user name and password that a user enters when accessing a system.

[0106] "Computer" refers to a server device that receives data sent from devices and performs authentication, analyzes instructions, distributes video, etc.

[0107] "Artificial intelligence device" refers to a robot or device equipped with AI that receives instructions from a computer and performs nursing care tasks based on those instructions.

[0108] "Photography device" refers to a device for photographing the status of nursing care work, such as a camera installed in an artificial intelligence device.

[0109] A "secure protocol" refers to a protocol used to ensure that data is transmitted securely. An example is HTTPS.

[0110] "Authentication Token" means temporary authentication information generated after login and used to authenticate subsequent communications and instructions.

[0111] The present invention relates to an AI-equipped nursing robot system for supporting nursing care work. By using this system, the efficiency and automation of nursing care work can be improved, and the burden on caregivers can be reduced. Detailed embodiments for carrying out the present invention will be described.

[0112] This system is operated by the user using a device, and is composed of a combination of multiple hardware and software components, including devices, computers, artificial intelligence devices, and imaging devices.

[0113] Hardware and software used

[0114] Device: An electronic device (e.g., laptop, smartphone, tablet) used to log in, input care instructions, and display images.

[0115] Computer: A server device (e.g., AWS EC2, Microsoft Azure) that receives data sent from devices and performs authentication, analyzes instructions, and distributes video.

[0116] Artificial intelligence device: A robot that receives instructions from a computer and performs caregiving tasks based on those instructions (e.g., SoftBank's Pepper, Riken-TRI Collaboration Center for Human-Interactive Robot Research's ROBEAR).

[0117] Camera: A camera that records the nursing care work being performed (e.g., Logitech C920).

[0118] Program processing and data processing

[0119] To access a system using a device, a user must first log in. The device displays a login screen to the user, prompting them to enter a username and password. After the user enters and submits this information, the device sends the information to a computer. The computer compares the received login information with a database to authenticate whether the user is legitimate. If authentication is successful, the computer generates an authentication token and sends it to the device. This token enables subsequent communication to be secure.

[0120] After logging in, the user inputs instructions for care work into the device. For example, specific instructions such as "move patient A from the wheelchair to the bed." The device then sends the instructions along with an authentication token to the computer. The computer analyzes the instructions and forwards them to the AI ​​device. The AI ​​device then begins the care work based on the received instructions.

[0121] While the caregiving task is being performed, the AI ​​device captures the situation with a camera and transmits the video in real time to a computer. The computer then distributes the received video to the device, where the user can review it. The user can input additional instructions into the device as needed and remotely manage the progress of the caregiving task.

[0122] Specific examples

[0123] For example, if a user inputs an instruction into the device to move patient A from his wheelchair to a bed at 9:00 a.m., the operation will proceed as follows:

[0124] 1. The user logs in to the device and enters the instruction "Move patient A from the wheelchair to the bed."

[0125] 2. The device sends the input instructions to the computer.

[0126] 3. The computer receives the instructions and forwards them to the artificial intelligence device.

[0127] 4. Based on the instructions received by the artificial intelligence device, it begins the process of moving Patient A from the wheelchair to the bed.

[0128] 5. While the AI ​​device is working, it captures its actions with a camera and transmits the footage to a computer in real time.

[0129] 6. The computer receives the video and distributes it to the device using a secure protocol.

[0130] 7. The user views the real-time video on the device and enters additional instructions.

[0131] Example prompt sentence:

[0132] We are currently designing an AI-powered nursing robot system to assist caregivers in their work. What technologies can we use to automate the following nursing tasks? Please provide details on the login and authentication process, the instructions and execution of nursing tasks, and the video streaming and monitoring. Please provide specific examples.

[0133] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0134] Step 1:

[0135] A user uses a device to enter login information (username and password) to access a system, and the information entered is collected by the terminal.

[0136] Step 2:

[0137] The device sends the collected login information to the server using a secure protocol (e.g., HTTPS). The input data is sent to the server as transmission data.

[0138] Step 3:

[0139] The server compares the received login information with the database and authenticates whether the user is a valid user. This authentication process verifies whether the entered login information matches the registered information in the database. If authentication is successful, the server generates an authentication token and sends it to the terminal as output data.

[0140] Step 4:

[0141] After logging in, the user inputs instructions for care work (e.g., "Move patient A from the wheelchair to the bed") into the device. The input instructions are collected by the terminal.

[0142] Step 5:

[0143] The device sends the collected instructions and the authentication token to the server using a secure protocol. At this stage, the input data are the instructions and the authentication token.

[0144] Step 6:

[0145] The server analyzes the received instructions and extracts the information necessary to understand the content of the instructions. This analysis process provides specific instruction information. Once the analysis is complete, it generates output data to transmit the instructions to the artificial intelligence device.

[0146] Step 7:

[0147] Based on the analysis results, the server transfers specific instructions to the AI ​​device, which receives the instructions and begins its caregiving duties.

[0148] Step 8:

[0149] The AI ​​device performs a care task based on the received instructions. For example, it may move a patient from a wheelchair to a bed. While the AI ​​device is performing the task, it takes pictures of the task with a camera. The photographed data is generated.

[0150] Step 9:

[0151] The AI ​​device transmits the captured video to a server in real time, and the input data is the captured video information.

[0152] Step 10:

[0153] The server then transmits the received video to the terminal using a secure protocol. The transmitted video is the output data.

[0154] Step 11:

[0155] The user checks the real-time video on the terminal. While monitoring this video, the user inputs additional instructions into the device as needed. The additional instructions are collected as input data.

[0156] Step 12:

[0157] The terminal sends additional instructions to the server, which then forwards the instructions to the AI ​​device. This cycle is repeated to continuously manage the care work.

[0158] (Application example 1)

[0159] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0160] In today's aging society, the shortage of caregivers and other care workers is becoming a serious problem. Furthermore, in order to perform caregiving and monitoring tasks efficiently and accurately, it is essential to reduce the burden on caregivers, automate tasks, and speed up monitoring. Therefore, real-time monitoring and rapid response when an abnormality occurs are required.

[0161] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0162] In this invention, the server includes: a means for a user to input login information using a terminal; a means for the terminal to transmit the login information to the server and for the server to authenticate the received login information; a means for the user to input instructions related to nursing care or monitoring work from the terminal; a means for the server to receive the input instructions and forward the instructions to the worker robot; a means for the worker robot to perform the nursing care or monitoring work based on the received instructions and record the execution status in a recording device; a means for the worker robot to transmit video recorded by the worker robot to the server in real time; a means for the server to distribute the received video to the terminal and for the terminal to display the video to the user; and a means for notifying the user when an abnormality is detected. This enables the automation and efficiency of nursing care and monitoring work, reduces the burden on workers, and enables rapid response when an abnormality occurs.

[0163] "User" means an individual or entity that uses a terminal to operate the System.

[0164] A "terminal" is an electronic device used to access and operate the system.

[0165] "Login Information" means identifying information, such as a username and password, for authenticating access to a system.

[0166] A "server" is a computer that is the core of the system and is a device that authenticates login information, receives and transfers instructions, and stores and distributes data.

[0167] "Authentication" is the process of verifying that a user has legitimate authority to access a system.

[0168] "Instructions" are information that the user inputs via the terminal to provide specific work content related to care work or monitoring work.

[0169] A "worker robot" is an automated machine equipped with artificial intelligence that performs nursing and monitoring tasks based on instructions it receives.

[0170] "Nursing care work" refers to the specific tasks involved in providing care to patients and elderly people.

[0171] "Surveillance work" refers to the regular or continuous observation of a facility or specific area to ensure its safety.

[0172] The "recording device" is a device that records the status of the work performed by the worker robot as video and data.

[0173] "Real-time" refers to the instantaneous transfer of data and video without delay.

[0174] "Distribution" means that the server sends the video and data it receives to the terminal and displays it.

[0175] "Notification" is the process of sending alerts and messages to users when the system detects an abnormality.

[0176] overview

[0177] The present invention is a system that allows a user to log in using a terminal and input instructions to have a worker robot perform nursing care or monitoring tasks. This system includes a server, a terminal, a worker robot, a recording device, and a real-time distribution and notification function.

[0178] System Configuration

[0179] server

[0180] The server acts as the center of the system and performs the following functions:

[0181] 1. Login authentication: Receives the login information entered by the user from the terminal and authenticates it by checking it against a database.

[0182] 2. Instruction transfer: Analyzes instructions received from the user and transfers them to the corresponding worker robot.

[0183] 3. Video distribution: Real-time video received from the worker robot is distributed to the terminal.

[0184] 4. Secure communication: Received data is communicated using a secure protocol (e.g. HTTPS).

[0185] 5. Abnormality notification: If the worker robot or user detects an abnormality, a notification is sent to the user.

[0186] Terminal

[0187] The terminal is an electronic device that allows users to operate the system and has the following functions:

[0188] 1. Entering login information: The user enters their login ID and password.

[0189] 2. Inputting instructions: Entering instructions regarding nursing care and monitoring tasks.

[0190] 3. Video display: Displays real-time video streamed from the server.

[0191] 4. Notification reception: Receive notifications sent from the server when an abnormality occurs.

[0192] Worker robot

[0193] Employee robots are automated machines equipped with AI that perform the following functions:

[0194] 1. Execute Instructions: Performs caregiving or supervisory tasks based on received instructions.

[0195] 2. Recording of execution status: The status of work is recorded in real time by a camera and sent to a server as video data.

[0196] Hardware and software used

[0197] Server: A computer with high-performance computing power (e.g., Amazon Web Services, Google Cloud Platform)

[0198] Devices: Electronic devices such as smartphones, tablets, and computers

[0199] Recording device: High-resolution camera (e.g. Logitech Webcam)

[0200] AI model: Generative AI model (e.g. GPT-3, BERT)

[0201] Secure protocol: HTTPS

[0202] Specific examples

[0203] If a user uses a terminal to input instructions to monitor a specific area at 9:00 a.m., the system operates as follows:

[0204] 1. Terminal: The user enters login information and accesses the system.

[0205] 2. Server: Authenticates the login information and allows the user access. When the user enters an instruction to "monitor a specific area," the server receives this instruction.

[0206] 3. Server: Analyzes the received instructions and forwards them to the appropriate worker robot.

[0207] 4. Worker robot: Based on instructions, it patrols designated areas and performs surveillance tasks. It records images of its patrols in real time with a camera.

[0208] 5. Server: Receives the video sent from the robot and distributes it to the terminal.

[0209] 6. Terminal: The user checks the real-time video and inputs additional instructions if an abnormality is detected. If an abnormality is detected, the user receives a notification.

[0210] Prompt Sentence Examples

[0211] To further refine the design using the generative AI model, use the following prompt:

[0212] Please explain the design of a security robot system. The system allows users to log in from their smartphones and instruct the AI-equipped robots on how to perform monitoring tasks. The robots will send real-time video footage to the smartphones. If an abnormality is detected, notifications will be sent to the users. This system will be built using Python, Flask, and OpenCV.

[0213] This prompt can be used to obtain detailed design information and implementation methods from the AI ​​model.

[0214] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0215] Step 1:

[0216] The user enters their login information using the device.

[0217] Input: User's login ID and password

[0218] Operation: The user enters login information into the device's login screen and presses the "Login" button.

[0219] Output: The device sends the login information to the server.

[0220] Step 2:

[0221] The server authenticates the received login information.

[0222] Input: Login ID and password sent from the device

[0223] How it works: The server checks the received login information against a database to determine if the user is legitimate.

[0224] Output: If authentication is successful, the server generates an authentication token and sends it to the terminal. If authentication fails, it returns an error message to the terminal.

[0225] Step 3:

[0226] The user inputs instructions regarding the care work or monitoring work from the terminal.

[0227] Input: Authentication token after logging in, instructions for care work or monitoring work (e.g., "Monitor a specific area")

[0228] Operation: The terminal receives instructions entered by the user and sends them to the server.

[0229] Output: Data containing instructions and an authentication token is sent to the server.

[0230] Step 4:

[0231] The server receives the input instructions and forwards the instructions to the worker robot.

[0232] Input: Instructions, authentication token

[0233] Operation: The server analyzes the received instructions and forwards them to the corresponding worker robot.

[0234] Output: The parsed instructions are sent to the worker robot.

[0235] Step 5:

[0236] The worker robot performs caregiving or monitoring tasks based on the received instructions.

[0237] Input: Instructions sent by the server

[0238] Action: The worker robot follows instructions and begins nursing or monitoring tasks.

[0239] Output: The execution status is recorded in real time by a camera.

[0240] Step 6:

[0241] The worker robot transmits the recorded video to a server in real time.

[0242] Input: Camera footage

[0243] Operation: The worker robot transmits the recorded video in real time to the server.

[0244] Output: Real-time video data is sent to the server.

[0245] Step 7:

[0246] The server distributes the received video to the terminal, and the terminal displays the video to the user.

[0247] Input: Real-time video sent from the worker robot

[0248] Operation: The server delivers the received video to the device using a secure protocol.

[0249] Output: The distributed real-time video is displayed on the terminal.

[0250] Step 8:

[0251] The server will notify the user when an abnormality is detected.

[0252] Input: Anomaly detection information or manual detection input by the user

[0253] Operation: When the server detects an abnormality, it sends a push notification to the user device.

[0254] Output: An abnormality notification is displayed on the user's terminal.

[0255] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0256] In addition to the conventional nursing care support system, the present invention provides a new technology that recognizes and responds to the emotions of users and patients. Hereinafter, embodiments of the present invention will be described in detail.

[0257] In this system, the user uses a terminal to give instructions on care tasks, and the AI ​​robot carries out the tasks based on those instructions. The system also captures the progress of the tasks with a camera and can be monitored remotely in real time. Furthermore, an emotion engine is used to recognize the emotions of the user and patient, optimizing the operation of the entire system.

[0258] Login and Authentication Process

[0259] Authentication process

[0260] When a user tries to access a system using a terminal, they must first log in. The terminal displays a login screen to the user and asks them to enter their login ID and password. Once the user enters the login information, the terminal sends it to the server. The server compares the received login information with a database and performs authentication. If this authentication is successful, the server sends an authentication token to the terminal, and the user gains access to the system.

[0261] Furthermore, the emotion engine analyzes the user's facial expressions and voice to recognize their emotional state at the time of login, which makes it possible to detect the user's stress level and fatigue at that time.

[0262] Directing and carrying out nursing care tasks

[0263] Sending instructions

[0264] After the user logs in from the device, an interface for inputting instructions for care work is displayed. For example, specific care instructions such as "move patient A from the wheelchair to the bed" are entered. The device sends this instruction along with an authentication token to the server. The server analyzes the received instructions and forwards them to the AI ​​robot.

[0265] In addition, when inputting instructions, the emotion engine analyzes the user's emotions and sends the results to the server, which can then transfer instructions to the AI ​​robot that take into account the user's emotional state.

[0266] Robot movement

[0267] The AI ​​robot receives instructions from the server and begins its nursing care tasks based on those instructions. For example, it safely performs tasks such as moving and assisting the patient. While working, the robot captures its movements with a camera and sends the footage to the server in real time. Furthermore, the AI ​​robot is equipped with an emotion engine that recognizes emotions from the patient's facial expressions and voice, and adjusts its tasks as necessary.

[0268] Video distribution and monitoring

[0269] Real-time monitoring

[0270] The server receives the video sent by the AI ​​robot. This video is then distributed to the device using a secure protocol. The device displays the video in real time, allowing the user to monitor it. The emotion engine continuously monitors the user's emotional state and issues an alert if an abnormality is detected.

[0271] Specific examples

[0272] For example, if a user inputs an instruction into a terminal to move patient A from a wheelchair to a bed at 9:00 a.m., the system operates as follows:

[0273] 1. The user logs in to the terminal and inputs the command, "Move patient A from the wheelchair to the bed." At this time, the emotion engine analyzes the user's facial expressions and voice and sends the emotional state to the server.

[0274] 2. The device sends the input instructions to the server, which then analyzes the received instruction data and emotion data.

[0275] 3. The server transfers instructions and emotional data to the AI ​​robot. If the user is under stress, the robot will adjust its work pace to take that situation into account.

[0276] 4. The AI ​​robot begins its nursing care tasks based on the instructions. When transferring Patient A from the wheelchair to the bed, the emotion engine also monitors the patient's emotional state.

[0277] 5. While the AI ​​robot is working, a camera captures its movements and sends the footage to a server in real time.

[0278] 6. The server delivers the video data to the device, which displays the video to the user in real time.

[0279] 7. The user monitors the video on the device to ensure that the care work is being carried out properly. The emotion engine also continuously monitors the user's emotions and issues an alert if there are any abnormalities.

[0280] This invention not only realizes automation and efficiency in nursing care work, dramatically reducing the burden on caregivers, but also provides a safer and more comfortable nursing environment through emotion recognition, making it an extremely useful system in an aging society.

[0281] The processing flow will be explained below.

[0282] Step 1:

[0283] The user accesses the login screen of the device, and the device displays a form for entering the login ID and password.

[0284] Step 2:

[0285] The user enters their login ID and password and clicks the send button.

[0286] Step 3:

[0287] The terminal encrypts the entered login ID and password and sends them to the server.

[0288] Step 4:

[0289] The server decrypts the encrypted login ID and password it receives and checks them against the database.

[0290] Step 5:

[0291] The server checks the authentication result, and if the authentication is successful, generates an authentication token and sends it to the device.

[0292] Step 6:

[0293] The device receives a successful authentication message and token, and displays a message to the user indicating that they have been granted access to the system. The emotion engine analyzes the user's facial expressions and voice to confirm their emotional state at that time.

[0294] Step 7:

[0295] The user accesses an interface that allows them to input instructions for nursing care tasks into the terminal. For example, they input an instruction to "move patient A from the wheelchair to the bed."

[0296] Step 8:

[0297] The emotion engine analyzes the facial expressions and voice of the user inputting instructions and generates emotion data.

[0298] Step 9:

[0299] The terminal transmits the input instruction data, emotion data, and authentication token to the server.

[0300] Step 10:

[0301] The server checks the received instruction data, emotion data, and authentication token, and analyzes the instruction content, taking into account the user's emotional state.

[0302] Step 11:

[0303] The server analyzes the instruction data and transfers it to the AI ​​robot. If the user is under stress, the instructions to the robot are adjusted.

[0304] Step 12:

[0305] The AI ​​robot receives and analyzes the instructions, then begins its caregiving tasks, such as moving patient A from a wheelchair to a bed.

[0306] Step 13:

[0307] The emotion engine analyzes the patient's facial expressions and voice to assess their emotional state and adjust the work as needed.

[0308] Step 14:

[0309] While the AI ​​robot is performing its nursing care duties, a camera will record what is happening.

[0310] Step 15:

[0311] The AI ​​robot sends the footage it captures in real time to a server.

[0312] Step 16:

[0313] The server delivers the received video data to the terminal using a secure protocol.

[0314] Step 17:

[0315] The terminal displays the video data received from the server to the user, and the emotion engine continuously monitors the user's emotional state.

[0316] Step 18:

[0317] Users can monitor the video footage via their devices to ensure that care work is being carried out properly, and if there is an abnormality, a warning is issued.

[0318] Step 19:

[0319] If necessary, the user inputs additional instructions, and the terminal transmits the data to the server again.

[0320] Step 20:

[0321] The server forwards the received additional instructions to the AI ​​robot, which then changes or performs additional nursing care tasks based on the instructions.

[0322] Example 2

[0323] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0324] Conventional nursing care support systems have difficulty recognizing the emotional states of caregivers and patients and responding appropriately. Furthermore, even though remote real-time monitoring is possible, there is a lack of systems that can optimize behavior based on emotions or detect abnormalities, making it difficult to fully ensure the safety and efficiency of nursing care work.

[0325] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input login information using a terminal; a means for the terminal to transmit the login information to the server and for the server to authenticate the received login information; a means having an emotion analysis engine for recognizing the emotional states of the user and the patient; a means for the user to input instructions related to the care work from the terminal; a means for the server to receive, analyze, and forward the input instructions to the AI ​​robot; a means for the AI ​​robot to perform the care work based on the received instructions and capture the performance status with a camera; a means for transmitting the video captured by the AI ​​robot to the server in real time; a means for the server to optimize the operation by taking into account the emotional states of the user and the patient via the emotion analysis engine; a means for the server to distribute the received video to the terminal, which then displays the video to the user and continuously monitors the user's emotional state; and a means for issuing an alert when an abnormality is detected. This improves the safety and efficiency of care work and enables more appropriate care that takes into account the emotional state.

[0326] "Login information" refers to the authentication data that a user enters when accessing a system, and typically consists of a user ID and password.

[0327] A "terminal" is a device used by a user to access the nursing care support system, input instructions, and monitor video footage, and includes PCs, tablets, smartphones, etc.

[0328] The "server" is a computer system that serves as the core of the nursing care support system and performs various processes such as login authentication, instruction analysis, data transfer, emotional state analysis, and real-time video distribution.

[0329] An "emotion analysis engine" is a software or hardware component that analyzes the facial expressions and voice of a user or patient to recognize and evaluate their emotional state.

[0330] "Instruction data" refers to data that includes specific instructions regarding care work input by the user via the terminal.

[0331] An "AI robot" is an autonomous robot that performs nursing care tasks based on instructions received from a server, and is equipped with a camera, emotion analysis engine, and other features.

[0332] The "camera" is a device installed on the AI ​​robot that captures the nursing care work being performed in real time and sends the footage to a server.

[0333] A "secure protocol" is a secure communication method that prevents data transmission and reception from being tampered with or intercepted by third parties, and includes SSL / TLS.

[0334] An "authentication token" is temporary data issued by the server when a user is successfully authenticated, and is used to verify that subsequent communications and instructions are legitimate.

[0335] "Real-time streaming" is a technology that transmits footage captured by an AI robot from a server to a terminal with almost no delay, allowing users to monitor it immediately.

[0336] "Abnormality detection" is a function that allows the system to detect abnormal patterns in the emotional state of a user or patient or the execution of care tasks and issue a warning.

[0337] In addition to the conventional nursing care support system, the present invention provides a technology that recognizes the emotions of users and patients and takes optimal measures according to their emotional states. Hereinafter, embodiments of the present invention will be described in detail.

[0338] This system consists of the following main components: a server, a terminal, an AI robot, a camera, and an emotion analysis engine. The functions and interactions of each component are described in detail below.

[0339] Server Roles

[0340] The server acts as the central part of the system and performs the following main tasks:

[0341] 1. Login authentication: The login information entered by the user from the terminal is compared with the database. If authentication is successful, an authentication token is issued.

[0342] 2. Receiving and analyzing instruction data: Receives and analyzes instructions for nursing care tasks entered by the user through the terminal.

[0343] 3. Transfer of instruction data: Based on the analysis results, appropriate instruction data is transferred to the AI ​​robot.

[0344] 4. Sentiment analysis and optimization: Use a sentiment analysis engine to analyze the emotional state of users and patients and optimize operations.

[0345] 5. Real-time video distribution: Receives video data sent from the AI ​​robot and distributes it to the terminal in real time.

[0346] 6. Anomaly detection and warning: If an abnormality is detected in the user or patient's condition, a warning will be issued to the terminal.

[0347] Device Role

[0348] The terminal is used by the user to access the system, enter commands, and monitor footage. Terminal functions include:

[0349] 1. Displaying a login screen: A login screen is displayed to the user and they are prompted to enter their login information.

[0350] 2. Providing an instruction input interface: An interface is provided for the user to input specific nursing care instructions.

[0351] 3. Real-time video display: Displays the video received from the server in real time to monitor the progress of nursing care work.

[0352] 4. Displaying a warning notification: When an abnormality is detected, a warning is displayed to the user.

[0353] The role of AI robots

[0354] The AI ​​robot performs care tasks based on instructions received from the server. Specific functions include:

[0355] 1. Carrying out care tasks: Carry out care tasks (e.g., moving or assisting patients) according to received instructions.

[0356] 2. Video recording: The nursing care work is recorded with a camera and the video is sent to the server in real time.

[0357] 3. Emotion Analysis: Use the built-in emotion analysis engine to analyze the patient's emotional state and optimize the progress of the work.

[0358] The role of sentiment analysis engines

[0359] The emotion analysis engine analyzes the facial expressions and voice of users and patients to recognize their emotional state, providing important data for optimizing the operation of the entire system.

[0360] Specific examples

[0361] For example, if a user inputs the instruction "Move patient A from his wheelchair to his bed" into the terminal at 9:00 AM, the system will perform the following specific operations:

[0362] 1. The user logs in to the device and inputs instructions. The emotion analysis engine analyzes the user's facial expressions and voice and sends their emotional state to the server.

[0363] 2. The terminal transmits the input instructions and emotion data to the server.

[0364] 3. The server analyzes the instruction data and emotional data and transmits them to the AI ​​robot. If stress is detected, the robot will adjust its work pace.

[0365] 4. The AI ​​robot will begin nursing care tasks based on instructions, while simultaneously analyzing the patient's emotions and optimizing its tasks accordingly.

[0366] 5. The AI ​​robot uses a camera to record its work and sends the footage to a server in real time.

[0367] 6. The server delivers the video data to the terminal and displays it to the user in real time.

[0368] 7. The user monitors the video through the device, and if the emotion analysis engine detects an abnormality, an alert is issued.

[0369] Prompt Sentence Examples

[0370] "Transfer patient A from wheelchair to bed at 9:00 AM"

[0371] "Please notify me of patient B's medication time."

[0372] "Check on patient C's safety"

[0373] This invention not only realizes automation and efficiency of nursing care work, reducing the burden on caregivers, but also makes it possible to provide a safer and more comfortable nursing care environment through emotion recognition.

[0374] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0375] Step 1:

[0376] Enter and submit your login information

[0377] The user accesses the login screen using the device and enters the login ID and password.

[0378] Input: The login ID and password entered by the user.

[0379] Output: Data format in which login information (ID and password) is sent to the terminal

[0380] Specific operation: The terminal receives the user's input and sends it to the server.

[0381] Step 2:

[0382] Authenticate your login details

[0383] The server compares the login information received from the terminal with the database to perform authentication.

[0384] Input: Login information sent from the device

[0385] Output: Authentication token or authentication failure message

[0386] Specific operation: The server checks the login information against the database, and if authentication is successful, it generates an authentication token and sends it to the terminal. If authentication fails, it returns an error message.

[0387] Step 3:

[0388] Emotional state analysis

[0389] The emotion analysis engine analyzes the user's facial expressions and voice to recognize their emotional state at the time of login.

[0390] Input: User's facial expression and voice data

[0391] Output: Emotional state information (stress level, fatigue level, etc.)

[0392] How it works: The emotion analysis engine monitors the user's face and voice in real time and evaluates their state using an emotion analysis algorithm. The evaluation results are sent to the server and recorded.

[0393] Step 4:

[0394] Entering nursing care instructions

[0395] The user inputs specific instructions regarding the care work from the terminal.

[0396] Input: Care work instructions entered by the user

[0397] Output: Instruction data (specific nursing care work content)

[0398] Specific operation: The terminal provides an interface for inputting instructions for nursing care work, and the user inputs specific instructions (e.g., "Move patient A from the wheelchair to the bed").

[0399] Step 5:

[0400] Sending and analyzing instruction data

[0401] Care work instructions and emotional state information are sent from the terminal to the server, which then analyzes them.

[0402] Input: Care instructions and emotional state information

[0403] Output: Analysis results (optimized instruction data)

[0404] Specific operation: The server analyzes the received instruction data and generates optimized instructions that take into account the emotional state information. These instructions are then forwarded to the AI ​​robot.

[0405] Step 6:

[0406] Carrying out nursing care tasks with AI robots

[0407] The AI ​​robot performs nursing care tasks based on instructions from the server.

[0408] Input: Optimized instruction data

[0409] Output: Status of nursing care work and video data

[0410] Specific actions: The AI ​​robot begins specific care tasks (e.g., moving a patient) based on the instructions it receives. The robot's actions are captured in real time by a camera.

[0411] Step 7:

[0412] Real-time video transmission and distribution

[0413] The video data captured by the AI ​​robot is sent to a server, which then distributes it to the device in real time.

[0414] Input: Real-time video data

[0415] Output: Real-time video streaming data

[0416] How it works: The AI ​​robot captures video of itself working with a camera and sends it in real time to a server, which then receives the data and distributes it to the device using a secure protocol.

[0417] Step 8:

[0418] Anomaly detection and warning

[0419] The server uses an emotion analysis engine to continuously monitor the emotional state of users and patients and issues an alert if an abnormality is detected.

[0420] Input: Emotional state information and care work execution state

[0421] Output: Warning notice

[0422] Specific operation: The server analyzes the data from the sentiment analysis engine, and if an anomaly is detected, it sends a warning message to the device. The user receives the warning on the device and can take appropriate action.

[0423] In this way, this system works by linking users, servers, AI robots, and emotion analysis engines to carry out nursing care tasks efficiently and safely.

[0424] (Application example 2)

[0425] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0426] Current work support systems only transfer and execute simple instructions without considering the emotional state of the user or target, which means that the safety and efficiency of the work environment cannot be sufficiently maintained.In addition, because they are unable to provide feedback based on real-time emotion recognition, stress and fatigue levels may be overlooked, which could have a negative impact on the health and safety of workers and targets.

[0427] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input login information using a terminal; a means for the terminal to transmit the login information to the server and for the server to authenticate the received login information; a means for the user to input instructions related to nursing care work from the terminal; a means for the server to receive the input instructions and forward the instructions to the AI ​​processing device; a means for the AI ​​processing device to perform work based on the received instructions and record the execution status using an optical device; a means for the AI ​​processing device to transmit the recorded video to the server in real time; a means for the server to distribute the received video to the terminal and for the terminal to display the video to the user; a means for analyzing the emotions of the user and the target person in real time using an emotion recognition engine; a means for adjusting the operation of the AI ​​processing device based on the target person's emotional state; and a means for analyzing the video from the optical device and issuing a safety warning based on the results of emotion recognition. This makes it possible to improve the safety and efficiency of the work environment.

[0428] A "user" is a person who operates the system and gives instructions for work.

[0429] "Terminal" refers to an electronic device operated by a user, including smart glasses and PCs.

[0430] "Login information" means the authentication information provided by a user to access the system, and specifically includes a user ID and password.

[0431] A "server" is a computer system on a network that manages the entire system and processes data.

[0432] An "authentication token" is authentication data for subsequent communications that is issued by the server after a user has successfully logged in.

[0433] An "AI processing device" is a device that uses AI technology to analyze instructions and automatically execute tasks.

[0434] An "optical device" is a device for recording images, such as a camera.

[0435] An "emotion recognition engine" is a technology or software that analyzes the emotional state of a user or target person in real time from their facial expressions and voice.

[0436] "Real-time" refers to data processing and communication occurring instantly, without delay.

[0437] "Target" refers to the person to whom the system provides services, and in the case of a care robot, it means the person being cared for.

[0438] "Safety warnings" are caution or warning messages issued for the safety of users or targets based on the analysis results of the emotion recognition engine.

[0439] This invention is a system for improving the safety and efficiency of a work environment. This system is composed of a user, a terminal, a server, an AI processing device, an optical device, and an emotion recognition engine.

[0440] First, the user enters login information using the terminal. The terminal then sends this login information to the server, which then performs authentication using the received login information. If authentication is successful, the server generates an authentication token and sends it to the terminal.

[0441] Next, the user can input instructions related to the work from the device. For example, a command to start a specific task may be given. The device then sends this instruction along with an authentication token to the server. The server then analyzes the received instruction and forwards it to the AI ​​processing device based on its content.

[0442] The AI ​​processing device performs tasks based on the received instructions and records its progress using an optical device (such as a camera). The recorded video is sent to a server in real time. The server then distributes the received video to a terminal, allowing users to monitor the video in real time through their terminal.

[0443] Furthermore, the emotional state of the user and the target can be analyzed in real time using an emotion recognition engine. The emotion recognition engine analyzes the facial expressions and voice data of the user and the target, and adjusts the operation of the AI ​​processing unit based on the analysis results. It is also possible to issue safety warnings as needed based on the analysis results.

[0444] For example, if a factory worker wears smart glasses, the glasses' camera and microphone can be used to analyze the worker's facial expressions and voice to recognize their emotional state in real time. Based on this recognition result, the system can determine whether the working environment is appropriate and issue a warning if necessary.

[0445] For example, if Worker A is wearing smart glasses at the start of his shift at 8 a.m. and the emotion recognition engine detects fatigue, the system will automatically issue a warning and prompt the worker to take a break, thus optimizing the working environment and improving safety and work efficiency.

[0446] An example of a prompt sentence is as follows:

[0447] "How can we monitor the emotions of factory workers in real time and issue warnings if fatigue or stress is detected to improve work efficiency?"

[0448] This allows the system embodying the invention to take into account the emotional state of the user and the subject, providing a safer and more efficient working environment.

[0449] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0450] Step 1:

[0451] The user enters login information using the terminal. The terminal displays the login screen, and the user enters their login ID and password. The terminal then sends the entered login information to the server.

[0452] Input: User login ID and password

[0453] Output: Login information sent to the server

[0454] Step 2:

[0455] The server authenticates the received login information. The server accesses a database and checks the received login information. If authentication is successful, the server generates an authentication token and sends it to the terminal.

[0456] Input: Login information

[0457] Output: Authentication token

[0458] Step 3:

[0459] The user inputs work instructions from the terminal. The authenticated terminal displays the work instruction input screen, and the user inputs specific work instructions (e.g., instructions for a specific task). The terminal then sends these instructions to the server along with an authentication token.

[0460] Input: Work instructions, authentication token

[0461] Output: Work instructions sent to the server

[0462] Step 4:

[0463] The server receives and analyzes the input instructions, and then transfers the instructions to the appropriate AI processing device based on their content.

[0464] Input: Work Instructions

[0465] Output: Instructions to the AI ​​processor

[0466] Step 5:

[0467] The AI ​​processing unit executes tasks based on the instructions it receives, such as a robot performing a specific action. The AI ​​processing unit continues to record the progress of the task using an optical device.

[0468] Input: Instructions from the server

[0469] Output: Work progress

[0470] Step 6:

[0471] The AI ​​processing device transmits the recorded video to the server in real time. The AI ​​processing device transmits the video data captured by the optical device to the server in real time.

[0472] Input: Recorded video

[0473] Output: Video data sent to the server

[0474] Step 7:

[0475] The server delivers the received video to the terminal. The server delivers the received video to the terminal in real time using a secure protocol. The terminal displays this video to the user.

[0476] Input: Video data

[0477] Output: The image displayed to the user

[0478] Step 8:

[0479] The emotion recognition engine analyzes the emotions of the user and the target. The emotion recognition model is run using video and audio data. Based on the analysis results, it determines whether the operation of the AI ​​processing unit needs to be adjusted.

[0480] Input: Video data, audio data

[0481] Output: Emotion analysis results

[0482] Step 9:

[0483] The emotion recognition engine issues safety warnings based on the analysis results. If the analysis results indicate that the user or target is in danger, the system will automatically issue a warning and prompt the user to take necessary measures.

[0484] Input: Sentiment analysis results

[0485] Output: Safety warning

[0486] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0487] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0488] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0489] [Second embodiment]

[0490] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0491] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0492] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0493] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0494] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0495] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0496] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0497] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0498] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0499] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0500] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0501] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0502] The present invention relates to a nursing robot system equipped with AI for supporting the work of a caregiver. Hereinafter, an embodiment of the present invention will be specifically described.

[0503] In this system, users use a terminal to give instructions on care tasks, and the AI ​​robot carries out the tasks based on those instructions. Furthermore, the progress of the tasks can be captured on a camera and monitored remotely in real time.

[0504] Login and Authentication Process

[0505] Authentication process

[0506] When a user tries to access a system using a terminal, they must first log in. The terminal displays a login screen to the user and asks them to enter their login ID and password. Once the user enters the login information, the terminal sends it to the server. The server compares the received login information with a database and authenticates whether the user is a legitimate user. If this authentication is successful, the server sends an authentication token to the terminal, and the user gains access to the system.

[0507] Directing and carrying out nursing care tasks

[0508] Sending instructions

[0509] After the user logs in from the device, an interface for inputting instructions for nursing care work is displayed. For example, specific nursing care instructions such as "move patient A from the wheelchair to the bed" are entered. The device sends this instruction along with an authentication token to the server. The server analyzes the received instructions and forwards them to the AI ​​robot.

[0510] Robot movement

[0511] The AI ​​robot receives instructions from the server and begins its nursing care tasks based on those instructions. For example, it safely performs tasks such as moving and providing care. While working, the robot captures its actions with a camera and sends the footage to the server in real time.

[0512] Video distribution and monitoring

[0513] Real-time monitoring

[0514] The server receives the video sent by the AI ​​robot. This video is then distributed to the terminal using a secure protocol. The terminal displays the video in real time, allowing the user to monitor it and issue additional instructions as needed.

[0515] Specific examples

[0516] For example, if a user inputs an instruction into a terminal to move patient A from a wheelchair to a bed at 9:00 a.m., the system operates as follows:

[0517] 1. The user logs in to the terminal and enters the instruction "Move patient A from the wheelchair to the bed."

[0518] 2. The terminal sends the entered instructions to the server.

[0519] 3. The server receives the instructions and forwards them to the AI ​​robot.

[0520] 4. The AI ​​robot begins its nursing care duties based on the instructions it receives.

[0521] 5. While the AI ​​robot is working, a camera captures its movements and sends the footage to a server in real time.

[0522] 6. The server receives the video and distributes it to the device using a secure protocol.

[0523] 7. The user checks the real-time video on the terminal and enters additional instructions as necessary.

[0524] This invention realizes automation and efficiency of nursing care work, dramatically reducing the burden on caregivers, making it an extremely useful system in an aging society.

[0525] The processing flow will be explained below.

[0526] Step 1:

[0527] The user accesses the login screen of the device, and the device displays a form for entering the login ID and password.

[0528] Step 2:

[0529] The user enters their login ID and password and clicks the send button.

[0530] Step 3:

[0531] The terminal encrypts the entered login ID and password and sends them to the server.

[0532] Step 4:

[0533] The server decrypts the encrypted login ID and password it receives and checks them against the database.

[0534] Step 5:

[0535] The server checks the authentication result, and if the authentication is successful, generates an authentication token and sends it to the device.

[0536] Step 6:

[0537] The terminal receives a successful authentication message and token and displays to the user that they have been granted access to the system.

[0538] Step 7:

[0539] The user accesses an interface that allows them to input instructions for nursing care tasks into the terminal. For example, they input an instruction to "move patient A from the wheelchair to the bed."

[0540] Step 8:

[0541] The terminal transmits the input instruction data and the authentication token to the server.

[0542] Step 9:

[0543] The server checks the received instruction data and authentication token and analyzes the instruction content.

[0544] Step 10:

[0545] The server then transfers the analyzed instruction data to the AI ​​robot.

[0546] Step 11:

[0547] The AI ​​robot receives and analyzes the instructions, then begins its caregiving tasks, such as moving patient A from a wheelchair to a bed.

[0548] Step 12:

[0549] While the AI ​​robot is performing its nursing care duties, a camera will record what is happening.

[0550] Step 13:

[0551] The AI ​​robot sends the footage it captures in real time to a server.

[0552] Step 14:

[0553] The server delivers the received video data to the terminal using a secure protocol.

[0554] Step 15:

[0555] The terminal displays the video data received from the server to the user.

[0556] Step 16:

[0557] The user monitors the video footage through the device to ensure that care work is being carried out properly.

[0558] Step 17:

[0559] If necessary, the user inputs additional instructions, and the terminal transmits the data to the server again.

[0560] Step 18:

[0561] The server forwards the received additional instructions to the AI ​​robot, which then changes or performs additional nursing care tasks based on the instructions.

[0562] Example 1

[0563] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0564] As the aging society progresses, the workload of caregivers is increasing. In particular, the diversity and heavy workload of caregiving tasks have become issues, and there is a need to improve work efficiency and reduce the burden. In addition, monitoring and giving instructions on caregiving tasks from remote locations is also very important, so a system that can grasp the situation in real time and respond quickly is required.

[0565] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0566] In this invention, the server includes: means for a user to input login information using a device; means for the device to transmit the login information to a computer and for the computer to authenticate the received login information; means for the user to input instructions related to nursing care tasks from the device; means for the computer to receive the input instructions and forward the instructions to an artificial intelligence device; means for the artificial intelligence device to perform the nursing care tasks based on the received instructions and capture the performance status with an imaging device; means for the artificial intelligence device to transmit the captured image to the computer in real time; means for the computer to distribute the received image to the device and for the device to display the image to the user; means for communicating the data received by the computer using a secure protocol; and means for generating an authentication token when a user logs in from the device and using the token to confirm the consistency of subsequent instructions. This automates and streamlines nursing care tasks, reduces the burden on caregivers, and enables remote monitoring and instruction of nursing care tasks in real time.

[0567] "User" refers to people such as caregivers and care staff who use the system.

[0568] "Device" refers to an electronic terminal used for logging in, inputting instructions for care work, displaying images, etc.

[0569] "Login information" refers to information such as a user name and password that a user enters when accessing a system.

[0570] "Computer" refers to a server device that receives data sent from devices and performs authentication, analyzes instructions, distributes video, etc.

[0571] "Artificial intelligence device" refers to a robot or device equipped with AI that receives instructions from a computer and performs nursing care tasks based on those instructions.

[0572] "Photography device" refers to a device for photographing the status of nursing care work, such as a camera installed in an artificial intelligence device.

[0573] A "secure protocol" refers to a protocol used to ensure that data is transmitted securely. An example is HTTPS.

[0574] "Authentication Token" means temporary authentication information generated after login and used to authenticate subsequent communications and instructions.

[0575] The present invention relates to an AI-equipped nursing robot system for supporting nursing care work. By using this system, the efficiency and automation of nursing care work can be improved, and the burden on caregivers can be reduced. Detailed embodiments for carrying out the present invention will be described.

[0576] This system is operated by the user using a device, and is composed of a combination of multiple hardware and software components, including devices, computers, artificial intelligence devices, and imaging devices.

[0577] Hardware and software used

[0578] Device: An electronic device (e.g., laptop, smartphone, tablet) used to log in, input care instructions, and display images.

[0579] Computer: A server device (e.g., AWS EC2, Microsoft Azure) that receives data sent from devices and performs authentication, analyzes instructions, and distributes video.

[0580] Artificial intelligence device: A robot that receives instructions from a computer and performs caregiving tasks based on those instructions (e.g., SoftBank's Pepper, Riken-TRI Collaboration Center for Human-Interactive Robot Research's ROBEAR).

[0581] Camera: A camera that records the nursing care work being performed (e.g., Logitech C920).

[0582] Program processing and data processing

[0583] To access a system using a device, a user must first log in. The device displays a login screen to the user, prompting them to enter a username and password. After the user enters and submits this information, the device sends the information to a computer. The computer compares the received login information with a database to authenticate whether the user is legitimate. If authentication is successful, the computer generates an authentication token and sends it to the device. This token enables subsequent communication to be secure.

[0584] After logging in, the user inputs instructions for care work into the device. For example, specific instructions such as "move patient A from the wheelchair to the bed." The device then sends the instructions along with an authentication token to the computer. The computer analyzes the instructions and forwards them to the AI ​​device. The AI ​​device then begins the care work based on the received instructions.

[0585] While the caregiving task is being performed, the AI ​​device captures the situation with a camera and transmits the video in real time to a computer. The computer then distributes the received video to the device, where the user can review it. The user can input additional instructions into the device as needed and remotely manage the progress of the caregiving task.

[0586] Specific examples

[0587] For example, if a user inputs an instruction into the device to move patient A from his wheelchair to a bed at 9:00 a.m., the operation will proceed as follows:

[0588] 1. The user logs in to the device and enters the instruction "Move patient A from the wheelchair to the bed."

[0589] 2. The device sends the input instructions to the computer.

[0590] 3. The computer receives the instructions and forwards them to the artificial intelligence device.

[0591] 4. Based on the instructions received by the artificial intelligence device, it begins the process of moving Patient A from the wheelchair to the bed.

[0592] 5. While the AI ​​device is working, it captures its actions with a camera and transmits the footage to a computer in real time.

[0593] 6. The computer receives the video and distributes it to the device using a secure protocol.

[0594] 7. The user views the real-time video on the device and enters additional instructions.

[0595] Example prompt sentence:

[0596] We are currently designing an AI-powered nursing robot system to assist caregivers in their work. What technologies can we use to automate the following nursing tasks? Please provide details on the login and authentication process, the instructions and execution of nursing tasks, and the video streaming and monitoring. Please provide specific examples.

[0597] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0598] Step 1:

[0599] A user uses a device to enter login information (username and password) to access a system, and the information entered is collected by the terminal.

[0600] Step 2:

[0601] The device sends the collected login information to the server using a secure protocol (e.g., HTTPS). The input data is sent to the server as transmission data.

[0602] Step 3:

[0603] The server compares the received login information with the database and authenticates whether the user is a valid user. This authentication process verifies whether the entered login information matches the registered information in the database. If authentication is successful, the server generates an authentication token and sends it to the terminal as output data.

[0604] Step 4:

[0605] After logging in, the user inputs instructions for care work (e.g., "Move patient A from the wheelchair to the bed") into the device. The input instructions are collected by the terminal.

[0606] Step 5:

[0607] The device sends the collected instructions and the authentication token to the server using a secure protocol. At this stage, the input data are the instructions and the authentication token.

[0608] Step 6:

[0609] The server analyzes the received instructions and extracts the information necessary to understand the content of the instructions. This analysis process provides specific instruction information. Once the analysis is complete, it generates output data to transmit the instructions to the artificial intelligence device.

[0610] Step 7:

[0611] Based on the analysis results, the server transfers specific instructions to the AI ​​device, which receives the instructions and begins its caregiving duties.

[0612] Step 8:

[0613] The AI ​​device performs a care task based on the received instructions. For example, it may move a patient from a wheelchair to a bed. While the AI ​​device is performing the task, it takes pictures of the task with a camera. The photographed data is generated.

[0614] Step 9:

[0615] The AI ​​device transmits the captured video to a server in real time, and the input data is the captured video information.

[0616] Step 10:

[0617] The server then transmits the received video to the terminal using a secure protocol. The transmitted video is the output data.

[0618] Step 11:

[0619] The user checks the real-time video on the terminal. While monitoring this video, the user inputs additional instructions into the device as needed. The additional instructions are collected as input data.

[0620] Step 12:

[0621] The terminal sends additional instructions to the server, which then forwards the instructions to the AI ​​device. This cycle is repeated to continuously manage the care work.

[0622] (Application example 1)

[0623] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0624] In today's aging society, the shortage of caregivers and other care workers is becoming a serious problem. Furthermore, in order to perform caregiving and monitoring tasks efficiently and accurately, it is essential to reduce the burden on caregivers, automate tasks, and speed up monitoring. Therefore, real-time monitoring and rapid response when an abnormality occurs are required.

[0625] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0626] In this invention, the server includes: a means for a user to input login information using a terminal; a means for the terminal to transmit the login information to the server and for the server to authenticate the received login information; a means for the user to input instructions related to nursing care or monitoring work from the terminal; a means for the server to receive the input instructions and forward the instructions to the worker robot; a means for the worker robot to perform the nursing care or monitoring work based on the received instructions and record the execution status in a recording device; a means for the worker robot to transmit video recorded by the worker robot to the server in real time; a means for the server to distribute the received video to the terminal and for the terminal to display the video to the user; and a means for notifying the user when an abnormality is detected. This enables the automation and efficiency of nursing care and monitoring work, reduces the burden on workers, and enables rapid response when an abnormality occurs.

[0627] "User" means an individual or entity that uses a terminal to operate the System.

[0628] A "terminal" is an electronic device used to access and operate the system.

[0629] "Login Information" means identifying information, such as a username and password, for authenticating access to a system.

[0630] A "server" is a computer that is the core of the system and is a device that authenticates login information, receives and transfers instructions, and stores and distributes data.

[0631] "Authentication" is the process of verifying that a user has legitimate authority to access a system.

[0632] "Instructions" are information that the user inputs via the terminal to provide specific work content related to care work or monitoring work.

[0633] A "worker robot" is an automated machine equipped with artificial intelligence that performs nursing and monitoring tasks based on instructions it receives.

[0634] "Nursing care work" refers to the specific tasks involved in providing care to patients and elderly people.

[0635] "Surveillance work" refers to the regular or continuous observation of a facility or specific area to ensure its safety.

[0636] The "recording device" is a device that records the status of the work performed by the worker robot as video and data.

[0637] "Real-time" refers to the instantaneous transfer of data and video without delay.

[0638] "Distribution" means that the server sends the video and data it receives to the terminal and displays it.

[0639] "Notification" is the process of sending alerts and messages to users when the system detects an abnormality.

[0640] overview

[0641] The present invention is a system that allows a user to log in using a terminal and input instructions to have a worker robot perform nursing care or monitoring tasks. This system includes a server, a terminal, a worker robot, a recording device, and a real-time distribution and notification function.

[0642] System Configuration

[0643] server

[0644] The server acts as the center of the system and performs the following functions:

[0645] 1. Login authentication: Receives the login information entered by the user from the terminal and authenticates it by checking it against a database.

[0646] 2. Instruction transfer: Analyzes instructions received from the user and transfers them to the corresponding worker robot.

[0647] 3. Video distribution: Real-time video received from the worker robot is distributed to the terminal.

[0648] 4. Secure communication: Received data is communicated using a secure protocol (e.g. HTTPS).

[0649] 5. Abnormality notification: If the worker robot or user detects an abnormality, a notification is sent to the user.

[0650] Terminal

[0651] The terminal is an electronic device that allows users to operate the system and has the following functions:

[0652] 1. Entering login information: The user enters their login ID and password.

[0653] 2. Inputting instructions: Entering instructions regarding nursing care and monitoring tasks.

[0654] 3. Video display: Displays real-time video streamed from the server.

[0655] 4. Notification reception: Receive notifications sent from the server when an abnormality occurs.

[0656] Worker robot

[0657] Employee robots are automated machines equipped with AI that perform the following functions:

[0658] 1. Execute Instructions: Performs caregiving or supervisory tasks based on received instructions.

[0659] 2. Recording of execution status: The status of work is recorded in real time by a camera and sent to a server as video data.

[0660] Hardware and software used

[0661] Server: A computer with high-performance computing power (e.g., Amazon Web Services, Google Cloud Platform)

[0662] Devices: Electronic devices such as smartphones, tablets, and computers

[0663] Recording device: High-resolution camera (e.g. Logitech Webcam)

[0664] AI model: Generative AI model (e.g. GPT-3, BERT)

[0665] Secure protocol: HTTPS

[0666] Specific examples

[0667] If a user uses a terminal to input instructions to monitor a specific area at 9:00 a.m., the system operates as follows:

[0668] 1. Terminal: The user enters login information and accesses the system.

[0669] 2. Server: Authenticates the login information and allows the user access. When the user enters an instruction to "monitor a specific area," the server receives this instruction.

[0670] 3. Server: Analyzes the received instructions and forwards them to the appropriate worker robot.

[0671] 4. Worker robot: Based on instructions, it patrols designated areas and performs surveillance tasks. It records images of its patrols in real time with a camera.

[0672] 5. Server: Receives the video sent from the robot and distributes it to the terminal.

[0673] 6. Terminal: The user checks the real-time video and inputs additional instructions if an abnormality is detected. If an abnormality is detected, the user receives a notification.

[0674] Prompt Sentence Examples

[0675] To further refine the design using the generative AI model, use the following prompt:

[0676] Please explain the design of a security robot system. The system allows users to log in from their smartphones and instruct the AI-equipped robots on how to perform monitoring tasks. The robots will send real-time video footage to the smartphones. If an abnormality is detected, notifications will be sent to the users. This system will be built using Python, Flask, and OpenCV.

[0677] This prompt can be used to obtain detailed design information and implementation methods from the AI ​​model.

[0678] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0679] Step 1:

[0680] The user enters their login information using the device.

[0681] Input: User's login ID and password

[0682] Operation: The user enters login information into the device's login screen and presses the "Login" button.

[0683] Output: The device sends the login information to the server.

[0684] Step 2:

[0685] The server authenticates the received login information.

[0686] Input: Login ID and password sent from the device

[0687] How it works: The server checks the received login information against a database to determine if the user is legitimate.

[0688] Output: If authentication is successful, the server generates an authentication token and sends it to the terminal. If authentication fails, it returns an error message to the terminal.

[0689] Step 3:

[0690] The user inputs instructions regarding the care work or monitoring work from the terminal.

[0691] Input: Authentication token after logging in, instructions for care work or monitoring work (e.g., "Monitor a specific area")

[0692] Operation: The terminal receives instructions entered by the user and sends them to the server.

[0693] Output: Data containing instructions and an authentication token is sent to the server.

[0694] Step 4:

[0695] The server receives the input instructions and forwards the instructions to the worker robot.

[0696] Input: Instructions, authentication token

[0697] Operation: The server analyzes the received instructions and forwards them to the corresponding worker robot.

[0698] Output: The parsed instructions are sent to the worker robot.

[0699] Step 5:

[0700] The worker robot performs caregiving or monitoring tasks based on the received instructions.

[0701] Input: Instructions sent by the server

[0702] Action: The worker robot follows instructions and begins nursing or monitoring tasks.

[0703] Output: The execution status is recorded in real time by a camera.

[0704] Step 6:

[0705] The worker robot transmits the recorded video to a server in real time.

[0706] Input: Camera footage

[0707] Operation: The worker robot transmits the recorded video in real time to the server.

[0708] Output: Real-time video data is sent to the server.

[0709] Step 7:

[0710] The server distributes the received video to the terminal, and the terminal displays the video to the user.

[0711] Input: Real-time video sent from the worker robot

[0712] Operation: The server delivers the received video to the device using a secure protocol.

[0713] Output: The distributed real-time video is displayed on the terminal.

[0714] Step 8:

[0715] The server will notify the user when an abnormality is detected.

[0716] Input: Anomaly detection information or manual detection input by the user

[0717] Operation: When the server detects an abnormality, it sends a push notification to the user device.

[0718] Output: An abnormality notification is displayed on the user's terminal.

[0719] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0720] In addition to the conventional nursing care support system, the present invention provides a new technology that recognizes and responds to the emotions of users and patients. Hereinafter, embodiments of the present invention will be described in detail.

[0721] In this system, the user uses a terminal to give instructions on care tasks, and the AI ​​robot carries out the tasks based on those instructions. The system also captures the progress of the tasks with a camera and can be monitored remotely in real time. Furthermore, an emotion engine is used to recognize the emotions of the user and patient, optimizing the operation of the entire system.

[0722] Login and Authentication Process

[0723] Authentication process

[0724] When a user tries to access a system using a terminal, they must first log in. The terminal displays a login screen to the user and asks them to enter their login ID and password. Once the user enters the login information, the terminal sends it to the server. The server compares the received login information with a database and performs authentication. If this authentication is successful, the server sends an authentication token to the terminal, and the user gains access to the system.

[0725] Furthermore, the emotion engine analyzes the user's facial expressions and voice to recognize their emotional state at the time of login, which makes it possible to detect the user's stress level and fatigue at that time.

[0726] Directing and carrying out nursing care tasks

[0727] Sending instructions

[0728] After the user logs in from the device, an interface for inputting instructions for care work is displayed. For example, specific care instructions such as "move patient A from the wheelchair to the bed" are entered. The device sends this instruction along with an authentication token to the server. The server analyzes the received instructions and forwards them to the AI ​​robot.

[0729] In addition, when inputting instructions, the emotion engine analyzes the user's emotions and sends the results to the server, which can then transfer instructions to the AI ​​robot that take into account the user's emotional state.

[0730] Robot movement

[0731] The AI ​​robot receives instructions from the server and begins its nursing care tasks based on those instructions. For example, it safely performs tasks such as moving and assisting the patient. While working, the robot captures its movements with a camera and sends the footage to the server in real time. Furthermore, the AI ​​robot is equipped with an emotion engine that recognizes emotions from the patient's facial expressions and voice, and adjusts its tasks as necessary.

[0732] Video distribution and monitoring

[0733] Real-time monitoring

[0734] The server receives the video sent by the AI ​​robot. This video is then distributed to the device using a secure protocol. The device displays the video in real time, allowing the user to monitor it. The emotion engine continuously monitors the user's emotional state and issues an alert if an abnormality is detected.

[0735] Specific examples

[0736] For example, if a user inputs an instruction into a terminal to move patient A from a wheelchair to a bed at 9:00 a.m., the system operates as follows:

[0737] 1. The user logs in to the terminal and inputs the command, "Move patient A from the wheelchair to the bed." At this time, the emotion engine analyzes the user's facial expressions and voice and sends the emotional state to the server.

[0738] 2. The device sends the input instructions to the server, which then analyzes the received instruction data and emotion data.

[0739] 3. The server transfers instructions and emotional data to the AI ​​robot. If the user is under stress, the robot will adjust its work pace to take that situation into account.

[0740] 4. The AI ​​robot begins its nursing care tasks based on the instructions. When transferring Patient A from the wheelchair to the bed, the emotion engine also monitors the patient's emotional state.

[0741] 5. While the AI ​​robot is working, a camera captures its movements and sends the footage to a server in real time.

[0742] 6. The server delivers the video data to the device, which displays the video to the user in real time.

[0743] 7. The user monitors the video on the device to ensure that the care work is being carried out properly. The emotion engine also continuously monitors the user's emotions and issues an alert if there are any abnormalities.

[0744] This invention not only realizes automation and efficiency in nursing care work, dramatically reducing the burden on caregivers, but also provides a safer and more comfortable nursing environment through emotion recognition, making it an extremely useful system in an aging society.

[0745] The processing flow will be explained below.

[0746] Step 1:

[0747] The user accesses the login screen of the device, and the device displays a form for entering the login ID and password.

[0748] Step 2:

[0749] The user enters their login ID and password and clicks the send button.

[0750] Step 3:

[0751] The terminal encrypts the entered login ID and password and sends them to the server.

[0752] Step 4:

[0753] The server decrypts the encrypted login ID and password it receives and checks them against the database.

[0754] Step 5:

[0755] The server checks the authentication result, and if the authentication is successful, generates an authentication token and sends it to the device.

[0756] Step 6:

[0757] The device receives a successful authentication message and token, and displays a message to the user indicating that they have been granted access to the system. The emotion engine analyzes the user's facial expressions and voice to confirm their emotional state at that time.

[0758] Step 7:

[0759] The user accesses an interface that allows them to input instructions for nursing care tasks into the terminal. For example, they input an instruction to "move patient A from the wheelchair to the bed."

[0760] Step 8:

[0761] The emotion engine analyzes the facial expressions and voice of the user inputting instructions and generates emotion data.

[0762] Step 9:

[0763] The terminal transmits the input instruction data, emotion data, and authentication token to the server.

[0764] Step 10:

[0765] The server checks the received instruction data, emotion data, and authentication token, and analyzes the instruction content, taking into account the user's emotional state.

[0766] Step 11:

[0767] The server analyzes the instruction data and transfers it to the AI ​​robot. If the user is under stress, the instructions to the robot are adjusted.

[0768] Step 12:

[0769] The AI ​​robot receives and analyzes the instructions, then begins its caregiving tasks, such as moving patient A from a wheelchair to a bed.

[0770] Step 13:

[0771] The emotion engine analyzes the patient's facial expressions and voice to assess their emotional state and adjust the work as needed.

[0772] Step 14:

[0773] While the AI ​​robot is performing its nursing care duties, a camera will record what is happening.

[0774] Step 15:

[0775] The AI ​​robot sends the footage it captures in real time to a server.

[0776] Step 16:

[0777] The server delivers the received video data to the terminal using a secure protocol.

[0778] Step 17:

[0779] The terminal displays the video data received from the server to the user, and the emotion engine continuously monitors the user's emotional state.

[0780] Step 18:

[0781] Users can monitor the video footage via their devices to ensure that care work is being carried out properly, and if there is an abnormality, a warning is issued.

[0782] Step 19:

[0783] If necessary, the user inputs additional instructions, and the terminal transmits the data to the server again.

[0784] Step 20:

[0785] The server forwards the received additional instructions to the AI ​​robot, which then changes or performs additional nursing care tasks based on the instructions.

[0786] Example 2

[0787] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0788] Conventional nursing care support systems have difficulty recognizing the emotional states of caregivers and patients and responding appropriately. Furthermore, even though remote real-time monitoring is possible, there is a lack of systems that can optimize behavior based on emotions or detect abnormalities, making it difficult to fully ensure the safety and efficiency of nursing care work.

[0789] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input login information using a terminal; a means for the terminal to transmit the login information to the server and for the server to authenticate the received login information; a means having an emotion analysis engine for recognizing the emotional states of the user and the patient; a means for the user to input instructions related to the care work from the terminal; a means for the server to receive, analyze, and forward the input instructions to the AI ​​robot; a means for the AI ​​robot to perform the care work based on the received instructions and capture the performance status with a camera; a means for transmitting the video captured by the AI ​​robot to the server in real time; a means for the server to optimize the operation by taking into account the emotional states of the user and the patient via the emotion analysis engine; a means for the server to distribute the received video to the terminal, which then displays the video to the user and continuously monitors the user's emotional state; and a means for issuing an alert when an abnormality is detected. This improves the safety and efficiency of care work and enables more appropriate care that takes into account the emotional state.

[0790] "Login information" refers to the authentication data that a user enters when accessing a system, and typically consists of a user ID and password.

[0791] A "terminal" is a device used by a user to access the nursing care support system, input instructions, and monitor video footage, and includes PCs, tablets, smartphones, etc.

[0792] The "server" is a computer system that serves as the core of the nursing care support system and performs various processes such as login authentication, instruction analysis, data transfer, emotional state analysis, and real-time video distribution.

[0793] An "emotion analysis engine" is a software or hardware component that analyzes the facial expressions and voice of a user or patient to recognize and evaluate their emotional state.

[0794] "Instruction data" refers to data that includes specific instructions regarding care work input by the user via the terminal.

[0795] An "AI robot" is an autonomous robot that performs nursing care tasks based on instructions received from a server, and is equipped with a camera, emotion analysis engine, and other features.

[0796] The "camera" is a device installed on the AI ​​robot that captures the nursing care work being performed in real time and sends the footage to a server.

[0797] A "secure protocol" is a secure communication method that prevents data transmission and reception from being tampered with or intercepted by third parties, and includes SSL / TLS.

[0798] An "authentication token" is temporary data issued by the server when a user is successfully authenticated, and is used to verify that subsequent communications and instructions are legitimate.

[0799] "Real-time streaming" is a technology that transmits footage captured by an AI robot from a server to a terminal with almost no delay, allowing users to monitor it immediately.

[0800] "Abnormality detection" is a function that allows the system to detect abnormal patterns in the emotional state of a user or patient or the execution of care tasks and issue a warning.

[0801] In addition to the conventional nursing care support system, the present invention provides a technology that recognizes the emotions of users and patients and takes optimal measures according to their emotional states. Hereinafter, embodiments of the present invention will be described in detail.

[0802] This system consists of the following main components: a server, a terminal, an AI robot, a camera, and an emotion analysis engine. The functions and interactions of each component are described in detail below.

[0803] Server Roles

[0804] The server acts as the central part of the system and performs the following main tasks:

[0805] 1. Login authentication: The login information entered by the user from the terminal is compared with the database. If authentication is successful, an authentication token is issued.

[0806] 2. Receiving and analyzing instruction data: Receives and analyzes instructions for nursing care tasks entered by the user through the terminal.

[0807] 3. Transfer of instruction data: Based on the analysis results, appropriate instruction data is transferred to the AI ​​robot.

[0808] 4. Sentiment analysis and optimization: Use a sentiment analysis engine to analyze the emotional state of users and patients and optimize operations.

[0809] 5. Real-time video distribution: Receives video data sent from the AI ​​robot and distributes it to the terminal in real time.

[0810] 6. Anomaly detection and warning: If an abnormality is detected in the user or patient's condition, a warning will be issued to the terminal.

[0811] Device Role

[0812] The terminal is used by the user to access the system, enter commands, and monitor footage. Terminal functions include:

[0813] 1. Displaying a login screen: A login screen is displayed to the user and they are prompted to enter their login information.

[0814] 2. Providing an instruction input interface: An interface is provided for the user to input specific nursing care instructions.

[0815] 3. Real-time video display: Displays the video received from the server in real time to monitor the progress of nursing care work.

[0816] 4. Displaying a warning notification: When an abnormality is detected, a warning is displayed to the user.

[0817] The role of AI robots

[0818] The AI ​​robot performs care tasks based on instructions received from the server. Specific functions include:

[0819] 1. Carrying out care tasks: Carry out care tasks (e.g., moving or assisting patients) according to received instructions.

[0820] 2. Video recording: The nursing care work is recorded with a camera and the video is sent to the server in real time.

[0821] 3. Emotion Analysis: Use the built-in emotion analysis engine to analyze the patient's emotional state and optimize the progress of the work.

[0822] The role of sentiment analysis engines

[0823] The emotion analysis engine analyzes the facial expressions and voice of users and patients to recognize their emotional state, providing important data for optimizing the operation of the entire system.

[0824] Specific examples

[0825] For example, if a user inputs the instruction "Move patient A from his wheelchair to his bed" into the terminal at 9:00 AM, the system will perform the following specific operations:

[0826] 1. The user logs in to the device and inputs instructions. The emotion analysis engine analyzes the user's facial expressions and voice and sends their emotional state to the server.

[0827] 2. The terminal transmits the input instructions and emotion data to the server.

[0828] 3. The server analyzes the instruction data and emotional data and transmits them to the AI ​​robot. If stress is detected, the robot will adjust its work pace.

[0829] 4. The AI ​​robot will begin nursing care tasks based on instructions, while simultaneously analyzing the patient's emotions and optimizing its tasks accordingly.

[0830] 5. The AI ​​robot uses a camera to record its work and sends the footage to a server in real time.

[0831] 6. The server delivers the video data to the terminal and displays it to the user in real time.

[0832] 7. The user monitors the video through the device, and if the emotion analysis engine detects an abnormality, an alert is issued.

[0833] Prompt Sentence Examples

[0834] "Transfer patient A from wheelchair to bed at 9:00 AM"

[0835] "Please notify me of patient B's medication time."

[0836] "Check on patient C's safety"

[0837] This invention not only realizes automation and efficiency of nursing care work, reducing the burden on caregivers, but also makes it possible to provide a safer and more comfortable nursing care environment through emotion recognition.

[0838] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0839] Step 1:

[0840] Enter and submit your login information

[0841] The user accesses the login screen using the device and enters the login ID and password.

[0842] Input: The login ID and password entered by the user.

[0843] Output: Data format in which login information (ID and password) is sent to the terminal

[0844] Specific operation: The terminal receives the user's input and sends it to the server.

[0845] Step 2:

[0846] Authenticate your login details

[0847] The server compares the login information received from the terminal with the database to perform authentication.

[0848] Input: Login information sent from the device

[0849] Output: Authentication token or authentication failure message

[0850] Specific operation: The server checks the login information against the database, and if authentication is successful, it generates an authentication token and sends it to the terminal. If authentication fails, it returns an error message.

[0851] Step 3:

[0852] Emotional state analysis

[0853] The emotion analysis engine analyzes the user's facial expressions and voice to recognize their emotional state at the time of login.

[0854] Input: User's facial expression and voice data

[0855] Output: Emotional state information (stress level, fatigue level, etc.)

[0856] How it works: The emotion analysis engine monitors the user's face and voice in real time and evaluates their state using an emotion analysis algorithm. The evaluation results are sent to the server and recorded.

[0857] Step 4:

[0858] Entering nursing care instructions

[0859] The user inputs specific instructions regarding the care work from the terminal.

[0860] Input: Care work instructions entered by the user

[0861] Output: Instruction data (specific nursing care work content)

[0862] Specific operation: The terminal provides an interface for inputting instructions for nursing care work, and the user inputs specific instructions (e.g., "Move patient A from the wheelchair to the bed").

[0863] Step 5:

[0864] Sending and analyzing instruction data

[0865] Care work instructions and emotional state information are sent from the terminal to the server, which then analyzes them.

[0866] Input: Care instructions and emotional state information

[0867] Output: Analysis results (optimized instruction data)

[0868] Specific operation: The server analyzes the received instruction data and generates optimized instructions that take into account the emotional state information. These instructions are then forwarded to the AI ​​robot.

[0869] Step 6:

[0870] Carrying out nursing care tasks with AI robots

[0871] The AI ​​robot performs nursing care tasks based on instructions from the server.

[0872] Input: Optimized instruction data

[0873] Output: Status of nursing care work and video data

[0874] Specific actions: The AI ​​robot begins specific care tasks (e.g., moving a patient) based on the instructions it receives. The robot's actions are captured in real time by a camera.

[0875] Step 7:

[0876] Real-time video transmission and distribution

[0877] The video data captured by the AI ​​robot is sent to a server, which then distributes it to the device in real time.

[0878] Input: Real-time video data

[0879] Output: Real-time video streaming data

[0880] How it works: The AI ​​robot captures video of itself working with a camera and sends it in real time to a server, which then receives the data and distributes it to the device using a secure protocol.

[0881] Step 8:

[0882] Anomaly detection and warning

[0883] The server uses an emotion analysis engine to continuously monitor the emotional state of users and patients and issues an alert if an abnormality is detected.

[0884] Input: Emotional state information and care work execution state

[0885] Output: Warning notice

[0886] Specific operation: The server analyzes the data from the sentiment analysis engine, and if an anomaly is detected, it sends a warning message to the device. The user receives the warning on the device and can take appropriate action.

[0887] In this way, this system works by linking users, servers, AI robots, and emotion analysis engines to carry out nursing care tasks efficiently and safely.

[0888] (Application example 2)

[0889] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0890] Current work support systems only transfer and execute simple instructions without considering the emotional state of the user or target, which means that the safety and efficiency of the work environment cannot be sufficiently maintained.In addition, because they are unable to provide feedback based on real-time emotion recognition, stress and fatigue levels may be overlooked, which could have a negative impact on the health and safety of workers and targets.

[0891] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input login information using a terminal; a means for the terminal to transmit the login information to the server and for the server to authenticate the received login information; a means for the user to input instructions related to nursing care work from the terminal; a means for the server to receive the input instructions and forward the instructions to the AI ​​processing device; a means for the AI ​​processing device to perform work based on the received instructions and record the execution status using an optical device; a means for the AI ​​processing device to transmit the recorded video to the server in real time; a means for the server to distribute the received video to the terminal and for the terminal to display the video to the user; a means for analyzing the emotions of the user and the target person in real time using an emotion recognition engine; a means for adjusting the operation of the AI ​​processing device based on the target person's emotional state; and a means for analyzing the video from the optical device and issuing a safety warning based on the results of emotion recognition. This makes it possible to improve the safety and efficiency of the work environment.

[0892] A "user" is a person who operates the system and gives instructions for work.

[0893] "Terminal" refers to an electronic device operated by a user, including smart glasses and PCs.

[0894] "Login information" means the authentication information provided by a user to access the system, and specifically includes a user ID and password.

[0895] A "server" is a computer system on a network that manages the entire system and processes data.

[0896] An "authentication token" is authentication data for subsequent communications that is issued by the server after a user has successfully logged in.

[0897] An "AI processing device" is a device that uses AI technology to analyze instructions and automatically execute tasks.

[0898] An "optical device" is a device for recording images, such as a camera.

[0899] An "emotion recognition engine" is a technology or software that analyzes the emotional state of a user or target person in real time from their facial expressions and voice.

[0900] "Real-time" refers to data processing and communication occurring instantly, without delay.

[0901] "Target" refers to the person to whom the system provides services, and in the case of a care robot, it means the person being cared for.

[0902] "Safety warnings" are caution or warning messages issued for the safety of users or targets based on the analysis results of the emotion recognition engine.

[0903] This invention is a system for improving the safety and efficiency of a work environment. This system is composed of a user, a terminal, a server, an AI processing device, an optical device, and an emotion recognition engine.

[0904] First, the user enters login information using the terminal. The terminal then sends this login information to the server, which then performs authentication using the received login information. If authentication is successful, the server generates an authentication token and sends it to the terminal.

[0905] Next, the user can input instructions related to the work from the device. For example, a command to start a specific task may be given. The device then sends this instruction along with an authentication token to the server. The server then analyzes the received instruction and forwards it to the AI ​​processing device based on its content.

[0906] The AI ​​processing device performs tasks based on the received instructions and records its progress using an optical device (such as a camera). The recorded video is sent to a server in real time. The server then distributes the received video to a terminal, allowing users to monitor the video in real time through their terminal.

[0907] Furthermore, the emotional state of the user and the target can be analyzed in real time using an emotion recognition engine. The emotion recognition engine analyzes the facial expressions and voice data of the user and the target, and adjusts the operation of the AI ​​processing unit based on the analysis results. It is also possible to issue safety warnings as needed based on the analysis results.

[0908] For example, if a factory worker wears smart glasses, the glasses' camera and microphone can be used to analyze the worker's facial expressions and voice to recognize their emotional state in real time. Based on this recognition result, the system can determine whether the working environment is appropriate and issue a warning if necessary.

[0909] For example, if Worker A is wearing smart glasses at the start of his shift at 8 a.m. and the emotion recognition engine detects fatigue, the system will automatically issue a warning and prompt the worker to take a break, thus optimizing the working environment and improving safety and work efficiency.

[0910] An example of a prompt sentence is as follows:

[0911] "How can we monitor the emotions of factory workers in real time and issue warnings if fatigue or stress is detected to improve work efficiency?"

[0912] This allows the system embodying the invention to take into account the emotional state of the user and the subject, providing a safer and more efficient working environment.

[0913] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0914] Step 1:

[0915] The user enters login information using the terminal. The terminal displays the login screen, and the user enters their login ID and password. The terminal then sends the entered login information to the server.

[0916] Input: User login ID and password

[0917] Output: Login information sent to the server

[0918] Step 2:

[0919] The server authenticates the received login information. The server accesses a database and checks the received login information. If authentication is successful, the server generates an authentication token and sends it to the terminal.

[0920] Input: Login information

[0921] Output: Authentication token

[0922] Step 3:

[0923] The user inputs work instructions from the terminal. The authenticated terminal displays the work instruction input screen, and the user inputs specific work instructions (e.g., instructions for a specific task). The terminal then sends these instructions to the server along with an authentication token.

[0924] Input: Work instructions, authentication token

[0925] Output: Work instructions sent to the server

[0926] Step 4:

[0927] The server receives and analyzes the input instructions, and then transfers the instructions to the appropriate AI processing device based on their content.

[0928] Input: Work Instructions

[0929] Output: Instructions to the AI ​​processor

[0930] Step 5:

[0931] The AI ​​processing unit executes tasks based on the instructions it receives, such as a robot performing a specific action. The AI ​​processing unit continues to record the progress of the task using an optical device.

[0932] Input: Instructions from the server

[0933] Output: Work progress

[0934] Step 6:

[0935] The AI ​​processing device transmits the recorded video to the server in real time. The AI ​​processing device transmits the video data captured by the optical device to the server in real time.

[0936] Input: Recorded video

[0937] Output: Video data sent to the server

[0938] Step 7:

[0939] The server delivers the received video to the terminal. The server delivers the received video to the terminal in real time using a secure protocol. The terminal displays this video to the user.

[0940] Input: Video data

[0941] Output: The image displayed to the user

[0942] Step 8:

[0943] The emotion recognition engine analyzes the emotions of the user and the target. The emotion recognition model is run using video and audio data. Based on the analysis results, it determines whether the operation of the AI ​​processing unit needs to be adjusted.

[0944] Input: Video data, audio data

[0945] Output: Emotion analysis results

[0946] Step 9:

[0947] The emotion recognition engine issues safety warnings based on the analysis results. If the analysis results indicate that the user or target is in danger, the system will automatically issue a warning and prompt the user to take necessary measures.

[0948] Input: Sentiment analysis results

[0949] Output: Safety warning

[0950] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0951] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0952] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0953] [Third embodiment]

[0954] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0955] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0956] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0957] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0958] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0959] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0960] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0961] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0962] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0963] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0964] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0965] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0966] The present invention relates to a nursing robot system equipped with AI for supporting the work of a caregiver. Hereinafter, an embodiment of the present invention will be specifically described.

[0967] In this system, users use a terminal to give instructions on care tasks, and the AI ​​robot carries out the tasks based on those instructions. Furthermore, the progress of the tasks can be captured on a camera and monitored remotely in real time.

[0968] Login and Authentication Process

[0969] Authentication process

[0970] When a user tries to access a system using a terminal, they must first log in. The terminal displays a login screen to the user and asks them to enter their login ID and password. Once the user enters the login information, the terminal sends it to the server. The server compares the received login information with a database and authenticates whether the user is a legitimate user. If this authentication is successful, the server sends an authentication token to the terminal, and the user gains access to the system.

[0971] Directing and carrying out nursing care tasks

[0972] Sending instructions

[0973] After the user logs in from the device, an interface for inputting instructions for nursing care work is displayed. For example, specific nursing care instructions such as "move patient A from the wheelchair to the bed" are entered. The device sends this instruction along with an authentication token to the server. The server analyzes the received instructions and forwards them to the AI ​​robot.

[0974] Robot movement

[0975] The AI ​​robot receives instructions from the server and begins its nursing care tasks based on those instructions. For example, it safely performs tasks such as moving and providing care. While working, the robot captures its actions with a camera and sends the footage to the server in real time.

[0976] Video distribution and monitoring

[0977] Real-time monitoring

[0978] The server receives the video sent by the AI ​​robot. This video is then distributed to the terminal using a secure protocol. The terminal displays the video in real time, allowing the user to monitor it and issue additional instructions as needed.

[0979] Specific examples

[0980] For example, if a user inputs an instruction into a terminal to move patient A from a wheelchair to a bed at 9:00 a.m., the system operates as follows:

[0981] 1. The user logs in to the terminal and enters the instruction "Move patient A from the wheelchair to the bed."

[0982] 2. The terminal sends the entered instructions to the server.

[0983] 3. The server receives the instructions and forwards them to the AI ​​robot.

[0984] 4. The AI ​​robot begins its nursing care duties based on the instructions it receives.

[0985] 5. While the AI ​​robot is working, a camera captures its movements and sends the footage to a server in real time.

[0986] 6. The server receives the video and distributes it to the device using a secure protocol.

[0987] 7. The user checks the real-time video on the terminal and enters additional instructions as necessary.

[0988] This invention realizes automation and efficiency of nursing care work, dramatically reducing the burden on caregivers, making it an extremely useful system in an aging society.

[0989] The processing flow will be explained below.

[0990] Step 1:

[0991] The user accesses the login screen of the device, and the device displays a form for entering the login ID and password.

[0992] Step 2:

[0993] The user enters their login ID and password and clicks the send button.

[0994] Step 3:

[0995] The terminal encrypts the entered login ID and password and sends them to the server.

[0996] Step 4:

[0997] The server decrypts the encrypted login ID and password it receives and checks them against the database.

[0998] Step 5:

[0999] The server checks the authentication result, and if the authentication is successful, generates an authentication token and sends it to the device.

[1000] Step 6:

[1001] The terminal receives a successful authentication message and token and displays to the user that they have been granted access to the system.

[1002] Step 7:

[1003] The user accesses an interface that allows them to input instructions for nursing care tasks into the terminal. For example, they input an instruction to "move patient A from the wheelchair to the bed."

[1004] Step 8:

[1005] The terminal transmits the input instruction data and the authentication token to the server.

[1006] Step 9:

[1007] The server checks the received instruction data and authentication token and analyzes the instruction content.

[1008] Step 10:

[1009] The server then transfers the analyzed instruction data to the AI ​​robot.

[1010] Step 11:

[1011] The AI ​​robot receives and analyzes the instructions, then begins its caregiving tasks, such as moving patient A from a wheelchair to a bed.

[1012] Step 12:

[1013] While the AI ​​robot is performing its nursing care duties, a camera will record what is happening.

[1014] Step 13:

[1015] The AI ​​robot sends the footage it captures in real time to a server.

[1016] Step 14:

[1017] The server delivers the received video data to the terminal using a secure protocol.

[1018] Step 15:

[1019] The terminal displays the video data received from the server to the user.

[1020] Step 16:

[1021] The user monitors the video footage through the device to ensure that care work is being carried out properly.

[1022] Step 17:

[1023] If necessary, the user inputs additional instructions, and the terminal transmits the data to the server again.

[1024] Step 18:

[1025] The server forwards the received additional instructions to the AI ​​robot, which then changes or performs additional nursing care tasks based on the instructions.

[1026] Example 1

[1027] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1028] As the aging society progresses, the workload of caregivers is increasing. In particular, the diversity and heavy workload of caregiving tasks have become issues, and there is a need to improve work efficiency and reduce the burden. In addition, monitoring and giving instructions on caregiving tasks from remote locations is also very important, so a system that can grasp the situation in real time and respond quickly is required.

[1029] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1030] In this invention, the server includes: means for a user to input login information using a device; means for the device to transmit the login information to a computer and for the computer to authenticate the received login information; means for the user to input instructions related to nursing care tasks from the device; means for the computer to receive the input instructions and forward the instructions to an artificial intelligence device; means for the artificial intelligence device to perform the nursing care tasks based on the received instructions and capture the performance status with an imaging device; means for the artificial intelligence device to transmit the captured image to the computer in real time; means for the computer to distribute the received image to the device and for the device to display the image to the user; means for communicating the data received by the computer using a secure protocol; and means for generating an authentication token when a user logs in from the device and using the token to confirm the consistency of subsequent instructions. This automates and streamlines nursing care tasks, reduces the burden on caregivers, and enables remote monitoring and instruction of nursing care tasks in real time.

[1031] "User" refers to people such as caregivers and care staff who use the system.

[1032] "Device" refers to an electronic terminal used for logging in, inputting instructions for care work, displaying images, etc.

[1033] "Login information" refers to information such as a user name and password that a user enters when accessing a system.

[1034] "Computer" refers to a server device that receives data sent from devices and performs authentication, analyzes instructions, distributes video, etc.

[1035] "Artificial intelligence device" refers to a robot or device equipped with AI that receives instructions from a computer and performs nursing care tasks based on those instructions.

[1036] "Photography device" refers to a device for photographing the status of nursing care work, such as a camera installed in an artificial intelligence device.

[1037] A "secure protocol" refers to a protocol used to ensure that data is transmitted securely. An example is HTTPS.

[1038] "Authentication Token" means temporary authentication information generated after login and used to authenticate subsequent communications and instructions.

[1039] The present invention relates to an AI-equipped nursing robot system for supporting nursing care work. By using this system, the efficiency and automation of nursing care work can be improved, and the burden on caregivers can be reduced. Detailed embodiments for carrying out the present invention will be described.

[1040] This system is operated by the user using a device, and is composed of a combination of multiple hardware and software components, including devices, computers, artificial intelligence devices, and imaging devices.

[1041] Hardware and software used

[1042] Device: An electronic device (e.g., laptop, smartphone, tablet) used to log in, input care instructions, and display images.

[1043] Computer: A server device (e.g., AWS EC2, Microsoft Azure) that receives data sent from devices and performs authentication, analyzes instructions, and distributes video.

[1044] Artificial intelligence device: A robot that receives instructions from a computer and performs caregiving tasks based on those instructions (e.g., SoftBank's Pepper, Riken-TRI Collaboration Center for Human-Interactive Robot Research's ROBEAR).

[1045] Camera: A camera that records the nursing care work being performed (e.g., Logitech C920).

[1046] Program processing and data processing

[1047] To access a system using a device, a user must first log in. The device displays a login screen to the user, prompting them to enter a username and password. After the user enters and submits this information, the device sends the information to a computer. The computer compares the received login information with a database to authenticate whether the user is legitimate. If authentication is successful, the computer generates an authentication token and sends it to the device. This token enables subsequent communication to be secure.

[1048] After logging in, the user inputs instructions for care work into the device. For example, specific instructions such as "move patient A from the wheelchair to the bed." The device then sends the instructions along with an authentication token to the computer. The computer analyzes the instructions and forwards them to the AI ​​device. The AI ​​device then begins the care work based on the received instructions.

[1049] While the caregiving task is being performed, the AI ​​device captures the situation with a camera and transmits the video in real time to a computer. The computer then distributes the received video to the device, where the user can review it. The user can input additional instructions into the device as needed and remotely manage the progress of the caregiving task.

[1050] Specific examples

[1051] For example, if a user inputs an instruction into the device to move patient A from his wheelchair to a bed at 9:00 a.m., the operation will proceed as follows:

[1052] 1. The user logs in to the device and enters the instruction "Move patient A from the wheelchair to the bed."

[1053] 2. The device sends the input instructions to the computer.

[1054] 3. The computer receives the instructions and forwards them to the artificial intelligence device.

[1055] 4. Based on the instructions received by the artificial intelligence device, it begins the process of moving Patient A from the wheelchair to the bed.

[1056] 5. While the AI ​​device is working, it captures its actions with a camera and transmits the footage to a computer in real time.

[1057] 6. The computer receives the video and distributes it to the device using a secure protocol.

[1058] 7. The user views the real-time video on the device and enters additional instructions.

[1059] Example prompt sentence:

[1060] We are currently designing an AI-powered nursing robot system to assist caregivers in their work. What technologies can we use to automate the following nursing tasks? Please provide details on the login and authentication process, the instructions and execution of nursing tasks, and the video streaming and monitoring. Please provide specific examples.

[1061] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1062] Step 1:

[1063] A user uses a device to enter login information (username and password) to access a system, and the information entered is collected by the terminal.

[1064] Step 2:

[1065] The device sends the collected login information to the server using a secure protocol (e.g., HTTPS). The input data is sent to the server as transmission data.

[1066] Step 3:

[1067] The server compares the received login information with the database and authenticates whether the user is a valid user. This authentication process verifies whether the entered login information matches the registered information in the database. If authentication is successful, the server generates an authentication token and sends it to the terminal as output data.

[1068] Step 4:

[1069] After logging in, the user inputs instructions for care work (e.g., "Move patient A from the wheelchair to the bed") into the device. The input instructions are collected by the terminal.

[1070] Step 5:

[1071] The device sends the collected instructions and the authentication token to the server using a secure protocol. At this stage, the input data are the instructions and the authentication token.

[1072] Step 6:

[1073] The server analyzes the received instructions and extracts the information necessary to understand the content of the instructions. This analysis process provides specific instruction information. Once the analysis is complete, it generates output data to transmit the instructions to the artificial intelligence device.

[1074] Step 7:

[1075] Based on the analysis results, the server transfers specific instructions to the AI ​​device, which receives the instructions and begins its caregiving duties.

[1076] Step 8:

[1077] The AI ​​device performs a care task based on the received instructions. For example, it may move a patient from a wheelchair to a bed. While the AI ​​device is performing the task, it takes pictures of the task with a camera. The photographed data is generated.

[1078] Step 9:

[1079] The AI ​​device transmits the captured video to a server in real time, and the input data is the captured video information.

[1080] Step 10:

[1081] The server then transmits the received video to the terminal using a secure protocol. The transmitted video is the output data.

[1082] Step 11:

[1083] The user checks the real-time video on the terminal. While monitoring this video, the user inputs additional instructions into the device as needed. The additional instructions are collected as input data.

[1084] Step 12:

[1085] The terminal sends additional instructions to the server, which then forwards the instructions to the AI ​​device. This cycle is repeated to continuously manage the care work.

[1086] (Application example 1)

[1087] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1088] In today's aging society, the shortage of caregivers and other care workers is becoming a serious problem. Furthermore, in order to perform caregiving and monitoring tasks efficiently and accurately, it is essential to reduce the burden on caregivers, automate tasks, and speed up monitoring. Therefore, real-time monitoring and rapid response when an abnormality occurs are required.

[1089] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1090] In this invention, the server includes: a means for a user to input login information using a terminal; a means for the terminal to transmit the login information to the server and for the server to authenticate the received login information; a means for the user to input instructions related to nursing care or monitoring work from the terminal; a means for the server to receive the input instructions and forward the instructions to the worker robot; a means for the worker robot to perform the nursing care or monitoring work based on the received instructions and record the execution status in a recording device; a means for the worker robot to transmit video recorded by the worker robot to the server in real time; a means for the server to distribute the received video to the terminal and for the terminal to display the video to the user; and a means for notifying the user when an abnormality is detected. This enables the automation and efficiency of nursing care and monitoring work, reduces the burden on workers, and enables rapid response when an abnormality occurs.

[1091] "User" means an individual or entity that uses a terminal to operate the System.

[1092] A "terminal" is an electronic device used to access and operate the system.

[1093] "Login Information" means identifying information, such as a username and password, for authenticating access to a system.

[1094] A "server" is a computer that is the core of the system and is a device that authenticates login information, receives and transfers instructions, and stores and distributes data.

[1095] "Authentication" is the process of verifying that a user has legitimate authority to access a system.

[1096] "Instructions" are information that the user inputs via the terminal to provide specific work content related to care work or monitoring work.

[1097] A "worker robot" is an automated machine equipped with artificial intelligence that performs nursing and monitoring tasks based on instructions it receives.

[1098] "Nursing care work" refers to the specific tasks involved in providing care to patients and elderly people.

[1099] "Surveillance work" refers to the regular or continuous observation of a facility or specific area to ensure its safety.

[1100] The "recording device" is a device that records the status of the work performed by the worker robot as video and data.

[1101] "Real-time" refers to the instantaneous transfer of data and video without delay.

[1102] "Distribution" means that the server sends the video and data it receives to the terminal and displays it.

[1103] "Notification" is the process of sending alerts and messages to users when the system detects an abnormality.

[1104] overview

[1105] The present invention is a system that allows a user to log in using a terminal and input instructions to have a worker robot perform nursing care or monitoring tasks. This system includes a server, a terminal, a worker robot, a recording device, and a real-time distribution and notification function.

[1106] System Configuration

[1107] server

[1108] The server acts as the center of the system and performs the following functions:

[1109] 1. Login authentication: Receives the login information entered by the user from the terminal and authenticates it by checking it against a database.

[1110] 2. Instruction transfer: Analyzes instructions received from the user and transfers them to the corresponding worker robot.

[1111] 3. Video distribution: Real-time video received from the worker robot is distributed to the terminal.

[1112] 4. Secure communication: Received data is communicated using a secure protocol (e.g. HTTPS).

[1113] 5. Abnormality notification: If the worker robot or user detects an abnormality, a notification is sent to the user.

[1114] Terminal

[1115] The terminal is an electronic device that allows users to operate the system and has the following functions:

[1116] 1. Entering login information: The user enters their login ID and password.

[1117] 2. Inputting instructions: Entering instructions regarding nursing care and monitoring tasks.

[1118] 3. Video display: Displays real-time video streamed from the server.

[1119] 4. Notification reception: Receive notifications sent from the server when an abnormality occurs.

[1120] Worker robot

[1121] Employee robots are automated machines equipped with AI that perform the following functions:

[1122] 1. Execute Instructions: Performs caregiving or supervisory tasks based on received instructions.

[1123] 2. Recording of execution status: The status of work is recorded in real time by a camera and sent to a server as video data.

[1124] Hardware and software used

[1125] Server: A computer with high-performance computing power (e.g., Amazon Web Services, Google Cloud Platform)

[1126] Devices: Electronic devices such as smartphones, tablets, and computers

[1127] Recording device: High-resolution camera (e.g. Logitech Webcam)

[1128] AI model: Generative AI model (e.g. GPT-3, BERT)

[1129] Secure protocol: HTTPS

[1130] Specific examples

[1131] If a user uses a terminal to input instructions to monitor a specific area at 9:00 a.m., the system operates as follows:

[1132] 1. Terminal: The user enters login information and accesses the system.

[1133] 2. Server: Authenticates the login information and allows the user access. When the user enters an instruction to "monitor a specific area," the server receives this instruction.

[1134] 3. Server: Analyzes the received instructions and forwards them to the appropriate worker robot.

[1135] 4. Worker robot: Based on instructions, it patrols designated areas and performs surveillance tasks. It records images of its patrols in real time with a camera.

[1136] 5. Server: Receives the video sent from the robot and distributes it to the terminal.

[1137] 6. Terminal: The user checks the real-time video and inputs additional instructions if an abnormality is detected. If an abnormality is detected, the user receives a notification.

[1138] Prompt Sentence Examples

[1139] To further refine the design using the generative AI model, use the following prompt:

[1140] Please explain the design of a security robot system. The system allows users to log in from their smartphones and instruct the AI-equipped robots on how to perform monitoring tasks. The robots will send real-time video footage to the smartphones. If an abnormality is detected, notifications will be sent to the users. This system will be built using Python, Flask, and OpenCV.

[1141] This prompt can be used to obtain detailed design information and implementation methods from the AI ​​model.

[1142] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1143] Step 1:

[1144] The user enters their login information using the device.

[1145] Input: User's login ID and password

[1146] Operation: The user enters login information into the device's login screen and presses the "Login" button.

[1147] Output: The device sends the login information to the server.

[1148] Step 2:

[1149] The server authenticates the received login information.

[1150] Input: Login ID and password sent from the device

[1151] How it works: The server checks the received login information against a database to determine if the user is legitimate.

[1152] Output: If authentication is successful, the server generates an authentication token and sends it to the terminal. If authentication fails, it returns an error message to the terminal.

[1153] Step 3:

[1154] The user inputs instructions regarding the care work or monitoring work from the terminal.

[1155] Input: Authentication token after logging in, instructions for care work or monitoring work (e.g., "Monitor a specific area")

[1156] Operation: The terminal receives instructions entered by the user and sends them to the server.

[1157] Output: Data containing instructions and an authentication token is sent to the server.

[1158] Step 4:

[1159] The server receives the input instructions and forwards the instructions to the worker robot.

[1160] Input: Instructions, authentication token

[1161] Operation: The server analyzes the received instructions and forwards them to the corresponding worker robot.

[1162] Output: The parsed instructions are sent to the worker robot.

[1163] Step 5:

[1164] The worker robot performs caregiving or monitoring tasks based on the received instructions.

[1165] Input: Instructions sent by the server

[1166] Action: The worker robot follows instructions and begins nursing or monitoring tasks.

[1167] Output: The execution status is recorded in real time by a camera.

[1168] Step 6:

[1169] The worker robot transmits the recorded video to a server in real time.

[1170] Input: Camera footage

[1171] Operation: The worker robot transmits the recorded video in real time to the server.

[1172] Output: Real-time video data is sent to the server.

[1173] Step 7:

[1174] The server distributes the received video to the terminal, and the terminal displays the video to the user.

[1175] Input: Real-time video sent from the worker robot

[1176] Operation: The server delivers the received video to the device using a secure protocol.

[1177] Output: The distributed real-time video is displayed on the terminal.

[1178] Step 8:

[1179] The server will notify the user when an abnormality is detected.

[1180] Input: Anomaly detection information or manual detection input by the user

[1181] Operation: When the server detects an abnormality, it sends a push notification to the user device.

[1182] Output: An abnormality notification is displayed on the user's terminal.

[1183] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1184] In addition to the conventional nursing care support system, the present invention provides a new technology that recognizes and responds to the emotions of users and patients. Hereinafter, embodiments of the present invention will be described in detail.

[1185] In this system, the user uses a terminal to give instructions on care tasks, and the AI ​​robot carries out the tasks based on those instructions. The system also captures the progress of the tasks with a camera and can be monitored remotely in real time. Furthermore, an emotion engine is used to recognize the emotions of the user and patient, optimizing the operation of the entire system.

[1186] Login and Authentication Process

[1187] Authentication process

[1188] When a user tries to access a system using a terminal, they must first log in. The terminal displays a login screen to the user and asks them to enter their login ID and password. Once the user enters the login information, the terminal sends it to the server. The server compares the received login information with a database and performs authentication. If this authentication is successful, the server sends an authentication token to the terminal, and the user gains access to the system.

[1189] Furthermore, the emotion engine analyzes the user's facial expressions and voice to recognize their emotional state at the time of login, which makes it possible to detect the user's stress level and fatigue at that time.

[1190] Directing and carrying out nursing care tasks

[1191] Sending instructions

[1192] After the user logs in from the device, an interface for inputting instructions for care work is displayed. For example, specific care instructions such as "move patient A from the wheelchair to the bed" are entered. The device sends this instruction along with an authentication token to the server. The server analyzes the received instructions and forwards them to the AI ​​robot.

[1193] In addition, when inputting instructions, the emotion engine analyzes the user's emotions and sends the results to the server, which can then transfer instructions to the AI ​​robot that take into account the user's emotional state.

[1194] Robot movement

[1195] The AI ​​robot receives instructions from the server and begins its nursing care tasks based on those instructions. For example, it safely performs tasks such as moving and assisting the patient. While working, the robot captures its movements with a camera and sends the footage to the server in real time. Furthermore, the AI ​​robot is equipped with an emotion engine that recognizes emotions from the patient's facial expressions and voice, and adjusts its tasks as necessary.

[1196] Video distribution and monitoring

[1197] Real-time monitoring

[1198] The server receives the video sent by the AI ​​robot. This video is then distributed to the device using a secure protocol. The device displays the video in real time, allowing the user to monitor it. The emotion engine continuously monitors the user's emotional state and issues an alert if an abnormality is detected.

[1199] Specific examples

[1200] For example, if a user inputs an instruction into a terminal to move patient A from a wheelchair to a bed at 9:00 a.m., the system operates as follows:

[1201] 1. The user logs in to the terminal and inputs the command, "Move patient A from the wheelchair to the bed." At this time, the emotion engine analyzes the user's facial expressions and voice and sends the emotional state to the server.

[1202] 2. The device sends the input instructions to the server, which then analyzes the received instruction data and emotion data.

[1203] 3. The server transfers instructions and emotional data to the AI ​​robot. If the user is under stress, the robot will adjust its work pace to take that situation into account.

[1204] 4. The AI ​​robot begins its nursing care tasks based on the instructions. When transferring Patient A from the wheelchair to the bed, the emotion engine also monitors the patient's emotional state.

[1205] 5. While the AI ​​robot is working, a camera captures its movements and sends the footage to a server in real time.

[1206] 6. The server delivers the video data to the device, which displays the video to the user in real time.

[1207] 7. The user monitors the video on the device to ensure that the care work is being carried out properly. The emotion engine also continuously monitors the user's emotions and issues an alert if there are any abnormalities.

[1208] This invention not only realizes automation and efficiency in nursing care work, dramatically reducing the burden on caregivers, but also provides a safer and more comfortable nursing environment through emotion recognition, making it an extremely useful system in an aging society.

[1209] The processing flow will be explained below.

[1210] Step 1:

[1211] The user accesses the login screen of the device, and the device displays a form for entering the login ID and password.

[1212] Step 2:

[1213] The user enters their login ID and password and clicks the send button.

[1214] Step 3:

[1215] The terminal encrypts the entered login ID and password and sends them to the server.

[1216] Step 4:

[1217] The server decrypts the encrypted login ID and password it receives and checks them against the database.

[1218] Step 5:

[1219] The server checks the authentication result, and if the authentication is successful, generates an authentication token and sends it to the device.

[1220] Step 6:

[1221] The device receives a successful authentication message and token, and displays a message to the user indicating that they have been granted access to the system. The emotion engine analyzes the user's facial expressions and voice to confirm their emotional state at that time.

[1222] Step 7:

[1223] The user accesses an interface that allows them to input instructions for nursing care tasks into the terminal. For example, they input an instruction to "move patient A from the wheelchair to the bed."

[1224] Step 8:

[1225] The emotion engine analyzes the facial expressions and voice of the user inputting instructions and generates emotion data.

[1226] Step 9:

[1227] The terminal transmits the input instruction data, emotion data, and authentication token to the server.

[1228] Step 10:

[1229] The server checks the received instruction data, emotion data, and authentication token, and analyzes the instruction content, taking into account the user's emotional state.

[1230] Step 11:

[1231] The server analyzes the instruction data and transfers it to the AI ​​robot. If the user is under stress, the instructions to the robot are adjusted.

[1232] Step 12:

[1233] The AI ​​robot receives and analyzes the instructions, then begins its caregiving tasks, such as moving patient A from a wheelchair to a bed.

[1234] Step 13:

[1235] The emotion engine analyzes the patient's facial expressions and voice to assess their emotional state and adjust the work as needed.

[1236] Step 14:

[1237] While the AI ​​robot is performing its nursing care duties, a camera will record what is happening.

[1238] Step 15:

[1239] The AI ​​robot sends the footage it captures in real time to a server.

[1240] Step 16:

[1241] The server delivers the received video data to the terminal using a secure protocol.

[1242] Step 17:

[1243] The terminal displays the video data received from the server to the user, and the emotion engine continuously monitors the user's emotional state.

[1244] Step 18:

[1245] Users can monitor the video footage via their devices to ensure that care work is being carried out properly, and if there is an abnormality, a warning is issued.

[1246] Step 19:

[1247] If necessary, the user inputs additional instructions, and the terminal transmits the data to the server again.

[1248] Step 20:

[1249] The server forwards the received additional instructions to the AI ​​robot, which then changes or performs additional nursing care tasks based on the instructions.

[1250] Example 2

[1251] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1252] Conventional nursing care support systems have difficulty recognizing the emotional states of caregivers and patients and responding appropriately. Furthermore, even though remote real-time monitoring is possible, there is a lack of systems that can optimize behavior based on emotions or detect abnormalities, making it difficult to fully ensure the safety and efficiency of nursing care work.

[1253] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input login information using a terminal; a means for the terminal to transmit the login information to the server and for the server to authenticate the received login information; a means having an emotion analysis engine for recognizing the emotional states of the user and the patient; a means for the user to input instructions related to the care work from the terminal; a means for the server to receive, analyze, and forward the input instructions to the AI ​​robot; a means for the AI ​​robot to perform the care work based on the received instructions and capture the performance status with a camera; a means for transmitting the video captured by the AI ​​robot to the server in real time; a means for the server to optimize the operation by taking into account the emotional states of the user and the patient via the emotion analysis engine; a means for the server to distribute the received video to the terminal, which then displays the video to the user and continuously monitors the user's emotional state; and a means for issuing an alert when an abnormality is detected. This improves the safety and efficiency of care work and enables more appropriate care that takes into account the emotional state.

[1254] "Login information" refers to the authentication data that a user enters when accessing a system, and typically consists of a user ID and password.

[1255] A "terminal" is a device used by a user to access the nursing care support system, input instructions, and monitor video footage, and includes PCs, tablets, smartphones, etc.

[1256] The "server" is a computer system that serves as the core of the nursing care support system and performs various processes such as login authentication, instruction analysis, data transfer, emotional state analysis, and real-time video distribution.

[1257] An "emotion analysis engine" is a software or hardware component that analyzes the facial expressions and voice of a user or patient to recognize and evaluate their emotional state.

[1258] "Instruction data" refers to data that includes specific instructions regarding care work input by the user via the terminal.

[1259] An "AI robot" is an autonomous robot that performs nursing care tasks based on instructions received from a server, and is equipped with a camera, emotion analysis engine, and other features.

[1260] The "camera" is a device installed on the AI ​​robot that captures the nursing care work being performed in real time and sends the footage to a server.

[1261] A "secure protocol" is a secure communication method that prevents data transmission and reception from being tampered with or intercepted by third parties, and includes SSL / TLS.

[1262] An "authentication token" is temporary data issued by the server when a user is successfully authenticated, and is used to verify that subsequent communications and instructions are legitimate.

[1263] "Real-time streaming" is a technology that transmits footage captured by an AI robot from a server to a terminal with almost no delay, allowing users to monitor it immediately.

[1264] "Abnormality detection" is a function that allows the system to detect abnormal patterns in the emotional state of a user or patient or the execution of care tasks and issue a warning.

[1265] In addition to the conventional nursing care support system, the present invention provides a technology that recognizes the emotions of users and patients and takes optimal measures according to their emotional states. Hereinafter, embodiments of the present invention will be described in detail.

[1266] This system consists of the following main components: a server, a terminal, an AI robot, a camera, and an emotion analysis engine. The functions and interactions of each component are described in detail below.

[1267] Server Roles

[1268] The server acts as the central part of the system and performs the following main tasks:

[1269] 1. Login authentication: The login information entered by the user from the terminal is compared with the database. If authentication is successful, an authentication token is issued.

[1270] 2. Receiving and analyzing instruction data: Receives and analyzes instructions for nursing care tasks entered by the user through the terminal.

[1271] 3. Transfer of instruction data: Based on the analysis results, appropriate instruction data is transferred to the AI ​​robot.

[1272] 4. Sentiment analysis and optimization: Use a sentiment analysis engine to analyze the emotional state of users and patients and optimize operations.

[1273] 5. Real-time video distribution: Receives video data sent from the AI ​​robot and distributes it to the terminal in real time.

[1274] 6. Anomaly detection and warning: If an abnormality is detected in the user or patient's condition, a warning will be issued to the terminal.

[1275] Device Role

[1276] The terminal is used by the user to access the system, enter commands, and monitor footage. Terminal functions include:

[1277] 1. Displaying a login screen: A login screen is displayed to the user and they are prompted to enter their login information.

[1278] 2. Providing an instruction input interface: An interface is provided for the user to input specific nursing care instructions.

[1279] 3. Real-time video display: Displays the video received from the server in real time to monitor the progress of nursing care work.

[1280] 4. Displaying a warning notification: When an abnormality is detected, a warning is displayed to the user.

[1281] The role of AI robots

[1282] The AI ​​robot performs care tasks based on instructions received from the server. Specific functions include:

[1283] 1. Carrying out care tasks: Carry out care tasks (e.g., moving or assisting patients) according to received instructions.

[1284] 2. Video recording: The nursing care work is recorded with a camera and the video is sent to the server in real time.

[1285] 3. Emotion Analysis: Use the built-in emotion analysis engine to analyze the patient's emotional state and optimize the progress of the work.

[1286] The role of sentiment analysis engines

[1287] The emotion analysis engine analyzes the facial expressions and voice of users and patients to recognize their emotional state, providing important data for optimizing the operation of the entire system.

[1288] Specific examples

[1289] For example, if a user inputs the instruction "Move patient A from his wheelchair to his bed" into the terminal at 9:00 AM, the system will perform the following specific operations:

[1290] 1. The user logs in to the device and inputs instructions. The emotion analysis engine analyzes the user's facial expressions and voice and sends their emotional state to the server.

[1291] 2. The terminal transmits the input instructions and emotion data to the server.

[1292] 3. The server analyzes the instruction data and emotional data and transmits them to the AI ​​robot. If stress is detected, the robot will adjust its work pace.

[1293] 4. The AI ​​robot will begin nursing care tasks based on instructions, while simultaneously analyzing the patient's emotions and optimizing its tasks accordingly.

[1294] 5. The AI ​​robot uses a camera to record its work and sends the footage to a server in real time.

[1295] 6. The server delivers the video data to the terminal and displays it to the user in real time.

[1296] 7. The user monitors the video through the device, and if the emotion analysis engine detects an abnormality, an alert is issued.

[1297] Prompt Sentence Examples

[1298] "Transfer patient A from wheelchair to bed at 9:00 AM"

[1299] "Please notify me of patient B's medication time."

[1300] "Check on patient C's safety"

[1301] This invention not only realizes automation and efficiency of nursing care work, reducing the burden on caregivers, but also makes it possible to provide a safer and more comfortable nursing care environment through emotion recognition.

[1302] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1303] Step 1:

[1304] Enter and submit your login information

[1305] The user accesses the login screen using the device and enters the login ID and password.

[1306] Input: The login ID and password entered by the user.

[1307] Output: Data format in which login information (ID and password) is sent to the terminal

[1308] Specific operation: The terminal receives the user's input and sends it to the server.

[1309] Step 2:

[1310] Authenticate your login details

[1311] The server compares the login information received from the terminal with the database to perform authentication.

[1312] Input: Login information sent from the device

[1313] Output: Authentication token or authentication failure message

[1314] Specific operation: The server checks the login information against the database, and if authentication is successful, it generates an authentication token and sends it to the terminal. If authentication fails, it returns an error message.

[1315] Step 3:

[1316] Emotional state analysis

[1317] The emotion analysis engine analyzes the user's facial expressions and voice to recognize their emotional state at the time of login.

[1318] Input: User's facial expression and voice data

[1319] Output: Emotional state information (stress level, fatigue level, etc.)

[1320] How it works: The emotion analysis engine monitors the user's face and voice in real time and evaluates their state using an emotion analysis algorithm. The evaluation results are sent to the server and recorded.

[1321] Step 4:

[1322] Entering nursing care instructions

[1323] The user inputs specific instructions regarding the care work from the terminal.

[1324] Input: Care work instructions entered by the user

[1325] Output: Instruction data (specific nursing care work content)

[1326] Specific operation: The terminal provides an interface for inputting instructions for nursing care work, and the user inputs specific instructions (e.g., "Move patient A from the wheelchair to the bed").

[1327] Step 5:

[1328] Sending and analyzing instruction data

[1329] Care work instructions and emotional state information are sent from the terminal to the server, which then analyzes them.

[1330] Input: Care instructions and emotional state information

[1331] Output: Analysis results (optimized instruction data)

[1332] Specific operation: The server analyzes the received instruction data and generates optimized instructions that take into account the emotional state information. These instructions are then forwarded to the AI ​​robot.

[1333] Step 6:

[1334] Carrying out nursing care tasks with AI robots

[1335] The AI ​​robot performs nursing care tasks based on instructions from the server.

[1336] Input: Optimized instruction data

[1337] Output: Status of nursing care work and video data

[1338] Specific actions: The AI ​​robot begins specific care tasks (e.g., moving a patient) based on the instructions it receives. The robot's actions are captured in real time by a camera.

[1339] Step 7:

[1340] Real-time video transmission and distribution

[1341] The video data captured by the AI ​​robot is sent to a server, which then distributes it to the device in real time.

[1342] Input: Real-time video data

[1343] Output: Real-time video streaming data

[1344] How it works: The AI ​​robot captures video of itself working with a camera and sends it in real time to a server, which then receives the data and distributes it to the device using a secure protocol.

[1345] Step 8:

[1346] Anomaly detection and warning

[1347] The server uses an emotion analysis engine to continuously monitor the emotional state of users and patients and issues an alert if an abnormality is detected.

[1348] Input: Emotional state information and care work execution state

[1349] Output: Warning notice

[1350] Specific operation: The server analyzes the data from the sentiment analysis engine, and if an anomaly is detected, it sends a warning message to the device. The user receives the warning on the device and can take appropriate action.

[1351] In this way, this system works by linking users, servers, AI robots, and emotion analysis engines to carry out nursing care tasks efficiently and safely.

[1352] (Application example 2)

[1353] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1354] Current work support systems only transfer and execute simple instructions without considering the emotional state of the user or target, which means that the safety and efficiency of the work environment cannot be sufficiently maintained.In addition, because they are unable to provide feedback based on real-time emotion recognition, stress and fatigue levels may be overlooked, which could have a negative impact on the health and safety of workers and targets.

[1355] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input login information using a terminal; a means for the terminal to transmit the login information to the server and for the server to authenticate the received login information; a means for the user to input instructions related to nursing care work from the terminal; a means for the server to receive the input instructions and forward the instructions to the AI ​​processing device; a means for the AI ​​processing device to perform work based on the received instructions and record the execution status using an optical device; a means for the AI ​​processing device to transmit the recorded video to the server in real time; a means for the server to distribute the received video to the terminal and for the terminal to display the video to the user; a means for analyzing the emotions of the user and the target person in real time using an emotion recognition engine; a means for adjusting the operation of the AI ​​processing device based on the target person's emotional state; and a means for analyzing the video from the optical device and issuing a safety warning based on the results of emotion recognition. This makes it possible to improve the safety and efficiency of the work environment.

[1356] A "user" is a person who operates the system and gives instructions for work.

[1357] "Terminal" refers to an electronic device operated by a user, including smart glasses and PCs.

[1358] "Login information" means the authentication information provided by a user to access the system, and specifically includes a user ID and password.

[1359] A "server" is a computer system on a network that manages the entire system and processes data.

[1360] An "authentication token" is authentication data for subsequent communications that is issued by the server after a user has successfully logged in.

[1361] An "AI processing device" is a device that uses AI technology to analyze instructions and automatically execute tasks.

[1362] An "optical device" is a device for recording images, such as a camera.

[1363] An "emotion recognition engine" is a technology or software that analyzes the emotional state of a user or target person in real time from their facial expressions and voice.

[1364] "Real-time" refers to data processing and communication occurring instantly, without delay.

[1365] "Target" refers to the person to whom the system provides services, and in the case of a care robot, it means the person being cared for.

[1366] "Safety warnings" are caution or warning messages issued for the safety of users or targets based on the analysis results of the emotion recognition engine.

[1367] This invention is a system for improving the safety and efficiency of a work environment. This system is composed of a user, a terminal, a server, an AI processing device, an optical device, and an emotion recognition engine.

[1368] First, the user enters login information using the terminal. The terminal then sends this login information to the server, which then performs authentication using the received login information. If authentication is successful, the server generates an authentication token and sends it to the terminal.

[1369] Next, the user can input instructions related to the work from the device. For example, a command to start a specific task may be given. The device then sends this instruction along with an authentication token to the server. The server then analyzes the received instruction and forwards it to the AI ​​processing device based on its content.

[1370] The AI ​​processing device performs tasks based on the received instructions and records its progress using an optical device (such as a camera). The recorded video is sent to a server in real time. The server then distributes the received video to a terminal, allowing users to monitor the video in real time through their terminal.

[1371] Furthermore, the emotional state of the user and the target can be analyzed in real time using an emotion recognition engine. The emotion recognition engine analyzes the facial expressions and voice data of the user and the target, and adjusts the operation of the AI ​​processing unit based on the analysis results. It is also possible to issue safety warnings as needed based on the analysis results.

[1372] For example, if a factory worker wears smart glasses, the glasses' camera and microphone can be used to analyze the worker's facial expressions and voice to recognize their emotional state in real time. Based on this recognition result, the system can determine whether the working environment is appropriate and issue a warning if necessary.

[1373] For example, if Worker A is wearing smart glasses at the start of his shift at 8 a.m. and the emotion recognition engine detects fatigue, the system will automatically issue a warning and prompt the worker to take a break, thus optimizing the working environment and improving safety and work efficiency.

[1374] An example of a prompt sentence is as follows:

[1375] "How can we monitor the emotions of factory workers in real time and issue warnings if fatigue or stress is detected to improve work efficiency?"

[1376] This allows the system embodying the invention to take into account the emotional state of the user and the subject, providing a safer and more efficient working environment.

[1377] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1378] Step 1:

[1379] The user enters login information using the terminal. The terminal displays the login screen, and the user enters their login ID and password. The terminal then sends the entered login information to the server.

[1380] Input: User login ID and password

[1381] Output: Login information sent to the server

[1382] Step 2:

[1383] The server authenticates the received login information. The server accesses a database and checks the received login information. If authentication is successful, the server generates an authentication token and sends it to the terminal.

[1384] Input: Login information

[1385] Output: Authentication token

[1386] Step 3:

[1387] The user inputs work instructions from the terminal. The authenticated terminal displays the work instruction input screen, and the user inputs specific work instructions (e.g., instructions for a specific task). The terminal then sends these instructions to the server along with an authentication token.

[1388] Input: Work instructions, authentication token

[1389] Output: Work instructions sent to the server

[1390] Step 4:

[1391] The server receives and analyzes the input instructions, and then transfers the instructions to the appropriate AI processing device based on their content.

[1392] Input: Work Instructions

[1393] Output: Instructions to the AI ​​processor

[1394] Step 5:

[1395] The AI ​​processing unit executes tasks based on the instructions it receives, such as a robot performing a specific action. The AI ​​processing unit continues to record the progress of the task using an optical device.

[1396] Input: Instructions from the server

[1397] Output: Work progress

[1398] Step 6:

[1399] The AI ​​processing device transmits the recorded video to the server in real time. The AI ​​processing device transmits the video data captured by the optical device to the server in real time.

[1400] Input: Recorded video

[1401] Output: Video data sent to the server

[1402] Step 7:

[1403] The server delivers the received video to the terminal. The server delivers the received video to the terminal in real time using a secure protocol. The terminal displays this video to the user.

[1404] Input: Video data

[1405] Output: The image displayed to the user

[1406] Step 8:

[1407] The emotion recognition engine analyzes the emotions of the user and the target. The emotion recognition model is run using video and audio data. Based on the analysis results, it determines whether the operation of the AI ​​processing unit needs to be adjusted.

[1408] Input: Video data, audio data

[1409] Output: Emotion analysis results

[1410] Step 9:

[1411] The emotion recognition engine issues safety warnings based on the analysis results. If the analysis results indicate that the user or target is in danger, the system will automatically issue a warning and prompt the user to take necessary measures.

[1412] Input: Sentiment analysis results

[1413] Output: Safety warning

[1414] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1415] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1416] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1417] [Fourth embodiment]

[1418] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1419] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1420] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1421] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1422] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1423] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1424] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1425] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1426] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1427] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1428] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1429] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1430] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1431] The present invention relates to a nursing robot system equipped with AI for supporting the work of a caregiver. Hereinafter, an embodiment of the present invention will be specifically described.

[1432] In this system, users use a terminal to give instructions on care tasks, and the AI ​​robot carries out the tasks based on those instructions. Furthermore, the progress of the tasks can be captured on a camera and monitored remotely in real time.

[1433] Login and Authentication Process

[1434] Authentication process

[1435] When a user tries to access a system using a terminal, they must first log in. The terminal displays a login screen to the user and asks them to enter their login ID and password. Once the user enters the login information, the terminal sends it to the server. The server compares the received login information with a database and authenticates whether the user is a legitimate user. If this authentication is successful, the server sends an authentication token to the terminal, and the user gains access to the system.

[1436] Directing and carrying out nursing care tasks

[1437] Sending instructions

[1438] After the user logs in from the device, an interface for inputting instructions for nursing care work is displayed. For example, specific nursing care instructions such as "move patient A from the wheelchair to the bed" are entered. The device sends this instruction along with an authentication token to the server. The server analyzes the received instructions and forwards them to the AI ​​robot.

[1439] Robot movement

[1440] The AI ​​robot receives instructions from the server and begins its nursing care tasks based on those instructions. For example, it safely performs tasks such as moving and providing care. While working, the robot captures its actions with a camera and sends the footage to the server in real time.

[1441] Video distribution and monitoring

[1442] Real-time monitoring

[1443] The server receives the video sent by the AI ​​robot. This video is then distributed to the terminal using a secure protocol. The terminal displays the video in real time, allowing the user to monitor it and issue additional instructions as needed.

[1444] Specific examples

[1445] For example, if a user inputs an instruction into a terminal to move patient A from a wheelchair to a bed at 9:00 a.m., the system operates as follows:

[1446] 1. The user logs in to the terminal and enters the instruction "Move patient A from the wheelchair to the bed."

[1447] 2. The terminal sends the entered instructions to the server.

[1448] 3. The server receives the instructions and forwards them to the AI ​​robot.

[1449] 4. The AI ​​robot begins its nursing care duties based on the instructions it receives.

[1450] 5. While the AI ​​robot is working, a camera captures its movements and sends the footage to a server in real time.

[1451] 6. The server receives the video and distributes it to the device using a secure protocol.

[1452] 7. The user checks the real-time video on the terminal and enters additional instructions as necessary.

[1453] This invention realizes automation and efficiency of nursing care work, dramatically reducing the burden on caregivers, making it an extremely useful system in an aging society.

[1454] The processing flow will be explained below.

[1455] Step 1:

[1456] The user accesses the login screen of the device, and the device displays a form for entering the login ID and password.

[1457] Step 2:

[1458] The user enters their login ID and password and clicks the send button.

[1459] Step 3:

[1460] The terminal encrypts the entered login ID and password and sends them to the server.

[1461] Step 4:

[1462] The server decrypts the encrypted login ID and password it receives and checks them against the database.

[1463] Step 5:

[1464] The server checks the authentication result, and if the authentication is successful, generates an authentication token and sends it to the device.

[1465] Step 6:

[1466] The terminal receives a successful authentication message and token and displays to the user that they have been granted access to the system.

[1467] Step 7:

[1468] The user accesses an interface that allows them to input instructions for nursing care tasks into the terminal. For example, they input an instruction to "move patient A from the wheelchair to the bed."

[1469] Step 8:

[1470] The terminal transmits the input instruction data and the authentication token to the server.

[1471] Step 9:

[1472] The server checks the received instruction data and authentication token and analyzes the instruction content.

[1473] Step 10:

[1474] The server then transfers the analyzed instruction data to the AI ​​robot.

[1475] Step 11:

[1476] The AI ​​robot receives and analyzes the instructions, then begins its caregiving tasks, such as moving patient A from a wheelchair to a bed.

[1477] Step 12:

[1478] While the AI ​​robot is performing its nursing care duties, a camera will record what is happening.

[1479] Step 13:

[1480] The AI ​​robot sends the footage it captures in real time to a server.

[1481] Step 14:

[1482] The server delivers the received video data to the terminal using a secure protocol.

[1483] Step 15:

[1484] The terminal displays the video data received from the server to the user.

[1485] Step 16:

[1486] The user monitors the video footage through the device to ensure that care work is being carried out properly.

[1487] Step 17:

[1488] If necessary, the user inputs additional instructions, and the terminal transmits the data to the server again.

[1489] Step 18:

[1490] The server forwards the received additional instructions to the AI ​​robot, which then changes or performs additional nursing care tasks based on the instructions.

[1491] Example 1

[1492] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1493] As the aging society progresses, the workload of caregivers is increasing. In particular, the diversity and heavy workload of caregiving tasks have become issues, and there is a need to improve work efficiency and reduce the burden. In addition, monitoring and giving instructions on caregiving tasks from remote locations is also very important, so a system that can grasp the situation in real time and respond quickly is required.

[1494] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1495] In this invention, the server includes: means for a user to input login information using a device; means for the device to transmit the login information to a computer and for the computer to authenticate the received login information; means for the user to input instructions related to nursing care tasks from the device; means for the computer to receive the input instructions and forward the instructions to an artificial intelligence device; means for the artificial intelligence device to perform the nursing care tasks based on the received instructions and capture the performance status with an imaging device; means for the artificial intelligence device to transmit the captured image to the computer in real time; means for the computer to distribute the received image to the device and for the device to display the image to the user; means for communicating the data received by the computer using a secure protocol; and means for generating an authentication token when a user logs in from the device and using the token to confirm the consistency of subsequent instructions. This automates and streamlines nursing care tasks, reduces the burden on caregivers, and enables remote monitoring and instruction of nursing care tasks in real time.

[1496] "User" refers to people such as caregivers and care staff who use the system.

[1497] "Device" refers to an electronic terminal used for logging in, inputting instructions for care work, displaying images, etc.

[1498] "Login information" refers to information such as a user name and password that a user enters when accessing a system.

[1499] "Computer" refers to a server device that receives data sent from devices and performs authentication, analyzes instructions, distributes video, etc.

[1500] "Artificial intelligence device" refers to a robot or device equipped with AI that receives instructions from a computer and performs nursing care tasks based on those instructions.

[1501] "Photography device" refers to a device for photographing the status of nursing care work, such as a camera installed in an artificial intelligence device.

[1502] A "secure protocol" refers to a protocol used to ensure that data is transmitted securely. An example is HTTPS.

[1503] "Authentication Token" means temporary authentication information generated after login and used to authenticate subsequent communications and instructions.

[1504] The present invention relates to an AI-equipped nursing robot system for supporting nursing care work. By using this system, the efficiency and automation of nursing care work can be improved, and the burden on caregivers can be reduced. Detailed embodiments for carrying out the present invention will be described.

[1505] This system is operated by the user using a device, and is composed of a combination of multiple hardware and software components, including devices, computers, artificial intelligence devices, and imaging devices.

[1506] Hardware and software used

[1507] Device: An electronic device (e.g., laptop, smartphone, tablet) used to log in, input care instructions, and display images.

[1508] Computer: A server device (e.g., AWS EC2, Microsoft Azure) that receives data sent from devices and performs authentication, analyzes instructions, and distributes video.

[1509] Artificial intelligence device: A robot that receives instructions from a computer and performs caregiving tasks based on those instructions (e.g., SoftBank's Pepper, Riken-TRI Collaboration Center for Human-Interactive Robot Research's ROBEAR).

[1510] Camera: A camera that records the nursing care work being performed (e.g., Logitech C920).

[1511] Program processing and data processing

[1512] To access a system using a device, a user must first log in. The device displays a login screen to the user, prompting them to enter a username and password. After the user enters and submits this information, the device sends the information to a computer. The computer compares the received login information with a database to authenticate whether the user is legitimate. If authentication is successful, the computer generates an authentication token and sends it to the device. This token enables subsequent communication to be secure.

[1513] After logging in, the user inputs instructions for care work into the device. For example, specific instructions such as "move patient A from the wheelchair to the bed." The device then sends the instructions along with an authentication token to the computer. The computer analyzes the instructions and forwards them to the AI ​​device. The AI ​​device then begins the care work based on the received instructions.

[1514] While the caregiving task is being performed, the AI ​​device captures the situation with a camera and transmits the video in real time to a computer. The computer then distributes the received video to the device, where the user can review it. The user can input additional instructions into the device as needed and remotely manage the progress of the caregiving task.

[1515] Specific examples

[1516] For example, if a user inputs an instruction into the device to move patient A from his wheelchair to a bed at 9:00 a.m., the operation will proceed as follows:

[1517] 1. The user logs in to the device and enters the instruction "Move patient A from the wheelchair to the bed."

[1518] 2. The device sends the input instructions to the computer.

[1519] 3. The computer receives the instructions and forwards them to the artificial intelligence device.

[1520] 4. Based on the instructions received by the artificial intelligence device, it begins the process of moving Patient A from the wheelchair to the bed.

[1521] 5. While the AI ​​device is working, it captures its actions with a camera and transmits the footage to a computer in real time.

[1522] 6. The computer receives the video and distributes it to the device using a secure protocol.

[1523] 7. The user views the real-time video on the device and enters additional instructions.

[1524] Example prompt sentence:

[1525] We are currently designing an AI-powered nursing robot system to assist caregivers in their work. What technologies can we use to automate the following nursing tasks? Please provide details on the login and authentication process, the instructions and execution of nursing tasks, and the video streaming and monitoring. Please provide specific examples.

[1526] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1527] Step 1:

[1528] A user uses a device to enter login information (username and password) to access a system, and the information entered is collected by the terminal.

[1529] Step 2:

[1530] The device sends the collected login information to the server using a secure protocol (e.g., HTTPS). The input data is sent to the server as transmission data.

[1531] Step 3:

[1532] The server compares the received login information with the database and authenticates whether the user is a valid user. This authentication process verifies whether the entered login information matches the registered information in the database. If authentication is successful, the server generates an authentication token and sends it to the terminal as output data.

[1533] Step 4:

[1534] After logging in, the user inputs instructions for care work (e.g., "Move patient A from the wheelchair to the bed") into the device. The input instructions are collected by the terminal.

[1535] Step 5:

[1536] The device sends the collected instructions and the authentication token to the server using a secure protocol. At this stage, the input data are the instructions and the authentication token.

[1537] Step 6:

[1538] The server analyzes the received instructions and extracts the information necessary to understand the content of the instructions. This analysis process provides specific instruction information. Once the analysis is complete, it generates output data to transmit the instructions to the artificial intelligence device.

[1539] Step 7:

[1540] Based on the analysis results, the server transfers specific instructions to the AI ​​device, which receives the instructions and begins its caregiving duties.

[1541] Step 8:

[1542] The AI ​​device performs a care task based on the received instructions. For example, it may move a patient from a wheelchair to a bed. While the AI ​​device is performing the task, it takes pictures of the task with a camera. The photographed data is generated.

[1543] Step 9:

[1544] The AI ​​device transmits the captured video to a server in real time, and the input data is the captured video information.

[1545] Step 10:

[1546] The server then transmits the received video to the terminal using a secure protocol. The transmitted video is the output data.

[1547] Step 11:

[1548] The user checks the real-time video on the terminal. While monitoring this video, the user inputs additional instructions into the device as needed. The additional instructions are collected as input data.

[1549] Step 12:

[1550] The terminal sends additional instructions to the server, which then forwards the instructions to the AI ​​device. This cycle is repeated to continuously manage the care work.

[1551] (Application example 1)

[1552] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1553] In today's aging society, the shortage of caregivers and other care workers is becoming a serious problem. Furthermore, in order to perform caregiving and monitoring tasks efficiently and accurately, it is essential to reduce the burden on caregivers, automate tasks, and speed up monitoring. Therefore, real-time monitoring and rapid response when an abnormality occurs are required.

[1554] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1555] In this invention, the server includes: a means for a user to input login information using a terminal; a means for the terminal to transmit the login information to the server and for the server to authenticate the received login information; a means for the user to input instructions related to nursing care or monitoring work from the terminal; a means for the server to receive the input instructions and forward the instructions to the worker robot; a means for the worker robot to perform the nursing care or monitoring work based on the received instructions and record the execution status in a recording device; a means for the worker robot to transmit video recorded by the worker robot to the server in real time; a means for the server to distribute the received video to the terminal and for the terminal to display the video to the user; and a means for notifying the user when an abnormality is detected. This enables the automation and efficiency of nursing care and monitoring work, reduces the burden on workers, and enables rapid response when an abnormality occurs.

[1556] "User" means an individual or entity that uses a terminal to operate the System.

[1557] A "terminal" is an electronic device used to access and operate the system.

[1558] "Login Information" means identifying information, such as a username and password, for authenticating access to a system.

[1559] A "server" is a computer that is the core of the system and is a device that authenticates login information, receives and transfers instructions, and stores and distributes data.

[1560] "Authentication" is the process of verifying that a user has legitimate authority to access a system.

[1561] "Instructions" are information that the user inputs via the terminal to provide specific work content related to care work or monitoring work.

[1562] A "worker robot" is an automated machine equipped with artificial intelligence that performs nursing and monitoring tasks based on instructions it receives.

[1563] "Nursing care work" refers to the specific tasks involved in providing care to patients and elderly people.

[1564] "Surveillance work" refers to the regular or continuous observation of a facility or specific area to ensure its safety.

[1565] The "recording device" is a device that records the status of the work performed by the worker robot as video and data.

[1566] "Real-time" refers to the instantaneous transfer of data and video without delay.

[1567] "Distribution" means that the server sends the video and data it receives to the terminal and displays it.

[1568] "Notification" is the process of sending alerts and messages to users when the system detects an abnormality.

[1569] overview

[1570] The present invention is a system that allows a user to log in using a terminal and input instructions to have a worker robot perform nursing care or monitoring tasks. This system includes a server, a terminal, a worker robot, a recording device, and a real-time distribution and notification function.

[1571] System Configuration

[1572] server

[1573] The server acts as the center of the system and performs the following functions:

[1574] 1. Login authentication: Receives the login information entered by the user from the terminal and authenticates it by checking it against a database.

[1575] 2. Instruction transfer: Analyzes instructions received from the user and transfers them to the corresponding worker robot.

[1576] 3. Video distribution: Real-time video received from the worker robot is distributed to the terminal.

[1577] 4. Secure communication: Received data is communicated using a secure protocol (e.g. HTTPS).

[1578] 5. Abnormality notification: If the worker robot or user detects an abnormality, a notification is sent to the user.

[1579] Terminal

[1580] The terminal is an electronic device that allows users to operate the system and has the following functions:

[1581] 1. Entering login information: The user enters their login ID and password.

[1582] 2. Inputting instructions: Entering instructions regarding nursing care and monitoring tasks.

[1583] 3. Video display: Displays real-time video streamed from the server.

[1584] 4. Notification reception: Receive notifications sent from the server when an abnormality occurs.

[1585] Worker robot

[1586] Employee robots are automated machines equipped with AI that perform the following functions:

[1587] 1. Execute Instructions: Performs caregiving or supervisory tasks based on received instructions.

[1588] 2. Recording of execution status: The status of work is recorded in real time by a camera and sent to a server as video data.

[1589] Hardware and software used

[1590] Server: A computer with high-performance computing power (e.g., Amazon Web Services, Google Cloud Platform)

[1591] Devices: Electronic devices such as smartphones, tablets, and computers

[1592] Recording device: High-resolution camera (e.g. Logitech Webcam)

[1593] AI model: Generative AI model (e.g. GPT-3, BERT)

[1594] Secure protocol: HTTPS

[1595] Specific examples

[1596] If a user uses a terminal to input instructions to monitor a specific area at 9:00 a.m., the system operates as follows:

[1597] 1. Terminal: The user enters login information and accesses the system.

[1598] 2. Server: Authenticates the login information and allows the user access. When the user enters an instruction to "monitor a specific area," the server receives this instruction.

[1599] 3. Server: Analyzes the received instructions and forwards them to the appropriate worker robot.

[1600] 4. Worker robot: Based on instructions, it patrols designated areas and performs surveillance tasks. It records images of its patrols in real time with a camera.

[1601] 5. Server: Receives the video sent from the robot and distributes it to the terminal.

[1602] 6. Terminal: The user checks the real-time video and inputs additional instructions if an abnormality is detected. If an abnormality is detected, the user receives a notification.

[1603] Prompt Sentence Examples

[1604] To further refine the design using the generative AI model, use the following prompt:

[1605] Please explain the design of a security robot system. The system allows users to log in from their smartphones and instruct the AI-equipped robots on how to perform monitoring tasks. The robots will send real-time video footage to the smartphones. If an abnormality is detected, notifications will be sent to the users. This system will be built using Python, Flask, and OpenCV.

[1606] This prompt can be used to obtain detailed design information and implementation methods from the AI ​​model.

[1607] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1608] Step 1:

[1609] The user enters their login information using the device.

[1610] Input: User's login ID and password

[1611] Operation: The user enters login information into the device's login screen and presses the "Login" button.

[1612] Output: The device sends the login information to the server.

[1613] Step 2:

[1614] The server authenticates the received login information.

[1615] Input: Login ID and password sent from the device

[1616] How it works: The server checks the received login information against a database to determine if the user is legitimate.

[1617] Output: If authentication is successful, the server generates an authentication token and sends it to the terminal. If authentication fails, it returns an error message to the terminal.

[1618] Step 3:

[1619] The user inputs instructions regarding the care work or monitoring work from the terminal.

[1620] Input: Authentication token after logging in, instructions for care work or monitoring work (e.g., "Monitor a specific area")

[1621] Operation: The terminal receives instructions entered by the user and sends them to the server.

[1622] Output: Data containing instructions and an authentication token is sent to the server.

[1623] Step 4:

[1624] The server receives the input instructions and forwards the instructions to the worker robot.

[1625] Input: Instructions, authentication token

[1626] Operation: The server analyzes the received instructions and forwards them to the corresponding worker robot.

[1627] Output: The parsed instructions are sent to the worker robot.

[1628] Step 5:

[1629] The worker robot performs caregiving or monitoring tasks based on the received instructions.

[1630] Input: Instructions sent by the server

[1631] Action: The worker robot follows instructions and begins nursing or monitoring tasks.

[1632] Output: The execution status is recorded in real time by a camera.

[1633] Step 6:

[1634] The worker robot transmits the recorded video to a server in real time.

[1635] Input: Camera footage

[1636] Operation: The worker robot transmits the recorded video in real time to the server.

[1637] Output: Real-time video data is sent to the server.

[1638] Step 7:

[1639] The server distributes the received video to the terminal, and the terminal displays the video to the user.

[1640] Input: Real-time video sent from the worker robot

[1641] Operation: The server delivers the received video to the device using a secure protocol.

[1642] Output: The distributed real-time video is displayed on the terminal.

[1643] Step 8:

[1644] The server will notify the user when an abnormality is detected.

[1645] Input: Anomaly detection information or manual detection input by the user

[1646] Operation: When the server detects an abnormality, it sends a push notification to the user device.

[1647] Output: An abnormality notification is displayed on the user's terminal.

[1648] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1649] In addition to the conventional nursing care support system, the present invention provides a new technology that recognizes and responds to the emotions of users and patients. Hereinafter, embodiments of the present invention will be described in detail.

[1650] In this system, the user uses a terminal to give instructions on care tasks, and the AI ​​robot carries out the tasks based on those instructions. The system also captures the progress of the tasks with a camera and can be monitored remotely in real time. Furthermore, an emotion engine is used to recognize the emotions of the user and patient, optimizing the operation of the entire system.

[1651] Login and Authentication Process

[1652] Authentication process

[1653] When a user tries to access a system using a terminal, they must first log in. The terminal displays a login screen to the user and asks them to enter their login ID and password. Once the user enters the login information, the terminal sends it to the server. The server compares the received login information with a database and performs authentication. If this authentication is successful, the server sends an authentication token to the terminal, and the user gains access to the system.

[1654] Furthermore, the emotion engine analyzes the user's facial expressions and voice to recognize their emotional state at the time of login, which makes it possible to detect the user's stress level and fatigue at that time.

[1655] Directing and carrying out nursing care tasks

[1656] Sending instructions

[1657] After the user logs in from the device, an interface for inputting instructions for care work is displayed. For example, specific care instructions such as "move patient A from the wheelchair to the bed" are entered. The device sends this instruction along with an authentication token to the server. The server analyzes the received instructions and forwards them to the AI ​​robot.

[1658] In addition, when inputting instructions, the emotion engine analyzes the user's emotions and sends the results to the server, which can then transfer instructions to the AI ​​robot that take into account the user's emotional state.

[1659] Robot movement

[1660] The AI ​​robot receives instructions from the server and begins its nursing care tasks based on those instructions. For example, it safely performs tasks such as moving and assisting the patient. While working, the robot captures its movements with a camera and sends the footage to the server in real time. Furthermore, the AI ​​robot is equipped with an emotion engine that recognizes emotions from the patient's facial expressions and voice, and adjusts its tasks as necessary.

[1661] Video distribution and monitoring

[1662] Real-time monitoring

[1663] The server receives the video sent by the AI ​​robot. This video is then distributed to the device using a secure protocol. The device displays the video in real time, allowing the user to monitor it. The emotion engine continuously monitors the user's emotional state and issues an alert if an abnormality is detected.

[1664] Specific examples

[1665] For example, if a user inputs an instruction into a terminal to move patient A from a wheelchair to a bed at 9:00 a.m., the system operates as follows:

[1666] 1. The user logs in to the terminal and inputs the command, "Move patient A from the wheelchair to the bed." At this time, the emotion engine analyzes the user's facial expressions and voice and sends the emotional state to the server.

[1667] 2. The device sends the input instructions to the server, which then analyzes the received instruction data and emotion data.

[1668] 3. The server transfers instructions and emotional data to the AI ​​robot. If the user is under stress, the robot will adjust its work pace to take that situation into account.

[1669] 4. The AI ​​robot begins its nursing care tasks based on the instructions. When transferring Patient A from the wheelchair to the bed, the emotion engine also monitors the patient's emotional state.

[1670] 5. While the AI ​​robot is working, a camera captures its movements and sends the footage to a server in real time.

[1671] 6. The server delivers the video data to the device, which displays the video to the user in real time.

[1672] 7. The user monitors the video on the device to ensure that the care work is being carried out properly. The emotion engine also continuously monitors the user's emotions and issues an alert if there are any abnormalities.

[1673] This invention not only realizes automation and efficiency in nursing care work, dramatically reducing the burden on caregivers, but also provides a safer and more comfortable nursing environment through emotion recognition, making it an extremely useful system in an aging society.

[1674] The processing flow will be explained below.

[1675] Step 1:

[1676] The user accesses the login screen of the device, and the device displays a form for entering the login ID and password.

[1677] Step 2:

[1678] The user enters their login ID and password and clicks the send button.

[1679] Step 3:

[1680] The terminal encrypts the entered login ID and password and sends them to the server.

[1681] Step 4:

[1682] The server decrypts the encrypted login ID and password it receives and checks them against the database.

[1683] Step 5:

[1684] The server checks the authentication result, and if the authentication is successful, generates an authentication token and sends it to the device.

[1685] Step 6:

[1686] The device receives a successful authentication message and token, and displays a message to the user indicating that they have been granted access to the system. The emotion engine analyzes the user's facial expressions and voice to confirm their emotional state at that time.

[1687] Step 7:

[1688] The user accesses an interface that allows them to input instructions for nursing care tasks into the terminal. For example, they input an instruction to "move patient A from the wheelchair to the bed."

[1689] Step 8:

[1690] The emotion engine analyzes the facial expressions and voice of the user inputting instructions and generates emotion data.

[1691] Step 9:

[1692] The terminal transmits the input instruction data, emotion data, and authentication token to the server.

[1693] Step 10:

[1694] The server checks the received instruction data, emotion data, and authentication token, and analyzes the instruction content, taking into account the user's emotional state.

[1695] Step 11:

[1696] The server analyzes the instruction data and transfers it to the AI ​​robot. If the user is under stress, the instructions to the robot are adjusted.

[1697] Step 12:

[1698] The AI ​​robot receives and analyzes the instructions, then begins its caregiving tasks, such as moving patient A from a wheelchair to a bed.

[1699] Step 13:

[1700] The emotion engine analyzes the patient's facial expressions and voice to assess their emotional state and adjust the work as needed.

[1701] Step 14:

[1702] While the AI ​​robot is performing its nursing care duties, a camera will record what is happening.

[1703] Step 15:

[1704] The AI ​​robot sends the footage it captures in real time to a server.

[1705] Step 16:

[1706] The server delivers the received video data to the terminal using a secure protocol.

[1707] Step 17:

[1708] The terminal displays the video data received from the server to the user, and the emotion engine continuously monitors the user's emotional state.

[1709] Step 18:

[1710] Users can monitor the video footage via their devices to ensure that care work is being carried out properly, and if there is an abnormality, a warning is issued.

[1711] Step 19:

[1712] If necessary, the user inputs additional instructions, and the terminal transmits the data to the server again.

[1713] Step 20:

[1714] The server forwards the received additional instructions to the AI ​​robot, which then changes or performs additional nursing care tasks based on the instructions.

[1715] Example 2

[1716] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1717] Conventional nursing care support systems have difficulty recognizing the emotional states of caregivers and patients and responding appropriately. Furthermore, even though remote real-time monitoring is possible, there is a lack of systems that can optimize behavior based on emotions or detect abnormalities, making it difficult to fully ensure the safety and efficiency of nursing care work.

[1718] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input login information using a terminal; a means for the terminal to transmit the login information to the server and for the server to authenticate the received login information; a means having an emotion analysis engine for recognizing the emotional states of the user and the patient; a means for the user to input instructions related to the care work from the terminal; a means for the server to receive, analyze, and forward the input instructions to the AI ​​robot; a means for the AI ​​robot to perform the care work based on the received instructions and capture the performance status with a camera; a means for transmitting the video captured by the AI ​​robot to the server in real time; a means for the server to optimize the operation by taking into account the emotional states of the user and the patient via the emotion analysis engine; a means for the server to distribute the received video to the terminal, which then displays the video to the user and continuously monitors the user's emotional state; and a means for issuing an alert when an abnormality is detected. This improves the safety and efficiency of care work and enables more appropriate care that takes into account the emotional state.

[1719] "Login information" refers to the authentication data that a user enters when accessing a system, and typically consists of a user ID and password.

[1720] A "terminal" is a device used by a user to access the nursing care support system, input instructions, and monitor video footage, and includes PCs, tablets, smartphones, etc.

[1721] The "server" is a computer system that serves as the core of the nursing care support system and performs various processes such as login authentication, instruction analysis, data transfer, emotional state analysis, and real-time video distribution.

[1722] An "emotion analysis engine" is a software or hardware component that analyzes the facial expressions and voice of a user or patient to recognize and evaluate their emotional state.

[1723] "Instruction data" refers to data that includes specific instructions regarding care work input by the user via the terminal.

[1724] An "AI robot" is an autonomous robot that performs nursing care tasks based on instructions received from a server, and is equipped with a camera, emotion analysis engine, and other features.

[1725] The "camera" is a device installed on the AI ​​robot that captures the nursing care work being performed in real time and sends the footage to a server.

[1726] A "secure protocol" is a secure communication method that prevents data transmission and reception from being tampered with or intercepted by third parties, and includes SSL / TLS.

[1727] An "authentication token" is temporary data issued by the server when a user is successfully authenticated, and is used to verify that subsequent communications and instructions are legitimate.

[1728] "Real-time streaming" is a technology that transmits footage captured by an AI robot from a server to a terminal with almost no delay, allowing users to monitor it immediately.

[1729] "Abnormality detection" is a function that allows the system to detect abnormal patterns in the emotional state of a user or patient or the execution of care tasks and issue a warning.

[1730] In addition to the conventional nursing care support system, the present invention provides a technology that recognizes the emotions of users and patients and takes optimal measures according to their emotional states. Hereinafter, embodiments of the present invention will be described in detail.

[1731] This system consists of the following main components: a server, a terminal, an AI robot, a camera, and an emotion analysis engine. The functions and interactions of each component are described in detail below.

[1732] Server Roles

[1733] The server acts as the central part of the system and performs the following main tasks:

[1734] 1. Login authentication: The login information entered by the user from the terminal is compared with the database. If authentication is successful, an authentication token is issued.

[1735] 2. Receiving and analyzing instruction data: Receives and analyzes instructions for nursing care tasks entered by the user through the terminal.

[1736] 3. Transfer of instruction data: Based on the analysis results, appropriate instruction data is transferred to the AI ​​robot.

[1737] 4. Sentiment analysis and optimization: Use a sentiment analysis engine to analyze the emotional state of users and patients and optimize operations.

[1738] 5. Real-time video distribution: Receives video data sent from the AI ​​robot and distributes it to the terminal in real time.

[1739] 6. Anomaly detection and warning: If an abnormality is detected in the user or patient's condition, a warning will be issued to the terminal.

[1740] Device Role

[1741] The terminal is used by the user to access the system, enter commands, and monitor footage. Terminal functions include:

[1742] 1. Displaying a login screen: A login screen is displayed to the user and they are prompted to enter their login information.

[1743] 2. Providing an instruction input interface: An interface is provided for the user to input specific nursing care instructions.

[1744] 3. Real-time video display: Displays the video received from the server in real time to monitor the progress of nursing care work.

[1745] 4. Displaying a warning notification: When an abnormality is detected, a warning is displayed to the user.

[1746] The role of AI robots

[1747] The AI ​​robot performs care tasks based on instructions received from the server. Specific functions include:

[1748] 1. Carrying out care tasks: Carry out care tasks (e.g., moving or assisting patients) according to received instructions.

[1749] 2. Video recording: The nursing care work is recorded with a camera and the video is sent to the server in real time.

[1750] 3. Emotion Analysis: Use the built-in emotion analysis engine to analyze the patient's emotional state and optimize the progress of the work.

[1751] The role of sentiment analysis engines

[1752] The emotion analysis engine analyzes the facial expressions and voice of users and patients to recognize their emotional state, providing important data for optimizing the operation of the entire system.

[1753] Specific examples

[1754] For example, if a user inputs the instruction "Move patient A from his wheelchair to his bed" into the terminal at 9:00 AM, the system will perform the following specific operations:

[1755] 1. The user logs in to the device and inputs instructions. The emotion analysis engine analyzes the user's facial expressions and voice and sends their emotional state to the server.

[1756] 2. The terminal transmits the input instructions and emotion data to the server.

[1757] 3. The server analyzes the instruction data and emotional data and transmits them to the AI ​​robot. If stress is detected, the robot will adjust its work pace.

[1758] 4. The AI ​​robot will begin nursing care tasks based on instructions, while simultaneously analyzing the patient's emotions and optimizing its tasks accordingly.

[1759] 5. The AI ​​robot uses a camera to record its work and sends the footage to a server in real time.

[1760] 6. The server delivers the video data to the terminal and displays it to the user in real time.

[1761] 7. The user monitors the video through the device, and if the emotion analysis engine detects an abnormality, an alert is issued.

[1762] Prompt Sentence Examples

[1763] "Transfer patient A from wheelchair to bed at 9:00 AM"

[1764] "Please notify me of patient B's medication time."

[1765] "Check on patient C's safety"

[1766] This invention not only realizes automation and efficiency of nursing care work, reducing the burden on caregivers, but also makes it possible to provide a safer and more comfortable nursing care environment through emotion recognition.

[1767] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1768] Step 1:

[1769] Enter and submit your login information

[1770] The user accesses the login screen using the device and enters the login ID and password.

[1771] Input: The login ID and password entered by the user.

[1772] Output: Data format in which login information (ID and password) is sent to the terminal

[1773] Specific operation: The terminal receives the user's input and sends it to the server.

[1774] Step 2:

[1775] Authenticate your login details

[1776] The server compares the login information received from the terminal with the database to perform authentication.

[1777] Input: Login information sent from the device

[1778] Output: Authentication token or authentication failure message

[1779] Specific operation: The server checks the login information against the database, and if authentication is successful, it generates an authentication token and sends it to the terminal. If authentication fails, it returns an error message.

[1780] Step 3:

[1781] Emotional state analysis

[1782] The emotion analysis engine analyzes the user's facial expressions and voice to recognize their emotional state at the time of login.

[1783] Input: User's facial expression and voice data

[1784] Output: Emotional state information (stress level, fatigue level, etc.)

[1785] How it works: The emotion analysis engine monitors the user's face and voice in real time and evaluates their state using an emotion analysis algorithm. The evaluation results are sent to the server and recorded.

[1786] Step 4:

[1787] Entering nursing care instructions

[1788] The user inputs specific instructions regarding the care work from the terminal.

[1789] Input: Care work instructions entered by the user

[1790] Output: Instruction data (specific nursing care work content)

[1791] Specific operation: The terminal provides an interface for inputting instructions for nursing care work, and the user inputs specific instructions (e.g., "Move patient A from the wheelchair to the bed").

[1792] Step 5:

[1793] Sending and analyzing instruction data

[1794] Care work instructions and emotional state information are sent from the terminal to the server, which then analyzes them.

[1795] Input: Care instructions and emotional state information

[1796] Output: Analysis results (optimized instruction data)

[1797] Specific operation: The server analyzes the received instruction data and generates optimized instructions that take into account the emotional state information. These instructions are then forwarded to the AI ​​robot.

[1798] Step 6:

[1799] Carrying out nursing care tasks with AI robots

[1800] The AI ​​robot performs nursing care tasks based on instructions from the server.

[1801] Input: Optimized instruction data

[1802] Output: Status of nursing care work and video data

[1803] Specific actions: The AI ​​robot begins specific care tasks (e.g., moving a patient) based on the instructions it receives. The robot's actions are captured in real time by a camera.

[1804] Step 7:

[1805] Real-time video transmission and distribution

[1806] The video data captured by the AI ​​robot is sent to a server, which then distributes it to the device in real time.

[1807] Input: Real-time video data

[1808] Output: Real-time video streaming data

[1809] How it works: The AI ​​robot captures video of itself working with a camera and sends it in real time to a server, which then receives the data and distributes it to the device using a secure protocol.

[1810] Step 8:

[1811] Anomaly detection and warning

[1812] The server uses an emotion analysis engine to continuously monitor the emotional state of users and patients and issues an alert if an abnormality is detected.

[1813] Input: Emotional state information and care work execution state

[1814] Output: Warning notice

[1815] Specific operation: The server analyzes the data from the sentiment analysis engine, and if an anomaly is detected, it sends a warning message to the device. The user receives the warning on the device and can take appropriate action.

[1816] In this way, this system works by linking users, servers, AI robots, and emotion analysis engines to carry out nursing care tasks efficiently and safely.

[1817] (Application example 2)

[1818] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1819] Current work support systems only transfer and execute simple instructions without considering the emotional state of the user or target, which means that the safety and efficiency of the work environment cannot be sufficiently maintained.In addition, because they are unable to provide feedback based on real-time emotion recognition, stress and fatigue levels may be overlooked, which could have a negative impact on the health and safety of workers and targets.

[1820] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input login information using a terminal; a means for the terminal to transmit the login information to the server and for the server to authenticate the received login information; a means for the user to input instructions related to nursing care work from the terminal; a means for the server to receive the input instructions and forward the instructions to the AI ​​processing device; a means for the AI ​​processing device to perform work based on the received instructions and record the execution status using an optical device; a means for the AI ​​processing device to transmit the recorded video to the server in real time; a means for the server to distribute the received video to the terminal and for the terminal to display the video to the user; a means for analyzing the emotions of the user and the target person in real time using an emotion recognition engine; a means for adjusting the operation of the AI ​​processing device based on the target person's emotional state; and a means for analyzing the video from the optical device and issuing a safety warning based on the results of emotion recognition. This makes it possible to improve the safety and efficiency of the work environment.

[1821] A "user" is a person who operates the system and gives instructions for work.

[1822] "Terminal" refers to an electronic device operated by a user, including smart glasses and PCs.

[1823] "Login information" means the authentication information provided by a user to access the system, and specifically includes a user ID and password.

[1824] A "server" is a computer system on a network that manages the entire system and processes data.

[1825] An "authentication token" is authentication data for subsequent communications that is issued by the server after a user has successfully logged in.

[1826] An "AI processing device" is a device that uses AI technology to analyze instructions and automatically execute tasks.

[1827] An "optical device" is a device for recording images, such as a camera.

[1828] An "emotion recognition engine" is a technology or software that analyzes the emotional state of a user or target person in real time from their facial expressions and voice.

[1829] "Real-time" refers to data processing and communication occurring instantly, without delay.

[1830] "Target" refers to the person to whom the system provides services, and in the case of a care robot, it means the person being cared for.

[1831] "Safety warnings" are caution or warning messages issued for the safety of users or targets based on the analysis results of the emotion recognition engine.

[1832] This invention is a system for improving the safety and efficiency of a work environment. This system is composed of a user, a terminal, a server, an AI processing device, an optical device, and an emotion recognition engine.

[1833] First, the user enters login information using the terminal. The terminal then sends this login information to the server, which then performs authentication using the received login information. If authentication is successful, the server generates an authentication token and sends it to the terminal.

[1834] Next, the user can input instructions related to the work from the device. For example, a command to start a specific task may be given. The device then sends this instruction along with an authentication token to the server. The server then analyzes the received instruction and forwards it to the AI ​​processing device based on its content.

[1835] The AI ​​processing device performs tasks based on the received instructions and records its progress using an optical device (such as a camera). The recorded video is sent to a server in real time. The server then distributes the received video to a terminal, allowing users to monitor the video in real time through their terminal.

[1836] Furthermore, the emotional state of the user and the target can be analyzed in real time using an emotion recognition engine. The emotion recognition engine analyzes the facial expressions and voice data of the user and the target, and adjusts the operation of the AI ​​processing unit based on the analysis results. It is also possible to issue safety warnings as needed based on the analysis results.

[1837] For example, if a factory worker wears smart glasses, the glasses' camera and microphone can be used to analyze the worker's facial expressions and voice to recognize their emotional state in real time. Based on this recognition result, the system can determine whether the working environment is appropriate and issue a warning if necessary.

[1838] For example, if Worker A is wearing smart glasses at the start of his shift at 8 a.m. and the emotion recognition engine detects fatigue, the system will automatically issue a warning and prompt the worker to take a break, thus optimizing the working environment and improving safety and work efficiency.

[1839] An example of a prompt sentence is as follows:

[1840] "How can we monitor the emotions of factory workers in real time and issue warnings if fatigue or stress is detected to improve work efficiency?"

[1841] This allows the system embodying the invention to take into account the emotional state of the user and the subject, providing a safer and more efficient working environment.

[1842] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1843] Step 1:

[1844] The user enters login information using the terminal. The terminal displays the login screen, and the user enters their login ID and password. The terminal then sends the entered login information to the server.

[1845] Input: User login ID and password

[1846] Output: Login information sent to the server

[1847] Step 2:

[1848] The server authenticates the received login information. The server accesses a database and checks the received login information. If authentication is successful, the server generates an authentication token and sends it to the terminal.

[1849] Input: Login information

[1850] Output: Authentication token

[1851] Step 3:

[1852] The user inputs work instructions from the terminal. The authenticated terminal displays the work instruction input screen, and the user inputs specific work instructions (e.g., instructions for a specific task). The terminal then sends these instructions to the server along with an authentication token.

[1853] Input: Work instructions, authentication token

[1854] Output: Work instructions sent to the server

[1855] Step 4:

[1856] The server receives and analyzes the input instructions, and then transfers the instructions to the appropriate AI processing device based on their content.

[1857] Input: Work Instructions

[1858] Output: Instructions to the AI ​​processor

[1859] Step 5:

[1860] The AI ​​processing unit executes tasks based on the instructions it receives, such as a robot performing a specific action. The AI ​​processing unit continues to record the progress of the task using an optical device.

[1861] Input: Instructions from the server

[1862] Output: Work progress

[1863] Step 6:

[1864] The AI ​​processing device transmits the recorded video to the server in real time. The AI ​​processing device transmits the video data captured by the optical device to the server in real time.

[1865] Input: Recorded video

[1866] Output: Video data sent to the server

[1867] Step 7:

[1868] The server delivers the received video to the terminal. The server delivers the received video to the terminal in real time using a secure protocol. The terminal displays this video to the user.

[1869] Input: Video data

[1870] Output: The image displayed to the user

[1871] Step 8:

[1872] The emotion recognition engine analyzes the emotions of the user and the target. The emotion recognition model is run using video and audio data. Based on the analysis results, it determines whether the operation of the AI ​​processing unit needs to be adjusted.

[1873] Input: Video data, audio data

[1874] Output: Emotion analysis results

[1875] Step 9:

[1876] The emotion recognition engine issues safety warnings based on the analysis results. If the analysis results indicate that the user or target is in danger, the system will automatically issue a warning and prompt the user to take necessary measures.

[1877] Input: Sentiment analysis results

[1878] Output: Safety warning

[1879] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1880] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1881] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1882] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1883] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1884] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1885] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1886] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1887] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1888] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1889] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1890] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1891] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1892] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1893] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1894] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1895] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1896] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1897] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1898] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1899] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1900] The following is further disclosed regarding the above embodiment.

[1901] (Claim 1)

[1902] A means for a user to input login information using a terminal;

[1903] a means for the terminal to transmit login information to a server, and for the server to authenticate the received login information;

[1904] A means for a user to input instructions regarding care work from a terminal;

[1905] a means for the server to receive the input instructions and forward the instructions to the AI ​​robot;

[1906] A means for the AI ​​robot to perform care work based on the received instructions and to take pictures of the execution status with a camera;

[1907] A means to send the images captured by the AI ​​robot to a server in real time,

[1908] The system includes a means for distributing the video received by the server to a terminal, and for the terminal to display the video to a user.

[1909] (Claim 2)

[1910] 10. The system of claim 1, wherein the server communicates the received data using a secure protocol.

[1911] (Claim 3)

[1912] 10. The system of claim 1, further comprising means for generating an authentication token when a user logs in from a terminal and using the token to verify the integrity of subsequent instructions.

[1913] "Example 1"

[1914] (Claim 1)

[1915] A means by which a user enters login information using the device;

[1916] means for the device to transmit login information to a computer and for the computer to authenticate the received login information;

[1917] a means for a user to input instructions regarding care work from the device;

[1918] means for the computer to receive input instructions and forward the instructions to the artificial intelligence device;

[1919] a means for executing a care work based on the instructions received by the artificial intelligence device and capturing an image of the execution status with an image capturing device;

[1920] a means for transmitting the images captured by the artificial intelligence device to a computer in real time;

[1921] The system includes a means for distributing the video received by the computer to the device, and for the device to display the video to the user.

[1922] (Claim 2)

[1923] 10. The system of claim 1, wherein the computer communicates the received data using a secure protocol.

[1924] (Claim 3)

[1925] 10. The system of claim 1, further comprising means for generating an authentication token when a user logs in from a device and using the token to verify the integrity of subsequent instructions.

[1926] "Application Example 1"

[1927] Rewritten Claims

[1928] (Claim 1)

[1929] A means for a user to input login information using a terminal;

[1930] a means for the terminal to transmit login information to a server, and for the server to authenticate the received login information;

[1931] A means for a user to input instructions regarding care work from a terminal;

[1932] a means for the server to receive the input instruction and transfer the instruction to the worker robot;

[1933] a means for causing the worker robot to perform a caregiving task or a monitoring task based on the received instruction, and recording the execution status with a recording device;

[1934] a means for transmitting the video recorded by the worker robot to a server in real time;

[1935] means for the server to distribute the received video to the terminal, and for the terminal to display the video to the user;

[1936] The system includes a means for issuing a notification when a user detects an abnormality.

[1937] (Claim 2)

[1938] 10. The system of claim 1, wherein the server communicates the received data using a secure protocol.

[1939] (Claim 3)

[1940] 10. The system of claim 1, further comprising means for generating an authentication code when a user logs in from a terminal and using the authentication code to verify the integrity of subsequent instructions.

[1941] "Example 2: Combining Emotion Engines"

[1942] (Claim 1)

[1943] A means for a user to input login information using a terminal;

[1944] a means for the terminal to transmit login information to a server, and for the server to authenticate the received login information;

[1945] means comprising an emotion analysis engine for recognizing the emotional state of a user and a patient;

[1946] A means for a user to input instructions regarding care work from a terminal;

[1947] A means for the server to receive the input instructions, analyze them, and forward them to the AI ​​robot;

[1948] A means for the AI ​​robot to perform care work based on the received instructions and to take pictures of the execution status with a camera;

[1949] A means to send the images captured by the AI ​​robot to a server in real time,

[1950] means for the server to take into account the emotional state of the user and the patient via an emotion analysis engine and optimize the operation;

[1951] means for transmitting the received video from the server to the terminal, and for the terminal to display the video to the user and continuously monitor the emotional state of the user;

[1952] A system that includes a means for issuing an alert when an abnormality is detected.

[1953] (Claim 2)

[1954] 10. The system of claim 1, wherein the server communicates the received data using a secure protocol.

[1955] (Claim 3)

[1956] 10. The system of claim 1, further comprising means for generating an authentication token when a user logs in from a terminal and using the token to verify the integrity of subsequent instructions.

[1957] "Application example 2 when combining emotion engines"

[1958] (Claim 1)

[1959] A means for a user to input login information using a terminal;

[1960] a means for the terminal to transmit login information to a server, and for the server to authenticate the received login information;

[1961] A means for a user to input instructions regarding care work from a terminal;

[1962] A server receives input instructions and transfers the instructions to an AI processing device;

[1963] A means for executing a task based on the received instructions by the AI ​​processing device and recording the execution status with an optical device;

[1964] A means for transmitting the video recorded by the AI ​​processing device to a server in real time;

[1965] means for the server to distribute the received video to the terminal, and for the terminal to display the video to the user;

[1966] means for analyzing the emotions of users and subjects in real time using an emotion recognition engine;

[1967] means for adjusting the operation of the AI ​​processing device based on the emotional state of the subject;

[1968] means for analyzing an image from the optical device and issuing a safety warning based on the result of emotion recognition;

[1969] A system including:

[1970] (Claim 2)

[1971] 10. The system of claim 1, wherein the server communicates the received data using a secure protocol.

[1972] (Claim 3)

[1973] 10. The system of claim 1, further comprising means for generating an authentication token when a user logs in from a terminal and using the token to verify the integrity of subsequent instructions. [Explanation of symbols]

[1974] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for a user to input login information using a terminal; a means for the terminal to transmit login information to a server, and for the server to authenticate the received login information; A means for a user to input instructions regarding care work from a terminal; a means for the server to receive the input instructions and forward the instructions to the AI ​​robot; A means for the AI ​​robot to perform care work based on the received instructions and to take pictures of the execution status with a camera; A means to send the images captured by the AI ​​robot to a server in real time, The system includes a means for distributing the video received by the server to a terminal, and for the terminal to display the video to a user.

2. 10. The system of claim 1, wherein the server communicates the received data using a secure protocol.

3. 10. The system of claim 1, further comprising means for generating an authentication token when a user logs in from a terminal and using the token to verify the integrity of subsequent instructions.

Images